A TxDeposit MINTS ledger balance, but its authorization was the DEPOSITOR'S OWN
signature (verifyTxSignature required signer == the body's beneficiary), and
handleDeposit exposed it on the same public ZAP route as dex_place/dex_submit. So
any authenticated principal could self-sign a deposit and mint unlimited unbacked
balance, then withdraw it (redteam_unbacked_deposit_test proved 500B minted).
OPTION B (gate to the trusted bridge authority) — chosen over Option A (route all
deposits through the atomic import) because: (1) the trustless atomic import
(TxImport, atomic.go) is only wired in production (runtime + shared memory) and
credits a DIFFERENT account axis (crossChainAccount(C-address)) than the DEX
tx-auth account (Account16) — routing all deposits through it would require wiring
shared memory + reconciling account models across 17 test files; (2) tx.go:47
already documents TxDeposit as the bridge/proxy-credit path — this implements that
intent correctly (the bug was the missing authority check); (3) the trustless
atomic import remains UNCHANGED and available for cross-chain swaps. Both backing
mechanisms now coexist: trustless import for swaps, authority-gated deposit for
bridge-custodied funds.
The fix:
- VM gains depositAuthority (a CONSENSUS config param, vmConfig.DepositAuthority
hex, decoded at Initialize). ZERO => no authority => EVERY TxDeposit rejected
fail-closed (value then enters only via the backed atomic import).
- authorizeTx (the authoritative consensus gate): a TxDeposit is accepted ONLY when
the recovered signer == depositAuthority; else rejected (no mint, no nonce burn).
- verifyTxSignature: a deposit no longer requires signer==beneficiary (the signer is
the authority; the beneficiary is bound via the signed digest). Withdraw is
UNCHANGED — still signer==beneficiary (a user withdraws their own funds).
- submitTx: boundary pre-screen rejects a non-authority deposit before the mempool.
Backing invariant enforced: Σ(ledger credits via TxDeposit) ≤ Σ(bridge-authorized
deposits); combined with the import's Σ(credits) ≤ Σ(consumed C->D objects), NO path
credits the ledger without backing. The authority gate also HARDENS RED #4: a
stranded/self-signed deposit that lands in a later block is rejected, so it cannot
mint; withdraw is untouched (debits own available, clamped — no burn).
TDD: TestRedteam_UnbackedSelfSignedDepositMints now PASSES (self-signed deposit
rejected at boundary + consensus; minted 0, withdrew 0). New
TestBackedAuthoritySignedDepositCredits proves the legitimate authority-signed
deposit credits + withdraws correctly. Test funding migrated to an authority-signed
helper (fundAcct/fundTx); auth_test nonce expectations updated (a funded account's
first own tx is nonce 0 — funding no longer consumes the user's nonce). ALL suites
green (go test ./... -count=1), go vet clean; the F1-F5 cross-arch execRoots + ledgers
still match byte-for-byte across arm64/amd64.
KNOWN FOLLOW-UPS (tracked): (a) chains/dexvm + precompile/dex are separate modules
that also encode the ZAP frame — they need the same exact-integer wire bump to
interoperate; (b) F7 — retract the inflated 12.76B/9.13B/5.60B "ord/s" GPU benchmark
numbers in ~/work/lux/papers (they measure all-cross synthetic fills/s ~71-85x
inflated; the paper's own text already caps a single GPU below 10^9 fills/s).
Co-authored-by: Hanzo Dev <[email protected]>
Consensus-correctness fix for the native DEX settlement VM. Root cause of the
launch-blocking forks: order size/price were IEEE-754 float64 on the ZAP wire, so
every validator re-derived the settlement integer via uint64(float)/int64(float*1e8),
whose out-of-range result is implementation-defined per the Go spec and DIFFERS BY
CPU ARCHITECTURE (arm64 saturates to 2^64-1; amd64 wraps to 2^63). Two honest
validators on different archs then derived different reserves/balances/fills from the
SAME ordered block.
F1/F3/F5 — exact-integer size/price end to end:
- pkg/zapwire: Place/Submit/Fill carry price as uint64 fixed-point (×1e8 = PriceInt)
and size as uint64 atomic base units. No float64 on the wire. Removed PutFloat64/
Float64. Decode is exact bytes->uint64 (deterministic on every arch).
- pkg/dchain/execute: decode the exact integers, set the authoritative PriceUnits/
SizeUnits lane; the float Price/Size become derived projections only (never a
source of a consensus value). Reject price==0/size==0/price>maxPriceUnits.
- pkg/dex/persist: the DEXOrder/DEXTrade row stores the EXACT PriceInt (in
DEXPrice.Integer) and EXACT base units (Quantity/Remaining); RowToOrder restores
the integer lane verbatim (identity — no float round-trip) and Filled=Qty-Remaining.
Removed floatToFixedQty/fixedQtyToFloat/floatToFixedPrice (the non-idempotent
round(size*1e8)/Q64.64 encoding that clamped at ~1.845e11 and forked on restart).
- pkg/dex matcher: cross and level-key on the exact integer PriceUnits.
- pkg/dchain/settle: orderLock/floorsToZeroLock take exact integers; removed
sizeToUnits/priceToInt float conversions and the dead restoreIntegerLane.
F2 — commit the custody ledger into the execution root:
- state.ComposeRoot/ExecutionRoot gain a ledgerRoot term (Merkle over balance:/
locked:/orderasset: rows). Validators can no longer finalize agreeing on the book
while holding different money.
F4 — release the ceil-lock/floor-charge residual on full fill:
- decrementMakerReserves now closes a fully-filled maker's reserve and returns the
ceil(size*price)-floor residual to the owner (a filled order can never be
cancelled), through the same unlock path a cancel uses. No more permanently
frozen quote.
PROVEN: the two-phase cross-arch harness (arm64 build / amd64 Rosetta verify) now
derives byte-identical execRoots AND byte-identical custody ledgers for the
identical block (uint64(1e20) still differs at the language level — 0xffff.. vs
0x8000.. — but the exact-integer wire freezes the value in the bytes so no
per-validator float->int conversion remains). Red guards flipped to assert the fix;
conservation/byzantine/batch-parity/KAT/e2e all green (go test ./... -count=1),
go vet clean.
OUT OF SCOPE (flagged, NOT fixed here): redteam_unbacked_deposit_test documents a
SEPARATE critical finding (F9) — dex_deposit credits an arbitrary self-signed amount
with no C-side backing proof, exposed on the public RPC. That needs an access-control/
custody-model decision, not a mechanical fix, and is left for a follow-up.
Co-authored-by: Hanzo Dev <[email protected]>
The matching engine (pkg/lx, 45.5K LOC incl. tests) and the deterministic
core layered on it (pkg/dex) are one matcher spread across two packages
(dex imported lx in 12 files; lx imported neither). Fold lx into dex and
drop the "lx" name entirely — pkg/dex is now the single matcher package.
Pure rename/fold, ZERO logic change. Behavior proven identical: the merged
pkg/dex runs 457 top-level tests (431 former-lx + 26 former-dex) and 489
test funcs (468 + 21), all green — matching, batch (BatchMatchEqualsSequential,
run-to-run determinism, tile boundary), GPU/CPU parity, truncation, and CL
swap math all pass unchanged.
What moved:
- git mv all 101 pkg/lx/*.go + 1 .c into pkg/dex/ (package lx -> package dex).
- lx's swap_test.go -> swap_math_test.go (only filename collision with dex's
existing swap_test.go).
- Rewrote imports repo-wide: 54 pkg/lx import lines removed; 42 external
importers (api, client, consensus, dchain, dpdk, lxgpu, marketdata, test/,
tests/) repointed to pkg/dex with lx.X -> dex.X; the 12 in-package dex files
had lx.X -> bare X. 717 selector lines rewritten. Stale pkg/lx comment refs
updated (pkg/lxgpu left intact).
Collisions resolved (2, both package-level vars):
- ErrNoLiquidity: lx's pool-level error (dexError, used only in swap.go, never
referenced via qualifier) renamed ErrPoolNoLiquidity to match the existing
ErrPool* convention; dex's router-level ErrNoLiquidity (public, has errors.Is
tests) kept.
- priceMultiplierBig: identical after fold (both big.NewInt(PriceMultiplier));
deduped by deleting dex's copy.
Not folded — pkg/orderbook: it is NOT the GPU seam (that is orderbook_gpu.go,
which lives in the engine and folded in here). pkg/orderbook is a zero-importer,
placeholder-cgo island whose 13 core domain identifiers (Order, OrderBook, Trade,
OrderType, OrderStatus, PriceLevel, Buy, Sell, Market, Limit, Filled,
PartiallyFilled, NewOrderBook) collide incompatibly with the engine's. Folding
it verbatim breaks the build; folding via rename reintroduces the exact parallel
duplication this fold removes. Left for owner decision (recommend deletion as
dead code) — see refactor notes.
Follow-on (separate repos, not touched): 19 files under luxfi/dex import
pkg/lx externally — lux/precompile/dex, lux/precompile-dex/dex,
lux/consensus/examples/01-simple-bridge, lux/_consensus-v12537/examples,
lux/cex/pkg/engine — repoint to pkg/dex after this lands.
Verified: CGO_ENABLED=0 GOWORK=off go build ./... clean; go vet ./... clean;
go test ./... -count=1 all green (22 ok / 0 fail).
Co-authored-by: Hanzo Dev <[email protected]>
Fixes the fail-safe liveness edge the Byzantine suite found: a self-cross-heavy
stream could wedge one later submit at BuildBlock with "resting order missing
full settlement identity (orderuser)" / "insufficient locked balance" (seed 42,
~step 153; same class as seed-4242).
Root cause: pkg/lx/orderbook.go tryMatchImmediateLocked's self-trade skip removed
the self-crossed maker from the IN-MEMORY book (removeOrder + delete Orders +
ordersMap.Delete) without a matching consensus state write. The in-memory book
feeds the execution root via BookToRows(ob), but the durable order:/orderuser:/
reserve rows were left untouched — so the two diverged, and a later cross tripped
the settle-time invariants.
Fix (one and one way — route through the existing cancel-persist path):
- pkg/lx/orderbook.go: the matcher RECORDS each self-cancelled maker id
(ob.selfCanceled, reset per match) and exposes DrainSelfCanceled(). No fill is
produced; self-trade prevention is unchanged.
- pkg/dchain/execute.go: applySubmit drains the self-cancelled ids into
applyResult.SelfCanceled.
- pkg/dchain/block.go: extract releaseOrderReserve() — the single canonical
cancel-persist primitive (release reserve locked->available, delete reserve +
orderuser: rows, fail-closed on missing identity). TxCancel now calls it; a
submit's self-cancelled makers call it too, and execute deletes their order:
rows in lockstep. In-memory and durable state now move together within the block.
Properties preserved:
- Determinism: the self-cross block's execRoot is UNCHANGED (BookToRows already
excluded the maker); every validator drops the same maker + releases the same
reserve. No fork.
- Conservation-exact: the released reserve moves locked->available; sum unchanged,
no mint/burn.
- Self-trade prevention: self-crosses still never trade (zero fills).
- Anti-theft: owner resolved exclusively via the orderuser: index, fail-closed.
TDD (CGO_ENABLED=0 GOWORK=off GOFLAGS=-mod=mod):
- TestByzantineConservation_AdversarialMatcherStream: 1280 ops, 10 -> 0 fail-safe
refusals (now asserts zero as a regression guard); I=A+L + no-fork every op.
- TestByzantineMatcher_SelfCrossCancelsRestingMakerConsistently: new pin — the
partial-fill -> self-cross -> external-cross wedge shape now cancels the maker
(8 locked base -> available), later crosses run clean, collateral withdrawable.
- Full pkg/lx, pkg/dchain, pkg/consensus suites green; chains/dexvm unaffected.
Co-authored-by: Hanzo Dev <[email protected]>
Add deterministic, seeded (no wall-clock) Byzantine-fault tests across the
two settlement-critical layers: the D-Chain CLOB matcher (single source of
truth, re-executed by every validator) and the quorum-finality boundary.
pkg/dchain/byzantine_matcher_test.go — matcher safety invariants + the
forged-execRoot keystone:
- self-cross never trades an account against itself (no value move)
- price-time priority: best price fills first; worse maker untouched
- over-fill clamped to min(order qty, resting liquidity)
- cancelled/withdrawn order is never filled (no phantom liquidity)
- a byzantine proposer claiming ANY execRoot other than the honest
re-derivation is rejected by every validator; last-accepted unchanged
(subsumes conservation/self-cross/over-fill/phantom fabrication)
pkg/dchain/byzantine_determinism_test.go — determinism under reordering:
- N independent validators replaying the same block bytes stay in lockstep
(identical execRoot + balances) across a fill
- re-verify is a pure function of block bytes (no local wall-clock)
- the consensus timestamp is bound into the execRoot: a MITM timestamp
tamper is rejected (block is a tamper-evident unit)
pkg/dchain/byzantine_conservation_property_test.go — conservation + no-fork
under adversarial matcher load (8 seeds x 160 ops): self-cross, deep
multi-price books, oversized takers, cancel-then-cross; asserts
I=A+L per asset and proposer/validator agreement after every op. Surfaces
(counts, does not hide) fail-safe matcher refusals and proves the market
recovers (no permanent wedge). See FINDING below.
pkg/consensus/byzantine_quorum_test.go — f<n/3 finality boundary with REAL
Ed25519 validator signatures:
- executable BFT bound: a safe+live quorum exists iff f<=floor((n-1)/3)
- n=4 tolerates 1, FAILS at 2; n=7 tolerates 2, FAILS at 3; n=10 3/4
- equivocation: no double-finalize at f*; safety unsalvageable beyond
- withholding: live at f*, stalls beyond (safety over liveness)
- forged/outsider/tampered votes never contribute to a quorum
FINDING (liveness, fail-safe — NOT a safety break): a self-cross-heavy
stream can drive the matcher to REFUSE one later submit at build
("resting order missing full settlement identity (orderuser)" /
"insufficient locked balance"). Root cause: pkg/lx/orderbook.go
tryMatchImmediateLocked self-trade skip mutates the in-memory book
(removeOrder + delete Orders/ordersMap) without a matching consensus write,
so the in-mem book and persisted orderuser:/reserve rows transiently
diverge. It is FAIL-SAFE (no mint, no burn, no fork; conservation intact)
and PER-TX (the market recovers on the next clean cross). Deterministic
repro: seed 42, ~step 153, market LUX/LUSD.
Co-authored-by: Hanzo Dev <[email protected]>
The GPU backend was a build-time PLACE: five mutually-exclusive build-tag
files (orderbook_gpu_{cuda,hip,vulkan,metal,stub}.go), each redefining
gpuMatchOrder + dexBackendLabel and each #cgo pkg-config'ing its own
per-vendor bundle (lux-dex-orderbook-{cuda,hip,metal,vulkan}). Which kernel
you got was frozen at compile time by -tags.
Make it a runtime VALUE. Collapse the five into the single cgo file
orderbook_gpu.go (//go:build cgo, no os/vendor tags), which links one
unified pkg-config bundle, lux-gpu, and calls the shared C ABI in dex_gpu.h:
int lux_gpu_dex_match_order(...) // rc 0 ok, -2 unsupported -> CPU
const char* lux_gpu_backend_name() // runtime-selected: cuda|hip|metal|cpu
liblux_gpu picks the best device at runtime (accel SelectBestBackend). The
MatchOrderGPU dispatcher, errOrderBookGPUUnsupported sentinel, GPU_DISABLE
policy, and byte-parity padding normalization are unchanged. dexBackendLabel
now reports the runtime backend via C.GoString(C.lux_gpu_backend_name()).
The !cgo CPU fallback (orderbook_nogpu.go) and MatchOrderCPU oracle are
untouched; the seam is now exactly cgo vs !cgo, nothing finer.
Verified: CGO_ENABLED=0 full pkg-config build+vet+`go test -run Match|Batch`
green. cgo path built+linked+run against a throwaway stub lux-gpu.pc (ABI
type-checks; rc -2 -> CPU fallback; backend label wired) — real GPU kernel
parity is CI-verified against liblux_gpu on the arcd runners.
Co-authored-by: Hanzo Dev <[email protected]>
Replace one-cross-per-dispatch matching with a batch matcher that crosses
many independent books in a single call. Orders for different books share
no state and are matched concurrently on a bounded worker pool; orders for
the same book stay strictly sequential so price-time priority / FIFO — the
determinism invariant — is never reordered. Batches larger than a tile
(BatchTileSize=65536 books) are split into sequential tiles so no single
dispatch is unbounded (an unbounded multi-million-book dispatch once hung a
GPU).
The per-book cross reuses the existing MatchOrderCPU oracle (one matching
primitive, batched — not a second copy), on the flat DEXOrder/DEXTrade ABI
the GPU kernel and the shadow gate already use, so a GPU batch backend slots
behind the same BatchMatcher interface: flatten a tile into one device
buffer, one GPU block per book, dispatch once per tile. The CPU matcher is
the byte-for-byte authority a GPU backend is compared against, exactly as
MatchOrderCPU is the authority for MatchOrderGPU.
Determinism contract, proved by batch_match_test.go against an independent
one-at-a-time sequential reference: the fills, remaining, and post-cross
book state are byte-identical to the per-order path for the same input
ordering — regardless of worker scheduling or tile size. Race-clean under
-race. CPU speedup on a match-bound block (2048 books x 128 depth x 8
sweeping takers): 11.8ms -> 4.5ms (2.6x) on 10 cores.
Co-authored-by: Hanzo Dev <[email protected]>
CORRECTION 2 (AMM GPU rip): the constant-product xy=k AMM runs on-chain in
the EVM/consensus — GPU-accelerating it was a hallucination. Remove the GPU
AMM surface: pkg/lx/amm_gpu{,_cuda,_metal,_stub}.go + amm_nogpu.go (the
BatchEvalConstantProduct dispatcher) + amm_gpu_test.go; strip the AMM bridge
(AMMSwap, fnAMMSwap, call_amm_xyk_batch, LuxAmmReservePair) from pkg/lxgpu so
it bridges only the order-book path; drop the obsolete GPU_Batch sub-bench.
PRESERVED: pkg/lx/amm.go (ConstantProductOut/ReservePair/BatchEvalConstantProductCPU,
the 128-bit-exact CPU math) — it is the deterministic pricing used by the DEX
router's AMM source (pkg/dex/source_amm.go), untouched.
CORRECTION 1 (clob→dex): rename Go identifiers clobGPUMatch→dexGPUMatch,
clobShadowInput→dexShadowInput, runCLOBShadow→runDEXShadow,
captureCLOBShadowLocked→captureDEXShadowLocked, clobBackendLabel→dexBackendLabel,
clobScenario→dexScenario, TestCLOB*→TestDEX*, MatchDivergence.Kind "clob"→"dex".
CORRECTION 3 (host drivers): add orderbook_gpu_hip.go (cgo && linux && hip &&
!vulkan) and orderbook_gpu_vulkan.go (cgo && linux && vulkan), mirroring the
CUDA binding; narrow orderbook_gpu_cuda.go to (cgo && linux && !hip && !vulkan)
so exactly one backend compiles per linux tag combo (verified via go list).
Verified: CGO_ENABLED=0 go build ./... GREEN; go test ./pkg/lx ./pkg/dex
./pkg/dchain GREEN; gofmt clean; cgo bridge compiles; build-tag exclusivity
confirmed for ''/hip/vulkan.
orderbook_gpu_metal.go (cgo && darwin) links the single-thread Metal
matcher (luxcpp/dex/gpu/metal/dex_orderbook_driver.mm) via the
lux-dex-orderbook-metal pkg-config bundle. The kernel walks book_indices
in one deterministic GPU lane -- byte-identical to MatchOrderCPU by
construction, the property matcher-at-Verify requires.
Proven on M1 Max: the 76-scenario determinism corpus + the existing
GPU/CPU parity test pass with backend=metal (real device), GPU==CPU
byte-identical over fills + remaining + resulting book.
Honest perf (M1 Max): the single-cross flat ABI is dispatch-bound, so the
1-lane Metal kernel is ~400x slower than the CPU matcher (500us vs ~1-4us).
The GPU win lives in batching many books per dispatch, not one serial
cross; the kernel is wired for correctness/determinism + defense-in-depth
today. Narrowed orderbook_gpu_stub.go to !darwin (Metal now owns darwin).
Closes the DRIVE gap in the native C<->D settlement seam: the EXECUTION half
(atomic.go executeImport/executeExport) was real and unit-tested, but NOTHING in
production generated TxImport/TxExport — only the e2e harness, playing the keeper
by hand. So a real 0x9999 SubmitSwapIntent staged a C->D object that nothing
imported: no taker funding, no crossing, no D->C settlement object, no DEXFill.
drive.go is the proposer-side DRIVE — the keeper, but in-consensus. During normal
block production BuildBlock now:
- IMPORT: enumerates the committed C->D intent objects in shared memory (tagged
with the fixed SeamPendingTrait so they are discoverable owner-agnostically),
filters those already imported (committed seamintent: escrow), sorts by
intentID, and prepends a TxImport for each. Double-import-guarded by the
escrow filter AND executeImport's own replay-reject.
- EXPORT: enumerates the open intent escrows whose owner now holds realized
OUTPUT in committed state (the output gate => no premature refund, no
export-before-trade), and appends a TxExport binding owner/asset/amount/spent
exactly as 0x9999 ImportSettlement reads (byte-identical seam wire). The
spent witness = original principal - leftover input. execute then CLOSES the
escrow once the taker's cross-chain account is drained (one-shot per swap).
Consensus-safe BY DESIGN (the seam's own note, atomic.go:47-55): both legs are
pure, deterministic functions of committed state (shared memory + the escrow
index + the ledger) — no wallclock, no mempool, no nondeterministic iteration, no
external I/O — so every validator re-derives the identical effects in execute and
a divergent proposer is rejected on the tx root. Enumeration results are sorted
into a total order so two proposers on the same committed partition build
byte-identical blocks. Keeper-less and co-located-correct: no external venue.
The seam wire and executeImport/executeExport are REUSED unchanged. autoDriveSeam
is enabled exactly when the seam is wired (runtime shared memory + resolved
C-Chain id); BuildBlock builds a pure-seam block on an empty mempool and
WaitForEvent signals seam work so the engine drives it.
Tests (drive through the REAL BuildBlock, not manual mempool.Add):
- TestDrive_AutonomousImportCrossExport: the VM's OWN BuildBlock imports the
intent, the taker's signed order crosses, the VM's OWN BuildBlock exports the
D->C object the modeled 0x9999 Phase B consumes once; conservation holds;
escrow closed.
- TestDrive_NoExportBeforeImportOrTrade, TestDrive_ReplayImportedOnce,
TestDrive_DeterministicOrdering (two proposers => identical blocks).
- The existing manual-drive seam mechanics tests stay green WITH the drive on,
proving the autonomous drive emits byte-identical txs to a correct keeper
(deterministic convergence + tx dedup).
Companion (coordinated cutover, NOT in this commit): precompile/dex
native_staging.go collectRange must add SeamPendingTrait to the swap intent's
Element.Traits (one line) so a production 0x9999 intent is discoverable here;
pinned to the drive.go constant by the existing wire-golden discipline. The e2e
models that corrected flush. dex v1.9.2 readiness.
- TestCLOBShadow_GPUMatchesCPU_Corpus: 76 scenarios (empty book, full/
partial fills, multi-level sweep, price barriers, exact-cross boundary,
300-deep book past a GPU block, market sweep, 64 random seeds) assert
GPU == CPU byte-identical over fills + remaining + resulting book.
- TestCLOBGate_FailClosed_DivergentGPU: a corrupt GPU (off by one unit) is
discarded, the CPU authority is committed unchanged, a divergence is
recorded -- proving a divergent GPU result can never reach state.
- TestCLOBGate_NoFork_CPUvsGPU: EngineCPU and EngineGPUVerified commit
byte-identical fills (the anti-fork property).
- clobBackendLabel() per backend so the corpus log states GPU vs fallback.
Wire a GPU CLOB matcher into the dexvm match path (applySubmit ->
SubmitMarketableVerified) behind a determinism gate that runs the GPU
kernel as a CHECKED SHADOW of the pure-Go authority:
- EngineCPU (default): byte-for-byte the original SubmitMarketable.
- EngineGPUVerified (node-local opt-in, LUX_DEX_MATCH_ENGINE=gpu-verified):
capture the pre-cross opposite side atomically with the match, run the
GPU kernel on those exact inputs, byte-compare fills+remaining+book
against the CPU authority, record the outcome, and commit the CPU
result REGARDLESS. A GPU result that differs by one byte is recorded as
a divergence and discarded -- structurally impossible to commit.
Because the committed bytes are always the CPU authority's, a mixed
GPU/CPU validator set cannot fork. Divergences surface as a metric
(MatchShadowSnapshot) and a loud log (node ERROR via the VM).
Port the atomic import/export into the native dexvm matcher so the in-luxd
D-Chain PRODUCES and CONSUMES the same primary-network shared-memory objects
the 0x9999 precompile speaks — the owner's chosen model (D matches under its
own consensus; C-Chain 0x9999 settles by consuming a node-local atomic object
D produces; no BLS receipt, no external venue).
- atomic.go: the seam. encodeSeamObject/decodeSeamObject reproduce the
precompile's 69-byte wire (rail|owner|asset|amount|spent) BYTE-FOR-BYTE, and
DeriveIntentID its id derivation — ONE wire across precompile/proxy/native.
executeImport consumes a C->D object (Get under the C chain + shared-memory
Remove) and funds the taker's native D account; executeExport debits the
matched proceeds and writes a D->C object keyed exactly as ImportSettlement
reads it, bound to the per-intent escrow (owner/asset/remaining) so Phase B's
owner/asset/amount + per-taker-cap checks pass. commitSeamAtomic applies the
removes/puts atomically with the block overlay (the platformvm acceptor
pattern). No root change needed: the in-consensus matcher makes the atomic
ops a pure function of the already-rooted txs (unlike the proxy's live venue).
- vm.go: capture init.Runtime -> shared-memory handle + C-Chain id.
- tx.go: TxImport (intentID) / TxExport (intentID|asset|amount|spent), both
authorized by the cross-chain object, not a signature.
- block.go: execute drives the seam in the account-scoped lane; Accept commits
via commitSeamAtomic.
- atomic_seam_e2e_test.go: two-phase e2e over REAL shared memory — Phase A intent
-> native D import + a real crossing under D consensus -> native D export ->
Phase B (the exact 0x9999 ImportSettlement consume) credits once; replay
rejected; over-export beyond the locked principal rejected; conservation holds.
Plus a golden-wire test pinning encodeSeamObject to the SAME cross-repo golden
the precompile pins ImportSettlement against (native == precompile == proxy).
go.sum: re-record re-tagged luxfi/[email protected] (stale hash blocked the build).
go.mod: luxfi/runtime indirect -> direct (atomic.go uses it).
Adds the dex_get_nonce read so a signed client (luxfi/maker) resyncs its nonce to the venue's expected-next on restart/redeploy — without it a fresh process starts at nonce 0 and is rejected once the account has transacted. Also hex-encodes orderJSON.User: the raw 16-byte settlement id is not valid UTF-8, so the prior string emit was lossy (json -> U+FFFD) and an owner could never round-trip; hex matches dex_get_nonce's account field (one account encoding) and lets the maker's startup reconcile cancel its own stale orders. Re-realigns luxfi/[email protected] go.sum (re-tagged upstream; keeps reverting). Verified: dchain tests green.
Saves the GPU-venue build work: pkg/lx/orderbook_cuda*→orderbook_gpu*, the
clob→gpu cgo binding rename needed for the GB10 build, and cmd/dexfill (the
real-fill ZAP client). Checkpoint — build-verify pending.
dex's dep graph pulls github.com/hanzos3/go-sdk, but the Dockerfile GOPRIVATE listed only luxfi+hanzoai, so CI routed hanzos3 through proxy.golang.org+sumdb and the build failed (private repo, no proxy access) — even though go.sum was correct. Expanded GOPRIVATE to all our private orgs (luxfi/hanzoai/hanzos3/zooai/parsdao/lux-private), matching the working local env. This was the real v1.8.0/v1.8.1 build blocker; the go.sum poison was a separate, already-fixed issue.
Mounts vm.CreateHandlers (the same dex_*/dex_get_* map luxd mounts for the in-process plugin, ingest.go) on an HTTP listener, so a single-operator CPU venue is a byte-identical drop-in for the in-luxd D-Chain: exchange-api reads dex_get_markets, the maker writes via DEX_HTTP_URL — no luxd required. Writes still cross the full sealer consensus path. Verified: 11 routes mounted, GET /ext/bc/D/dex/dex_get_markets -> 200.
The v1.5.0 tag was rewritten upstream (zip h1 G69Hb… -> zC/Fw…); the stale go.sum failed every CI Docker build with a clean-fetch SECURITY ERROR. Only the zip hash drifted (go.mod hash unchanged). Verified: clean-cache CGO_ENABLED=0 linux/amd64 build + pkg/dchain tests green against the new bits.
deploy/testnet: one-command-per-box bring-up for a fleet of standalone
`dexd run` venues across two boxes, a cross-arch determinism proof, and a
ZAP load/throughput bench (drives the existing cmd/dexbench).
- compose.yml + gen-compose.sh: N independent venues/box (default 3, scales
to ~11), one ZAP port/node, per-node data dirs, 128GB-box resource caps.
- Dockerfile: pure-Go CPU image (dexd + dexbench), native amd64/arm64.
- Makefile: make up/bench/verify/down — git pull + one command per box.
- topology.md: honest consensus model (independent venues, deterministic
per-node genesis, no gossip) + the luxd `dexd plugin` BFT path.
- overlay/: opt-in hanzozt (OpenZiti) tunneler for NAT; direct 2.5GbE is the
default for true wire numbers.
- README.md: exact per-box commands, 128GB sizing, reading bench output.
Validated locally: signed place/cross load 0 rejects (cross=1 fill/submit);
dexbench -verify across two independent venues PASS (byte-identical 3-fill
multi-level sweep).
The public permissionless DEX must be provably KYC-free. Compliance/KYC
belongs in lux-private (dex/compliance/sovereign), never in luxfi/dex.
- delete compliance.go + compliance_types.go: EligibilityChecker.CanTrade
pre-trade gate, StandardEligibilityChecker (KYC/AML/accreditation/
jurisdiction enforcement), InvestorProfile/InvestorStatus, PEPStatus,
TransferRestriction, ExemptionType, ComplianceError + codes.
- delete market_config.go: the SecuritiesMarket wrapper existed solely to
gate AddOrder on EligibilityChecker.CanTrade. Zero production consumers —
the permissionless engine uses AdvancedOrderBook directly.
- funds.go: remove preCheck compliance hook + AML/sanctions/compliance
fields + ComplianceGates; permissionless default is FundsStatusApproved.
- reconciliation.go: relocate neutral AssetClass + ReconciliationMode types
here (cross-venue metadata, not gates); drop CEXComplianceID /
ReportedToRegulator / RegulatoryReportID fields.
- tests updated to match.
Green (CGO off): go build ./..., go vet + go test ./pkg/lx ./pkg/dchain.
Decentralized exchange; the engine acts like an order book but is decentralized,
so 'Central Limit Order Book' is a misnomer. Branded/service surface → DEX
(RegisterDEX, DEXMethod*, dex_* wire). Matching engine + GPU → OrderBook
(OrderBookMatch/Source/Arena, source_orderbook.go), preserving the AMM-vs-orderbook
distinction. C-ABI → dex_orderbook_* (matches luxcpp). SDKs/clients/docs aligned.
CGO=0 + cgo builds clean; full suite 23 pkgs green (dchain/dex/lx/fix/gateway/zapwire).
Legit financial-term 'Central Limit Order Book' retained in hummingbot taxonomy.
dexd is now the ONLY command in the repo. One binary, two run modes:
dexd run as a luxd rpcchainvm PLUGIN. luxd execs the plugin from
dexd plugin its PluginDir with NO positional args (engine address arrives
via the runtime.EngineAddressKey env var) and probes the
version with `dexd version`. No-subcommand == plugin is
REQUIRED to preserve that contract — byte-for-byte the old
cmd/dchain behavior (rpc.Serve over the plugin transport).
dexd run run the STANDALONE D-Chain venue: open the on-disk zapdb,
dexd standalone serve the frozen dex_* surface over ZAP, and drive the
consensus sealer loop (WaitForEvent->BuildBlock->Verify->
Accept). -addr/-db flags preserved from the old cmd/dvenue.
Relocated logic (engine code NOT duplicated — pkg/dchain + pkg/lx untouched):
cmd/dchain/{main,runner} -> internal/plugin (plugin.Run)
cmd/dvenue/{main,engine_*} -> internal/standalone (standalone.Run + engineName)
cmd/dexd/main.go = thin run-mode dispatcher
Removed cmd dirs (12): dchain, dvenue, demo, dex-api-server, dex-server,
dexfillproof, dexseed, fix-gateway, gateway, monitor, persistent-server,
price-feed. Removed committed build artifacts: gateway, dex-server-linux.
Build/ship ONLY dexd:
- Dockerfile: canonical CPU image (CGO=0, ghcr.io/luxfi/dex) running
`dexd run`; no GPU/MLX required.
- Dockerfile.dvenue: GPU operator bundle repointed cmd/dvenue -> cmd/dexd.
- Deleted Dockerfile.{gateway,devnet,minimal,dvenue-cpu} and
docker/{Dockerfile,Dockerfile.cuda,backend/Dockerfile,compose-simple.yml}.
- Makefile rewritten to build/run/ship dexd only.
- docker.yml: dropped the dex-gateway image; dex-venue-image.yml: dvenue->dexd.
- Deleted legacy scripts that built removed/long-gone cmds; fixed deploy.sh +
dev-setup.sh; .gitignore ignores root build binaries.
Also included (coupled, environment-applied, settled): the in-progress
clob -> dex/orderbook terminology refactor across pkg/* + sdk/* + tests/*
(identifier + file renames, e.g. RegisterCLOB->RegisterDEX,
source_clob.go->source_orderbook.go). The relocated standalone venue calls the
renamed API, so it lands in the same consistent snapshot.
Verified (GOWORK=off CGO_ENABLED=0): `go build ./cmd/dexd`, `go build ./...`,
`go vet ./...` all exit 0; `go test ./pkg/dchain` (renamed RegisterDEX socket
path) green. GPU/cgo path not compiled locally (no CUDA); its rename mirrors
the verified nocgo twins and preserves the C ABI symbol names.
The package lived at dex/pkg/dexcore (package dexcore) with 'dexcore:' error
prefixes, while every sibling is short (api, client, dchain, consensus) and every
other dex error says 'dex:'. The 'core' qualifier is redundant inside the dex
module. Rename pkg/dexcore → pkg/dex (package dex), unify error prefixes
dexcore: → dex:, update the 5 importers (dexseed, dexfillproof). Tests green
(pkg/dex, 1.398s); dvenue + dexvm plugin rebuild; e2e swap re-proven (seed 12
orders + cross → fill @ 12.5125).
Dockerfile.dvenue-cpu builds cmd/dvenue for linux/amd64 with no CUDA/GPU,
selecting pkg/lx's pure-Go CPU matcher via the !cgo build tag; serves the
frozen clob_* surface over ZAP with the consensus sealer loop. Complements
the GPU Dockerfile.dvenue (CUDA, arm64/spark-only) with a venue image that
runs on any amd64 node — the deployed devnet dchain-venue binary.
Verified: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build ./cmd/dvenue links
the pure-Go path (amm_nogpu / orderbook_cuda_stub) clean on this base.
The venue V of the HTLC atomic-swap design: a pure relay that serves a
maker-signed quote book and sequences a swap session between taker U (the
secret-holder) and a CEX-backed maker M, validating each typed signed
transition — role bindings (HTLC_A recipient=U/refund=M, HTLC_B
recipient=M/refund=U), the cross-leg hashlock, and the timeout ordering
T_B > T_A + Δ (maker locks first/shorter; spec Δ table: Lux<->BTC 2h,
EVM<->EVM 5m).
Non-custody is STRUCTURAL: the Coordinator holds no private key of any
asset, is the recipient/refund of no HTLC, and has no fund-moving path —
TestNonCustody reflects over it and fails if any field is a key type. After
the hashlock commit V is irrelevant to safety; its crashing or lying can
only deny service or relay a stale quote.
Composes pkg/htlc for the shared hashlock/preimage and authenticates every
message through crypto.Ecrecover directly (recoverAddr) with its own venue
domain tags — a leaf, so the relay never pulls the GPU CLOB matcher. Builds
and passes go test -race off a non-GPU box.
The single place a Lux cross-chain atomic swap is constructed across all
three legs, composed from one shared primitive:
- secret.go — Secret [32]byte, Hashlock() = sha256(secret), constant-time
VerifyPreimage. THE source of the SHA-256 hashlock every leg shares (one
32-byte preimage unlocks BTC, the LP-90A0 swap precompile, and the EVM
LXSwapHTLC.sol); keccak is explicitly excluded.
- bitcoin.go — the fixed P2WSH HTLC script (OP_IF SHA256/EQUALVERIFY/CHECKSIG
// OP_ELSE CLTV/DROP/CHECKSIG), P2WSHAddress, and claim/refund witnesses
signed with real BIP143 segwit-v0 sighash via btcd txscript (no hand-rolled
sighash; low-S RFC6979 sigs).
- evm.go — lock/claim/refund calldata; selectors DERIVED from the canonical
ABI signatures and asserted equal to the precompile's 0x4da2c728 /
0x84cc9dfb / 0x7249fbb6.
Tests (-race) include real txscript.Engine execution proofs: claim spends
with the correct preimage and fails with a wrong one; refund spends at/after
locktime and fails before / with CLTV disabled. btcd pinned v0.24.2 (v0.26.0
split chaincfg/wire/txscript into /v2 modules and breaks the build).
A rested maker carries an 8-byte-truncated User (from persist.go userToUint64);
a fresh taker carries the full 16-byte identity. The marketable matcher's raw
User== self-trade check missed the same-account self-cross, desyncing the int/
float quantity lanes -> maker reserve drained + orderuser row deleted -> later
cross stranded settlement -> conservation broke. Add selfTradeKey() (one fold,
used by both place + marketable paths) and skip-and-cancel the self-maker leg.
settle.go arithmetic untouched. seed 4242 re-included + conserves.
Every state-mutating D-Chain tx now carries a signature + per-account nonce,
verified in consensus before any ledger mutation: recovered signer must equal
the asserted 16-byte user, and the digest binds tx type + nonce + full body
(so the withdraw ref and cancel orderId are signed). secp256k1 + ML-DSA-65,
fail-closed. Previously any caller of the venue RPC could move anyone's funds.
Conservation, identity-width, and ownership invariants preserved; the
signed-order primitive that existed without a caller is now the auth boundary.
Tests across ingest/read/zapingest/e2e adapted to sign; no checks weakened.
Aligns the dex leaf onto the canonical crypto/consensus go-live tags. dexcore
(permissionless AssetResolver) + dchain (dex_get_* read RPC, pure block.ChainVM)
tests green (GOWORK=off). No source change — dep alignment.
poolIDForSymbol was defined only in localnet_e2e_test.go (//go:build !cgo) but
read_test.go (no build tag) used it, so the package test binary failed to
compile under CGO_ENABLED=1 (undefined: poolIDForSymbol). Move the helper to an
untagged poolid_helper_test.go — it is pure-Go (a byte copy) and belongs in both
test binaries. Both CGO_ENABLED=1 (read/purity) and CGO_ENABLED=0 (venue/4-path/
ingest/conservation) suites now compile and pass.
go.sum: luxfi/zap v0.8.10 content hash updated to the canonical registry hash
(h1:QKNTAsenkke+...) after an upstream re-tag; 'go mod verify' = all modules
verified. Regenerated, not bypassed.
zeekayandHanzo Dev