lux/dex - dex - Hanzo Git

254 Commits 3 Branches 138 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
zeekayandHanzo Dev	4f8df54e87	fix(dex): exact-integer ZAP wire + ledger-in-root — kill cross-arch/restart execRoot forks, silent money desync, and ceil/floor strand Consensus-correctness fix for the native DEX settlement VM. Root cause of the launch-blocking forks: order size/price were IEEE-754 float64 on the ZAP wire, so every validator re-derived the settlement integer via uint64(float)/int64(float1e8), whose out-of-range result is implementation-defined per the Go spec and DIFFERS BY CPU ARCHITECTURE (arm64 saturates to 2^64-1; amd64 wraps to 2^63). Two honest validators on different archs then derived different reserves/balances/fills from the SAME ordered block. F1/F3/F5 — exact-integer size/price end to end: - pkg/zapwire: Place/Submit/Fill carry price as uint64 fixed-point (×1e8 = PriceInt) and size as uint64 atomic base units. No float64 on the wire. Removed PutFloat64/ Float64. Decode is exact bytes->uint64 (deterministic on every arch). - pkg/dchain/execute: decode the exact integers, set the authoritative PriceUnits/ SizeUnits lane; the float Price/Size become derived projections only (never a source of a consensus value). Reject price==0/size==0/price>maxPriceUnits. - pkg/dex/persist: the DEXOrder/DEXTrade row stores the EXACT PriceInt (in DEXPrice.Integer) and EXACT base units (Quantity/Remaining); RowToOrder restores the integer lane verbatim (identity — no float round-trip) and Filled=Qty-Remaining. Removed floatToFixedQty/fixedQtyToFloat/floatToFixedPrice (the non-idempotent round(size1e8)/Q64.64 encoding that clamped at ~1.845e11 and forked on restart). - pkg/dex matcher: cross and level-key on the exact integer PriceUnits. - pkg/dchain/settle: orderLock/floorsToZeroLock take exact integers; removed sizeToUnits/priceToInt float conversions and the dead restoreIntegerLane. F2 — commit the custody ledger into the execution root: - state.ComposeRoot/ExecutionRoot gain a ledgerRoot term (Merkle over balance:/ locked:/orderasset: rows). Validators can no longer finalize agreeing on the book while holding different money. F4 — release the ceil-lock/floor-charge residual on full fill: - decrementMakerReserves now closes a fully-filled maker's reserve and returns the ceil(size*price)-floor residual to the owner (a filled order can never be cancelled), through the same unlock path a cancel uses. No more permanently frozen quote. PROVEN: the two-phase cross-arch harness (arm64 build / amd64 Rosetta verify) now derives byte-identical execRoots AND byte-identical custody ledgers for the identical block (uint64(1e20) still differs at the language level — 0xffff.. vs 0x8000.. — but the exact-integer wire freezes the value in the bytes so no per-validator float->int conversion remains). Red guards flipped to assert the fix; conservation/byzantine/batch-parity/KAT/e2e all green (go test ./... -count=1), go vet clean. OUT OF SCOPE (flagged, NOT fixed here): redteam_unbacked_deposit_test documents a SEPARATE critical finding (F9) — dex_deposit credits an arbitrary self-signed amount with no C-side backing proof, exposed on the public RPC. That needs an access-control/ custody-model decision, not a mechanical fix, and is left for a follow-up. Co-authored-by: Hanzo Dev <[email protected]>	2026-07-02 09:42:54 -07:00
zeekay	21301212f9	refactor(cmd): consolidate cmd/* into one binary — dexd dexd is now the ONLY command in the repo. One binary, two run modes: dexd run as a luxd rpcchainvm PLUGIN. luxd execs the plugin from dexd plugin its PluginDir with NO positional args (engine address arrives via the runtime.EngineAddressKey env var) and probes the version with `dexd version`. No-subcommand == plugin is REQUIRED to preserve that contract — byte-for-byte the old cmd/dchain behavior (rpc.Serve over the plugin transport). dexd run run the STANDALONE D-Chain venue: open the on-disk zapdb, dexd standalone serve the frozen dex_* surface over ZAP, and drive the consensus sealer loop (WaitForEvent->BuildBlock->Verify-> Accept). -addr/-db flags preserved from the old cmd/dvenue. Relocated logic (engine code NOT duplicated — pkg/dchain + pkg/lx untouched): cmd/dchain/{main,runner} -> internal/plugin (plugin.Run) cmd/dvenue/{main,engine_} -> internal/standalone (standalone.Run + engineName) cmd/dexd/main.go = thin run-mode dispatcher Removed cmd dirs (12): dchain, dvenue, demo, dex-api-server, dex-server, dexfillproof, dexseed, fix-gateway, gateway, monitor, persistent-server, price-feed. Removed committed build artifacts: gateway, dex-server-linux. Build/ship ONLY dexd: - Dockerfile: canonical CPU image (CGO=0, ghcr.io/luxfi/dex) running `dexd run`; no GPU/MLX required. - Dockerfile.dvenue: GPU operator bundle repointed cmd/dvenue -> cmd/dexd. - Deleted Dockerfile.{gateway,devnet,minimal,dvenue-cpu} and docker/{Dockerfile,Dockerfile.cuda,backend/Dockerfile,compose-simple.yml}. - Makefile rewritten to build/run/ship dexd only. - docker.yml: dropped the dex-gateway image; dex-venue-image.yml: dvenue->dexd. - Deleted legacy scripts that built removed/long-gone cmds; fixed deploy.sh + dev-setup.sh; .gitignore ignores root build binaries. Also included (coupled, environment-applied, settled): the in-progress clob -> dex/orderbook terminology refactor across pkg/ + sdk/* + tests/* (identifier + file renames, e.g. RegisterCLOB->RegisterDEX, source_clob.go->source_orderbook.go). The relocated standalone venue calls the renamed API, so it lands in the same consistent snapshot. Verified (GOWORK=off CGO_ENABLED=0): `go build ./cmd/dexd`, `go build ./...`, `go vet ./...` all exit 0; `go test ./pkg/dchain` (renamed RegisterDEX socket path) green. GPU/cgo path not compiled locally (no CUDA); its rename mirrors the verified nocgo twins and preserves the C ABI symbol names.	2026-06-26 12:24:34 -07:00
zeekay	baa275e80f	feat(dex): FIX 4.4 acceptor + unified 4-path trading e2e Adds the missing SERVER side of the FIX trading path and a unified e2e proving REAL fills over all four DEX access paths against ONE book. The decomplection: the D-Chain CLOB (pkg/dchain) is the single matcher, reached over the single frozen clob_* ZAP surface (pkg/zapwire). Every access path is a thin wire dialect onto that surface — none owns a matcher. This commit adds the two missing dialects (FIX, a focused WS venue handler) and the shared seam they translate through. New: - pkg/venue: ONE Venue interface (ensure/place/cancel/submit = the clob_* ops) + ONE production ZAP-backed impl. The seam FIX, WS, and the 0x9010 precompile all route through. Dials the SAME rpc.Listen socket dvenue serves. - pkg/fix: dependency-free FIX 4.4 codec (BodyLength tag 9 + CheckSum tag 10 computed AND validated; spec-correct emit/parse) + a TCP acceptor (Logon/Heartbeat/TestRequest/Logout, monotonic seq, NewOrderSingle->clob_submit->ExecutionReport, OrderCancelRequest-> clob_cancel). Pure protocol front-end onto Venue; no matcher. - pkg/api/venue_ws.go: minimal gorilla/websocket /ws handler that translates a JSON order to Venue.Submit and streams fills back — the WS dialect onto the SAME venue (kept orthogonal to the existing TradingEngine product WS server). - cmd/fix-gateway: runs the acceptor in front of a live venue (-venue clob_* endpoint), the production form of the FIX path. - pkg/dchain/fourpath_e2e_test.go: stands up ONE venue + ONE LUX/LUSD book and drives a real limit-order -> match -> FILL over EACH path (V4 precompile engine_zap frame \| ZAP native \| FIX \| WS) against the SAME resting asks, asserting real consensus fills, per-path custody changes, and GLOBAL conservation (available+locked invariant). V4 leg: exercised at the engine_zap WIRE boundary — the exact 66-byte clob_submit frame engine_zap.Swap emits (asserted byte-identical to zapwire.EncodeSubmit) + the V4 BalanceDelta sign convention derived from the consensus fills. A live luxd EVM (StateDB replay-idempotency, gas, PoolManager) is what the full on-chain precompile call additionally needs; the matcher boundary is proven in-process here. No deps added; no existing files modified (go.mod/go.sum untouched). Builds CGO_ENABLED=0; race-clean.	2026-06-19 13:05:58 -07:00

Author

SHA1

Message

Date

zeekayandHanzo Dev

4f8df54e87

fix(dex): exact-integer ZAP wire + ledger-in-root — kill cross-arch/restart execRoot forks, silent money desync, and ceil/floor strand

Consensus-correctness fix for the native DEX settlement VM. Root cause of the
launch-blocking forks: order size/price were IEEE-754 float64 on the ZAP wire, so
every validator re-derived the settlement integer via uint64(float)/int64(float*1e8),
whose out-of-range result is implementation-defined per the Go spec and DIFFERS BY
CPU ARCHITECTURE (arm64 saturates to 2^64-1; amd64 wraps to 2^63). Two honest
validators on different archs then derived different reserves/balances/fills from the
SAME ordered block.

F1/F3/F5 — exact-integer size/price end to end:
- pkg/zapwire: Place/Submit/Fill carry price as uint64 fixed-point (×1e8 = PriceInt)
  and size as uint64 atomic base units. No float64 on the wire. Removed PutFloat64/
  Float64. Decode is exact bytes->uint64 (deterministic on every arch).
- pkg/dchain/execute: decode the exact integers, set the authoritative PriceUnits/
  SizeUnits lane; the float Price/Size become derived projections only (never a
  source of a consensus value). Reject price==0/size==0/price>maxPriceUnits.
- pkg/dex/persist: the DEXOrder/DEXTrade row stores the EXACT PriceInt (in
  DEXPrice.Integer) and EXACT base units (Quantity/Remaining); RowToOrder restores
  the integer lane verbatim (identity — no float round-trip) and Filled=Qty-Remaining.
  Removed floatToFixedQty/fixedQtyToFloat/floatToFixedPrice (the non-idempotent
  round(size*1e8)/Q64.64 encoding that clamped at ~1.845e11 and forked on restart).
- pkg/dex matcher: cross and level-key on the exact integer PriceUnits.
- pkg/dchain/settle: orderLock/floorsToZeroLock take exact integers; removed
  sizeToUnits/priceToInt float conversions and the dead restoreIntegerLane.

F2 — commit the custody ledger into the execution root:
- state.ComposeRoot/ExecutionRoot gain a ledgerRoot term (Merkle over balance:/
  locked:/orderasset: rows). Validators can no longer finalize agreeing on the book
  while holding different money.

F4 — release the ceil-lock/floor-charge residual on full fill:
- decrementMakerReserves now closes a fully-filled maker's reserve and returns the
  ceil(size*price)-floor residual to the owner (a filled order can never be
  cancelled), through the same unlock path a cancel uses. No more permanently
  frozen quote.

PROVEN: the two-phase cross-arch harness (arm64 build / amd64 Rosetta verify) now
derives byte-identical execRoots AND byte-identical custody ledgers for the
identical block (uint64(1e20) still differs at the language level — 0xffff.. vs
0x8000.. — but the exact-integer wire freezes the value in the bytes so no
per-validator float->int conversion remains). Red guards flipped to assert the fix;
conservation/byzantine/batch-parity/KAT/e2e all green (go test ./... -count=1),
go vet clean.

OUT OF SCOPE (flagged, NOT fixed here): redteam_unbacked_deposit_test documents a
SEPARATE critical finding (F9) — dex_deposit credits an arbitrary self-signed amount
with no C-side backing proof, exposed on the public RPC. That needs an access-control/
custody-model decision, not a mechanical fix, and is left for a follow-up.

Co-authored-by: Hanzo Dev <[email protected]>

2026-07-02 09:42:54 -07:00

zeekay

21301212f9

refactor(cmd): consolidate cmd/* into one binary — dexd

dexd is now the ONLY command in the repo. One binary, two run modes:

  dexd            run as a luxd rpcchainvm PLUGIN. luxd execs the plugin from
  dexd plugin     its PluginDir with NO positional args (engine address arrives
                  via the runtime.EngineAddressKey env var) and probes the
                  version with `dexd version`. No-subcommand == plugin is
                  REQUIRED to preserve that contract — byte-for-byte the old
                  cmd/dchain behavior (rpc.Serve over the plugin transport).
  dexd run        run the STANDALONE D-Chain venue: open the on-disk zapdb,
  dexd standalone serve the frozen dex_* surface over ZAP, and drive the
                  consensus sealer loop (WaitForEvent->BuildBlock->Verify->
                  Accept). -addr/-db flags preserved from the old cmd/dvenue.

Relocated logic (engine code NOT duplicated — pkg/dchain + pkg/lx untouched):
  cmd/dchain/{main,runner}   -> internal/plugin     (plugin.Run)
  cmd/dvenue/{main,engine_*} -> internal/standalone (standalone.Run + engineName)
  cmd/dexd/main.go            = thin run-mode dispatcher

Removed cmd dirs (12): dchain, dvenue, demo, dex-api-server, dex-server,
dexfillproof, dexseed, fix-gateway, gateway, monitor, persistent-server,
price-feed. Removed committed build artifacts: gateway, dex-server-linux.

Build/ship ONLY dexd:
  - Dockerfile: canonical CPU image (CGO=0, ghcr.io/luxfi/dex) running
    `dexd run`; no GPU/MLX required.
  - Dockerfile.dvenue: GPU operator bundle repointed cmd/dvenue -> cmd/dexd.
  - Deleted Dockerfile.{gateway,devnet,minimal,dvenue-cpu} and
    docker/{Dockerfile,Dockerfile.cuda,backend/Dockerfile,compose-simple.yml}.
  - Makefile rewritten to build/run/ship dexd only.
  - docker.yml: dropped the dex-gateway image; dex-venue-image.yml: dvenue->dexd.
  - Deleted legacy scripts that built removed/long-gone cmds; fixed deploy.sh +
    dev-setup.sh; .gitignore ignores root build binaries.

Also included (coupled, environment-applied, settled): the in-progress
clob -> dex/orderbook terminology refactor across pkg/* + sdk/* + tests/*
(identifier + file renames, e.g. RegisterCLOB->RegisterDEX,
source_clob.go->source_orderbook.go). The relocated standalone venue calls the
renamed API, so it lands in the same consistent snapshot.

Verified (GOWORK=off CGO_ENABLED=0): `go build ./cmd/dexd`, `go build ./...`,
`go vet ./...` all exit 0; `go test ./pkg/dchain` (renamed RegisterDEX socket
path) green. GPU/cgo path not compiled locally (no CUDA); its rename mirrors
the verified nocgo twins and preserves the C ABI symbol names.

2026-06-26 12:24:34 -07:00

zeekay

baa275e80f

feat(dex): FIX 4.4 acceptor + unified 4-path trading e2e

Adds the missing SERVER side of the FIX trading path and a unified e2e
proving REAL fills over all four DEX access paths against ONE book.

The decomplection: the D-Chain CLOB (pkg/dchain) is the single matcher,
reached over the single frozen clob_* ZAP surface (pkg/zapwire). Every
access path is a thin wire dialect onto that surface — none owns a
matcher. This commit adds the two missing dialects (FIX, a focused WS
venue handler) and the shared seam they translate through.

New:
- pkg/venue: ONE Venue interface (ensure/place/cancel/submit = the
  clob_* ops) + ONE production ZAP-backed impl. The seam FIX, WS, and
  the 0x9010 precompile all route through. Dials the SAME rpc.Listen
  socket dvenue serves.
- pkg/fix: dependency-free FIX 4.4 codec (BodyLength tag 9 + CheckSum
  tag 10 computed AND validated; spec-correct emit/parse) + a TCP
  acceptor (Logon/Heartbeat/TestRequest/Logout, monotonic seq,
  NewOrderSingle->clob_submit->ExecutionReport, OrderCancelRequest->
  clob_cancel). Pure protocol front-end onto Venue; no matcher.
- pkg/api/venue_ws.go: minimal gorilla/websocket /ws handler that
  translates a JSON order to Venue.Submit and streams fills back —
  the WS dialect onto the SAME venue (kept orthogonal to the existing
  TradingEngine product WS server).
- cmd/fix-gateway: runs the acceptor in front of a live venue
  (-venue clob_* endpoint), the production form of the FIX path.
- pkg/dchain/fourpath_e2e_test.go: stands up ONE venue + ONE LUX/LUSD
  book and drives a real limit-order -> match -> FILL over EACH path
  (V4 precompile engine_zap frame | ZAP native | FIX | WS) against the
  SAME resting asks, asserting real consensus fills, per-path custody
  changes, and GLOBAL conservation (available+locked invariant).

V4 leg: exercised at the engine_zap WIRE boundary — the exact 66-byte
clob_submit frame engine_zap.Swap emits (asserted byte-identical to
zapwire.EncodeSubmit) + the V4 BalanceDelta sign convention derived from
the consensus fills. A live luxd EVM (StateDB replay-idempotency, gas,
PoolManager) is what the full on-chain precompile call additionally
needs; the matcher boundary is proven in-process here.

No deps added; no existing files modified (go.mod/go.sum untouched).
Builds CGO_ENABLED=0; race-clean.

2026-06-19 13:05:58 -07:00