mirror of
https://github.com/luxfi/dex.git
synced 2026-09-24 00:26:31 +00:00
lx: make CGO=1 build cross-platform; honest secp256k1 GPU fallback
CGO_ENABLED=1 go build ./... failed to link every binary importing pkg/lx: signed_order_gpu.go unconditionally force-linked the Apple-only libsecp256k1_metal.a via a hardcoded build-tree -L path and the signed-order Metal anchor. That archive is not built by default and has no pkg-config bundle, so the link aborted with 'library secp256k1_metal not found'. The GPU-vs-CPU choice must be a runtime decision, never a hard build-time link dependency on an absent accelerator. - Gate the secp256k1 Metal force-link anchor (.go + .c) behind the opt-in 'lux_secp256k1_metal' build tag and move the Metal/Foundation LDFLAGS there. Default CGO builds now link and run the secp256k1 CPU ecrecover pipeline (via the working lux-crypto-secp256k1 .pc); the Metal accelerator, when its archive is built and the tag is set, is still resolved at runtime by cpp/ecrecover.cpp via dlsym gated on LUX_SECP256K1_METALLIB / LUX_SECP256K1_BACKEND. - signed_order_gpu.go keeps only the CPU pkg-config; the anchor file documents the build + opt-in recipe. - signed_order_cpu.go: verifyOrdersCPU returns nil for empty input, matching the cgo BatchVerifyOrders contract so CPU and cgo paths agree byte-for-byte (fixes TestBatchVerifyOrders_Empty under CGO=0). - README: remove the non-existent 'dex_gpu' build-tag claim; document the real model — AMM/CLOB GPU paths live in pkg/lx, bind the luxcpp DEX kernels via pkg-config (lux-dex-amm-metal / -amm-cuda / -clob-cuda), and select GPU-vs-CPU at runtime via crypto/backend detection with a logged, honest CPU fallback. The DEX AMM Metal bridge (amm_xyk_batch_metal) now compiles, links, and its dispatch is byte-identical to the CPU oracle (parity test green under CGO=1). CLOB GPU primitive (MatchOrderGPU) stays a flat-buffer parity/backtest primitive, intentionally not wired into the live linked-list MatchOrders() engine. Verified: CGO_ENABLED=0 go test ./... green (16 pkgs, 1360 cases); CGO_ENABLED=1 go build ./... exit 0.
This commit is contained in:
@@ -12,15 +12,46 @@ JSON-RPC / WebSocket / gRPC SDKs.
|
||||
|
||||
This repository is the **public, pure-Go reference implementation** of
|
||||
the Lux DEX matching engine. It is fully functional, runs standalone,
|
||||
and underpins every Lux DEX deployment.
|
||||
and underpins every Lux DEX deployment. Built with `CGO_ENABLED=0` it is
|
||||
a self-contained pure-Go binary with zero native dependencies.
|
||||
|
||||
For commercial deployments that need hardware acceleration (NUMA-aware
|
||||
C++ order book, CUDA / Metal batched verification, FPGA fast paths)
|
||||
the same Go interfaces are implemented by `lux-private/dex` and
|
||||
selected at build time via the `dex_gpu` build tag. The accelerated
|
||||
backend fails closed unless the operator's environment carries a Lux
|
||||
commercial license token whose scope list includes `dex`. Contact
|
||||
`[email protected]` for commercial licensing.
|
||||
### GPU acceleration is a runtime decision, not a build tag
|
||||
|
||||
The batched GPU paths (constant-product AMM curve evaluation and the
|
||||
flat-buffer CLOB match primitive) live **in this repository** under
|
||||
`pkg/lx/` and are compiled into the `CGO_ENABLED=1` build. They bind to
|
||||
the luxcpp DEX kernels through pkg-config:
|
||||
|
||||
| pkg-config bundle | backend | consumed by |
|
||||
| -------------------- | -------------- | ------------------------------- |
|
||||
| `lux-dex-amm-metal` | Metal (macOS) | `pkg/lx/amm_gpu_metal.go` |
|
||||
| `lux-dex-amm-cuda` | CUDA (Linux) | `pkg/lx/amm_gpu_cuda.go` |
|
||||
| `lux-dex-clob-cuda` | CUDA (Linux) | `pkg/lx/orderbook_cuda.go` |
|
||||
|
||||
These bundles ship from `luxcpp/dex` (`cmake --install`). Whether a GPU is
|
||||
actually used is decided **at runtime** by `github.com/luxfi/crypto/backend`
|
||||
detection, never by a build tag:
|
||||
|
||||
- `GPU_DISABLE=1` forces the CPU path (recorded as `reason=disabled`).
|
||||
- No Metal/CUDA device, or the runtime kernel/metallib is absent → the
|
||||
per-platform dispatch returns its `unsupported` sentinel and the call
|
||||
transparently falls back to the pure-Go CPU CLOB / AMM oracle (recorded
|
||||
as `reason=unsupported`). The fallback is logged — a CPU result is never
|
||||
mislabeled as GPU.
|
||||
- macOS resolves the AMM metallib at runtime via `LUX_DEX_AMM_METALLIB`
|
||||
(default: the installed `share/lux/dex/amm_xyk.metallib`).
|
||||
|
||||
There is no `dex_gpu` build tag. The one optional opt-in tag is
|
||||
`lux_secp256k1_metal`, which force-links the Apple-only batched
|
||||
secp256k1-ecrecover Metal archive; without it the signed-order batch
|
||||
verifier runs the secp256k1 **CPU** pipeline. See
|
||||
`pkg/lx/signed_order_metal_anchor_darwin.go` for the build + opt-in recipe.
|
||||
|
||||
For commercial deployments that need the heavier accelerators (NUMA-aware
|
||||
C++ order book, FPGA fast paths) the same Go interfaces are implemented by
|
||||
the private `lux-private/dex` tier, which fails closed unless the operator's
|
||||
environment carries a Lux commercial license token whose scope list
|
||||
includes `dex`. Contact `[email protected]` for commercial licensing.
|
||||
|
||||
## Features
|
||||
|
||||
|
||||
@@ -13,6 +13,11 @@ import (
|
||||
// order it recomputes the signing hash, recovers the pubkey from the sig,
|
||||
// derives the 20-byte address and compares with order.Sender.
|
||||
func verifyOrdersCPU(orders []SignedOrder) ([]bool, error) {
|
||||
// Empty input round-trips as nil, matching the cgo BatchVerifyOrders
|
||||
// contract (signed_order_gpu.go) so both paths agree byte-for-byte.
|
||||
if len(orders) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
out := make([]bool, len(orders))
|
||||
for i := range orders {
|
||||
hash, err := orders[i].SigningHash()
|
||||
|
||||
+12
-15
@@ -10,13 +10,15 @@ package lx
|
||||
// The C primitive lives in luxcpp/crypto:
|
||||
// header: $LUXCPP_PREFIX/include/lux/crypto/secp256k1.h
|
||||
// body: libsecp256k1_cpu.a in $LUXCPP_PREFIX/lib (CPU oracle)
|
||||
// metal: libsecp256k1_metal.a in luxcpp/crypto/build/secp256k1
|
||||
// (still consumed from the build tree — Metal-only, no .pc yet)
|
||||
// metal: libsecp256k1_metal.a (Apple-only GPU accelerator; OPTIONAL)
|
||||
//
|
||||
// The CPU/oracle entry point is `secp256k1_ecrecover_address_batch`: it
|
||||
// takes (hash, r||s||v) tuples and writes 20-byte Ethereum addresses out.
|
||||
// We then compare each recovered address with order.Sender and emit the
|
||||
// per-order bool.
|
||||
// per-order bool. This entry point is ALWAYS the CPU pipeline; the Metal
|
||||
// accelerator, when present, is resolved by cpp/ecrecover.cpp via
|
||||
// dlsym(RTLD_DEFAULT, ...) at RUNTIME — backend selection is a runtime
|
||||
// decision, never a build tag.
|
||||
//
|
||||
// CPU linkage goes through the lux-crypto-secp256k1 pkg-config bundle
|
||||
// (libsecp256k1_cpu + libkeccak_cpu). Build + install once with:
|
||||
@@ -26,21 +28,16 @@ package lx
|
||||
// cmake --build $HOME/work/luxcpp/crypto/build \
|
||||
// --target secp256k1_cpu keccak_cpu
|
||||
// cmake --install $HOME/work/luxcpp/crypto/build
|
||||
//
|
||||
// The Metal accelerator archive (libsecp256k1_metal.a) is NOT in the
|
||||
// lux-crypto-secp256k1 .pc bundle and is not built by default. Its
|
||||
// force-link anchor lives in signed_order_metal_anchor_darwin.go behind the
|
||||
// `lux_secp256k1_metal` build tag, so a default CGO build links and runs
|
||||
// CPU-only without requiring the archive to exist. See that file's header
|
||||
// for how to build the archive and opt in.
|
||||
|
||||
/*
|
||||
#cgo pkg-config: lux-crypto-secp256k1
|
||||
//
|
||||
// Metal driver: dispatch in cpp/ecrecover.cpp resolves the GPU symbol via
|
||||
// dlsym(RTLD_DEFAULT, ...). The anchor file signed_order_metal_anchor_darwin.c
|
||||
// takes its address so the linker pulls libsecp256k1_metal.a into the final
|
||||
// binary. Set LUX_SECP256K1_METALLIB to enable GPU dispatch; unset or
|
||||
// LUX_SECP256K1_BACKEND=cpu uses the CPU pipeline.
|
||||
//
|
||||
// libsecp256k1_metal is NOT in the lux-crypto-secp256k1 .pc bundle (Apple
|
||||
// only, Metal/Foundation frameworks required). Keep the build-tree path
|
||||
// until a separate lux-crypto-secp256k1-metal .pc lands.
|
||||
#cgo darwin LDFLAGS: -L${SRCDIR}/../../../../luxcpp/crypto/build/secp256k1
|
||||
#cgo darwin LDFLAGS: -lsecp256k1_metal -framework Metal -framework Foundation
|
||||
|
||||
#include <stdint.h>
|
||||
#include <stddef.h>
|
||||
|
||||
@@ -3,6 +3,12 @@
|
||||
//
|
||||
// Anchor for the secp256k1 Metal driver symbol. See cpp/ecrecover.cpp in
|
||||
// luxcpp/crypto for the dispatch logic that resolves this symbol via dlsym.
|
||||
//
|
||||
// Gated by the `lux_secp256k1_metal` build tag (matching the .go anchor) so
|
||||
// cgo only compiles this — and only then references the Metal-only extern
|
||||
// symbol — when the operator has opted into linking libsecp256k1_metal.a.
|
||||
|
||||
//go:build cgo && darwin && lux_secp256k1_metal
|
||||
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
@@ -1,11 +1,40 @@
|
||||
// Copyright (C) 2025-2026, Lux Industries Inc. All rights reserved.
|
||||
// See the file LICENSE for licensing terms.
|
||||
|
||||
//go:build cgo && darwin
|
||||
//go:build cgo && darwin && lux_secp256k1_metal
|
||||
|
||||
// Force-link anchor for the OPTIONAL secp256k1 Metal accelerator archive
|
||||
// (libsecp256k1_metal.a), Apple Silicon only. Compiled only under the
|
||||
// `lux_secp256k1_metal` build tag, so a default CGO build never requires the
|
||||
// archive to exist and runs the secp256k1 CPU pipeline.
|
||||
//
|
||||
// To enable GPU dispatch:
|
||||
//
|
||||
// 1. Build the Metal archive in luxcpp/crypto:
|
||||
// cmake --build $HOME/work/luxcpp/crypto/build --target secp256k1_metal
|
||||
// (emits libsecp256k1_metal.a under the build/secp256k1 tree)
|
||||
//
|
||||
// 2. Build the DEX with the tag, pointing the linker at the archive dir:
|
||||
// CGO_ENABLED=1 \
|
||||
// CGO_LDFLAGS="-L$HOME/work/luxcpp/crypto/build/secp256k1" \
|
||||
// go build -tags lux_secp256k1_metal ./...
|
||||
//
|
||||
// 3. At runtime export LUX_SECP256K1_METALLIB=<path/to/secp256k1.metallib>
|
||||
// (and leave LUX_SECP256K1_BACKEND unset / != "cpu"). The dispatch in
|
||||
// cpp/ecrecover.cpp dlsym's the GPU symbol this anchor keeps alive.
|
||||
//
|
||||
// Backend selection stays a RUNTIME decision: even with the archive linked,
|
||||
// an unset LUX_SECP256K1_METALLIB or LUX_SECP256K1_BACKEND=cpu uses the CPU
|
||||
// pipeline.
|
||||
|
||||
package lx
|
||||
|
||||
/*
|
||||
// Apple-only accelerator; pulls in the Metal/Foundation frameworks. The
|
||||
// archive directory is supplied by the operator via CGO_LDFLAGS (see
|
||||
// header) so this file carries no build-tree path of its own.
|
||||
#cgo darwin LDFLAGS: -lsecp256k1_metal -framework Metal -framework Foundation
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
// Defined in signed_order_metal_anchor_darwin.c.
|
||||
|
||||
Reference in New Issue
Block a user