lx: make CGO=1 build cross-platform; honest secp256k1 GPU fallback

CGO_ENABLED=1 go build ./... failed to link every binary importing
pkg/lx: signed_order_gpu.go unconditionally force-linked the Apple-only
libsecp256k1_metal.a via a hardcoded build-tree -L path and the
signed-order Metal anchor. That archive is not built by default and has
no pkg-config bundle, so the link aborted with 'library secp256k1_metal
not found'. The GPU-vs-CPU choice must be a runtime decision, never a
hard build-time link dependency on an absent accelerator.

- Gate the secp256k1 Metal force-link anchor (.go + .c) behind the
  opt-in 'lux_secp256k1_metal' build tag and move the Metal/Foundation
  LDFLAGS there. Default CGO builds now link and run the secp256k1 CPU
  ecrecover pipeline (via the working lux-crypto-secp256k1 .pc); the
  Metal accelerator, when its archive is built and the tag is set, is
  still resolved at runtime by cpp/ecrecover.cpp via dlsym gated on
  LUX_SECP256K1_METALLIB / LUX_SECP256K1_BACKEND.
- signed_order_gpu.go keeps only the CPU pkg-config; the anchor file
  documents the build + opt-in recipe.
- signed_order_cpu.go: verifyOrdersCPU returns nil for empty input,
  matching the cgo BatchVerifyOrders contract so CPU and cgo paths agree
  byte-for-byte (fixes TestBatchVerifyOrders_Empty under CGO=0).
- README: remove the non-existent 'dex_gpu' build-tag claim; document
  the real model — AMM/CLOB GPU paths live in pkg/lx, bind the luxcpp
  DEX kernels via pkg-config (lux-dex-amm-metal / -amm-cuda / -clob-cuda),
  and select GPU-vs-CPU at runtime via crypto/backend detection with a
  logged, honest CPU fallback.

The DEX AMM Metal bridge (amm_xyk_batch_metal) now compiles, links, and
its dispatch is byte-identical to the CPU oracle (parity test green
under CGO=1). CLOB GPU primitive (MatchOrderGPU) stays a flat-buffer
parity/backtest primitive, intentionally not wired into the live
linked-list MatchOrders() engine.

Verified: CGO_ENABLED=0 go test ./... green (16 pkgs, 1360 cases);
CGO_ENABLED=1 go build ./... exit 0.
This commit is contained in:
zooqueen
2026-06-21 22:37:05 -07:00
parent 982777ec74
commit f2e3705e96
5 changed files with 92 additions and 24 deletions
+39 -8
View File
@@ -12,15 +12,46 @@ JSON-RPC / WebSocket / gRPC SDKs.
This repository is the **public, pure-Go reference implementation** of
the Lux DEX matching engine. It is fully functional, runs standalone,
and underpins every Lux DEX deployment.
and underpins every Lux DEX deployment. Built with `CGO_ENABLED=0` it is
a self-contained pure-Go binary with zero native dependencies.
For commercial deployments that need hardware acceleration (NUMA-aware
C++ order book, CUDA / Metal batched verification, FPGA fast paths)
the same Go interfaces are implemented by `lux-private/dex` and
selected at build time via the `dex_gpu` build tag. The accelerated
backend fails closed unless the operator's environment carries a Lux
commercial license token whose scope list includes `dex`. Contact
`[email protected]` for commercial licensing.
### GPU acceleration is a runtime decision, not a build tag
The batched GPU paths (constant-product AMM curve evaluation and the
flat-buffer CLOB match primitive) live **in this repository** under
`pkg/lx/` and are compiled into the `CGO_ENABLED=1` build. They bind to
the luxcpp DEX kernels through pkg-config:
| pkg-config bundle | backend | consumed by |
| -------------------- | -------------- | ------------------------------- |
| `lux-dex-amm-metal` | Metal (macOS) | `pkg/lx/amm_gpu_metal.go` |
| `lux-dex-amm-cuda` | CUDA (Linux) | `pkg/lx/amm_gpu_cuda.go` |
| `lux-dex-clob-cuda` | CUDA (Linux) | `pkg/lx/orderbook_cuda.go` |
These bundles ship from `luxcpp/dex` (`cmake --install`). Whether a GPU is
actually used is decided **at runtime** by `github.com/luxfi/crypto/backend`
detection, never by a build tag:
- `GPU_DISABLE=1` forces the CPU path (recorded as `reason=disabled`).
- No Metal/CUDA device, or the runtime kernel/metallib is absent → the
per-platform dispatch returns its `unsupported` sentinel and the call
transparently falls back to the pure-Go CPU CLOB / AMM oracle (recorded
as `reason=unsupported`). The fallback is logged — a CPU result is never
mislabeled as GPU.
- macOS resolves the AMM metallib at runtime via `LUX_DEX_AMM_METALLIB`
(default: the installed `share/lux/dex/amm_xyk.metallib`).
There is no `dex_gpu` build tag. The one optional opt-in tag is
`lux_secp256k1_metal`, which force-links the Apple-only batched
secp256k1-ecrecover Metal archive; without it the signed-order batch
verifier runs the secp256k1 **CPU** pipeline. See
`pkg/lx/signed_order_metal_anchor_darwin.go` for the build + opt-in recipe.
For commercial deployments that need the heavier accelerators (NUMA-aware
C++ order book, FPGA fast paths) the same Go interfaces are implemented by
the private `lux-private/dex` tier, which fails closed unless the operator's
environment carries a Lux commercial license token whose scope list
includes `dex`. Contact `[email protected]` for commercial licensing.
## Features
+5
View File
@@ -13,6 +13,11 @@ import (
// order it recomputes the signing hash, recovers the pubkey from the sig,
// derives the 20-byte address and compares with order.Sender.
func verifyOrdersCPU(orders []SignedOrder) ([]bool, error) {
// Empty input round-trips as nil, matching the cgo BatchVerifyOrders
// contract (signed_order_gpu.go) so both paths agree byte-for-byte.
if len(orders) == 0 {
return nil, nil
}
out := make([]bool, len(orders))
for i := range orders {
hash, err := orders[i].SigningHash()
+12 -15
View File
@@ -10,13 +10,15 @@ package lx
// The C primitive lives in luxcpp/crypto:
// header: $LUXCPP_PREFIX/include/lux/crypto/secp256k1.h
// body: libsecp256k1_cpu.a in $LUXCPP_PREFIX/lib (CPU oracle)
// metal: libsecp256k1_metal.a in luxcpp/crypto/build/secp256k1
// (still consumed from the build tree — Metal-only, no .pc yet)
// metal: libsecp256k1_metal.a (Apple-only GPU accelerator; OPTIONAL)
//
// The CPU/oracle entry point is `secp256k1_ecrecover_address_batch`: it
// takes (hash, r||s||v) tuples and writes 20-byte Ethereum addresses out.
// We then compare each recovered address with order.Sender and emit the
// per-order bool.
// per-order bool. This entry point is ALWAYS the CPU pipeline; the Metal
// accelerator, when present, is resolved by cpp/ecrecover.cpp via
// dlsym(RTLD_DEFAULT, ...) at RUNTIME — backend selection is a runtime
// decision, never a build tag.
//
// CPU linkage goes through the lux-crypto-secp256k1 pkg-config bundle
// (libsecp256k1_cpu + libkeccak_cpu). Build + install once with:
@@ -26,21 +28,16 @@ package lx
// cmake --build $HOME/work/luxcpp/crypto/build \
// --target secp256k1_cpu keccak_cpu
// cmake --install $HOME/work/luxcpp/crypto/build
//
// The Metal accelerator archive (libsecp256k1_metal.a) is NOT in the
// lux-crypto-secp256k1 .pc bundle and is not built by default. Its
// force-link anchor lives in signed_order_metal_anchor_darwin.go behind the
// `lux_secp256k1_metal` build tag, so a default CGO build links and runs
// CPU-only without requiring the archive to exist. See that file's header
// for how to build the archive and opt in.
/*
#cgo pkg-config: lux-crypto-secp256k1
//
// Metal driver: dispatch in cpp/ecrecover.cpp resolves the GPU symbol via
// dlsym(RTLD_DEFAULT, ...). The anchor file signed_order_metal_anchor_darwin.c
// takes its address so the linker pulls libsecp256k1_metal.a into the final
// binary. Set LUX_SECP256K1_METALLIB to enable GPU dispatch; unset or
// LUX_SECP256K1_BACKEND=cpu uses the CPU pipeline.
//
// libsecp256k1_metal is NOT in the lux-crypto-secp256k1 .pc bundle (Apple
// only, Metal/Foundation frameworks required). Keep the build-tree path
// until a separate lux-crypto-secp256k1-metal .pc lands.
#cgo darwin LDFLAGS: -L${SRCDIR}/../../../../luxcpp/crypto/build/secp256k1
#cgo darwin LDFLAGS: -lsecp256k1_metal -framework Metal -framework Foundation
#include <stdint.h>
#include <stddef.h>
@@ -3,6 +3,12 @@
//
// Anchor for the secp256k1 Metal driver symbol. See cpp/ecrecover.cpp in
// luxcpp/crypto for the dispatch logic that resolves this symbol via dlsym.
//
// Gated by the `lux_secp256k1_metal` build tag (matching the .go anchor) so
// cgo only compiles this — and only then references the Metal-only extern
// symbol — when the operator has opted into linking libsecp256k1_metal.a.
//go:build cgo && darwin && lux_secp256k1_metal
#include <stddef.h>
#include <stdint.h>
+30 -1
View File
@@ -1,11 +1,40 @@
// Copyright (C) 2025-2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//go:build cgo && darwin
//go:build cgo && darwin && lux_secp256k1_metal
// Force-link anchor for the OPTIONAL secp256k1 Metal accelerator archive
// (libsecp256k1_metal.a), Apple Silicon only. Compiled only under the
// `lux_secp256k1_metal` build tag, so a default CGO build never requires the
// archive to exist and runs the secp256k1 CPU pipeline.
//
// To enable GPU dispatch:
//
// 1. Build the Metal archive in luxcpp/crypto:
// cmake --build $HOME/work/luxcpp/crypto/build --target secp256k1_metal
// (emits libsecp256k1_metal.a under the build/secp256k1 tree)
//
// 2. Build the DEX with the tag, pointing the linker at the archive dir:
// CGO_ENABLED=1 \
// CGO_LDFLAGS="-L$HOME/work/luxcpp/crypto/build/secp256k1" \
// go build -tags lux_secp256k1_metal ./...
//
// 3. At runtime export LUX_SECP256K1_METALLIB=<path/to/secp256k1.metallib>
// (and leave LUX_SECP256K1_BACKEND unset / != "cpu"). The dispatch in
// cpp/ecrecover.cpp dlsym's the GPU symbol this anchor keeps alive.
//
// Backend selection stays a RUNTIME decision: even with the archive linked,
// an unset LUX_SECP256K1_METALLIB or LUX_SECP256K1_BACKEND=cpu uses the CPU
// pipeline.
package lx
/*
// Apple-only accelerator; pulls in the Metal/Foundation frameworks. The
// archive directory is supplied by the operator via CGO_LDFLAGS (see
// header) so this file carries no build-tree path of its own.
#cgo darwin LDFLAGS: -lsecp256k1_metal -framework Metal -framework Foundation
#include <stddef.h>
// Defined in signed_order_metal_anchor_darwin.c.