lx: decomplect DEX GPU backend into one runtime-selected lux-gpu binding

The GPU backend was a build-time PLACE: five mutually-exclusive build-tag
files (orderbook_gpu_{cuda,hip,vulkan,metal,stub}.go), each redefining
gpuMatchOrder + dexBackendLabel and each #cgo pkg-config'ing its own
per-vendor bundle (lux-dex-orderbook-{cuda,hip,metal,vulkan}). Which kernel
you got was frozen at compile time by -tags.

Make it a runtime VALUE. Collapse the five into the single cgo file
orderbook_gpu.go (//go:build cgo, no os/vendor tags), which links one
unified pkg-config bundle, lux-gpu, and calls the shared C ABI in dex_gpu.h:

  int  lux_gpu_dex_match_order(...)   // rc 0 ok, -2 unsupported -> CPU
  const char* lux_gpu_backend_name()  // runtime-selected: cuda|hip|metal|cpu

liblux_gpu picks the best device at runtime (accel SelectBestBackend). The
MatchOrderGPU dispatcher, errOrderBookGPUUnsupported sentinel, GPU_DISABLE
policy, and byte-parity padding normalization are unchanged. dexBackendLabel
now reports the runtime backend via C.GoString(C.lux_gpu_backend_name()).
The !cgo CPU fallback (orderbook_nogpu.go) and MatchOrderCPU oracle are
untouched; the seam is now exactly cgo vs !cgo, nothing finer.

Verified: CGO_ENABLED=0 full pkg-config build+vet+`go test -run Match|Batch`
green. cgo path built+linked+run against a throwaway stub lux-gpu.pc (ABI
type-checks; rc -2 -> CPU fallback; backend label wired) — real GPU kernel
parity is CI-verified against liblux_gpu on the arcd runners.

Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
2026-07-01 19:52:36 -07:00
co-authored by Hanzo Dev
parent 2a9169184f
commit ab531a264b
6 changed files with 115 additions and 429 deletions
+115 -21
View File
@@ -5,36 +5,58 @@
package lx
// orderbook_gpu.go is the ONE cgo binding for the DEX OrderBook match_order
// kernel. There is deliberately NO per-OS / per-vendor build-tag file: the
// backend is a RUNTIME value, not a build-time place. This file links a single
// unified pkg-config bundle, lux-gpu, whose liblux_gpu selects the best
// available device at runtime (accel's SelectBestBackend: CUDA > HIP > Metal >
// CPU) and exposes it behind one stable C ABI (dex_gpu.h):
//
// int lux_gpu_dex_match_order(...) // the matcher; rc 0 ok, -2 unsupported
// const char* lux_gpu_backend_name() // the runtime-selected backend name
//
// The kernel is the SINGLE-THREAD deterministic matcher whose output is byte-
// identical to MatchOrderCPU by construction. That contract is enforced on
// every run by orderbook_gpu_test.go and the determinism corpus in
// match_verify_test.go, and byte-verified live against the CPU authority by the
// GPU shadow gate in match_verify.go.
//
// CGo off → orderbook_nogpu.go provides a CPU-only MatchOrderGPU and a
// dexBackendLabel of "cpu (cgo disabled)". One and only one of each symbol per
// build; the seam is cgo vs !cgo, nothing finer.
/*
#cgo pkg-config: lux-gpu
#include <stdint.h>
#include "dex_gpu.h"
*/
import "C"
import (
"fmt"
"runtime"
"unsafe"
"github.com/luxfi/crypto/backend"
)
// MatchOrderGPU matches one incoming taker against a subset of book
// orders identified by bookIndices, dispatching to the host GPU when
// available and falling back to MatchOrderCPU otherwise. Output is
// MatchOrderGPU matches one incoming taker against a subset of book orders
// identified by bookIndices, dispatching to the runtime-selected GPU backend
// when available and falling back to MatchOrderCPU otherwise. Output is
// byte-equal across paths after padding normalization — that contract is
// enforced by orderbook_gpu_test.go on every run.
//
// This is the flat-buffer matching primitive used for parity testing,
// batch backtests, and future GPU-resident architectures. The live
// OrderBook.MatchOrders() engine retains its richer order-type and
// linked-list semantics and is NOT touched by this dispatch.
//
// Backend selection at compile time (CGo on):
//
// linux → orderbook_gpu_cuda.go → CUDA
// other → orderbook_gpu_stub.go → returns errOrderBookGPUUnsupported (caller falls back to CPU)
// This is the flat-buffer matching primitive used for parity testing, batch
// backtests, and future GPU-resident architectures. The live
// OrderBook.MatchOrders() engine retains its richer order-type and linked-list
// semantics and is NOT touched by this dispatch.
//
// Fallback policy (recorded via backend.RecordFallback):
//
// GPU_DISABLE=1 → reason "disabled"
// gpuMatchOrder returns sentinel → reason "unsupported"
// other gpuMatchOrder error → surfaced to caller (no fallback)
//
// CGo off → orderbook_nogpu.go provides a CPU-only implementation of this
// entry point. One and only one MatchOrderGPU symbol per build.
// GPU_DISABLE=1 → reason "disabled"
// lux_gpu_dex_match_order rc == -2 → reason "unsupported"
// other rc → surfaced to caller (no fallback)
//
// Returns: trades emitted, remaining quantity on the incoming order.
// `book` may be mutated in place to reflect updated Remaining / Status.
@@ -67,7 +89,79 @@ func MatchOrderGPU(
return trades, remaining, nil
}
// errOrderBookGPUUnsupported is returned by the per-platform gpuMatchOrder
// when no GPU dispatch is available on this host. The dispatcher above
// turns that into a transparent CPU fallback rather than surfacing it.
// errOrderBookGPUUnsupported is returned by gpuMatchOrder when liblux_gpu
// reports no usable device dispatch (rc -2). The dispatcher turns that into a
// transparent CPU fallback rather than surfacing it.
var errOrderBookGPUUnsupported = fmt.Errorf("lx: orderBook GPU dispatch not available on this host")
// dexBackendLabel reports the runtime-selected DEX GPU backend for corpus
// logging ("cuda" | "hip" | "metal" | "cpu"), as chosen by liblux_gpu.
func dexBackendLabel() string { return C.GoString(C.lux_gpu_backend_name()) }
// gpuMatchOrder calls the unified liblux_gpu matcher. Returns
// errOrderBookGPUUnsupported when the library reports rc -2 (no usable device)
// so the dispatcher falls back to MatchOrderCPU. Semantics are identical to the
// CPU oracle: byte-equal trades + remaining, with `book` mutated in place.
func gpuMatchOrder(
incoming *DEXOrder,
book []DEXOrder,
bookIndices []uint32,
tradeIDBase uint64,
timestamp uint64,
) ([]DEXTrade, uint64, error) {
if len(bookIndices) == 0 {
return nil, incoming.Quantity, nil
}
tradesOut := make([]DEXTrade, len(bookIndices))
var (
tradesWritten uint32
remaining uint64
)
var pinner runtime.Pinner
defer pinner.Unpin()
pinner.Pin(incoming)
pinner.Pin(&book[0])
pinner.Pin(&bookIndices[0])
pinner.Pin(&tradesOut[0])
pinner.Pin(&tradesWritten)
pinner.Pin(&remaining)
rc := C.lux_gpu_dex_match_order(
(*C.DEXOrder)(unsafe.Pointer(incoming)),
(*C.DEXOrder)(unsafe.Pointer(&book[0])),
(*C.uint32_t)(unsafe.Pointer(&bookIndices[0])),
C.uint32_t(len(bookIndices)),
(*C.DEXTrade)(unsafe.Pointer(&tradesOut[0])),
C.uint64_t(tradeIDBase),
C.uint64_t(timestamp),
(*C.uint32_t)(unsafe.Pointer(&tradesWritten)),
(*C.uint64_t)(unsafe.Pointer(&remaining)),
)
runtime.KeepAlive(incoming)
runtime.KeepAlive(book)
runtime.KeepAlive(bookIndices)
runtime.KeepAlive(tradesOut)
switch rc {
case 0:
// Padding normalization to match the CPU oracle's ClearPadding output,
// so raw bytewise comparisons (EncodeTrade/EncodeRow) are deterministic.
// The unified driver already zeros the padding for emitted trades and
// touched book rows; this is belt-and-braces against a future kernel
// revision that changes the contract.
trades := tradesOut[:int(tradesWritten)]
for i := range trades {
trades[i].ClearPadding()
}
for i := range book {
book[i].ClearPadding()
}
return trades, remaining, nil
case -2:
return nil, 0, errOrderBookGPUUnsupported
default:
return nil, 0, fmt.Errorf("lx: lux_gpu_dex_match_order rc=%d", int(rc))
}
}
-97
View File
@@ -1,97 +0,0 @@
// Copyright (C) 2025-2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//go:build cgo && linux && !hip && !vulkan
package lx
// CUDA backend for the DEX OrderBook match_order kernel. This is the default
// linux GPU backend; build with -tags hip or -tags vulkan to select those
// instead (the tags are mutually exclusive — see orderbook_gpu_{hip,vulkan}.go).
// Linked via
// lux-dex-orderbook-cuda pkg-config bundle
// -> libdex_orderbook_cuda.a (luxcpp/dex/gpu/cuda/dex_orderbook_host.cu
// + luxcpp/cuda/kernels/gpu/dex_swap.cu)
/*
#cgo pkg-config: lux-dex-orderbook-cuda
#include <stdint.h>
#include "dex_orderbook_host.h"
*/
import "C"
import (
"fmt"
"runtime"
"unsafe"
)
// dexBackendLabel reports the linked DEX GPU backend for corpus logging.
func dexBackendLabel() string { return "cuda" }
// gpuMatchOrder is the CUDA implementation called by orderbook_gpu.go.
// Returns errOrderBookGPUUnsupported when no NVIDIA device is present so the
// dispatcher falls back to MatchOrderCPU.
func gpuMatchOrder(
incoming *DEXOrder,
book []DEXOrder,
bookIndices []uint32,
tradeIDBase uint64,
timestamp uint64,
) ([]DEXTrade, uint64, error) {
if len(bookIndices) == 0 {
return nil, incoming.Quantity, nil
}
tradesOut := make([]DEXTrade, len(bookIndices))
var (
tradesWritten uint32
remaining uint64
)
var pinner runtime.Pinner
defer pinner.Unpin()
pinner.Pin(incoming)
pinner.Pin(&book[0])
pinner.Pin(&bookIndices[0])
pinner.Pin(&tradesOut[0])
pinner.Pin(&tradesWritten)
pinner.Pin(&remaining)
rc := C.dex_orderbook_match_order_host(
(*C.DEXOrder)(unsafe.Pointer(incoming)),
(*C.DEXOrder)(unsafe.Pointer(&book[0])),
(*C.uint32_t)(unsafe.Pointer(&bookIndices[0])),
C.uint32_t(len(bookIndices)),
(*C.DEXTrade)(unsafe.Pointer(&tradesOut[0])),
C.uint64_t(tradeIDBase),
C.uint64_t(timestamp),
(*C.uint32_t)(unsafe.Pointer(&tradesWritten)),
(*C.uint64_t)(unsafe.Pointer(&remaining)),
)
runtime.KeepAlive(incoming)
runtime.KeepAlive(book)
runtime.KeepAlive(bookIndices)
runtime.KeepAlive(tradesOut)
switch rc {
case 0:
// Padding normalization: the host driver already zeros the
// padding bytes for emitted trades and touched book rows, but
// belt-and-braces here in case a future kernel revision
// changes the contract.
trades := tradesOut[:int(tradesWritten)]
for i := range trades {
trades[i].ClearPadding()
}
for i := range book {
book[i].ClearPadding()
}
return trades, remaining, nil
case -2:
return nil, 0, errOrderBookGPUUnsupported
default:
return nil, 0, fmt.Errorf("lx: dex_orderbook_match_order_host rc=%d", int(rc))
}
}
-96
View File
@@ -1,96 +0,0 @@
// Copyright (C) 2025-2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//go:build cgo && linux && hip && !vulkan
package lx
// HIP/ROCm backend for the DEX OrderBook match_order kernel (AMD GPUs).
// Opt-in via -tags hip (mutually exclusive with the default CUDA backend and
// the -tags vulkan backend). Linked via
// lux-dex-orderbook-hip pkg-config bundle
// -> libdex_orderbook_hip.a (luxcpp/dex/gpu/hip/dex_orderbook_hip.cpp)
//
// The kernel is the SINGLE-THREAD deterministic matcher whose output is byte-
// identical to MatchOrderCPU by construction (see dex_orderbook_hip.cpp). That
// contract is enforced on every run by orderbook_gpu_test.go and the
// determinism corpus in match_verify_test.go.
/*
#cgo pkg-config: lux-dex-orderbook-hip
#include <stdint.h>
#include "dex_orderbook_hip.h"
*/
import "C"
import (
"fmt"
"runtime"
"unsafe"
)
// dexBackendLabel reports the linked DEX GPU backend for corpus logging.
func dexBackendLabel() string { return "hip" }
// gpuMatchOrder is the HIP implementation called by orderbook_gpu.go.
// Returns errOrderBookGPUUnsupported when no AMD/ROCm device is present so the
// dispatcher falls back to MatchOrderCPU.
func gpuMatchOrder(
incoming *DEXOrder,
book []DEXOrder,
bookIndices []uint32,
tradeIDBase uint64,
timestamp uint64,
) ([]DEXTrade, uint64, error) {
if len(bookIndices) == 0 {
return nil, incoming.Quantity, nil
}
tradesOut := make([]DEXTrade, len(bookIndices))
var (
tradesWritten uint32
remaining uint64
)
var pinner runtime.Pinner
defer pinner.Unpin()
pinner.Pin(incoming)
pinner.Pin(&book[0])
pinner.Pin(&bookIndices[0])
pinner.Pin(&tradesOut[0])
pinner.Pin(&tradesWritten)
pinner.Pin(&remaining)
rc := C.dex_orderbook_match_order_hip(
(*C.DEXOrder)(unsafe.Pointer(incoming)),
(*C.DEXOrder)(unsafe.Pointer(&book[0])),
(*C.uint32_t)(unsafe.Pointer(&bookIndices[0])),
C.uint32_t(len(bookIndices)),
(*C.DEXTrade)(unsafe.Pointer(&tradesOut[0])),
C.uint64_t(tradeIDBase),
C.uint64_t(timestamp),
(*C.uint32_t)(unsafe.Pointer(&tradesWritten)),
(*C.uint64_t)(unsafe.Pointer(&remaining)),
)
runtime.KeepAlive(incoming)
runtime.KeepAlive(book)
runtime.KeepAlive(bookIndices)
runtime.KeepAlive(tradesOut)
switch rc {
case 0:
trades := tradesOut[:int(tradesWritten)]
for i := range trades {
trades[i].ClearPadding()
}
for i := range book {
book[i].ClearPadding()
}
return trades, remaining, nil
case -2:
return nil, 0, errOrderBookGPUUnsupported
default:
return nil, 0, fmt.Errorf("lx: dex_orderbook_match_order_hip rc=%d", int(rc))
}
}
-96
View File
@@ -1,96 +0,0 @@
// Copyright (C) 2025-2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//go:build cgo && darwin
package lx
// Metal backend for the DEX OrderBook match_order kernel (Apple Silicon).
// Linked via the lux-dex-orderbook-metal pkg-config bundle
// -> libdex_orderbook_metal.a (luxcpp/dex/gpu/metal/dex_orderbook_driver.mm)
//
// The kernel is a SINGLE-THREAD sequential matcher whose output is byte-
// identical to MatchOrderCPU by construction (see dex_orderbook_driver.mm).
// That contract is enforced on every run by orderbook_gpu_test.go and the
// determinism corpus in match_verify_test.go.
/*
#cgo pkg-config: lux-dex-orderbook-metal
#include <stdint.h>
#include "dex_orderbook_driver.h"
*/
import "C"
import (
"fmt"
"runtime"
"unsafe"
)
// dexBackendLabel reports the linked DEX GPU backend for corpus logging.
func dexBackendLabel() string { return "metal" }
// gpuMatchOrder is the Metal implementation called by orderbook_gpu.go. Returns
// errOrderBookGPUUnsupported when no Metal device is present / the shader failed
// to compile, so the dispatcher falls back to MatchOrderCPU.
func gpuMatchOrder(
incoming *DEXOrder,
book []DEXOrder,
bookIndices []uint32,
tradeIDBase uint64,
timestamp uint64,
) ([]DEXTrade, uint64, error) {
if len(bookIndices) == 0 {
return nil, incoming.Quantity, nil
}
tradesOut := make([]DEXTrade, len(bookIndices))
var (
tradesWritten uint32
remaining uint64
)
var pinner runtime.Pinner
defer pinner.Unpin()
pinner.Pin(incoming)
pinner.Pin(&book[0])
pinner.Pin(&bookIndices[0])
pinner.Pin(&tradesOut[0])
pinner.Pin(&tradesWritten)
pinner.Pin(&remaining)
rc := C.dex_orderbook_match_order_metal(
(*C.DEXOrder)(unsafe.Pointer(incoming)),
(*C.DEXOrder)(unsafe.Pointer(&book[0])),
(*C.uint32_t)(unsafe.Pointer(&bookIndices[0])),
C.uint32_t(len(bookIndices)),
(*C.DEXTrade)(unsafe.Pointer(&tradesOut[0])),
C.uint64_t(tradeIDBase),
C.uint64_t(timestamp),
(*C.uint32_t)(unsafe.Pointer(&tradesWritten)),
(*C.uint64_t)(unsafe.Pointer(&remaining)),
)
runtime.KeepAlive(incoming)
runtime.KeepAlive(book)
runtime.KeepAlive(bookIndices)
runtime.KeepAlive(tradesOut)
switch rc {
case 0:
// Padding normalization to match the CPU oracle's ClearPadding output, so
// raw bytewise comparisons (EncodeTrade/EncodeRow) are deterministic.
trades := tradesOut[:int(tradesWritten)]
for i := range trades {
trades[i].ClearPadding()
}
for i := range book {
book[i].ClearPadding()
}
return trades, remaining, nil
case -2:
return nil, 0, errOrderBookGPUUnsupported
default:
return nil, 0, fmt.Errorf("lx: dex_orderbook_match_order_metal rc=%d", int(rc))
}
}
-22
View File
@@ -1,22 +0,0 @@
// Copyright (C) 2025-2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//go:build cgo && !linux && !darwin
package lx
// dexBackendLabel reports the linked DEX GPU backend for corpus logging.
func dexBackendLabel() string { return "cpu-fallback (no DEX GPU backend on this OS)" }
// gpuMatchOrder has no GPU backend on this OS. The dispatcher in
// orderbook_gpu.go turns errOrderBookGPUUnsupported into a transparent
// fallback to MatchOrderCPU.
func gpuMatchOrder(
incoming *DEXOrder,
book []DEXOrder,
bookIndices []uint32,
tradeIDBase uint64,
timestamp uint64,
) ([]DEXTrade, uint64, error) {
return nil, 0, errOrderBookGPUUnsupported
}
-97
View File
@@ -1,97 +0,0 @@
// Copyright (C) 2025-2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//go:build cgo && linux && vulkan
package lx
// Vulkan-compute backend for the DEX OrderBook match_order kernel. Opt-in via
// -tags vulkan (mutually exclusive with the default CUDA backend and -tags hip;
// vulkan wins if both hip and vulkan are set). Linked via
// lux-dex-orderbook-vulkan pkg-config bundle
// -> libdex_orderbook_vulkan.a (luxcpp/dex/gpu/vulkan/dex_orderbook_vulkan.cpp
// + the embedded SPIR-V from dex_orderbook.comp)
//
// The shader is the SINGLE-INVOCATION deterministic matcher whose output is
// byte-identical to MatchOrderCPU by construction (see dex_orderbook_vulkan.cpp).
// That contract is enforced on every run by orderbook_gpu_test.go and the
// determinism corpus in match_verify_test.go.
/*
#cgo pkg-config: lux-dex-orderbook-vulkan
#include <stdint.h>
#include "dex_orderbook_vulkan.h"
*/
import "C"
import (
"fmt"
"runtime"
"unsafe"
)
// dexBackendLabel reports the linked DEX GPU backend for corpus logging.
func dexBackendLabel() string { return "vulkan" }
// gpuMatchOrder is the Vulkan implementation called by orderbook_gpu.go.
// Returns errOrderBookGPUUnsupported when no Vulkan device (or no shaderInt64
// support) is present so the dispatcher falls back to MatchOrderCPU.
func gpuMatchOrder(
incoming *DEXOrder,
book []DEXOrder,
bookIndices []uint32,
tradeIDBase uint64,
timestamp uint64,
) ([]DEXTrade, uint64, error) {
if len(bookIndices) == 0 {
return nil, incoming.Quantity, nil
}
tradesOut := make([]DEXTrade, len(bookIndices))
var (
tradesWritten uint32
remaining uint64
)
var pinner runtime.Pinner
defer pinner.Unpin()
pinner.Pin(incoming)
pinner.Pin(&book[0])
pinner.Pin(&bookIndices[0])
pinner.Pin(&tradesOut[0])
pinner.Pin(&tradesWritten)
pinner.Pin(&remaining)
rc := C.dex_orderbook_match_order_vulkan(
(*C.DEXOrder)(unsafe.Pointer(incoming)),
(*C.DEXOrder)(unsafe.Pointer(&book[0])),
(*C.uint32_t)(unsafe.Pointer(&bookIndices[0])),
C.uint32_t(len(bookIndices)),
(*C.DEXTrade)(unsafe.Pointer(&tradesOut[0])),
C.uint64_t(tradeIDBase),
C.uint64_t(timestamp),
(*C.uint32_t)(unsafe.Pointer(&tradesWritten)),
(*C.uint64_t)(unsafe.Pointer(&remaining)),
)
runtime.KeepAlive(incoming)
runtime.KeepAlive(book)
runtime.KeepAlive(bookIndices)
runtime.KeepAlive(tradesOut)
switch rc {
case 0:
trades := tradesOut[:int(tradesWritten)]
for i := range trades {
trades[i].ClearPadding()
}
for i := range book {
book[i].ClearPadding()
}
return trades, remaining, nil
case -2:
return nil, 0, errOrderBookGPUUnsupported
default:
return nil, 0, fmt.Errorf("lx: dex_orderbook_match_order_vulkan rc=%d", int(rc))
}
}