Files
dex/Dockerfile.dvenue
zeekay 3e4d9f5517 cmd: drop demo/bench/test sprawl; consolidate venue to one dvenue
Drop 31 demo/benchmark/experiment one-off cmds whose coverage lives in
package tests (bench-all, dag-network, dclient, dexe2e, e2e-fix-zmq,
fix-benchmark, hybrid-auto, latency-benchmark, memory-*, mlx-memory-analysis,
multi-node, multi-node-test, nats-*, perf-test, stress-test, test-client,
zmq-*, dvenue-cancel, dvenue-custody, venueprobe, empty api-server/server/trader).

Keep production binaries: dvenue, dchain, dex-server, dex-api-server, gateway,
price-feed, monitor, persistent-server, demo (each referenced by a
Dockerfile/Makefile/script/k8s manifest or a documented entrypoint).

Consolidate cmd/dvenue + cmd/dvenue-cpu into one tag-free cmd/dvenue: the two
mains were byte-identical except a build tag and a log label; the GPU-vs-CPU
matcher is already selected inside pkg/lx by its own cgo-tagged files, so the
main needs no tag. CGO_ENABLED alone picks the matcher; one binary builds in
both modes. engine_{cgo,nocgo}.go carry only the engine display label. dvenue
now builds under the default CGO_ENABLED=0 gate (was tag-excluded before).

Dockerfile.dvenue: drop the dvenue-custody build+copy (cmd removed).
Makefile build-tools: point at real tools (price-feed, monitor) instead of the
removed cmd/trader and never-existent cmd/lx-metrics.

pkg/price: fix the three CChainSource tests. They were gated on requireLuxd
(a luxd TCP dial) but CChainSource never dials luxd — fetch() is pure simulate().
The gate skipped only when luxd was down, masking the real defect on hosts where
luxd runs. Rewrite the tests to drive the source's true contract deterministically
via src.poll() (no ticker race) and assert against the symbols the source actually
polls (cchainTokens), plus not-found for untracked symbols. Remove the dead
requireLuxd helper and its net import.
2026-06-19 11:59:33 -07:00

181 lines
9.4 KiB
Docker

# ---------------------------------------------------------------------------
# D-Chain VENUE image — ghcr.io/luxfi/dchain-venue
#
# The single-operator private build that actually serves the D-Chain. It is NOT
# one of the 5 public luxd validators and is NOT the public ghcr.io/luxfi/node
# image (which excludes the DexVM). It bundles TWO binaries the operator venue
# StatefulSet runs from one image:
#
# /usr/local/bin/dvenue — the ZAP matching engine (luxfi/dex cmd/dvenue,
# //go:build cgo, built CGO_ENABLED=1). This is the
# real CLOB:
# it links the GPU matching kernels (libdex_clob_cuda
# + libamm_xyk_cuda) and the secp256k1 ecrecover
# bundle via pkg-config. The 0x9010 C-Chain precompile
# and the chains/dexvm atomic proxy dial it.
# /luxd/build/luxd — the dchain-tagged node (`-tags dchain`), which
# links the chains/dexvm PROXY so the venue node
# serves the P-chain-registered D-Chain. The proxy
# relays to the dvenue socket; it is pure-Go (no GPU).
#
# ARCH: linux/arm64 ONLY. The matcher's `cgo && linux` path in luxfi/dex pkg/lx
# (orderbook_cuda.go + amm_gpu_cuda.go) has no non-CUDA fallback on any arch, so
# the venue can only be built where CUDA exists — the arcd fleet's one CUDA host
# is spark (NVIDIA GB10, sm_121, aarch64). There is no linux/amd64 CUDA runner.
#
# PRIVATE DEPS: the three pkg-config libs are fetched as a per-arch tarball from
# the luxcpp/dex `dex-gpu-release` workflow (luxdex-venue-<ver>-linux-arm64.tar.gz),
# extracted to /usr/local — exactly how node/Dockerfile fetches luxcpp/cevm.
# That tarball carries lux-dex-clob-cuda.pc, lux-dex-amm-cuda.pc and
# lux-crypto-secp256k1.pc plus their libs/headers under a GNU prefix.
# ---------------------------------------------------------------------------
ARG GO_VERSION=1.26.4
# CUDA 12.x devel image (arm64/sbsa variant) provides nvcc-free libcudart for
# the cgo link of dvenue. Runtime stage uses the matching -runtime image.
ARG CUDA_VERSION=12.4.1
ARG UBUNTU_VERSION=22.04
# ============= Builder Stage (arm64, CUDA devel) ================
# CUDA devel base gives libcudart + headers so cgo can link -lcudart. Go is
# installed on top (the project requires Go 1.26.x, newer than apt ships).
FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION} AS builder
ARG GO_VERSION
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc g++ libc6-dev make git ca-certificates wget pkg-config xz-utils \
&& rm -rf /var/lib/apt/lists/*
# Install Go (arm64).
RUN wget -q "https://go.dev/dl/go${GO_VERSION}.linux-arm64.tar.gz" \
&& tar -C /usr/local -xzf "go${GO_VERSION}.linux-arm64.tar.gz" \
&& rm "go${GO_VERSION}.linux-arm64.tar.gz"
ENV PATH="/usr/local/go/bin:${PATH}"
# luxfi module resolution: skip checksum DB (tags may be rewritten), proxy for
# public deps, direct for luxfi/*.
ENV GONOSUMCHECK=github.com/luxfi/*
ENV GONOSUMDB=github.com/luxfi/*
ENV GOPROXY=https://proxy.golang.org,direct
ENV GONOPROXY=github.com/luxfi/*
ENV GOFLAGS="-mod=mod"
ENV GOEXPERIMENT=jsonv2
# Private luxfi/* modules resolve via git url.insteadOf using a BuildKit secret
# (same shape as node/Dockerfile). The token is written to /etc/gitconfig on
# this throwaway builder stage; only the compiled binaries reach the runtime.
RUN --mount=type=secret,id=ghtok,required=false \
if [ -s /run/secrets/ghtok ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/ghtok)@github.com/".insteadOf "https://github.com/"; \
fi
# --- Fetch the venue GPU dependency bundle (the private link surface) --------
# Published by luxcpp/dex .github/workflows/dex-gpu-release.yml. Extracting to
# /usr/local puts the .pc files at /usr/local/lib/pkgconfig and the libs at
# /usr/local/lib, so pkg-config resolves lux-dex-clob-cuda / lux-dex-amm-cuda /
# lux-crypto-secp256k1 at the cgo link step below.
#
# CI/RELEASE GAP (until the dex-gpu-release workflow has run on a tag): the URL
# below 404s. The build then fails at the dvenue link step with a clear
# pkg-config error. Run luxcpp/dex dex-gpu-release first (or point VENUE_DEPS_URL
# at the workflow artifact) to populate it.
ARG VENUE_DEPS_VERSION=v0.1.1
ARG VENUE_DEPS_URL=https://github.com/luxcpp/dex/releases/download/${VENUE_DEPS_VERSION}/luxdex-venue-${VENUE_DEPS_VERSION}-linux-arm64.tar.gz
RUN --mount=type=secret,id=ghtok,required=false \
AUTH=""; [ -s /run/secrets/ghtok ] && AUTH="--header=Authorization: Bearer $(cat /run/secrets/ghtok)"; \
wget -q $AUTH "${VENUE_DEPS_URL}" -O /tmp/venue-deps.tar.gz \
&& tar -xzf /tmp/venue-deps.tar.gz -C /usr/local \
&& rm /tmp/venue-deps.tar.gz \
&& ldconfig 2>/dev/null || true
ENV PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
# --- Build dvenue (the GPU matcher) from luxfi/dex ---------------------------
# cmd/dvenue is `//go:build cgo`: CGO_ENABLED=1 (which sets the `cgo` build tag)
# selects pkg/lx's GPU matcher (orderbook_cuda + amm_gpu_cuda + signed_order_gpu,
# all gated on `cgo`), satisfied by the pkg-config libs above. `-tags cgo` is
# explicit-for-readability (CGO_ENABLED=1 already implies it). The pure-Go CPU
# venue is cmd/dvenue-cpu (//go:build !cgo) — not built here.
WORKDIR /build/dex
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=1 GOOS=linux GOARCH=arm64 \
go build -trimpath -tags "cgo" -ldflags="-s -w" \
-o /out/dvenue ./cmd/dvenue
# --- Build the dchain-tagged luxd (the proxy node) ---------------------------
# Cloned from luxfi/node at a pinned release and built `-tags dchain` so it
# links the chains/dexvm proxy (pure-Go; published chains dep). The dchain tag
# is appended to the build profile's tags so nattraversal etc. are preserved.
ARG NODE_VERSION=v1.30.12
RUN git clone --depth 1 --branch ${NODE_VERSION} https://github.com/luxfi/node.git /build/node
WORKDIR /build/node
RUN go mod download
# Tag set = the public node image's `minimal` profile (`nattraversal`) PLUS
# `dchain` so the chains/dexvm proxy links. ZAP transport is in the default
# build (not gated). Deliberately NO `grpc`: the `grpc` tag flips
# luxfi/zapdb/pb to the protobuf variant (badgerpb4.pb.go), which drops the
# package-level Marshal/Unmarshal that zapdb/table needs — `grpc` does not
# compile against zapdb v1.10.1 (verified). The venue node needs no gRPC plugin
# transport, so `minimal,dchain` is both correct and sufficient.
#
# CGO_ENABLED=0: the dchain luxd's only cgo edge is the optional luxcpp/cevm
# C-Chain backend; the dexvm proxy + pure-Go EVM need no native libs (the
# build graph pulls only runtime/cgo). Building pure-Go keeps the venue node
# free of the cevm fetch — the GPU/cgo surface lives entirely in dvenue.
RUN GITCOMMIT=$(git rev-parse HEAD) && \
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 \
go build -trimpath -tags "nattraversal,dchain" -ldflags="-s -w \
-X github.com/luxfi/node/version.GitCommit=${GITCOMMIT}" \
-o /luxd/build/luxd ./main
# --- Build the EVM (C-Chain) plugin ------------------------------------------
# node registers the optional VMs (incl. the dexvm proxy) + P/X in-process, but
# the C-Chain EVM is an EXTERNAL plugin (same as the public node image's EVM
# Plugin Stage). The venue --track-all-chains, and the 0x9010 dex precompile
# (NewZAPEngine(dex-zap-endpoint) -> the dvenue socket) lives in this plugin, so
# the venue MUST ship it. Built from the pinned luxfi/evm release, CGO=0,
# written to /luxd/build/plugins/<EVM_VM_ID> — startup.sh seeds it into
# /data/plugins (--plugin-dir).
ARG EVM_VERSION=v1.99.25
ARG EVM_VM_ID=mgj786NP7uDwBCcq6YwThhaN8FLyybkCa4zBWTQbNgmK6k9A6
RUN git clone --depth 1 --branch ${EVM_VERSION} https://github.com/luxfi/evm.git /build/evm && \
cd /build/evm && \
mkdir -p /luxd/build/plugins && \
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOFLAGS=-mod=mod \
go build -trimpath -ldflags="-s -w" -o /luxd/build/plugins/${EVM_VM_ID} ./plugin && \
chmod +x /luxd/build/plugins/${EVM_VM_ID}
# ============= Runtime Stage (arm64, CUDA runtime) ================
# CUDA -runtime image carries libcudart for the GPU kernels at run time.
FROM nvidia/cuda:${CUDA_VERSION}-runtime-ubuntu${UBUNTU_VERSION} AS execution
RUN apt-get update && apt-get install -y --no-install-recommends \
curl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# Carry the GPU/crypto shared libs the dvenue binary may dlopen / rpath into.
COPY --from=builder /usr/local/lib /usr/local/lib
COPY --from=builder /usr/local/include /usr/local/include
RUN ldconfig 2>/dev/null || true
# Binaries: the dchain luxd (venue node) + the EVM (C-Chain) plugin it loads +
# dvenue (ZAP matcher). startup.sh seeds /luxd/build/plugins/* into /data/plugins.
COPY --from=builder /luxd/build/luxd /luxd/build/luxd
COPY --from=builder /luxd/build/plugins /luxd/build/plugins
COPY --from=builder /out/dvenue /usr/local/bin/dvenue
# Non-root (matches the operator StatefulSet securityContext runAsUser 1000).
RUN groupadd -g 1000 venue && useradd -u 1000 -g venue -m -s /usr/sbin/nologin venue \
&& mkdir -p /data && chown -R venue:venue /data /luxd
USER venue
ENV PATH="/luxd/build:/usr/local/bin:${PATH}"
WORKDIR /luxd/build
# 9650 luxd HTTP RPC, 9651 luxd staking, 9099 dvenue ZAP socket.
EXPOSE 9650 9651 9099
# Default to the venue node; the operator overrides the `zap` container's
# command to /usr/local/bin/dvenue.
CMD ["/luxd/build/luxd"]