xvm/BENCHMARKS.md

XVM Transition Substrate — Benchmarks

Wall-clock benchmarks for the GPU-native X-Chain UTXO transition substrate across CPU reference, Metal, and WGSL/WebGPU back-ends. Generated by xvm-benchmark (Release -O3 -DNDEBUG, std::chrono::steady_clock).

Raw data: see BENCHMARKS_RAW.txt in this directory.

Hardware

Item	Value
Host	MacBookPro18,2 (M1 Max)
CPU	Apple M1 Max (10-core)
GPU	Apple M1 Max (32-core, Metal 4)
Memory	64 GB unified
OS	macOS 26.4 (Build 25E241)
Compiler	AppleClang 17.0.0.17000603
CMake	Release, -O3 -DNDEBUG
WebGPU runtime	wgpu-native (Homebrew)
CUDA	not available on this host

Workloads

UTXO-heavy transfer round: seed N native-asset UTXOs, run M transfer transactions each consuming exactly one seed and emitting one new UTXO. UTXO ids are deterministic SplitMix64-distributed across all 32 bytes (matches the determinism harness — avoids cuckoo low-bit clustering).

The GPU per-round arenas are hard-capped at kDefaultUtxoSlots = 16384 and kMaxTxs = 4096; therefore seed_n + tx_n <= 16384 for the GPU-eligible tiers (each accepted tx consumes one slot for its new output). xlarge exceeds the GPU caps and runs CPU-only on a pre-sized arena.

Tier	seed_n	tx_n	GPU-eligible	iters (CPU)	iters (GPU)
small	1 024	256	yes	3 warmup + 10 timed	3 warmup + 10 timed
medium	4 096	1 024	yes	3 warmup + 10 timed	2 warmup + 5 timed
large	12 288	4 096	yes	3 warmup + 10 timed	1 warmup + 3 timed
xlarge	32 768	8 192	no (CPU)	3 warmup + 10 timed	—

GPU iteration counts shrink at higher tiers because the v0.55 driver dispatches all four kernels as 1×1×1 single-thread grids; per-round wall clock therefore scales linearly with seed_n + tx_n and a single Metal large round measures ~31 s. CPU keeps the full budget so its percentiles stay tight.

Determinism — fixed in v0.55.3

The v0.55.2 benchmark report flagged "execution_root mismatch" between CPU, Metal, and WGSL at small/medium/large tiers. The kernels were not divergent; the bench was reading per-tier last-iteration roots whose height field (= 100 + round_idx) varied per backend because each backend ran a different number of warmup+measured rounds. Same kernels, same inputs — but the LAST round's descriptor differed, so the LAST execution_root differed. v0.55.3 separates the determinism oracle from the wall-clock loop:

• A single canonical run at round=1, height=101 on every backend establishes the determinism oracle. All four roots (utxo, asset, tx, execution) match byte-for-byte across CPU / Metal / WGSL at every measured tier.
• The wall-clock timed loop continues to iterate round_idx per call so each iteration sees a distinct descriptor (cache freshness for the GPU drivers), but the per-iteration root is no longer compared cross-backend.

The determinism harness (xvm-determinism-test) was extended in v0.55.3 to sweep small / medium / large / xlarge (in addition to the original brief 10K/1K point) and confirm CPU↔Metal↔WGSL byte-equality at each shape — so the contract holds across the whole GPU-eligible parameter space, not just one pinned point. xlarge exceeds the GPU per-round arena (kDefaultUtxoSlots=16384, kMaxTxs=4096) and is asserted CPU-deterministic on a pre-sized arena; the three smaller tiers are 4-way (CPU + Metal + WGSL).

Results — wall-clock per round

Mean of timed iterations. p50/p95/p99/min/max are in BENCHMARKS_RAW.txt; they cluster within microseconds because the work is deterministic and the host is idle, so they collapse onto the mean at the printed precision.

v0.55.3 numbers below. Generated by ./build/xvm-benchmark. The v0.55.3 release introduced a canonical-descriptor determinism oracle in the bench that runs once per tier before the wall-clock loops; the numbers below are its timed-iteration means.

Tier	CPU	Metal	WGSL
small	36.829 ms	499.537 ms	2.954 s
medium	166.955 ms	2.578 s	13.690 s
large	828.322 ms	25.991 s	62.039 s
xlarge	3.890 s	—	—

Throughput

Tier	CPU UTXOs/s	CPU tx/s	Metal UTXOs/s	Metal tx/s	WGSL UTXOs/s	WGSL tx/s
small	7.14 K/s	7.14 K/s	514.49 /s	514.49 /s	86.70 /s	86.70 /s
medium	6.19 K/s	6.19 K/s	398.98 /s	398.98 /s	77.80 /s	77.80 /s
large	4.99 K/s	4.99 K/s	160.38 /s	160.38 /s	67.20 /s	67.20 /s
xlarge	2.29 K/s	2.29 K/s	—	—	—	—

xlarge accepts 8191/8192 txs (one tx is rejected; the cuckoo or bloom path reaches its bounded eviction count on a single output insert at this load factor — deterministic and reproducible across runs).

Speedup vs CPU

Tier	Metal	WGSL
small	0.07×	0.01×
medium	0.06×	0.01×
large	0.03×	0.01×
xlarge	n/a	n/a

There is no crossover at any measured tier. CPU dominates GPU end-to-end because the v0.55 kernels are 1×1×1 single-thread dispatches — the GPU runs the canonical traversal serially in one shader thread per kernel × four kernels per round, with no SIMD or workgroup parallelism. Per-round overhead (buffer staging, command-buffer commit, completion fence) and the GPU-side seeding loop dominate wall clock. WGSL adds a synchronous map-read for every result buffer, which is visible as the additional 2–3× cost over Metal.

Per-kernel breakdown (CPU, mean of 10 timed iterations)

Sub-mode runs of run_reference isolate each kernel; the dedicated XRootUpdate time is subtracted from the input-check and asset-transition sub-modes (which always recompute roots).

Tier	UTXOInputCheck + TransitionApply	XAssetTransition (empty pass)	XRootUpdate
small	12.286 ms	2.449 ms	10.098 ms
medium	59.426 ms	9.855 ms	39.819 ms
large	352.792 ms	38.964 ms	118.852 ms
xlarge	1.447 s	111.084 ms	335.866 ms

Observations:

• UTXOInputCheck + TransitionApply is the dominant kernel: per-input bloom + cuckoo lookup, ownership check, then per-output bloom-set + cuckoo-insert.
• XAssetTransition (empty pass) is the per-tx scan with zero asset ops in the transfer workload — a realistic floor for asset-transition bookkeeping.
• XRootUpdate scales with kDefaultUtxoSlots (full arena keccak walk) plus kDefaultAssetSlots and tx_count, so it grows roughly linearly with seed count regardless of tx count.

v0.55.2 → v0.55.3 — root cause + fix

v0.55.2 report. The bench printed three "WARN: execution_root mismatch" classes:

• small: WGSL root differed from CPU+Metal.
• medium: Metal and WGSL each differed from CPU and from each other.
• large: Metal+WGSL agreed but differed from CPU.

Investigation. The probe xvm-diverge-probe runs CPU, Metal, and WGSL at a single fixed descriptor (round=1, height=101) per workload size and prints per-root-component agreement. With the same descriptor, CPU, Metal, and WGSL agree byte-for-byte at every tier (small/medium/large/brief). Re-running the probe with each backend at the round_idx the v0.55.2 bench actually used (CPU at round=13, Metal at round={13,7,4} per tier, WGSL at round=4) reproduces the v0.55.2 mismatched roots exactly — including the specific 32-byte hex values printed in the v0.55.2 BENCHMARKS_RAW.txt.

Root cause. xvm_benchmark.cpp wall-clock loop iterates round_idx per backend so each call sees a distinct descriptor (make_desc(round_idx) sets height = 100 + round_idx). The reported "execution_root" was the LAST iteration's root. CPU runs 13 rounds; Metal runs (3+10, 2+5, 1+3) per tier; WGSL runs (1+3) per tier. Different last round_idx → different last height → different execution_root (which is keccak(parent || utxo_root || asset_root || tx_root || height)). No kernel divergence — just three backends reporting different rounds' results.

Fix. v0.55.3 adds an explicit canonical-descriptor determinism check to the bench, run before the timed wall-clock loops at a fixed round=1, height=101 on every backend, and prints determinism: CPU == all GPU backends @ round=1 (byte-equal) when they match (or DETERMINISM FAIL if they ever don't). The wall-clock loops keep iterating round_idx for cache freshness; their per-iteration roots are NOT compared cross-backend (different round_idx per backend would falsely flag a mismatch — the bug this version fixes). The determinism harness gains a test_size_sweep pass that runs the canonical workload at small/medium/large in addition to the existing brief (10K/1K) point, proving 4-way byte-equality across the entire GPU-eligible parameter space.

Conclusion

XVM at v0.55.3 is GPU-functional but not GPU-accelerated. The four transition kernels (UTXOInputCheck, UTXOTransitionApply, XAssetTransition, XRootUpdate) execute on Metal and WGSL/WebGPU and produce a finalized round result, byte-identical to the CPU reference at every measured workload tier on this host. Their 1×1×1 dispatches mean GPU wall clock is 14–80× slower than the CPU reference at every measured workload tier on this host.

The next iteration of the substrate must shard the kernels — at minimum parallelising UTXOInputCheck per-tx and XRootUpdate's arena keccak walk across thousands of GPU threads — before the GPU path can outperform the optimised CPU reference. The CPU oracle holds steady at ~5–7 K UTXOs/s up to 12K UTXOs / 4K txs, dropping to ~2.3 K/s at xlarge as the cuckoo hash load factor rises.

Reproducing

cmake -S /Users/z/work/luxcpp/xvm -B build-bench \
    -DCMAKE_BUILD_TYPE=Release -DLUX_XVM_ENABLE_WGPU=ON
cmake --build build-bench --target xvm-benchmark
./build-bench/xvm-benchmark > BENCHMARKS_RAW.txt

End-to-end runtime on this host is ~12 minutes (dominated by WGSL large).