mirror of
https://github.com/luxfi/zap.git
synced 2026-10-30 15:04:16 +00:00
zeekay
d14b104790
Three orthogonal additions, one wire format. v2 package — generic ad-hoc schema dispatch (~/work/lux/zap/v2): - README + IMPOSSIBILITY + BENCH_RESULTS document the v1↔v2 split. - builder/field/iter/list/pool/registry/schema/view + codegen package emit v1-equivalent fast paths from declarative .zap sources. - _compile_fail_test/ + examples/ cover negative and positive cases. - v2_test.go exercises the generic dispatch path. - doc.go in v1 marks v1 deprecated for *new* schema authoring; existing v1 buffers stay parseable by both v1 and v2. bufpool — pooled read slabs (bufpool.go + bufpool_test.go): - Message gains optional *bufRef; Release() returns the slab to its pool when sourced from the pooled read path, GC-managed otherwise. - ParseHeader returns (data, rootOffset) without allocating *Message, used by zapv2 to avoid two allocations per parse. - bench_read_test.go + node_quic_stream_test.go cover both paths. Transport allocator + UMA/RDMA cgo (transport/*): - Transport.Caps()/Buffer interface: capability + slab-typed buffer contract. Bytes() vs DevicePtr() decoupled so UMA and GPUDirect can share the same call surface. - DPDK PMD + GPUDirect RDMA + CUDA UMA real cgo backends gated behind build tags; stub paths remain default on darwin / non-Linux. - transport_test.go covers Caps reporting + Buffer lifecycle on the default transport. LLM.md + go.mod follow the additions.
32 lines
1.3 KiB
Go
32 lines
1.3 KiB
Go
// Copyright (C) 2025-2026, Lux Industries Inc. All rights reserved.
|
|
// See the file LICENSE for licensing terms.
|
|
|
|
package transport
|
|
|
|
// dpdk.go is the cross-platform doc comment for the DPDK PMD-backed
|
|
// transport. The factory lives in:
|
|
//
|
|
// dpdk_linux_real.go — //go:build cgo && linux && dpdk
|
|
// Real rte_eal_init probe + EAL bring-up.
|
|
// dpdk_linux.go — //go:build cgo && linux && !dpdk
|
|
// Clean "not available" when the dpdk tag isn't set.
|
|
// dpdk_other.go — //go:build !cgo || !linux
|
|
// Clean "not available" off-Linux or without cgo.
|
|
//
|
|
// What DPDK buys us (vs the kernel network stack):
|
|
//
|
|
// - Userspace polling instead of interrupts — sub-microsecond ack
|
|
// latency under load.
|
|
// - Hugepage-backed mempool — TLB cost amortised across the entire
|
|
// ring.
|
|
// - Lockless burst RX/TX — 100 Gbps line rate per worker thread.
|
|
//
|
|
// Why DPDK and not just GPUDirect: DPDK works on any modern NIC (not
|
|
// just Mellanox). On a Liquidity validator with an Intel E810 or
|
|
// Broadcom NetXtreme NIC, DPDK is the right path. On a DGX with a
|
|
// CX-7, GPUDirect is the right path. UMA is the right path on Apple
|
|
// Silicon and Grace-Hopper / GB10 (no discrete NIC↔GPU memory split).
|
|
//
|
|
// Pick() chooses in this order: gpudirect > dpdk > uma > default.
|
|
// Operators force a specific transport via ZAP_TRANSPORT.
|