Add 10 new Lux skill MDX pages, update navigation

New: cpp, coreth, exchange-api, fhe-coprocessor, genesis,
kms, ledger, lpm, staking, zapdb

Updated meta.json with 72+ skills in 17 categories.
This commit is contained in:
2026-03-13 17:46:54 -07:00
parent d6a14414cd
commit 818f1fb22d
11 changed files with 1968 additions and 1 deletions
@@ -0,0 +1,149 @@
---
title: "Lux Coreth - C-Chain EVM (DEPRECATED)"
description: "Coreth ("core Ethereum") was the original Go EVM implementation that powered the Lux C-Chain. As of January 2025, it was consolidated with the Subnet EVM codebase into the unified package. The..."
---
## Overview
**Coreth is deprecated and merged into `github.com/luxfi/evm`.** All new development MUST use `github.com/luxfi/evm`. This repository is archived for historical reference only.
Coreth ("core Ethereum") was the original Go EVM implementation that powered the Lux C-Chain. As of January 2025, it was consolidated with the Subnet EVM codebase into the unified `github.com/luxfi/evm` package. The deprecation commit is `ac7aec973` in the coreth repo. The README explicitly directs to the evm repo, and `DEPRECATED.md` provides a full migration guide and timeline.
The coreth repo still compiles (Go 1.26.1, module `github.com/luxfi/coreth`) and received post-deprecation maintenance fixes through early 2026 (dependency bumps, build fixes), but no new features. The final meaningful commits were block import persistence fixes and txpool sync improvements. The `github.com/luxfi/evm` module preserves all coreth functionality with identical package paths (just replace `coreth` with `evm` in imports).
### Relationship to `github.com/luxfi/evm`
| Aspect | coreth (deprecated) | evm (canonical) |
|--------|-------------------|-----------------|
| Module | `github.com/luxfi/coreth` | `github.com/luxfi/evm` |
| C-Chain support | Yes | Yes |
| Subnet EVM support | No | Yes |
| Warp messaging | Yes | Yes |
| FHE precompiles | Yes (via `luxfi/fhe`) | Yes (via `luxfi/fhe`) |
| ZK precompiles | No | Yes (blake3, zk, dead address) |
| Bridge precompile | No | Yes |
| Native asset transfers | Yes (`nativeasset/` dir) | No (removed; uses Warp) |
| State upgrades | No | Yes (`stateupgrade/` dir) |
| `api/` package | No | Yes |
| `commontype/` package | No | Yes |
| Active development | No (archived) | Yes |
Both repos share nearly identical dependency sets (`luxfi/consensus`, `luxfi/geth`, `luxfi/crypto`, `luxfi/node`, `luxfi/warp`, etc.) at the same versions. The evm repo has slightly newer versions of some dependencies (e.g., `luxfi/database` v1.17.43 vs v1.17.42, `luxfi/accel` v1.0.5 vs v1.0.1).
## When to use
- **Do NOT use coreth for new code.** Use `github.com/luxfi/evm` instead.
- Reference coreth only when:
- Migrating legacy code that still imports `github.com/luxfi/coreth`
- Investigating historical C-Chain behavior or block format changes
- Understanding the block import persistence fix (vm.go `PostImportCallback` consensus state update)
- Tracing the origin of C-Chain-specific features like atomic transactions and native asset transfers
## Hard requirements
1. NEVER use `github.com/luxfi/coreth` in new Go modules. Use `github.com/luxfi/evm`.
2. NEVER use `go-ethereum` directly. Use `github.com/luxfi/geth`.
3. When migrating, replace all `coreth` imports with `evm` -- the package paths are identical after the module prefix.
## Quick reference
| Item | Value |
|------|-------|
| Module (DEPRECATED) | `github.com/luxfi/coreth` |
| Replacement module | `github.com/luxfi/evm` |
| Go version | 1.26.1 |
| Deprecation date | January 2025 |
| Deprecation commit | `ac7aec973` |
| C-Chain ID | 96369 (mainnet), 96370 (testnet) |
| Genesis hash | `0x3f4fa2a0b0ce089f52bf0ae9199c75ffdd76ecafc987794050cb0d286f1ec61e` |
| Final status | Security fixes only through Q2 2025, then archive-only |
## Core Concepts
### Migration Guide
Replace go.mod dependency:
```
// Remove
require github.com/luxfi/coreth v1.x.x
// Add
require github.com/luxfi/evm v0.8.x
```
Replace all imports:
```go
// OLD (deprecated)
import "github.com/luxfi/coreth/core"
import "github.com/luxfi/coreth/params"
import "github.com/luxfi/coreth/plugin/evm"
import "github.com/luxfi/coreth/eth"
import "github.com/luxfi/coreth/ethclient"
// NEW (canonical)
import "github.com/luxfi/evm/core"
import "github.com/luxfi/evm/params"
import "github.com/luxfi/evm/plugin/evm"
import "github.com/luxfi/evm/eth"
import "github.com/luxfi/evm/ethclient"
```
The API surface is identical. No code changes needed beyond import paths.
### Why Coreth Was Deprecated
1. **Single implementation**: Eliminates divergence between C-Chain (coreth) and Subnet EVM code paths
2. **Simplified maintenance**: One codebase to maintain, test, and audit
3. **Consistent behavior**: All EVM chains on Lux use identical code
4. **New features**: Bridge precompiles, ZK precompiles, state upgrades, and `commontype`/`api` packages were added only to the unified evm repo
### C-Chain Differences from Ethereum
These behaviors are preserved in `github.com/luxfi/evm` (C-Chain mode):
- **Atomic transactions**: Cross-chain transfers between X-Chain and C-Chain via `ExtData` in block body
- **Block timing**: Blocks can have the same timestamp as parent (async Consensusman consensus, no fixed interval)
- **Difficulty**: Always 1 (Consensusman does not use PoW difficulty)
- **RANDOM opcode**: Returns block difficulty (always 1), no Eth2 randomness beacon equivalent
- **Block header extensions**: `ExtDataHash`, `BaseFee`, `ExtDataGasUsed`, `BlockGasCost`
- **Block body extensions**: `Version` and `ExtData` fields
### Historical Block Import Fix
The `PostImportCallback` fix (documented in coreth LLM.md, vm.go lines 710-758) resolved a critical issue where `admin_importChain` stored blocks in the database but did not update `chain.State.lastAcceptedBlock`. This caused the chain to appear frozen after import. The fix wraps the imported eth block and calls `SetLastAcceptedBlock()` on the consensus layer. This fix is carried forward in `github.com/luxfi/evm`.
### Key luxfi Dependencies (coreth final versions)
```
github.com/luxfi/consensus v1.22.67
github.com/luxfi/crypto v1.17.40
github.com/luxfi/geth v1.16.73
github.com/luxfi/database v1.17.42
github.com/luxfi/node v1.23.4
github.com/luxfi/precompile v0.4.7
github.com/luxfi/warp v1.18.5
github.com/luxfi/fhe v1.7.6
github.com/luxfi/lattice/v7 v7.0.0
```
## Troubleshooting
**"coreth" imports in existing code** -- Run `find . -name "*.go" | xargs grep "luxfi/coreth"` to find all references. Replace with `luxfi/evm`. Run `go mod tidy` after updating.
**Build failures after migration** -- The `nativeasset/` package exists in coreth but not in evm. If your code imports it, you need to refactor to use Warp messaging for cross-chain asset transfers.
**go.mod replace directive** -- If you had `replace github.com/luxfi/coreth => ../coreth`, change to `replace github.com/luxfi/evm => ../evm`.
**Conflicting dependency versions** -- The evm repo uses slightly newer versions of some packages (database v1.17.43 vs v1.17.42, accel v1.0.5 vs v1.0.1). Run `go mod tidy` to resolve.
## Related Skills
- `lux/lux-evm.md` -- The canonical replacement for coreth. Use this for all new work.
- `lux/lux-node.md` -- Lux node that runs the EVM plugin
- `lux/lux-warp.md` -- Cross-chain messaging (replaces atomic transactions for subnets)
- `lux/lux-precompile.md` -- Custom precompile development
- `lux/lux-fhe.md` -- FHE library used by both coreth and evm
---
Last Updated: 2026-03-13
+220
View File
@@ -0,0 +1,220 @@
---
title: "Lux C++ - High-Performance C++ Libraries for Blockchain & AI"
description: "### Why LuxCPP?"
---
## Overview
LuxCPP is a **modular C++ library suite** providing GPU-accelerated cryptography, FHE, lattice operations, DEX matching, and networking for the Lux blockchain ecosystem. It consists of 8 componentized libraries with Metal, CUDA, and WebGPU GPU backends, all built via CMake + Conan.
### Why LuxCPP?
- **Multi-backend GPU**: Metal (Apple Silicon), CUDA (NVIDIA), WebGPU (Dawn), CPU (SIMD) — compile-time or runtime backend selection
- **Full FHE stack**: TFHE/CKKS/BGV via OpenFHE integration with GPU-accelerated NTT
- **Post-quantum crypto**: BLS12-381 pairings, ML-DSA (Dilithium), lattice-based NTT
- **DEX engine**: C++ CLOB matching engine for sub-microsecond order processing
- **Zero-copy FFI**: C API boundary (`lux/gpu.h`) for Go, Rust, Python, WASM bindings
### Tech Stack
- **Language**: C++20
- **Build**: CMake 3.26+ with Conan 2.x package manager
- **GPU**: Metal (MLX), CUDA (CCCL), Dawn (WebGPU/WGSL)
- **Version**: 1.0.0
- **License**: BSD-3-Clause
- **Repo**: `github.com/luxfi/luxcpp`
### Components
| Component | Library | Purpose |
|-----------|---------|---------|
| `lux-gpu` | `libluxgpu` | GPU compute dispatch (Metal/CUDA/WebGPU/CPU) |
| `lux-crypto` | `libluxcrypto` | BLS12-381, ML-DSA, secp256k1, hashing |
| `lux-lattice` | `libluxlattice` | NTT, ring operations, RLWE |
| `lux-fhe` | `libluxfhe` | TFHE/CKKS/BGV (OpenFHE-based) |
| `lux-cuda` | `libluxcuda` | CUDA kernels (proprietary) |
| `lux-http` | `libluxhttp` | HTTP/REST framework |
| `lux-grpc` | `libluxgrpc` | gRPC services |
| `lux-dex` | `libluxdex` | DEX matching engine |
### Dependency Hierarchy
```
lux-gpu ← Foundation (cross-platform GPU acceleration)
▲
lux-crypto ← BLS pairings, post-quantum (depends on gpu)
lux-lattice ← NTT acceleration (depends on gpu)
▲
lux-fhe ← TFHE/CKKS/BGV (depends on crypto + lattice)
```
## When to use
- Building GPU-accelerated blockchain operations (ZK proofs, FHE, NTT)
- Integrating C++ crypto into Go/Rust services via FFI
- Running high-performance DEX matching engines
- Accelerating lattice-based post-quantum cryptography
- Building cross-platform GPU compute (macOS Metal + Linux CUDA)
## Hard requirements
1. **CMake 3.26+** for build
2. **Conan 2.x** for dependency management
3. **C++20 compiler** (Clang 15+ or GCC 13+)
4. **Metal SDK** (macOS) or **CUDA Toolkit** (Linux/Windows) for GPU backends
5. **Dawn** (optional, for WebGPU backend)
## Quick reference
| Item | Value |
|------|-------|
| Build system | CMake + Conan |
| Version | 1.0.0 |
| License | BSD-3-Clause |
| C++ standard | C++20 |
| GPU backends | Metal, CUDA, WebGPU (Dawn), CPU (SIMD) |
| Install prefix | `include/lux/<pkg>/`, `lib/liblux<pkg>.*` |
| CMake targets | `lux::gpu`, `lux::crypto`, `lux::lattice`, `lux::fhe` |
| pkg-config | `lux-gpu`, `lux-crypto`, etc. |
| Repo | `github.com/luxfi/luxcpp` |
## One-file quickstart
### Build all components
```bash
# Install dependencies
conan install . --output-folder=build --build=missing
# Configure and build
cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE=Release
cmake --build .
# Run tests
ctest --test-dir .
```
### Platform-specific profiles
```bash
# macOS Apple Silicon (Metal backend)
conan install . -pr:b=default -pr:h=profiles/macos-arm64
# Linux x86_64 (CUDA backend)
conan install . -pr:b=default -pr:h=profiles/linux-x86_64
# Windows x86_64
conan install . -pr:b=default -pr:h=profiles/windows-x86_64
```
### Use from CMake
```cmake
find_package(lux-gpu REQUIRED)
target_link_libraries(myapp lux::gpu)
find_package(lux-crypto REQUIRED)
target_link_libraries(myapp lux::crypto)
```
## Core Concepts
### Directory Structure
```
luxcpp/
├── gpu/ # lux-gpu: GPU compute library
│ ├── include/lux/gpu.h # C API (stable FFI boundary)
│ ├── src/ # Core dispatch, CPU backend, ZK ops
│ ├── webgpu/ # Dawn WebGPU backend + WGSL kernels
│ ├── kernels/cpu/ # CPU kernel implementations
│ ├── benchmarks/ # Performance tests
│ └── test/ # Unit tests
├── crypto/ # lux-crypto: Cryptographic primitives
├── lattice/ # lux-lattice: NTT, ring operations
├── fhe/ # lux-fhe: FHE (OpenFHE-based)
├── dex/ # lux-dex: Matching engine
├── http/ # lux-http: HTTP/REST
├── grpc/ # lux-grpc: gRPC services
├── session/ # Session-related C++ components
├── consensus/ # Consensus C++ components
├── cuda/ # CUDA kernel sources
├── metal/ # Metal shader sources
├── webgpu/ # WebGPU/WGSL shader sources
├── lux-accel/ # Umbrella acceleration package
├── lux-cuda/ # CUDA-specific build
├── lux-gpu/ # GPU-specific build
├── lux-metal/ # Metal-specific build
├── lux-webgpu/ # WebGPU-specific build
├── mlx-c-api/ # MLX C API bindings
├── third_party/ # Vendored dependencies
├── cmake/ # CMake modules
├── profiles/ # Conan build profiles
├── scripts/ # Build & install scripts
├── conanfile.py # Conan package definition
└── LIBRARY_CONTRACT.md # Standard layout specification
```
### GPU Backend Selection
The GPU library auto-selects the best available backend at compile time, with runtime fallback:
| Backend | Platform | Acceleration |
|---------|----------|-------------|
| Metal | macOS (Apple Silicon) | MLX integration |
| CUDA | Linux/Windows (NVIDIA) | CCCL/cuBLAS |
| Dawn | Cross-platform | WebGPU via WGSL shaders |
| CPU | All platforms | SIMD (AVX2/NEON) fallback |
### Install Layout Convention
All libraries follow a canonical install layout:
- Headers: `include/lux/<pkg>/*.h`
- Libraries: `lib/liblux<pkg>.{dylib,so,a}`
- CMake config: `lib/cmake/lux-<pkg>/`
- pkg-config: `lib/pkgconfig/lux-<pkg>.pc`
### Conan Build Options
```python
# Component toggles
with_gpu=True # GPU compute
with_fhe=True # FHE operations
with_crypto=True # Cryptographic primitives
with_lattice=True # Lattice crypto
with_dex=True # DEX matching engine
with_http=True # HTTP framework
with_grpc=False # gRPC (opt-in, large dep)
# Backend toggles
with_metal=True # Metal (macOS)
with_cuda=True # CUDA (Linux/Windows)
with_webgpu=True # WebGPU (Dawn)
with_openmp=True # OpenMP parallelism
# Features
with_tests=False # Build tests
with_benchmarks=False # Build benchmarks
embed_kernels=True # Embed kernel sources in binary
```
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| Metal not found | Wrong platform | Metal only on macOS with Xcode |
| CUDA not found | Missing toolkit | Install CUDA Toolkit 12+ |
| Conan errors | Wrong version | Requires Conan 2.x (not 1.x) |
| Dawn build fails | Missing deps | `git submodule update --init` in Dawn |
| Link errors | Missing component | Check `find_package(lux-<pkg>)` in CMakeLists |
## Related Skills
- `lux/lux-gpu.md` — Go bindings for GPU acceleration (wraps this C++ code)
- `lux/lux-fhe.md` — Go FHE library (uses lattice from here)
- `lux/lux-lattice.md` — Go lattice crypto (EPFL, complements C++ NTT)
- `lux/lux-crypto.md` — Go crypto primitives
- `lux/lux-dex.md` — Go/Rust DEX engine (C++ matching core)
- `lux/lux-accel.md` — Rust acceleration (complements C++)
@@ -0,0 +1,186 @@
---
title: "Lux Exchange API - GraphQL Proxy and Token Data Service"
description: "## When to use"
---
## Overview
Lux Exchange API (`github.com/luxfi/exchange-api`) is a **TypeScript Express server** that serves as a GraphQL proxy for the Lux DEX frontend. It intercepts Uniswap-compatible GraphQL queries, handles LUX/ZOO chain queries natively using Blockscout and subgraph data sources, and proxies all other chain queries to the upstream Uniswap API. Works alongside `exchange-proxy` (Go CORS proxy for raw Uniswap endpoints) and `exchange-sdk` (TypeScript order matching engine library).
## When to use
- Adding new token metadata to the Lux DEX token list
- Modifying how LUX/ZOO chain data is fetched or presented
- Adjusting cache TTLs for price/volume data
- Adding new GraphQL operation handlers for native chains
- Debugging token price display issues in the exchange frontend
## Hard requirements
1. ALWAYS use `github.com/luxfi/*` package paths -- NEVER `go-ethereum` or `ava-labs`
2. Native chain handling is determined by `NATIVE_CHAINS` set: `LUX` and `ZOO`
3. Stablecoins (USDT, USDC, LUSD, DAI, BUSD) are force-priced to $1.00
4. Prices and volumes exceeding 1e12 are capped to 0 (subgraph overflow artifacts)
5. Unhandled native chain operations fall through to Uniswap proxy
## Quick reference
| Item | Value |
|------|-------|
| Repo | `github.com/luxfi/exchange-api` |
| Language | TypeScript (ES2022, strict mode) |
| Runtime | Node.js 22 |
| Framework | Express 4.21 |
| Default Port | 4000 |
| Build | `npm install && npm run build` |
| Dev | `npm run dev` (tsx watch) |
| Docker | Multi-stage alpine, `node dist/index.js` |
## Core Concepts
### Three-Service Architecture
| Service | Repo | Language | Port | Purpose |
|---------|------|----------|------|---------|
| **exchange-api** | `github.com/luxfi/exchange-api` | TypeScript | 4000 | GraphQL proxy, native chain data |
| **exchange-proxy** | `github.com/luxfi/exchange-proxy` | Go | 8088 | CORS proxy for raw Uniswap APIs |
| **exchange-sdk** | `github.com/luxfi/exchange-sdk` | TypeScript | - | Order matching engine library |
### Request Flow
```
DEX Frontend
|
v
exchange-api :4000
POST /v1/graphql
|
+-- chain=LUX or ZOO? --> Native handler (Blockscout + Subgraph V2/V3)
|
+-- other chain? -------> Proxy to Uniswap API (with caching)
```
### Native GraphQL Operations
| Operation | Handler | Data Source |
|-----------|---------|-------------|
| `TopTokens` / `TopTokens100` / `TopTokensSparkline` | `handleTopTokens` | V2+V3 subgraph merged, Blockscout fallback |
| `Token` / `TokenPrice` / `SimpleToken` | `handleToken` | Subgraph + Blockscout + known token list |
| `TopV2Pairs` | `handleTopV2Pairs` | V2 subgraph |
| `TopV3Pools` | `handleTopV3Pools` | V3 subgraph |
| `V2Transactions` / `V3Transactions` | `handleTransactions` | V2 subgraph swaps |
| `HistoricalProtocolVolume` / `DailyProtocolTvl` | stub | Returns empty arrays |
| `IsV3SubgraphStale` | stub | Returns `false` |
### Data Sources
| Source | Default URL | Purpose |
|--------|------------|---------|
| Blockscout | `https://api-explore.lux.network` | ERC-20 token list, on-chain stats |
| V2 Subgraph | `https://subgraph.lux.network/subgraphs/name/luxfi/uniswap-v2` | V2 AMM pairs, tokens, swaps |
| V3 Subgraph | `https://subgraph.lux.network/subgraphs/name/luxfi/uniswap-v3` | V3 pools, tokens, swaps |
| Uniswap API | `https://interface.gateway.uniswap.org/v1/graphql` | Non-native chain proxy target |
### Cache TTLs
| Tier | Duration | What |
|------|----------|------|
| SHORT | 30s | Prices, volumes, stats |
| MEDIUM | 5min | Token lists, pool lists |
| LONG | 1hr | Token metadata, logos |
| PROXY | 60s | Proxied Uniswap responses |
### Environment Variables
| Variable | Default | Purpose |
|----------|---------|---------|
| `PORT` | `4000` | Listen port |
| `BLOCKSCOUT_API` | `https://api-explore.lux.network` | Blockscout API base |
| `SUBGRAPH_URL` | `https://subgraph.lux.network/.../uniswap-v2` | V2 subgraph |
| `SUBGRAPH_V3_URL` | `https://subgraph.lux.network/.../uniswap-v3` | V3 subgraph |
| `UNISWAP_API` | `https://interface.gateway.uniswap.org/v1/graphql` | Upstream Uniswap |
### Known Tokens (lux-tokens.ts)
20 LUX mainnet tokens with verified on-chain addresses:
| Symbol | Address | Decimals |
|--------|---------|----------|
| LUX | Native (0x0...0) | 18 |
| WLUX | `0x4888...8b3E` | 18 |
| USDC | `0xF85C...95A6` | 6 |
| LETH | `0x60E0...d9ba` | 18 |
| LBTC | `0x1E48...4f4e` | 8 |
| LSOL | `0x26B4...e9e7F` | 18 |
Plus LUSD, LZOO, LAVAX, LBLAST, LBNB, LBOME, LBONK, LCELO, LDOGS, LPOL, LTON, CYRUS, MELANIA, TRUMP, Z.
### Exchange Proxy (Go, :8088)
CORS reverse proxy that forwards requests to Uniswap upstream APIs with `Origin: https://app.uniswap.org` header injection:
| Route Prefix | Upstream |
|-------------|----------|
| `/uniswap/` | `https://api.uniswap.org` |
| `/liquidity/` | `https://liquidity.backend-prod.api.uniswap.org` |
| `/gateway/` | `https://interface.gateway.uniswap.org` |
| `/conversion/` | `https://entry-gateway.backend-prod.api.uniswap.org` |
In-memory SHA256-keyed cache with tiered TTLs (5s quotes, 30s pools, 5min tokens, 10s GraphQL).
### Exchange SDK (TypeScript, npm)
Order matching engine library (`@hanzo/[email protected]`):
- **Book**: AVL tree + Fibonacci heap order book with bid/ask heaps
- **Order**: Limit/market orders with side (ASK/BID) and status tracking
- **Trade**: Fill execution with matched/rejected/new orders
- **Candle**: OHLCV candlestick aggregation (1min, 1hr, 1day, 1week)
- **Servers**: HTTP REST + Socket.IO WebSocket transports
## File Structure
```
exchange-api/
src/
index.ts -- Express server, routes, CORS
graphql.ts -- GraphQL handler, operation dispatch, Uniswap proxy
blockscout.ts -- Blockscout + subgraph V2/V3 data fetchers
cache.ts -- In-memory TTL cache with periodic cleanup
lux-tokens.ts -- 20 known LUX mainnet tokens with metadata
Dockerfile -- Multi-stage Node 22 alpine
package.json -- @luxfi/[email protected]
tsconfig.json -- ES2022, strict, commonjs
exchange-proxy/
main.go -- HTTP server with graceful shutdown
proxy/proxy.go -- CORS proxy, in-memory cache, upstream routing
go.mod -- github.com/luxfi/exchange-proxy, Go 1.23
exchange-sdk/
src/
Book.ts -- Order book (AVL + Fibonacci heap)
Order.ts -- Order model (limit/market, bid/ask)
Trade.ts -- Trade execution model
Candle.ts -- OHLCV candlestick builder
server.ts -- Demo server with random order generation
servers/ -- HTTP and Socket.IO transports
clients/ -- Client implementations
package.json -- @hanzo/[email protected]
```
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| Token price shows $0 | Subgraph has no `derivedETH` | Add token to `LUX_TOKENS` with fallback data |
| Insane price display | Subgraph decimal overflow | Already capped at 1e12, check raw subgraph |
| CORS errors from frontend | Missing exchange-proxy | Deploy exchange-proxy alongside exchange-api |
| Uniswap proxy 502 | Upstream rate limit | Increase cache TTL or add API key |
| V3 data missing | V3 subgraph not synced | Falls back to V2 data automatically |
## Related Skills
- `lux/lux-exchange.md` -- DEX frontend and omnichain routing
- `lux/lux-dex.md` -- CLOB engine (high-performance order matching)
- `lux/lux-evm.md` -- EVM with DEX precompiles
@@ -0,0 +1,185 @@
---
title: "Lux FHE Coprocessor"
description: "The coprocessor is a Go service with three binaries: gateway (blockchain event listener), scheduler (worker coordination), and tfhe-worker (FHE compute). Jobs flow through a Redis queue. Ciphertext..."
---
## Overview
The FHE Coprocessor (`github.com/luxfi/fhe-coprocessor`) is a distributed computation engine that processes Fully Homomorphic Encryption operations off-chain on behalf of the Lux blockchain. It listens for FHE events emitted by on-chain smart contracts (via the FHEVM precompiles), executes TFHE operations on encrypted ciphertexts using worker pools, and submits results back to the chain.
The coprocessor is a Go service with three binaries: **gateway** (blockchain event listener), **scheduler** (worker coordination), and **tfhe-worker** (FHE compute). Jobs flow through a Redis queue. Ciphertext blobs are stored on disk (content-addressed by SHA-256 hash) with optional Redis or in-memory caching. The actual FHE computation is performed via `github.com/luxfi/fhe`, which provides TFHE operations (add, sub, mul, comparison, bitwise, shifts, rotations, min/max, negation) on serialized ciphertexts.
This is the Go-native coprocessor. The Rust-based coprocessor lives in `github.com/luxfi/fhevm/coprocessor/` (see `lux/lux-fhevm.md`). Both serve the same architectural role -- offloading FHE computation from the EVM -- but target different deployment contexts.
## When to use
- Deploying FHE compute infrastructure for Lux EVM chains
- Scaling TFHE computation horizontally across multiple workers
- Building or modifying the blockchain-to-FHE event pipeline
- Setting up a local FHE development environment with Docker Compose
- Integrating a new queue backend (NATS JetStream is already implemented alongside Redis)
- Adding new FHE operation types to the worker execution pipeline
## Hard requirements
1. Go 1.26.1
2. `github.com/luxfi/fhe` v1.7.5 -- the TFHE library. NEVER use Zama tfhe-rs or any other FHE library for the Go coprocessor.
3. `github.com/luxfi/lattice/v7` v7.0.0 (indirect, via luxfi/fhe) -- NEVER use raw Lattigo.
4. Redis 7+ for the job queue and optional ciphertext storage.
5. NATS (optional) -- the queue package implements both Redis and NATS JetStream backends.
6. A TFHE server key file for real computation. Without `-server-key`, the worker starts with a mock key (development only).
## Quick reference
| Item | Value |
|------|-------|
| Module | `github.com/luxfi/fhe-coprocessor` |
| Go version | 1.26.1 |
| Binaries | `tfhe-worker`, `gateway`, `scheduler` |
| Gateway port | 8080 |
| Scheduler port | 8081 |
| Worker metrics port | 9090 |
| Queue backends | Redis (primary), NATS JetStream |
| Storage backends | File (default), Redis, Memory, Cached (memory + any backend) |
| Key dependency | `github.com/luxfi/fhe` v1.7.5 |
## Core Concepts
### Architecture
```
Blockchain Events
|
v
+---------+ Redis/NATS +-----------+
| Gateway | -------------> | Scheduler |
| (:8080) | job queue | (:8081) |
+---------+ +-----------+
| |
| ciphertext | worker registration
| storage | heartbeat / cleanup
v |
+---------+ +--------+--------+
| File/ | | Workers |
| Redis |<------------>| (tfhe-worker) |
| Storage | load/store | N goroutines |
+---------+ ciphertexts | per process |
+-----------------+
|
github.com/luxfi/fhe
(TFHE operations)
```
### Three Binaries
**gateway** -- Listens to blockchain events (FHE compute, store, decrypt). Converts events to jobs and pushes them onto the Redis queue. Exposes `/health`, `/status` (last processed block), and `/job/{id}` HTTP endpoints. Uses a `BlockchainClient` interface (currently a mock; real implementation connects to Lux EVM RPC).
**scheduler** -- Coordinates workers via HTTP registration (`POST /workers`), heartbeat (`POST /heartbeat`), and stale cleanup (configurable period/timeout). Provides `/submit` for manual job submission and `/status` for cluster capacity. Does not perform FHE computation itself.
**tfhe-worker** -- Runs a pool of N goroutines (default 4) that pop jobs from Redis, load ciphertext operands from storage, execute the TFHE operation via `pkg/fhe.Adapter`, store the result, and update job status. Exposes Prometheus-format metrics at `/metrics`.
### FHE Operations (20 opcodes)
| Code | Op | Type | Description |
|------|----|------|-------------|
| 0 | Add | Binary | Addition |
| 1 | Sub | Binary | Subtraction |
| 2 | Mul | Binary | Multiplication |
| 3 | Not | Unary | Bitwise NOT |
| 4 | And | Binary | Bitwise AND |
| 5 | Or | Binary | Bitwise OR |
| 6 | Xor | Binary | Bitwise XOR |
| 7 | Eq | Binary | Equality |
| 8 | Ne | Binary | Not equal |
| 9 | Gt | Binary | Greater than |
| 10 | Ge | Binary | Greater or equal |
| 11 | Lt | Binary | Less than |
| 12 | Le | Binary | Less or equal |
| 13 | Shl | Binary | Shift left |
| 14 | Shr | Binary | Shift right |
| 15 | Rotl | Binary | Rotate left |
| 16 | Rotr | Binary | Rotate right |
| 17 | Min | Binary | Minimum |
| 18 | Max | Binary | Maximum |
| 19 | Neg | Unary | Negation |
### Job Lifecycle
1. Gateway receives blockchain event (EventCompute, EventStore, or EventDecrypt)
2. For compute events: creates a `Job` with operation code and ciphertext handles, pushes to Redis list `fhe:queue:<name>`
3. Worker pops job via `BRPOP`, updates status to `StatusProcessing`
4. Worker loads LHS (and optionally RHS) ciphertext from file storage
5. Worker calls `fhe.Adapter.Execute()` which deserializes ciphertexts, runs the TFHE operation via `github.com/luxfi/fhe`, and serializes the result
6. Worker stores result ciphertext, updates job to `StatusCompleted` with result handle
7. On failure: job status becomes `StatusFailed` with error message
### Storage Layer
Ciphertexts are content-addressed (SHA-256 hash of data = handle). Four implementations:
- **FileStorage** (default): Sharded directories (first 2 chars of hash). Atomic writes via temp file + rename. Path: `<base>/<hash[:2]>/<hash>`.
- **MemoryStorage**: In-memory map with capacity limit (MB). Used for testing and as cache layer.
- **RedisStorage**: Redis-backed with configurable TTL (default 24h). Key prefix `fhe:ct:`.
- **CachedStorage**: Wraps any backend with MemoryStorage as read-through/write-through cache.
### Queue Layer
The `Queue` interface supports Push, Pop (blocking), Update, Get, and Close. Two implementations:
- **RedisQueue**: Uses Redis lists (`LPUSH`/`BRPOP`) for job ordering and `SET` with 24h TTL for job metadata. Pipeline-based Push for atomicity.
- **NATSQueue**: NATS JetStream with `WorkQueuePolicy` retention. Subject: `fhe.jobs.*`. Pull-based consumption with explicit Ack/Nak. Note: `Get()` is not implemented (use separate storage layer).
### Docker Deployment
```bash
# Start all services (gateway, scheduler, 2 workers, Redis)
docker compose up -d
# Scale workers
docker compose up -d --scale worker=4
# With monitoring (Prometheus + Grafana)
docker compose --profile monitoring up -d
# Build images
make docker
```
The Dockerfile uses multi-stage builds with separate targets per binary (`worker`, `gateway`, `scheduler`). Each runs as non-root user `fhe`.
### Local Development
```bash
make build # Build all 3 binaries to bin/
make test # go test -v -race ./...
make test-coverage # Coverage report to coverage.html
make fmt # gofumpt
make lint # golangci-lint
make dev # Start Redis + all 3 services locally
make stop # Kill all local services
```
## Troubleshooting
**"no server key provided, using mock key"** -- The worker starts without real FHE computation capability. Pass `-server-key /path/to/server.key` with a valid TFHE server key generated by `github.com/luxfi/fhe`.
**Redis connection refused** -- Ensure Redis is running on the configured address. Default is `localhost:6379`. Use `-redis` flag or start via `docker run -d -p 6379:6379 redis:alpine`.
**Worker not processing jobs** -- Check that gateway and worker use the same `-queue` name (default: `default`). Verify Redis connectivity from both processes. Check scheduler `/workers` endpoint to confirm worker registration.
**Ciphertext not found errors** -- Gateway and workers must share the same storage path (`-storage`). In Docker Compose, the `ciphertext-storage` volume is shared between gateway and worker containers.
**NATS queue Get() returns error** -- The NATS implementation does not support `Get()` by design. Use Redis queue or add a separate KV store for job lookup.
**docker-compose.yml naming** -- The repo still uses `docker-compose.yml` filename. For new deployments, rename to `compose.yml` per project conventions.
## Related Skills
- `lux/lux-fhe.md` -- Core Go FHE library (`github.com/luxfi/fhe`) that powers the worker's TFHE operations
- `lux/lux-fhevm.md` -- Rust-based FHEVM coprocessor (alternative architecture using Zama tfhe crate)
- `lux/lux-evm.md` -- EVM with FHE precompiles that emit the events this coprocessor consumes
- `lux/lux-lattice.md` -- Underlying lattice cryptography primitives
---
Last Updated: 2026-03-13
@@ -0,0 +1,193 @@
---
title: "Lux Genesis"
description: "## When to use"
---
## Overview
`github.com/luxfi/genesis` is a Go module that provides canonical genesis configurations for all Lux blockchain networks. It contains embedded genesis JSON for mainnet, testnet, devnet, and local/custom networks, along with CLI tools for generating, extracting, and validating genesis state. The package supports the primary C-Chain (EVM), P-Chain (Platform), X-Chain (Exchange), and subnet chains (Zoo, SPC, Hanzo, Pars).
## When to use
- Bootstrapping a new Lux node with correct genesis state
- Generating custom genesis configurations for local or private networks
- Extracting or migrating genesis data between network environments
- Validating node keys against expected genesis allocations
- Looking up chain IDs, network IDs, or treasury addresses for any Lux network
- Building tooling that needs to programmatically load genesis JSON by network ID
## Hard requirements
- Go 1.26.1+
- Module path: `github.com/luxfi/genesis`
- All dependencies use `github.com/luxfi/*` packages exclusively -- never `go-ethereum` or `ava-labs`
- Go package versions stay at v1.x.x (no v2+ bumps)
- Never expose or commit private keys, mnemonics, or EWOQ keys
- Treasury address `0x9011E888251AB053B7bD1cdB598Db4f9DEd94714` is canonical across all networks
## Quick reference
### Build and test
```bash
cd ~/work/lux/genesis
make build # Produces bin/genesis
make test # go test -v -race ./...
make test-coverage
make lint # golangci-lint
make tidy # go mod tidy
```
### CLI usage
```bash
# Generate local genesis from keys
genesis -network local -output primary.json
# Generate mainnet genesis preserving original C-Chain
genesis -network mainnet -cchain mainnet/cchain.json -output primary.json
# Generate from BIP39 mnemonic with 5 validators
LUX_MNEMONIC="your mnemonic" genesis -validators 5 -output primary.json
# Load from existing genesis directory
genesis -genesis-dir ./mainnet -output primary.json
```
### CLI flags
| Flag | Default | Description |
|------|---------|-------------|
| `-network` | `local` | Network: mainnet, testnet, local, custom |
| `-network-id` | 0 | Explicit network ID (overrides -network) |
| `-keys-dir` | `~/.lux/keys` | Directory containing node keys |
| `-genesis-dir` | - | Directory containing genesis component files |
| `-output` | stdout | Output file path |
| `-allocation` | 1000000000 | Allocation per validator in nLUX |
| `-validators` | 5 | Number of validators for mnemonic-based genesis |
| `-cchain` | - | Path to existing C-Chain genesis to preserve |
| `-format` | `pretty` | Output format: json, pretty |
### Environment variables
| Variable | Purpose |
|----------|---------|
| `LUX_KEYS_DIR` | Directory containing node keys |
| `LUX_MNEMONIC` | BIP39 mnemonic for key derivation |
| `LUX_PRIVATE_KEY` | Single private key (hex) |
| `LUX_GENESIS_DIR` | Directory containing genesis component files |
| `LUX_NETWORK_ID` | Network ID |
| `LUX_PCHAIN_ALLOCS` | JSON string of P-Chain allocations |
| `LUX_PCHAIN_ALLOCS_FILE` | Path to P-Chain allocations JSON file |
## Core Concepts
### Network IDs and Chain IDs
| Network | Network ID | C-Chain ID | Purpose |
|---------|------------|------------|---------|
| Mainnet | 1 | 96369 | Production network |
| Testnet | 2 | 96368 | Public test network |
| Devnet | 3 | 96370 | Fast iteration dev network |
| Local/Custom | 1337 | 1337 | Local development |
### Subnet chains
| Chain | Mainnet ID | Testnet ID | Devnet ID | Treasury |
|-------|-----------|-----------|-----------|----------|
| Zoo | 200200 | 200201 | 200202 | 2T ZOO |
| SPC | 36911 | 36910 | 36912 | 1B SPC |
| Hanzo | 36963 | 36962 | 36964 | 1B AI |
| Pars | 494949 | 494950 | 494951 | 2T PARS |
### Directory structure
```
genesis/
cmd/
genesis/ # Main genesis generator CLI
checkkeys/ # Validate node keys against genesis
extract-genesis/ # Extract genesis from running node
find-validators/ # Find validators in genesis state
regenerate-genesis/ # Regenerate genesis from components
pkg/genesis/ # Core library
allocations.go # Token allocation logic
blockchain.go # Blockchain genesis types
chain_mapping.go # Chain ID mapping
config.go # Genesis config builder
keys.go # Key derivation and loading
network.go # Network parameter resolution
params.go # Network parameters
types.go # Core types
validator_keys.go # Validator key management
configs/ # Embedded genesis JSON (22 network configs)
mainnet/ # C-Chain, P-Chain, network, bootstrappers
testnet/ # C-Chain, genesis
devnet/ # C-Chain
local/ # C-Chain, genesis
zoo-mainnet/ # Zoo subnet genesis
spc-mainnet/ # SPC subnet genesis
hanzo-mainnet/ # Hanzo subnet genesis
pars-mainnet/ # Pars subnet genesis
... # testnet/devnet variants for each subnet
builder/ # Genesis builder with tests
chains/ # Subnet chain definitions (ai, spc, zoo)
mainnet/ # Production genesis exports
testnet/ # Test network genesis exports
keys/mainnet/ # Mainnet key material
```
### Key dependencies
| Package | Version | Purpose |
|---------|---------|---------|
| `github.com/luxfi/constants` | v1.4.4 | Network ID constants |
| `github.com/luxfi/ids` | v1.2.9 | Node/chain identifiers |
| `github.com/luxfi/crypto` | v1.17.40 | Cryptographic primitives |
| `github.com/luxfi/vm` | v1.0.33 | Virtual machine types |
| `github.com/luxfi/utxo` | v0.2.4 | UTXO model types |
| `github.com/luxfi/go-bip32` | v1.0.2 | HD key derivation |
| `github.com/luxfi/go-bip39` | v1.1.2 | Mnemonic generation |
| `github.com/luxfi/tls` | v1.0.3 | TLS certificate handling |
### Dynamic P-Chain allocations
The `configs` package supports runtime P-Chain allocation overrides without modifying embedded genesis. Three methods, checked in order:
1. `LUX_PCHAIN_ALLOCS` environment variable (JSON string)
2. `LUX_PCHAIN_ALLOCS_FILE` environment variable (path to JSON file)
3. `~/.lux/genesis/{network}/pchain.json` file system override
C-Chain genesis remains embedded and immutable.
### Programmatic usage
```go
import "github.com/luxfi/genesis/configs"
// Load genesis for mainnet
genesisBytes, err := configs.GetGenesis(configs.MainnetID)
// Load with custom P-Chain allocations
allocs := []genesis.AllocationJSON{...}
genesisBytes, err := configs.GetGenesisWithAllocations(configs.MainnetID, allocs)
```
## Troubleshooting
- **"failed to build config"**: Ensure one of `LUX_KEYS_DIR`, `LUX_MNEMONIC`, or `LUX_PRIVATE_KEY` is set, or that `~/.lux/keys/` exists with valid key files.
- **Wrong chain ID after genesis**: Check whether `-network-id` was passed explicitly. If not, the network name resolves to the P-Chain network ID (1, 2, 3, 1337), not the C-Chain ID (96369, etc.).
- **Hanzo testnet chain ID mismatch**: Hanzo testnet was deployed with chain ID 36964 (devnet ID) due to a deployment error. The intended ID is 36962. Code and configs reflect the actual deployed value.
- **C-Chain genesis not matching**: Use `-cchain` flag to preserve the original C-Chain genesis from an existing file rather than generating a new one.
- **Module not found**: Run `go mod tidy` and ensure Go 1.26.1+ is installed.
## Related Skills
- `lux-staking` -- TLS certificate parsing and signature verification for node identity
- `lux-kms` -- Secret management for node keys and validator credentials
- `lux-node` -- The node implementation that consumes these genesis configs
- `lux-cli` -- CLI tool that uses genesis configs for network management
---
Last Updated: 2026-03-13
+244
View File
@@ -0,0 +1,244 @@
---
title: "Lux KMS"
description: "## When to use"
---
## Overview
`github.com/luxfi/kms` is an open-source, enterprise-grade Key Management Service providing centralized secret management, cryptographic key operations, PKI (Private Certificate Authority), SSH certificate signing, and HSM integration. The platform consists of a Node.js/Fastify backend, a React/TypeScript frontend, a Go-based Kubernetes operator, and Helm charts for deployment. It is the secret management backbone for the Lux ecosystem, deployed at `kms.hanzo.ai` (Hanzo's hosted instance) and self-hostable via Docker or Kubernetes.
## When to use
- Managing application secrets (API keys, database credentials, tokens) across environments
- Encrypting and decrypting data with AES-128/256-GCM or RSA-2048/4096
- Signing and verifying data with RSA-PSS, RSA-PKCS1v15, or ECDSA
- Generating and rotating cryptographic keys (symmetric and asymmetric)
- Issuing and managing X.509 certificates via internal PKI
- Issuing signed SSH certificates for infrastructure access
- Syncing secrets to external platforms (GitHub Actions, Vercel, AWS, Terraform, Ansible)
- Deploying a Kubernetes operator to inject secrets into workloads
- Scanning codebases for leaked secrets (140+ secret types)
## Hard requirements
- Node.js 20+ for backend and frontend development
- Go 1.26+ for the Kubernetes operator (`k8-operator/`)
- PostgreSQL for primary storage, Redis for caching
- Docker and Docker Compose for local development and production deployment
- Root encryption key must be configured via `ENCRYPTION_KEY` environment variable or HSM
- All secrets encrypted at rest with AES-GCM; TLS 1.3 for transit
- RBAC enforced on all key and secret operations
- Audit logging enabled for all cryptographic operations
## Quick reference
### Local development
```bash
cd ~/work/lux/kms
# Full stack via Docker Compose
cp .env.example .env
docker compose -f docker-compose.dev.yml up
# Backend only
cd backend && npm install && npm run dev
# Frontend only
cd frontend && npm install && npm run dev
# Run tests
npm test
npm run test:e2e
```
### Production deployment
```bash
# Docker standalone
docker build -f Dockerfile.standalone-kms -t lux-kms .
docker compose -f docker-compose.prod.yml up -d
# Kubernetes via Helm
helm install lux-kms ./helm-charts/kms-standalone-postgres
# Kubernetes operator
cd k8-operator && make deploy
```
### Secret scanning
```bash
# Scan git history for leaked secrets
kms scan --verbose
# Install pre-commit hook
kms scan install --pre-commit-hook
```
### Database migrations
```bash
cd backend
npm run migration:create -- create_new_table
npm run migration:up
npm run migration:down
```
## Core Concepts
### Architecture
```
Frontend (React 18 / Vite / TypeScript)
|
Backend API (Node.js / Fastify / TypeScript)
|
+-- KMS Service Layer (key generation, crypto ops, access control, audit)
|
+-- PostgreSQL (primary storage) + Redis (cache)
|
+-- HSM Provider (PKCS#11: AWS CloudHSM, Thales Luna, SoftHSM)
```
### Key hierarchy
1. **Root Key** -- Protected by HSM or `ENCRYPTION_KEY` env var
2. **Organization KEK** -- Key Encryption Key per organization
3. **Project KEK** -- Project-specific encryption key
4. **Data Keys** -- Actual encryption/signing keys for application use
### Supported algorithms
| Category | Algorithms |
|----------|-----------|
| Symmetric encryption | AES-128-GCM, AES-256-GCM |
| Asymmetric encryption | RSA-2048, RSA-4096, ECC-P256, ECC-P384 |
| Signing | RSA-PSS (SHA-256/384/512), RSA-PKCS1v15, ECDSA |
| Key derivation | PBKDF2, Argon2 |
| FHE (planned) | TFHE-Binary, TFHE-Integer |
### API endpoints
| Method | Path | Purpose |
|--------|------|---------|
| POST | `/api/v1/kms/keys` | Create new key |
| GET | `/api/v1/kms/keys` | List keys |
| GET | `/api/v1/kms/keys/{keyId}` | Get key metadata |
| PATCH | `/api/v1/kms/keys/{keyId}` | Update key |
| DELETE | `/api/v1/kms/keys/{keyId}` | Delete key |
| POST | `/api/v1/kms/keys/{keyId}/encrypt` | Encrypt data |
| POST | `/api/v1/kms/keys/{keyId}/decrypt` | Decrypt data |
| POST | `/api/v1/kms/keys/{keyId}/sign` | Sign data |
| POST | `/api/v1/kms/keys/{keyId}/verify` | Verify signature |
| POST | `/api/v1/kms/keys/{keyId}/generate-data-key` | Generate data encryption key |
| POST | `/api/v1/kmip` | KMIP 2.0+ protocol endpoint |
### Directory structure
```
kms/
backend/
src/
services/
kms/ # Core KMS service
kms-service.ts # Main key operation logic (40K lines)
kms-types.ts # Type definitions
kms-fns.ts # Utility functions
kms-key-dal.ts # Database access layer for keys
kms-root-config-dal.ts # Root key configuration
internal-kms-dal.ts # Internal KMS operations
auth/ # Authentication service
certificate/ # X.509 certificate management
certificate-authority/ # Private CA
identity/ # Machine identity management
identity-aws-auth/ # AWS IAM auth
identity-gcp-auth/ # GCP auth
identity-azure-auth/ # Azure auth
identity-kubernetes-auth/ # K8s auth
cmek/ # Customer-managed encryption keys
... # 70+ service directories
frontend/ # React 18 / Vite / Tailwind / Radix UI / i18next (12 languages)
k8-operator/ # Go-based Kubernetes operator
go.mod # github.com/luxfi/kms/k8-operator (Go 1.26)
main.go # Operator entrypoint
controllers/ # Reconciliation controllers
api/ # CRD API types
internal/ # Internal operator logic
helm-charts/
kms-standalone-postgres/ # Full KMS + PostgreSQL chart
kms-gateway/ # Gateway-only chart
secrets-operator/ # Secrets operator chart
docs/ # Mintlify documentation (MDX)
docker-swarm/ # Swarm deployment configs
examples/ # Integration examples
sink/ # Event sink for audit log streaming
```
### Kubernetes operator
The K8s operator (`k8-operator/`) is a Go application using controller-runtime. Key dependencies:
| Package | Version | Purpose |
|---------|---------|---------|
| `github.com/luxfi/kms-go` | v0.5.98 | KMS Go SDK |
| `sigs.k8s.io/controller-runtime` | v0.14.4 | K8s controller framework |
| `k8s.io/api` | v0.26.1 | K8s API types |
| `k8s.io/client-go` | v0.26.1 | K8s client |
It reconciles `KMSSecret` CRDs to sync secrets from the KMS platform into Kubernetes Secrets, with automatic reloading of deployments when secrets change.
### Authentication methods
- Universal Auth (API key + secret)
- Kubernetes Service Account Auth
- AWS IAM Auth
- GCP IAM Auth
- Azure AD Auth
- OIDC Auth
- JWT Auth
- LDAP Auth
- Alibaba Cloud Auth
### Environment variables (core)
| Variable | Required | Purpose |
|----------|----------|---------|
| `ENCRYPTION_KEY` | Yes | 16-byte hex key for platform encryption |
| `AUTH_SECRET` | Yes | JWT signing secret |
| `DB_CONNECTION_URI` | Yes | PostgreSQL connection string |
| `REDIS_URL` | Yes | Redis connection string |
| `SITE_URL` | Yes | Base URL for the KMS frontend |
| `SMTP_HOST` | No | Mail server for notifications |
| `HSM_ENABLED` | No | Enable PKCS#11 HSM backend |
| `HSM_LIBRARY_PATH` | No | Path to PKCS#11 shared library |
| `HSM_SLOT` | No | HSM slot number |
| `HSM_PIN` | No | HSM PIN/password |
### Monitoring
| Endpoint | Purpose |
|----------|---------|
| `/health` | Health check |
| `/readiness` | Readiness probe |
| `/metrics` | Prometheus metrics |
## Troubleshooting
- **HSM connection failed**: Verify `HSM_LIBRARY_PATH` points to the correct PKCS#11 `.so` file, check slot number with `softhsm2-util --show-slots`, and confirm PIN is correct.
- **Key generation failed**: Check that the root encryption key is configured (`ENCRYPTION_KEY` env var) and that PostgreSQL is reachable.
- **"KMSSecret not syncing"**: Ensure the K8s operator is running, the `KMSSecret` CRD references valid project/environment paths, and the operator's Universal Auth credentials are active.
- **Frontend won't start**: Run `npm install` in the `frontend/` directory. Requires Node.js 20+. Check that `SITE_URL` is set correctly.
- **Database migration errors**: Run `npm run migration:up` from the `backend/` directory. For fresh installs, ensure the PostgreSQL database exists and `DB_CONNECTION_URI` is correct.
- **Port conflicts**: Default ports are 8080 (backend API) and 3000 (frontend dev). Check with `lsof -i :8080`.
- **Docker build fails**: Use `Dockerfile.standalone-kms` for production. Ensure `--platform linux/amd64` for K8s deployments.
## Related Skills
- `lux-genesis` -- Genesis configurations that reference KMS for secure key storage
- `lux-staking` -- Staking certificates whose private keys should be stored in KMS
- `lux-node` -- Node deployments that pull secrets from KMS via the K8s operator
---
Last Updated: 2026-03-13
@@ -0,0 +1,195 @@
---
title: "Lux Ledger - Hardware Wallet Integration"
description: "## When to use"
---
## Overview
Lux Ledger (`github.com/luxfi/ledger`) is a **Go client library** for interacting with the Lux app on Ledger hardware wallets (Nano S, Nano X, Nano S Plus, Stax, Flex). It implements the APDU protocol over USB HID for key derivation (BIP32/BIP44), address generation, transaction signing, hash signing, and multi-signature support. The repository also contains the **Rust source for the Ledger app itself** in `/rust/`.
## When to use
- Integrating Ledger hardware wallet signing into Go applications
- Implementing the `keychain.Ledger` interface for wallet/signing flows
- Building or modifying the Ledger app firmware (Rust, in `/rust/`)
- Debugging Ledger device communication issues (APDU protocol)
- Adding new Ledger device model support
## Hard requirements
1. ALWAYS use `github.com/luxfi/*` packages -- NEVER `go-ethereum` or `ava-labs`
2. NEVER bump Go package versions above v1.x.x
3. Tests require a **physical Ledger device** with the Lux app installed and unlocked
4. HRP (human-readable part) must be <= 83 characters
5. Build tags: `ledger_mock` and `ledger_zemu` exclude real HID transport
## Quick reference
| Item | Value |
|------|-------|
| Module | `github.com/luxfi/ledger` |
| Go | 1.26.1 |
| License | Apache 2.0 |
| Package Type | Library (no binary) |
| Default HRP | `lux` |
| Derivation Path | `m/44'/60'/0'` (Ethereum-compatible) |
| Default Chain ID | `2q9e4r6Mu3U68nU1fYjgbR6JvwrRx36CohpAX5UQxse55x1Q5` |
| Ledger App CLA | `0x80` |
## Core Concepts
### Entry Points
```go
// Recommended: implements keychain.Ledger interface
ledger, err := ledger.NewLedger()
defer ledger.Disconnect()
// Or directly
app, err := ledger.FindLedgerLuxApp()
defer app.Close()
```
### keychain.Ledger Interface
The library implements the `keychain.Ledger` interface for seamless wallet integration:
| Method | Signature | Purpose |
|--------|-----------|---------|
| `Address` | `(hrp string, index uint32) (ids.ShortID, error)` | Get address at derivation index |
| `GetAddresses` | `(indices []uint32) ([]ids.ShortID, error)` | Get multiple addresses |
| `Sign` | `(message []byte, index uint32) ([]byte, error)` | Sign raw message |
| `SignHash` | `(hash []byte, index uint32) ([]byte, error)` | Sign pre-hashed data (32 bytes) |
| `SignTransaction` | `(hash []byte, indices []uint32) ([][]byte, error)` | Multi-address transaction signing |
| `Disconnect` | `() error` | Close device connection |
### Low-Level API
| Method | Purpose |
|--------|---------|
| `GetVersion()` | Get app version (major.minor.patch) |
| `GetPubKey(path, show, hrp, chainID)` | Get public key, hash, and bech32 address |
| `SignFull(pathPrefix, signerPaths, message, changePaths)` | Sign full transaction with multiple paths |
| `SignHashFull(pathPrefix, signerPaths, hash)` | Sign a 32-byte hash with multiple paths |
| `VerifyMultipleSignatures(resp, hash, rootPath, paths, hrp, chainID)` | Verify all signatures from a multi-sign |
### APDU Instructions
| Instruction | Code | Purpose |
|-------------|------|---------|
| `INS_GET_VERSION` | `0x00` | Get app version |
| `INS_WALLET_ID` | `0x01` | Get wallet identifier |
| `INS_GET_ADDR` | `0x02` | Get address/pubkey |
| `INS_GET_EXTENDED_PUBLIC_KEY` | `0x03` | Get extended public key |
| `INS_SIGN_HASH` | `0x04` | Sign a hash |
| `INS_SIGN` | `0x05` | Sign a transaction |
| `INS_SIGN_MSG` | `0x06` | Sign a message |
### Supported Devices
| Device | Product ID (high byte) | Interface |
|--------|----------------------|-----------|
| Ledger Nano S | `0x10` | 0 |
| Ledger Nano X | `0x40` | 0 |
| Ledger Nano S Plus | `0x50` | 0 |
| Ledger Stax | `0x60` | 0 |
| Ledger Flex | `0x70` | 0 |
All use USB HID with vendor ID `0x2c97`, channel `0x0101`, packet size 64 bytes.
### Signature Verification
Uses `btcec/v2` secp256k1 ECDSA verification with malleable signature rejection:
```go
// 64-byte [R || S] format
ok := ledger.VerifySignature(pubkey, hash, signature)
```
### Key Dependencies
```
github.com/luxfi/[email protected] -- USB HID device access
github.com/luxfi/[email protected] -- Lux ID types (ShortID)
github.com/luxfi/[email protected] -- Cryptographic primitives
github.com/btcsuite/btcd/btcec/v2 -- secp256k1 ECDSA
github.com/mr-tron/base58 -- Base58 encoding (chain IDs)
go.uber.org/zap -- Structured logging
github.com/stretchr/testify -- Test assertions
```
## File Structure
```
ledger/
lux.go -- Main API: FindLedgerLuxApp, keychain.Ledger impl
types.go -- LedgerLux, VersionInfo, ResponseSign, ResponseAddr
common.go -- Path/HRP/ChainID serialization, signature helpers
hid.go -- USB HID transport (LedgerAdmin, LedgerDevice)
apdu.go -- APDU command wrapping/unwrapping
logger.go -- zap-based logger (LEDGER_LOG_LEVEL env)
lux_test.go -- Integration tests (require physical device)
common_test.go -- Unit tests (path serialization, version check)
rust/ -- Ledger app source (Rust workspace)
app/ -- Main Ledger app crate
app-derive/ -- Derive macros
Cargo.toml -- Workspace config (resolver v2)
deps/ -- Build dependencies
hfuzz/ -- Fuzz testing
zemu/ -- Zemu device emulator tests
docs/ -- Documentation
go-docs/ -- Go-specific documentation
```
### Rust Ledger App
The `/rust/` directory contains the Ledger app firmware:
- Workspace with `app` and `app-derive` crates
- Build profiles: dev (abort on panic), release (size-optimized, `opt-level = "z"`)
- Testing via Zemu emulator and Miri
- Fuzz testing via `cargo-hfuzz`
- Build: `make` (requires BOLOS SDK or Docker via `deps/dockerized_build.mk`)
## Build and Test
```bash
# Go library (unit tests, no device needed)
go test -short ./...
# Go library (full tests, requires Ledger device)
go test -v ./...
# Rust Ledger app
cd rust && cargo build
cd rust && cargo test
# Zemu emulator tests
cd rust/zemu && yarn test
# Fuzz testing
cd rust/hfuzz && cargo hfuzz run apdu
```
### Logging
Set `LEDGER_LOG_LEVEL` environment variable: `debug`, `info` (default), `warn`, `error`.
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| `device not found` | No Ledger connected or locked | Connect device, unlock, open Lux app |
| `CLA not supported` | Lux app not open on device | Open the Lux app on the Ledger |
| `APDU commands < 5` | Malformed command | Check message construction |
| `timeout reading from device` | Device busy or unresponsive | Disconnect and reconnect, 20s timeout |
| `wrong hash size` | Hash not 32 bytes | Pass SHA-256 digest (32 bytes) |
| `invalid path` | Bad BIP32 path format | Use format `m/44'/60'/0'/0/0` |
| `read channel closed` | USB connection dropped | Reconnect device |
| `Version required error` | App version too old | Update Ledger app firmware |
## Related Skills
- `lux/lux-wallet.md` -- Wallet implementation (uses ledger as signing backend)
- `lux/lux-hsm.md` -- HSM integration (software signing alternative)
- `lux/lux-crypto.md` -- Cryptographic primitives
+228
View File
@@ -0,0 +1,228 @@
---
title: "Lux Plugin Manager (LPM)"
description: "Status: Alpha."
---
## Overview
LPM is a command-line tool for managing virtual machine binaries and subnet definitions for the Lux Network. It tracks plugin repositories (git-based), downloads and builds VM binaries, installs them to the Lux node plugin directory, and can join subnets by installing all required VMs. The default plugin source is `luxfi/plugins-core`, but users can add custom repositories.
**Status**: Alpha.
## When to use
- Installing VM plugins for the Lux node
- Joining subnets that require specific VM binaries
- Managing custom plugin repositories for VM distribution
- Upgrading installed VM binaries to latest versions
- Building and linking local VM source code to the plugin directory
## Hard requirements
1. ALWAYS use `github.com/luxfi/*` packages -- NEVER `go-ethereum` or `ava-labs`
2. Go packages stay at v1.x.x (module is `github.com/luxfi/lpm`, no version suffix)
3. NEVER use EWOQ keys
4. Requires a running Lux node for VM hot-reload via admin API (optional; works offline too)
## Quick reference
| Item | Value |
|------|-------|
| Repo | `github.com/luxfi/lpm` |
| Module | `github.com/luxfi/lpm` |
| Go version | 1.26.1 |
| Binary | `build/lpm` |
| Default plugin repo | `luxfi/plugins-core` (master branch) |
| Plugin dir | `~/.lux/plugins` |
| LPM state dir | `~/.lpm` |
| License | See LICENSE |
## Core Concepts
### Directory structure
```
luxfi/lpm/
main/
main.go -- Entry point
cmd/
root.go -- Root cobra command, global flags, config init
install.go -- install-vm command
install_github.go -- install-github (from GitHub releases)
install_gitlab.go -- install-gitlab (from GitLab releases)
install_url.go -- install-url (from arbitrary URL)
install_source.go -- install-source (build from local source)
uninstall.go -- uninstall-vm command
join_subnet.go -- join-subnet command
link.go -- link command (symlink local binary)
update.go -- update command (fetch latest definitions)
upgrade.go -- upgrade command (rebuild/reinstall VMs)
add_repository.go -- add-repository command
remove_repository.go -- remove-repository command
list_repositories.go -- list-repositories command
lpm/
(core LPM logic)
workflow/
workflow.go -- Workflow orchestration
install.go -- Install workflow
installer.go -- Installer interface + impls
uninstall.go -- Uninstall workflow
upgrade_vm.go -- Upgrade workflow
add_repository.go -- Repository management
link.go -- Local binary linking
executor.go -- Command execution
config/ -- Configuration types
constant/
constants.go -- AppName, CoreAlias, CoreURL, CoreBranch
types/ -- Shared type definitions
state/ -- Persistent state management
engine/ -- Plugin engine
git/ -- Git operations (go-git)
url/ -- URL handling
checksum/ -- Checksum verification
admin/ -- Lux node admin API client
internal/
vmid/ -- VM ID utilities
pkg/ -- Shared utilities
util/ -- Helper functions
scripts/
build.sh -- Build script
lint.sh -- Linter script
```
### Key dependencies
```
github.com/luxfi/codec v1.1.3 -- Serialization
github.com/luxfi/ids v1.2.9 -- ID types (VM IDs, subnet IDs)
github.com/luxfi/sdk v1.16.46 -- Lux SDK (admin API client)
github.com/luxfi/crypto v1.17.40 -- Crypto primitives (indirect)
github.com/go-git/go-git v5.16.4 -- Git repository operations
github.com/spf13/cobra v1.10.2 -- CLI framework
github.com/spf13/viper v1.21.0 -- Configuration
github.com/spf13/afero v1.15.0 -- Filesystem abstraction (testable)
```
### Commands
```bash
# Repository management
lpm add-repository --alias luxfi/core --url https://github.com/luxfi/plugins-core.git --branch master
lpm remove-repository --alias luxfi/core
lpm list-repositories
lpm update # Fetch latest definitions from all repos
# VM installation
lpm install-vm --vm spacesvm # Install by alias (partial or fully qualified)
lpm install-vm --vm luxfi/core:spacesvm # Fully qualified (repo:vm)
lpm uninstall-vm --vm spacesvm # Remove VM binary from plugin dir
# Alternative install sources
lpm install-github --repo org/repo --release v1.0.0
lpm install-gitlab --repo org/repo --release v1.0.0
lpm install-url --url https://example.com/vm.tar.gz
lpm install-source --path /path/to/source
# Subnet operations
lpm join-subnet --subnet spaces # Install all VMs required by a subnet
# Upgrade
lpm upgrade # Upgrade all installed VMs
lpm upgrade --vm spacesvm # Upgrade specific VM
# Link local build
lpm link --vm myvm --path /path/to/binary
```
### Global flags
```
--config-file Path to LPM config file
--lpm-path LPM artifacts directory (default: ~/.lpm)
--plugin-path Lux plugin directory (default: ~/.lux/plugins)
--credentials-file Path to credentials file (for private repos)
--admin-api-endpoint Lux admin API (default: 127.0.0.1:9650/ext/admin)
```
### Credentials file format (for private repos)
```yaml
username: your-username
password: <github-personal-access-token>
```
### Build and test
```bash
# Build
make build # Output: build/lpm
./scripts/build.sh # Alternative
# Test
make test # go test -v -race -coverprofile=coverage.out ./...
make test-short # Short tests only
make coverage # HTML coverage report
# Lint
make lint # Runs scripts/lint.sh
# Release (multi-platform)
make release # linux/amd64, linux/arm64, darwin/amd64, darwin/arm64, windows/amd64
# Install
make install # Copy to $GOPATH/bin
# Docker
make docker # Build luxfi/lpm:VERSION image
# CI
make ci # deps, verify, check (fmt, lint, test)
```
### Install from release binary
```bash
curl -sSfL https://raw.githubusercontent.com/luxfi/pm/master/scripts/install.sh | sh -s
cd bin && export PATH=$PWD:$PATH
```
### Constants (constant/constants.go)
| Constant | Value |
|----------|-------|
| AppName | `lpm` |
| CoreAlias | `luxfi/plugins-core` |
| CoreURL | `https://github.com/luxfi/plugins-core.git` |
| CoreBranch | `master` |
| QualifiedNameDelimiter | `:` |
| AliasDelimiter | `/` |
| DefaultNetwork | `testnet` |
### VM naming
- Partial alias: `spacesvm` (searches all tracked repos)
- Fully qualified: `luxfi/core:spacesvm` (org/repo:vm)
- When multiple repos have the same VM alias, you must use the fully qualified name
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| `Node was offline` after install | Lux node not running | VMs will be loaded on next node start |
| Permission errors on Mac | Downloaded binary from GitHub UI | Use curl install script instead |
| Multiple matches for VM alias | Same VM name in multiple repos | Use fully qualified name: `repo:vm` |
| Private repo auth failure | Missing or invalid credentials | Use `--credentials-file` with valid PAT |
| Build script fails | Missing Go toolchain | Install Go 1.26+ |
| Plugin directory not found | Custom node install path | Set `--plugin-path` to correct location |
## Related Skills
- `lux/lux-node.md` -- Lux node (loads plugins from the plugin directory)
- `lux/lux-plugins.md` -- Plugin repository format and definitions
- `lux/lux-vm.md` -- Virtual machine interface and development
- `lux/lux-cli.md` -- Lux CLI (alternative node management tool)
- `lux/lux-sdk.md` -- Lux SDK (admin API used by LPM)
---
Last Updated: 2026-03-13
@@ -0,0 +1,148 @@
---
title: "Lux Staking"
description: "## When to use"
---
## Overview
`github.com/luxfi/staking` is a focused Go package that handles TLS certificate parsing, public key extraction, and signature verification for the Lux blockchain staking layer. It provides a minimal, performance-oriented alternative to Go's `crypto/x509` package, parsing only the fields needed for node identity validation: the public key and raw certificate bytes. Validators on the Lux network use TLS certificates as their identity, and this package is the foundation for verifying those identities.
## When to use
- Parsing staking TLS certificates from Lux validator nodes
- Verifying signatures from validators (RSA-PKCS1v15 or ECDSA with SHA-256)
- Loading TLS key pairs from PEM files or byte slices for node startup
- Validating RSA public key well-formedness (modulus size, exponent)
- Working with node identity at the certificate level without full x509 overhead
## Hard requirements
- Go 1.26.1+
- Module path: `github.com/luxfi/staking`
- All dependencies use `github.com/luxfi/*` packages exclusively -- never `go-ethereum` or `ava-labs`
- Go package versions stay at v1.x.x (no v2+ bumps)
- RSA keys must be 2048-bit or 4096-bit with public exponent 65537
- ECDSA keys must use P-256 curve
- Certificates must not exceed 2 KiB (`MaxCertificateLen = 2 * constants.KiB`)
## Quick reference
### Build and test
```bash
cd ~/work/lux/staking
go build ./...
go test -v ./...
go test -race ./...
```
### Package API
```go
import "github.com/luxfi/staking"
// Parse a raw ASN.1 certificate (extracts public key only)
cert, err := staking.ParseCertificate(rawBytes)
// Load TLS certificate from PEM files
tlsCert, err := staking.LoadTLSCertFromFiles(keyPath, certPath)
// Load TLS certificate from PEM byte slices
tlsCert, err := staking.LoadTLSCertFromBytes(keyBytes, certBytes)
// Verify a signature against a certificate
err := staking.CheckSignature(cert, message, signature)
// Validate RSA public key constraints
err := staking.ValidateRSAPublicKeyIsWellFormed(rsaPubKey)
```
## Core Concepts
### Certificate type
The `Certificate` type is an alias for `ids.Certificate` from `github.com/luxfi/ids`. It holds:
- `Raw []byte` -- the original ASN.1 DER-encoded certificate
- `PublicKey crypto.PublicKey` -- the extracted RSA or ECDSA public key
This is intentionally minimal. Unlike `x509.Certificate`, it does not parse validity periods, extensions, subject names, or issuer chains. Lux staking certificates are self-signed and identified solely by their public key.
### Supported algorithms
| Algorithm | Key Size | Use Case |
|-----------|----------|----------|
| RSA-PKCS1v15 + SHA-256 | 2048 or 4096 bit | Signature verification |
| ECDSA + SHA-256 | P-256 | Signature verification |
### RSA key validation rules
The `ValidateRSAPublicKeyIsWellFormed` function enforces:
1. Modulus must be positive
2. Modulus bit length must be exactly 2048 or 4096
3. Modulus must be odd (not even)
4. Public exponent must be exactly 65537
### ASN.1 parsing
The package defines OIDs for the two supported public key algorithms:
- RSA: `1.2.840.113549.1.1.1` (`oidPublicKeyRSA`)
- ECDSA: `1.2.840.10045.2.1` (`oidPublicKeyECDSA`)
Parsing uses `golang.org/x/crypto/cryptobyte` for efficient ASN.1 traversal without full x509 overhead.
### File structure
```
staking/
asn1.go # OID definitions, SHA-256 init check
certificate.go # Certificate type alias (ids.Certificate)
parse.go # ParseCertificate, parsePublicKey, ValidateRSAPublicKeyIsWellFormed
tls.go # LoadTLSCertFromBytes, LoadTLSCertFromFiles
verify.go # CheckSignature (RSA + ECDSA verification)
go.mod # Module definition
```
### Dependencies
| Package | Version | Purpose |
|---------|---------|---------|
| `github.com/luxfi/constants` | v1.4.2 | MaxCertificateLen size constant |
| `github.com/luxfi/ids` | v1.2.9 | Certificate type definition |
| `golang.org/x/crypto` | v0.47.0 | ASN.1 cryptobyte parsing |
### Error types
All errors are package-level variables with the `staking:` prefix:
| Error | Cause |
|-------|-------|
| `ErrCertificateTooLarge` | Certificate exceeds 2 KiB |
| `ErrMalformedCertificate` | Invalid ASN.1 SEQUENCE at top level |
| `ErrMalformedTBSCertificate` | Invalid TBS certificate structure |
| `ErrMalformedSPKI` | Invalid Subject Public Key Info |
| `ErrInvalidRSAPublicKey` | Nil RSA public key |
| `ErrUnsupportedRSAModulusBitLen` | RSA modulus not 2048 or 4096 bits |
| `ErrRSAModulusIsEven` | RSA modulus is even (invalid) |
| `ErrUnsupportedRSAPublicExponent` | RSA exponent is not 65537 |
| `ErrFailedUnmarshallingEllipticCurvePoint` | Invalid ECDSA P-256 point |
| `ErrUnknownPublicKeyAlgorithm` | Neither RSA nor ECDSA |
| `ErrUnsupportedAlgorithm` | CheckSignature: unknown key type |
| `ErrECDSAVerificationFailure` | ECDSA signature did not verify |
## Troubleshooting
- **"certificate length is greater than 2048"**: The raw DER certificate exceeds 2 KiB. Lux staking certificates should be minimal self-signed certs. Check if extra extensions or long subject names are inflating the size.
- **"unsupported RSA modulus bitlen"**: Only 2048-bit and 4096-bit RSA keys are accepted. Generate a new key pair with the correct size.
- **"unknown public key algorithm"**: The certificate uses an algorithm other than RSA or ECDSA P-256. Lux only supports these two for staking identity.
- **"ECDSA verification failure"**: The message or signature bytes do not match. Ensure the message is the exact bytes that were signed, not a hex-encoded or base64-encoded representation.
- **LoadTLSCertFromFiles argument order**: The function takes `(keyPath, certPath)`, not `(certPath, keyPath)`. This matches the convention where the private key comes first.
## Related Skills
- `lux-genesis` -- Genesis configuration that establishes initial validators and their certificates
- `lux-kms` -- Key management for generating and storing staking key pairs
- `lux-node` -- The node implementation that uses staking certificates for P2P identity
---
Last Updated: 2026-03-13
+206
View File
@@ -0,0 +1,206 @@
---
title: "Lux ZapDB - Embeddable Key-Value Store"
description: "ZapDB serves as the low-level storage engine across the Lux blockchain stack and is the backend for the Hanzo ORM's ZAP driver."
---
## Overview
ZapDB is a fork of BadgerDB (dgraph-io/badger) published as `github.com/luxfi/zapdb/v4`. It is an embeddable, persistent, high-performance key-value database written in pure Go. ZapDB uses an LSM tree with a separate value log (WiscKey design) to achieve high read and write throughput simultaneously, with concurrent ACID transactions using serializable snapshot isolation (SSI).
ZapDB serves as the low-level storage engine across the Lux blockchain stack and is the backend for the Hanzo ORM's ZAP driver.
## When to use
- Embedding a persistent key-value store in Go services on the Lux stack
- Storing blockchain state, transaction indices, or consensus metadata
- Building applications that need ACID transactions with SSI guarantees
- Needing an SSD-optimized pure Go database (no CGo required)
- Using the Hanzo ORM with the ZAP backend driver
## Hard requirements
1. ALWAYS import `github.com/luxfi/zapdb/v4` -- NEVER `github.com/dgraph-io/badger`
2. Go packages stay at v4.x.x; v4.0.0 and v4.3.0 are retracted
3. The Go package name is `badger` (import path ends with zapdb/v4, but package is `badger`)
4. Close the DB with `db.Close()` to prevent data corruption
5. All mutations must happen inside transactions (`db.Update` or `db.NewTransaction`)
## Quick reference
| Item | Value |
|------|-------|
| Repo | `github.com/luxfi/zapdb` |
| Module | `github.com/luxfi/zapdb/v4` |
| Go version | 1.26.1 |
| Package name | `badger` |
| License | Apache 2.0 |
| Retracted | v4.0.0, v4.3.0 |
| CLI binary | `badger` (in `badger/` subdir) |
## Core Concepts
### Directory structure
```
luxfi/zapdb/
db.go -- DB type: Open, Close, transaction management
txn.go -- Transaction: Get, Set, Delete, Commit, Discard
iterator.go -- Iterator: prefix scan, seek, key-value access
batch.go -- WriteBatch for bulk writes
stream.go -- Concurrent key-range streaming
merge.go -- MergeOperator for read-modify-write
backup.go -- Protobuf backup/restore
errors.go -- All exported error sentinels
options.go -- Options with builder pattern (WithX methods)
compaction.go -- LSM compaction logic
value.go -- Value log management
levels.go -- LSM level management
memtable.go -- In-memory table (skiplist-backed)
key_registry.go -- Encryption key rotation
options/ -- CompressionType, ChecksumVerificationMode
table/ -- SSTable implementation
skl/ -- Skiplist (memtable backend)
pb/ -- Protobuf definitions
fb/ -- FlatBuffers definitions
y/ -- Internal utilities (WaterMark, Slice)
trie/ -- Trie data structure
badger/ -- CLI tool
main.go -- CLI entry point (jemalloc, pprof)
cmd/ -- CLI subcommands
backup.go, restore.go, info.go, flatten.go,
bench.go, stream.go, rotate.go, bank.go
```
### Basic usage
```go
import badger "github.com/luxfi/zapdb/v4"
// Open database
opts := badger.DefaultOptions("/tmp/zapdb")
db, err := badger.Open(opts)
if err != nil {
return err
}
defer db.Close()
// Write
err = db.Update(func(txn *badger.Txn) error {
return txn.Set([]byte("key"), []byte("value"))
})
// Read
err = db.View(func(txn *badger.Txn) error {
item, err := txn.Get([]byte("key"))
if err != nil {
return err
}
return item.Value(func(val []byte) error {
fmt.Printf("value: %s\n", val)
return nil
})
})
```
### Options
```go
// Default: good for most use cases
opts := badger.DefaultOptions("/path/to/data")
// LSM-only mode (values collocated with keys, less disk usage)
opts := badger.LSMOnlyOptions("/path/to/data")
// In-memory mode (no persistence)
opts := badger.DefaultOptions("").WithInMemory(true)
// Common tuning
opts = opts.
WithSyncWrites(true). // Durability on hard reboot
WithCompression(options.ZSTD). // ZSTD compression
WithBlockCacheSize(512 << 20). // 512MB block cache
WithNumCompactors(4). // Compaction workers
WithEncryptionKey(key). // AES encryption (16/24/32 bytes)
WithReadOnly(true) // Read-only mode
```
### Key features
| Feature | Details |
|---------|---------|
| Design | LSM tree + value log (WiscKey paper) |
| Transactions | ACID with SSI, concurrent read-write |
| Compression | Snappy (default), ZSTD, None |
| Encryption | AES-128/192/256 with key rotation |
| TTL | Per-key expiration via `txn.SetEntry` |
| Versioning | Multi-version (configurable versions per key) |
| Streaming | Concurrent iteration over key ranges |
| Merge ops | Read-modify-write via MergeOperator |
| Backup | Protobuf-encoded incremental backup/restore |
| Pure Go | No CGo dependency |
### Error sentinels
Key errors to handle (from `errors.go`):
- `ErrKeyNotFound` -- Key does not exist
- `ErrConflict` -- Transaction conflict, retry
- `ErrTxnTooBig` -- Too many writes in one transaction
- `ErrReadOnlyTxn` -- Write attempted in read-only txn
- `ErrDiscardedTxn` -- Reusing a discarded transaction
- `ErrDBClosed` -- Operation after Close
- `ErrEmptyKey` -- Empty key not allowed
- `ErrBlockedWrites` -- Writes blocked during DropAll
### CLI tool
```bash
cd badger && go install .
badger backup --dir /path/to/db # Create backup
badger restore --dir /path/to/db # Restore from backup
badger info --dir /path/to/db # Show DB info
badger flatten --dir /path/to/db # Flatten LSM tree
badger stream --dir /path/to/db # Stream keys
badger rotate --dir /path/to/db # Rotate encryption key
badger bench read --dir /path/to/db # Read benchmark
badger bench write --dir /path/to/db # Write benchmark
```
### Build and test
```bash
# Build
go build ./...
# Run tests (requires jemalloc for full test suite on some platforms)
make test # Runs test.sh
go test ./... # Without jemalloc
# Build CLI
make badger # Compiles badger binary (downloads jemalloc if needed)
# Dependencies
make dependency # Install build deps (apt-based)
```
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| `ErrConflict` on commit | Read-write conflict in concurrent txns | Retry the transaction |
| `Value log GC didn't result in cleanup` | No reclaimable space | Normal; GC only runs when beneficial |
| Slow reads | Block cache too small | Increase `WithBlockCacheSize` |
| High disk usage | Value log not GC'd | Call `db.RunValueLogGC(0.5)` periodically |
| `ErrTruncateNeeded` | Corrupt value log after crash | Open with truncate option or restore from backup |
| Data directory locked | Another process has DB open | Close other instance or use `WithBypassLockGuard` (unsafe) |
| Memory usage too high | Large memtables or cache | Reduce `WithMemTableSize` and `WithBlockCacheSize` |
## Related Skills
- `lux/lux-database.md` -- Database layer overview for the Lux stack
- `lux/lux-node.md` -- Lux node (uses ZapDB for state storage)
---
Last Updated: 2026-03-13
+14 -1
View File
@@ -1,25 +1,33 @@
{
"title": "Lux Skills Reference",
"description": "Comprehensive reference for all 60 Lux skills",
"description": "Comprehensive reference for all 72+ Lux skills",
"icon": "BookOpen",
"pages": [
"index",
"---Core Blockchain---",
"lux-node",
"lux-evm",
"lux-coreth",
"lux-consensus",
"lux-dex",
"lux-vm",
"lux-spacesvm",
"lux-session",
"lux-p2p",
"lux-genesis",
"lux-staking",
"---Cryptography & Security---",
"lux-crypto",
"lux-fhe",
"lux-fhevm",
"lux-fhe-coprocessor",
"lux-lattice",
"lux-corona",
"lux-safe",
"lux-precompile",
"lux-mpc",
"lux-hsm",
"lux-kms",
"---Threshold Signatures---",
"lux-threshold",
"lux-teleport",
@@ -29,6 +37,8 @@
"lux-netrunner",
"lux-build",
"lux-plugins",
"lux-lpm",
"lux-ledger",
"---Wallets---",
"lux-wallet",
"lux-wwallet",
@@ -37,6 +47,7 @@
"lux-finance",
"---DeFi & Markets---",
"lux-exchange",
"lux-exchange-api",
"lux-market",
"lux-liquid",
"lux-oracle",
@@ -49,6 +60,7 @@
"---GPU & Acceleration---",
"lux-gpu",
"lux-accel",
"lux-cpp",
"---Infrastructure & Ops---",
"lux-universe",
"lux-stack",
@@ -56,6 +68,7 @@
"lux-operator",
"lux-monitoring",
"lux-database",
"lux-zapdb",
"lux-migrate",
"---AI & Compute---",
"lux-ai",