mirror of
https://github.com/luxfi/docs.git
synced 2026-09-24 00:25:32 +00:00
Add 10 new Lux skill MDX pages, update navigation
New: cpp, coreth, exchange-api, fhe-coprocessor, genesis, kms, ledger, lpm, staking, zapdb Updated meta.json with 72+ skills in 17 categories.
This commit is contained in:
@@ -0,0 +1,149 @@
|
||||
---
|
||||
title: "Lux Coreth - C-Chain EVM (DEPRECATED)"
|
||||
description: "Coreth ("core Ethereum") was the original Go EVM implementation that powered the Lux C-Chain. As of January 2025, it was consolidated with the Subnet EVM codebase into the unified package. The..."
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
**Coreth is deprecated and merged into `github.com/luxfi/evm`.** All new development MUST use `github.com/luxfi/evm`. This repository is archived for historical reference only.
|
||||
|
||||
Coreth ("core Ethereum") was the original Go EVM implementation that powered the Lux C-Chain. As of January 2025, it was consolidated with the Subnet EVM codebase into the unified `github.com/luxfi/evm` package. The deprecation commit is `ac7aec973` in the coreth repo. The README explicitly directs to the evm repo, and `DEPRECATED.md` provides a full migration guide and timeline.
|
||||
|
||||
The coreth repo still compiles (Go 1.26.1, module `github.com/luxfi/coreth`) and received post-deprecation maintenance fixes through early 2026 (dependency bumps, build fixes), but no new features. The final meaningful commits were block import persistence fixes and txpool sync improvements. The `github.com/luxfi/evm` module preserves all coreth functionality with identical package paths (just replace `coreth` with `evm` in imports).
|
||||
|
||||
### Relationship to `github.com/luxfi/evm`
|
||||
|
||||
| Aspect | coreth (deprecated) | evm (canonical) |
|
||||
|--------|-------------------|-----------------|
|
||||
| Module | `github.com/luxfi/coreth` | `github.com/luxfi/evm` |
|
||||
| C-Chain support | Yes | Yes |
|
||||
| Subnet EVM support | No | Yes |
|
||||
| Warp messaging | Yes | Yes |
|
||||
| FHE precompiles | Yes (via `luxfi/fhe`) | Yes (via `luxfi/fhe`) |
|
||||
| ZK precompiles | No | Yes (blake3, zk, dead address) |
|
||||
| Bridge precompile | No | Yes |
|
||||
| Native asset transfers | Yes (`nativeasset/` dir) | No (removed; uses Warp) |
|
||||
| State upgrades | No | Yes (`stateupgrade/` dir) |
|
||||
| `api/` package | No | Yes |
|
||||
| `commontype/` package | No | Yes |
|
||||
| Active development | No (archived) | Yes |
|
||||
|
||||
Both repos share nearly identical dependency sets (`luxfi/consensus`, `luxfi/geth`, `luxfi/crypto`, `luxfi/node`, `luxfi/warp`, etc.) at the same versions. The evm repo has slightly newer versions of some dependencies (e.g., `luxfi/database` v1.17.43 vs v1.17.42, `luxfi/accel` v1.0.5 vs v1.0.1).
|
||||
|
||||
## When to use
|
||||
|
||||
- **Do NOT use coreth for new code.** Use `github.com/luxfi/evm` instead.
|
||||
- Reference coreth only when:
|
||||
- Migrating legacy code that still imports `github.com/luxfi/coreth`
|
||||
- Investigating historical C-Chain behavior or block format changes
|
||||
- Understanding the block import persistence fix (vm.go `PostImportCallback` consensus state update)
|
||||
- Tracing the origin of C-Chain-specific features like atomic transactions and native asset transfers
|
||||
|
||||
## Hard requirements
|
||||
|
||||
1. NEVER use `github.com/luxfi/coreth` in new Go modules. Use `github.com/luxfi/evm`.
|
||||
2. NEVER use `go-ethereum` directly. Use `github.com/luxfi/geth`.
|
||||
3. When migrating, replace all `coreth` imports with `evm` -- the package paths are identical after the module prefix.
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Module (DEPRECATED) | `github.com/luxfi/coreth` |
|
||||
| Replacement module | `github.com/luxfi/evm` |
|
||||
| Go version | 1.26.1 |
|
||||
| Deprecation date | January 2025 |
|
||||
| Deprecation commit | `ac7aec973` |
|
||||
| C-Chain ID | 96369 (mainnet), 96370 (testnet) |
|
||||
| Genesis hash | `0x3f4fa2a0b0ce089f52bf0ae9199c75ffdd76ecafc987794050cb0d286f1ec61e` |
|
||||
| Final status | Security fixes only through Q2 2025, then archive-only |
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Migration Guide
|
||||
|
||||
Replace go.mod dependency:
|
||||
```
|
||||
// Remove
|
||||
require github.com/luxfi/coreth v1.x.x
|
||||
|
||||
// Add
|
||||
require github.com/luxfi/evm v0.8.x
|
||||
```
|
||||
|
||||
Replace all imports:
|
||||
```go
|
||||
// OLD (deprecated)
|
||||
import "github.com/luxfi/coreth/core"
|
||||
import "github.com/luxfi/coreth/params"
|
||||
import "github.com/luxfi/coreth/plugin/evm"
|
||||
import "github.com/luxfi/coreth/eth"
|
||||
import "github.com/luxfi/coreth/ethclient"
|
||||
|
||||
// NEW (canonical)
|
||||
import "github.com/luxfi/evm/core"
|
||||
import "github.com/luxfi/evm/params"
|
||||
import "github.com/luxfi/evm/plugin/evm"
|
||||
import "github.com/luxfi/evm/eth"
|
||||
import "github.com/luxfi/evm/ethclient"
|
||||
```
|
||||
|
||||
The API surface is identical. No code changes needed beyond import paths.
|
||||
|
||||
### Why Coreth Was Deprecated
|
||||
|
||||
1. **Single implementation**: Eliminates divergence between C-Chain (coreth) and Subnet EVM code paths
|
||||
2. **Simplified maintenance**: One codebase to maintain, test, and audit
|
||||
3. **Consistent behavior**: All EVM chains on Lux use identical code
|
||||
4. **New features**: Bridge precompiles, ZK precompiles, state upgrades, and `commontype`/`api` packages were added only to the unified evm repo
|
||||
|
||||
### C-Chain Differences from Ethereum
|
||||
|
||||
These behaviors are preserved in `github.com/luxfi/evm` (C-Chain mode):
|
||||
|
||||
- **Atomic transactions**: Cross-chain transfers between X-Chain and C-Chain via `ExtData` in block body
|
||||
- **Block timing**: Blocks can have the same timestamp as parent (async Consensusman consensus, no fixed interval)
|
||||
- **Difficulty**: Always 1 (Consensusman does not use PoW difficulty)
|
||||
- **RANDOM opcode**: Returns block difficulty (always 1), no Eth2 randomness beacon equivalent
|
||||
- **Block header extensions**: `ExtDataHash`, `BaseFee`, `ExtDataGasUsed`, `BlockGasCost`
|
||||
- **Block body extensions**: `Version` and `ExtData` fields
|
||||
|
||||
### Historical Block Import Fix
|
||||
|
||||
The `PostImportCallback` fix (documented in coreth LLM.md, vm.go lines 710-758) resolved a critical issue where `admin_importChain` stored blocks in the database but did not update `chain.State.lastAcceptedBlock`. This caused the chain to appear frozen after import. The fix wraps the imported eth block and calls `SetLastAcceptedBlock()` on the consensus layer. This fix is carried forward in `github.com/luxfi/evm`.
|
||||
|
||||
### Key luxfi Dependencies (coreth final versions)
|
||||
|
||||
```
|
||||
github.com/luxfi/consensus v1.22.67
|
||||
github.com/luxfi/crypto v1.17.40
|
||||
github.com/luxfi/geth v1.16.73
|
||||
github.com/luxfi/database v1.17.42
|
||||
github.com/luxfi/node v1.23.4
|
||||
github.com/luxfi/precompile v0.4.7
|
||||
github.com/luxfi/warp v1.18.5
|
||||
github.com/luxfi/fhe v1.7.6
|
||||
github.com/luxfi/lattice/v7 v7.0.0
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"coreth" imports in existing code** -- Run `find . -name "*.go" | xargs grep "luxfi/coreth"` to find all references. Replace with `luxfi/evm`. Run `go mod tidy` after updating.
|
||||
|
||||
**Build failures after migration** -- The `nativeasset/` package exists in coreth but not in evm. If your code imports it, you need to refactor to use Warp messaging for cross-chain asset transfers.
|
||||
|
||||
**go.mod replace directive** -- If you had `replace github.com/luxfi/coreth => ../coreth`, change to `replace github.com/luxfi/evm => ../evm`.
|
||||
|
||||
**Conflicting dependency versions** -- The evm repo uses slightly newer versions of some packages (database v1.17.43 vs v1.17.42, accel v1.0.5 vs v1.0.1). Run `go mod tidy` to resolve.
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux/lux-evm.md` -- The canonical replacement for coreth. Use this for all new work.
|
||||
- `lux/lux-node.md` -- Lux node that runs the EVM plugin
|
||||
- `lux/lux-warp.md` -- Cross-chain messaging (replaces atomic transactions for subnets)
|
||||
- `lux/lux-precompile.md` -- Custom precompile development
|
||||
- `lux/lux-fhe.md` -- FHE library used by both coreth and evm
|
||||
|
||||
---
|
||||
|
||||
Last Updated: 2026-03-13
|
||||
@@ -0,0 +1,220 @@
|
||||
---
|
||||
title: "Lux C++ - High-Performance C++ Libraries for Blockchain & AI"
|
||||
description: "### Why LuxCPP?"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
LuxCPP is a **modular C++ library suite** providing GPU-accelerated cryptography, FHE, lattice operations, DEX matching, and networking for the Lux blockchain ecosystem. It consists of 8 componentized libraries with Metal, CUDA, and WebGPU GPU backends, all built via CMake + Conan.
|
||||
|
||||
### Why LuxCPP?
|
||||
|
||||
- **Multi-backend GPU**: Metal (Apple Silicon), CUDA (NVIDIA), WebGPU (Dawn), CPU (SIMD) — compile-time or runtime backend selection
|
||||
- **Full FHE stack**: TFHE/CKKS/BGV via OpenFHE integration with GPU-accelerated NTT
|
||||
- **Post-quantum crypto**: BLS12-381 pairings, ML-DSA (Dilithium), lattice-based NTT
|
||||
- **DEX engine**: C++ CLOB matching engine for sub-microsecond order processing
|
||||
- **Zero-copy FFI**: C API boundary (`lux/gpu.h`) for Go, Rust, Python, WASM bindings
|
||||
|
||||
### Tech Stack
|
||||
|
||||
- **Language**: C++20
|
||||
- **Build**: CMake 3.26+ with Conan 2.x package manager
|
||||
- **GPU**: Metal (MLX), CUDA (CCCL), Dawn (WebGPU/WGSL)
|
||||
- **Version**: 1.0.0
|
||||
- **License**: BSD-3-Clause
|
||||
- **Repo**: `github.com/luxfi/luxcpp`
|
||||
|
||||
### Components
|
||||
|
||||
| Component | Library | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| `lux-gpu` | `libluxgpu` | GPU compute dispatch (Metal/CUDA/WebGPU/CPU) |
|
||||
| `lux-crypto` | `libluxcrypto` | BLS12-381, ML-DSA, secp256k1, hashing |
|
||||
| `lux-lattice` | `libluxlattice` | NTT, ring operations, RLWE |
|
||||
| `lux-fhe` | `libluxfhe` | TFHE/CKKS/BGV (OpenFHE-based) |
|
||||
| `lux-cuda` | `libluxcuda` | CUDA kernels (proprietary) |
|
||||
| `lux-http` | `libluxhttp` | HTTP/REST framework |
|
||||
| `lux-grpc` | `libluxgrpc` | gRPC services |
|
||||
| `lux-dex` | `libluxdex` | DEX matching engine |
|
||||
|
||||
### Dependency Hierarchy
|
||||
|
||||
```
|
||||
lux-gpu ← Foundation (cross-platform GPU acceleration)
|
||||
▲
|
||||
lux-crypto ← BLS pairings, post-quantum (depends on gpu)
|
||||
lux-lattice ← NTT acceleration (depends on gpu)
|
||||
▲
|
||||
lux-fhe ← TFHE/CKKS/BGV (depends on crypto + lattice)
|
||||
```
|
||||
|
||||
## When to use
|
||||
|
||||
- Building GPU-accelerated blockchain operations (ZK proofs, FHE, NTT)
|
||||
- Integrating C++ crypto into Go/Rust services via FFI
|
||||
- Running high-performance DEX matching engines
|
||||
- Accelerating lattice-based post-quantum cryptography
|
||||
- Building cross-platform GPU compute (macOS Metal + Linux CUDA)
|
||||
|
||||
## Hard requirements
|
||||
|
||||
1. **CMake 3.26+** for build
|
||||
2. **Conan 2.x** for dependency management
|
||||
3. **C++20 compiler** (Clang 15+ or GCC 13+)
|
||||
4. **Metal SDK** (macOS) or **CUDA Toolkit** (Linux/Windows) for GPU backends
|
||||
5. **Dawn** (optional, for WebGPU backend)
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Build system | CMake + Conan |
|
||||
| Version | 1.0.0 |
|
||||
| License | BSD-3-Clause |
|
||||
| C++ standard | C++20 |
|
||||
| GPU backends | Metal, CUDA, WebGPU (Dawn), CPU (SIMD) |
|
||||
| Install prefix | `include/lux/<pkg>/`, `lib/liblux<pkg>.*` |
|
||||
| CMake targets | `lux::gpu`, `lux::crypto`, `lux::lattice`, `lux::fhe` |
|
||||
| pkg-config | `lux-gpu`, `lux-crypto`, etc. |
|
||||
| Repo | `github.com/luxfi/luxcpp` |
|
||||
|
||||
## One-file quickstart
|
||||
|
||||
### Build all components
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
conan install . --output-folder=build --build=missing
|
||||
|
||||
# Configure and build
|
||||
cd build
|
||||
cmake .. -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake \
|
||||
-DCMAKE_BUILD_TYPE=Release
|
||||
cmake --build .
|
||||
|
||||
# Run tests
|
||||
ctest --test-dir .
|
||||
```
|
||||
|
||||
### Platform-specific profiles
|
||||
|
||||
```bash
|
||||
# macOS Apple Silicon (Metal backend)
|
||||
conan install . -pr:b=default -pr:h=profiles/macos-arm64
|
||||
|
||||
# Linux x86_64 (CUDA backend)
|
||||
conan install . -pr:b=default -pr:h=profiles/linux-x86_64
|
||||
|
||||
# Windows x86_64
|
||||
conan install . -pr:b=default -pr:h=profiles/windows-x86_64
|
||||
```
|
||||
|
||||
### Use from CMake
|
||||
|
||||
```cmake
|
||||
find_package(lux-gpu REQUIRED)
|
||||
target_link_libraries(myapp lux::gpu)
|
||||
|
||||
find_package(lux-crypto REQUIRED)
|
||||
target_link_libraries(myapp lux::crypto)
|
||||
```
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
luxcpp/
|
||||
├── gpu/ # lux-gpu: GPU compute library
|
||||
│ ├── include/lux/gpu.h # C API (stable FFI boundary)
|
||||
│ ├── src/ # Core dispatch, CPU backend, ZK ops
|
||||
│ ├── webgpu/ # Dawn WebGPU backend + WGSL kernels
|
||||
│ ├── kernels/cpu/ # CPU kernel implementations
|
||||
│ ├── benchmarks/ # Performance tests
|
||||
│ └── test/ # Unit tests
|
||||
├── crypto/ # lux-crypto: Cryptographic primitives
|
||||
├── lattice/ # lux-lattice: NTT, ring operations
|
||||
├── fhe/ # lux-fhe: FHE (OpenFHE-based)
|
||||
├── dex/ # lux-dex: Matching engine
|
||||
├── http/ # lux-http: HTTP/REST
|
||||
├── grpc/ # lux-grpc: gRPC services
|
||||
├── session/ # Session-related C++ components
|
||||
├── consensus/ # Consensus C++ components
|
||||
├── cuda/ # CUDA kernel sources
|
||||
├── metal/ # Metal shader sources
|
||||
├── webgpu/ # WebGPU/WGSL shader sources
|
||||
├── lux-accel/ # Umbrella acceleration package
|
||||
├── lux-cuda/ # CUDA-specific build
|
||||
├── lux-gpu/ # GPU-specific build
|
||||
├── lux-metal/ # Metal-specific build
|
||||
├── lux-webgpu/ # WebGPU-specific build
|
||||
├── mlx-c-api/ # MLX C API bindings
|
||||
├── third_party/ # Vendored dependencies
|
||||
├── cmake/ # CMake modules
|
||||
├── profiles/ # Conan build profiles
|
||||
├── scripts/ # Build & install scripts
|
||||
├── conanfile.py # Conan package definition
|
||||
└── LIBRARY_CONTRACT.md # Standard layout specification
|
||||
```
|
||||
|
||||
### GPU Backend Selection
|
||||
|
||||
The GPU library auto-selects the best available backend at compile time, with runtime fallback:
|
||||
|
||||
| Backend | Platform | Acceleration |
|
||||
|---------|----------|-------------|
|
||||
| Metal | macOS (Apple Silicon) | MLX integration |
|
||||
| CUDA | Linux/Windows (NVIDIA) | CCCL/cuBLAS |
|
||||
| Dawn | Cross-platform | WebGPU via WGSL shaders |
|
||||
| CPU | All platforms | SIMD (AVX2/NEON) fallback |
|
||||
|
||||
### Install Layout Convention
|
||||
|
||||
All libraries follow a canonical install layout:
|
||||
- Headers: `include/lux/<pkg>/*.h`
|
||||
- Libraries: `lib/liblux<pkg>.{dylib,so,a}`
|
||||
- CMake config: `lib/cmake/lux-<pkg>/`
|
||||
- pkg-config: `lib/pkgconfig/lux-<pkg>.pc`
|
||||
|
||||
### Conan Build Options
|
||||
|
||||
```python
|
||||
# Component toggles
|
||||
with_gpu=True # GPU compute
|
||||
with_fhe=True # FHE operations
|
||||
with_crypto=True # Cryptographic primitives
|
||||
with_lattice=True # Lattice crypto
|
||||
with_dex=True # DEX matching engine
|
||||
with_http=True # HTTP framework
|
||||
with_grpc=False # gRPC (opt-in, large dep)
|
||||
|
||||
# Backend toggles
|
||||
with_metal=True # Metal (macOS)
|
||||
with_cuda=True # CUDA (Linux/Windows)
|
||||
with_webgpu=True # WebGPU (Dawn)
|
||||
with_openmp=True # OpenMP parallelism
|
||||
|
||||
# Features
|
||||
with_tests=False # Build tests
|
||||
with_benchmarks=False # Build benchmarks
|
||||
embed_kernels=True # Embed kernel sources in binary
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| Metal not found | Wrong platform | Metal only on macOS with Xcode |
|
||||
| CUDA not found | Missing toolkit | Install CUDA Toolkit 12+ |
|
||||
| Conan errors | Wrong version | Requires Conan 2.x (not 1.x) |
|
||||
| Dawn build fails | Missing deps | `git submodule update --init` in Dawn |
|
||||
| Link errors | Missing component | Check `find_package(lux-<pkg>)` in CMakeLists |
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux/lux-gpu.md` — Go bindings for GPU acceleration (wraps this C++ code)
|
||||
- `lux/lux-fhe.md` — Go FHE library (uses lattice from here)
|
||||
- `lux/lux-lattice.md` — Go lattice crypto (EPFL, complements C++ NTT)
|
||||
- `lux/lux-crypto.md` — Go crypto primitives
|
||||
- `lux/lux-dex.md` — Go/Rust DEX engine (C++ matching core)
|
||||
- `lux/lux-accel.md` — Rust acceleration (complements C++)
|
||||
@@ -0,0 +1,186 @@
|
||||
---
|
||||
title: "Lux Exchange API - GraphQL Proxy and Token Data Service"
|
||||
description: "## When to use"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Lux Exchange API (`github.com/luxfi/exchange-api`) is a **TypeScript Express server** that serves as a GraphQL proxy for the Lux DEX frontend. It intercepts Uniswap-compatible GraphQL queries, handles LUX/ZOO chain queries natively using Blockscout and subgraph data sources, and proxies all other chain queries to the upstream Uniswap API. Works alongside `exchange-proxy` (Go CORS proxy for raw Uniswap endpoints) and `exchange-sdk` (TypeScript order matching engine library).
|
||||
|
||||
## When to use
|
||||
|
||||
- Adding new token metadata to the Lux DEX token list
|
||||
- Modifying how LUX/ZOO chain data is fetched or presented
|
||||
- Adjusting cache TTLs for price/volume data
|
||||
- Adding new GraphQL operation handlers for native chains
|
||||
- Debugging token price display issues in the exchange frontend
|
||||
|
||||
## Hard requirements
|
||||
|
||||
1. ALWAYS use `github.com/luxfi/*` package paths -- NEVER `go-ethereum` or `ava-labs`
|
||||
2. Native chain handling is determined by `NATIVE_CHAINS` set: `LUX` and `ZOO`
|
||||
3. Stablecoins (USDT, USDC, LUSD, DAI, BUSD) are force-priced to $1.00
|
||||
4. Prices and volumes exceeding 1e12 are capped to 0 (subgraph overflow artifacts)
|
||||
5. Unhandled native chain operations fall through to Uniswap proxy
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Repo | `github.com/luxfi/exchange-api` |
|
||||
| Language | TypeScript (ES2022, strict mode) |
|
||||
| Runtime | Node.js 22 |
|
||||
| Framework | Express 4.21 |
|
||||
| Default Port | 4000 |
|
||||
| Build | `npm install && npm run build` |
|
||||
| Dev | `npm run dev` (tsx watch) |
|
||||
| Docker | Multi-stage alpine, `node dist/index.js` |
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Three-Service Architecture
|
||||
|
||||
| Service | Repo | Language | Port | Purpose |
|
||||
|---------|------|----------|------|---------|
|
||||
| **exchange-api** | `github.com/luxfi/exchange-api` | TypeScript | 4000 | GraphQL proxy, native chain data |
|
||||
| **exchange-proxy** | `github.com/luxfi/exchange-proxy` | Go | 8088 | CORS proxy for raw Uniswap APIs |
|
||||
| **exchange-sdk** | `github.com/luxfi/exchange-sdk` | TypeScript | - | Order matching engine library |
|
||||
|
||||
### Request Flow
|
||||
|
||||
```
|
||||
DEX Frontend
|
||||
|
|
||||
v
|
||||
exchange-api :4000
|
||||
POST /v1/graphql
|
||||
|
|
||||
+-- chain=LUX or ZOO? --> Native handler (Blockscout + Subgraph V2/V3)
|
||||
|
|
||||
+-- other chain? -------> Proxy to Uniswap API (with caching)
|
||||
```
|
||||
|
||||
### Native GraphQL Operations
|
||||
|
||||
| Operation | Handler | Data Source |
|
||||
|-----------|---------|-------------|
|
||||
| `TopTokens` / `TopTokens100` / `TopTokensSparkline` | `handleTopTokens` | V2+V3 subgraph merged, Blockscout fallback |
|
||||
| `Token` / `TokenPrice` / `SimpleToken` | `handleToken` | Subgraph + Blockscout + known token list |
|
||||
| `TopV2Pairs` | `handleTopV2Pairs` | V2 subgraph |
|
||||
| `TopV3Pools` | `handleTopV3Pools` | V3 subgraph |
|
||||
| `V2Transactions` / `V3Transactions` | `handleTransactions` | V2 subgraph swaps |
|
||||
| `HistoricalProtocolVolume` / `DailyProtocolTvl` | stub | Returns empty arrays |
|
||||
| `IsV3SubgraphStale` | stub | Returns `false` |
|
||||
|
||||
### Data Sources
|
||||
|
||||
| Source | Default URL | Purpose |
|
||||
|--------|------------|---------|
|
||||
| Blockscout | `https://api-explore.lux.network` | ERC-20 token list, on-chain stats |
|
||||
| V2 Subgraph | `https://subgraph.lux.network/subgraphs/name/luxfi/uniswap-v2` | V2 AMM pairs, tokens, swaps |
|
||||
| V3 Subgraph | `https://subgraph.lux.network/subgraphs/name/luxfi/uniswap-v3` | V3 pools, tokens, swaps |
|
||||
| Uniswap API | `https://interface.gateway.uniswap.org/v1/graphql` | Non-native chain proxy target |
|
||||
|
||||
### Cache TTLs
|
||||
|
||||
| Tier | Duration | What |
|
||||
|------|----------|------|
|
||||
| SHORT | 30s | Prices, volumes, stats |
|
||||
| MEDIUM | 5min | Token lists, pool lists |
|
||||
| LONG | 1hr | Token metadata, logos |
|
||||
| PROXY | 60s | Proxied Uniswap responses |
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|----------|---------|---------|
|
||||
| `PORT` | `4000` | Listen port |
|
||||
| `BLOCKSCOUT_API` | `https://api-explore.lux.network` | Blockscout API base |
|
||||
| `SUBGRAPH_URL` | `https://subgraph.lux.network/.../uniswap-v2` | V2 subgraph |
|
||||
| `SUBGRAPH_V3_URL` | `https://subgraph.lux.network/.../uniswap-v3` | V3 subgraph |
|
||||
| `UNISWAP_API` | `https://interface.gateway.uniswap.org/v1/graphql` | Upstream Uniswap |
|
||||
|
||||
### Known Tokens (lux-tokens.ts)
|
||||
|
||||
20 LUX mainnet tokens with verified on-chain addresses:
|
||||
|
||||
| Symbol | Address | Decimals |
|
||||
|--------|---------|----------|
|
||||
| LUX | Native (0x0...0) | 18 |
|
||||
| WLUX | `0x4888...8b3E` | 18 |
|
||||
| USDC | `0xF85C...95A6` | 6 |
|
||||
| LETH | `0x60E0...d9ba` | 18 |
|
||||
| LBTC | `0x1E48...4f4e` | 8 |
|
||||
| LSOL | `0x26B4...e9e7F` | 18 |
|
||||
|
||||
Plus LUSD, LZOO, LAVAX, LBLAST, LBNB, LBOME, LBONK, LCELO, LDOGS, LPOL, LTON, CYRUS, MELANIA, TRUMP, Z.
|
||||
|
||||
### Exchange Proxy (Go, :8088)
|
||||
|
||||
CORS reverse proxy that forwards requests to Uniswap upstream APIs with `Origin: https://app.uniswap.org` header injection:
|
||||
|
||||
| Route Prefix | Upstream |
|
||||
|-------------|----------|
|
||||
| `/uniswap/` | `https://api.uniswap.org` |
|
||||
| `/liquidity/` | `https://liquidity.backend-prod.api.uniswap.org` |
|
||||
| `/gateway/` | `https://interface.gateway.uniswap.org` |
|
||||
| `/conversion/` | `https://entry-gateway.backend-prod.api.uniswap.org` |
|
||||
|
||||
In-memory SHA256-keyed cache with tiered TTLs (5s quotes, 30s pools, 5min tokens, 10s GraphQL).
|
||||
|
||||
### Exchange SDK (TypeScript, npm)
|
||||
|
||||
Order matching engine library (`@hanzo/[email protected]`):
|
||||
|
||||
- **Book**: AVL tree + Fibonacci heap order book with bid/ask heaps
|
||||
- **Order**: Limit/market orders with side (ASK/BID) and status tracking
|
||||
- **Trade**: Fill execution with matched/rejected/new orders
|
||||
- **Candle**: OHLCV candlestick aggregation (1min, 1hr, 1day, 1week)
|
||||
- **Servers**: HTTP REST + Socket.IO WebSocket transports
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
exchange-api/
|
||||
src/
|
||||
index.ts -- Express server, routes, CORS
|
||||
graphql.ts -- GraphQL handler, operation dispatch, Uniswap proxy
|
||||
blockscout.ts -- Blockscout + subgraph V2/V3 data fetchers
|
||||
cache.ts -- In-memory TTL cache with periodic cleanup
|
||||
lux-tokens.ts -- 20 known LUX mainnet tokens with metadata
|
||||
Dockerfile -- Multi-stage Node 22 alpine
|
||||
package.json -- @luxfi/[email protected]
|
||||
tsconfig.json -- ES2022, strict, commonjs
|
||||
|
||||
exchange-proxy/
|
||||
main.go -- HTTP server with graceful shutdown
|
||||
proxy/proxy.go -- CORS proxy, in-memory cache, upstream routing
|
||||
go.mod -- github.com/luxfi/exchange-proxy, Go 1.23
|
||||
|
||||
exchange-sdk/
|
||||
src/
|
||||
Book.ts -- Order book (AVL + Fibonacci heap)
|
||||
Order.ts -- Order model (limit/market, bid/ask)
|
||||
Trade.ts -- Trade execution model
|
||||
Candle.ts -- OHLCV candlestick builder
|
||||
server.ts -- Demo server with random order generation
|
||||
servers/ -- HTTP and Socket.IO transports
|
||||
clients/ -- Client implementations
|
||||
package.json -- @hanzo/[email protected]
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| Token price shows $0 | Subgraph has no `derivedETH` | Add token to `LUX_TOKENS` with fallback data |
|
||||
| Insane price display | Subgraph decimal overflow | Already capped at 1e12, check raw subgraph |
|
||||
| CORS errors from frontend | Missing exchange-proxy | Deploy exchange-proxy alongside exchange-api |
|
||||
| Uniswap proxy 502 | Upstream rate limit | Increase cache TTL or add API key |
|
||||
| V3 data missing | V3 subgraph not synced | Falls back to V2 data automatically |
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux/lux-exchange.md` -- DEX frontend and omnichain routing
|
||||
- `lux/lux-dex.md` -- CLOB engine (high-performance order matching)
|
||||
- `lux/lux-evm.md` -- EVM with DEX precompiles
|
||||
@@ -0,0 +1,185 @@
|
||||
---
|
||||
title: "Lux FHE Coprocessor"
|
||||
description: "The coprocessor is a Go service with three binaries: gateway (blockchain event listener), scheduler (worker coordination), and tfhe-worker (FHE compute). Jobs flow through a Redis queue. Ciphertext..."
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The FHE Coprocessor (`github.com/luxfi/fhe-coprocessor`) is a distributed computation engine that processes Fully Homomorphic Encryption operations off-chain on behalf of the Lux blockchain. It listens for FHE events emitted by on-chain smart contracts (via the FHEVM precompiles), executes TFHE operations on encrypted ciphertexts using worker pools, and submits results back to the chain.
|
||||
|
||||
The coprocessor is a Go service with three binaries: **gateway** (blockchain event listener), **scheduler** (worker coordination), and **tfhe-worker** (FHE compute). Jobs flow through a Redis queue. Ciphertext blobs are stored on disk (content-addressed by SHA-256 hash) with optional Redis or in-memory caching. The actual FHE computation is performed via `github.com/luxfi/fhe`, which provides TFHE operations (add, sub, mul, comparison, bitwise, shifts, rotations, min/max, negation) on serialized ciphertexts.
|
||||
|
||||
This is the Go-native coprocessor. The Rust-based coprocessor lives in `github.com/luxfi/fhevm/coprocessor/` (see `lux/lux-fhevm.md`). Both serve the same architectural role -- offloading FHE computation from the EVM -- but target different deployment contexts.
|
||||
|
||||
## When to use
|
||||
|
||||
- Deploying FHE compute infrastructure for Lux EVM chains
|
||||
- Scaling TFHE computation horizontally across multiple workers
|
||||
- Building or modifying the blockchain-to-FHE event pipeline
|
||||
- Setting up a local FHE development environment with Docker Compose
|
||||
- Integrating a new queue backend (NATS JetStream is already implemented alongside Redis)
|
||||
- Adding new FHE operation types to the worker execution pipeline
|
||||
|
||||
## Hard requirements
|
||||
|
||||
1. Go 1.26.1
|
||||
2. `github.com/luxfi/fhe` v1.7.5 -- the TFHE library. NEVER use Zama tfhe-rs or any other FHE library for the Go coprocessor.
|
||||
3. `github.com/luxfi/lattice/v7` v7.0.0 (indirect, via luxfi/fhe) -- NEVER use raw Lattigo.
|
||||
4. Redis 7+ for the job queue and optional ciphertext storage.
|
||||
5. NATS (optional) -- the queue package implements both Redis and NATS JetStream backends.
|
||||
6. A TFHE server key file for real computation. Without `-server-key`, the worker starts with a mock key (development only).
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Module | `github.com/luxfi/fhe-coprocessor` |
|
||||
| Go version | 1.26.1 |
|
||||
| Binaries | `tfhe-worker`, `gateway`, `scheduler` |
|
||||
| Gateway port | 8080 |
|
||||
| Scheduler port | 8081 |
|
||||
| Worker metrics port | 9090 |
|
||||
| Queue backends | Redis (primary), NATS JetStream |
|
||||
| Storage backends | File (default), Redis, Memory, Cached (memory + any backend) |
|
||||
| Key dependency | `github.com/luxfi/fhe` v1.7.5 |
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Blockchain Events
|
||||
|
|
||||
v
|
||||
+---------+ Redis/NATS +-----------+
|
||||
| Gateway | -------------> | Scheduler |
|
||||
| (:8080) | job queue | (:8081) |
|
||||
+---------+ +-----------+
|
||||
| |
|
||||
| ciphertext | worker registration
|
||||
| storage | heartbeat / cleanup
|
||||
v |
|
||||
+---------+ +--------+--------+
|
||||
| File/ | | Workers |
|
||||
| Redis |<------------>| (tfhe-worker) |
|
||||
| Storage | load/store | N goroutines |
|
||||
+---------+ ciphertexts | per process |
|
||||
+-----------------+
|
||||
|
|
||||
github.com/luxfi/fhe
|
||||
(TFHE operations)
|
||||
```
|
||||
|
||||
### Three Binaries
|
||||
|
||||
**gateway** -- Listens to blockchain events (FHE compute, store, decrypt). Converts events to jobs and pushes them onto the Redis queue. Exposes `/health`, `/status` (last processed block), and `/job/{id}` HTTP endpoints. Uses a `BlockchainClient` interface (currently a mock; real implementation connects to Lux EVM RPC).
|
||||
|
||||
**scheduler** -- Coordinates workers via HTTP registration (`POST /workers`), heartbeat (`POST /heartbeat`), and stale cleanup (configurable period/timeout). Provides `/submit` for manual job submission and `/status` for cluster capacity. Does not perform FHE computation itself.
|
||||
|
||||
**tfhe-worker** -- Runs a pool of N goroutines (default 4) that pop jobs from Redis, load ciphertext operands from storage, execute the TFHE operation via `pkg/fhe.Adapter`, store the result, and update job status. Exposes Prometheus-format metrics at `/metrics`.
|
||||
|
||||
### FHE Operations (20 opcodes)
|
||||
|
||||
| Code | Op | Type | Description |
|
||||
|------|----|------|-------------|
|
||||
| 0 | Add | Binary | Addition |
|
||||
| 1 | Sub | Binary | Subtraction |
|
||||
| 2 | Mul | Binary | Multiplication |
|
||||
| 3 | Not | Unary | Bitwise NOT |
|
||||
| 4 | And | Binary | Bitwise AND |
|
||||
| 5 | Or | Binary | Bitwise OR |
|
||||
| 6 | Xor | Binary | Bitwise XOR |
|
||||
| 7 | Eq | Binary | Equality |
|
||||
| 8 | Ne | Binary | Not equal |
|
||||
| 9 | Gt | Binary | Greater than |
|
||||
| 10 | Ge | Binary | Greater or equal |
|
||||
| 11 | Lt | Binary | Less than |
|
||||
| 12 | Le | Binary | Less or equal |
|
||||
| 13 | Shl | Binary | Shift left |
|
||||
| 14 | Shr | Binary | Shift right |
|
||||
| 15 | Rotl | Binary | Rotate left |
|
||||
| 16 | Rotr | Binary | Rotate right |
|
||||
| 17 | Min | Binary | Minimum |
|
||||
| 18 | Max | Binary | Maximum |
|
||||
| 19 | Neg | Unary | Negation |
|
||||
|
||||
### Job Lifecycle
|
||||
|
||||
1. Gateway receives blockchain event (EventCompute, EventStore, or EventDecrypt)
|
||||
2. For compute events: creates a `Job` with operation code and ciphertext handles, pushes to Redis list `fhe:queue:<name>`
|
||||
3. Worker pops job via `BRPOP`, updates status to `StatusProcessing`
|
||||
4. Worker loads LHS (and optionally RHS) ciphertext from file storage
|
||||
5. Worker calls `fhe.Adapter.Execute()` which deserializes ciphertexts, runs the TFHE operation via `github.com/luxfi/fhe`, and serializes the result
|
||||
6. Worker stores result ciphertext, updates job to `StatusCompleted` with result handle
|
||||
7. On failure: job status becomes `StatusFailed` with error message
|
||||
|
||||
### Storage Layer
|
||||
|
||||
Ciphertexts are content-addressed (SHA-256 hash of data = handle). Four implementations:
|
||||
|
||||
- **FileStorage** (default): Sharded directories (first 2 chars of hash). Atomic writes via temp file + rename. Path: `<base>/<hash[:2]>/<hash>`.
|
||||
- **MemoryStorage**: In-memory map with capacity limit (MB). Used for testing and as cache layer.
|
||||
- **RedisStorage**: Redis-backed with configurable TTL (default 24h). Key prefix `fhe:ct:`.
|
||||
- **CachedStorage**: Wraps any backend with MemoryStorage as read-through/write-through cache.
|
||||
|
||||
### Queue Layer
|
||||
|
||||
The `Queue` interface supports Push, Pop (blocking), Update, Get, and Close. Two implementations:
|
||||
|
||||
- **RedisQueue**: Uses Redis lists (`LPUSH`/`BRPOP`) for job ordering and `SET` with 24h TTL for job metadata. Pipeline-based Push for atomicity.
|
||||
- **NATSQueue**: NATS JetStream with `WorkQueuePolicy` retention. Subject: `fhe.jobs.*`. Pull-based consumption with explicit Ack/Nak. Note: `Get()` is not implemented (use separate storage layer).
|
||||
|
||||
### Docker Deployment
|
||||
|
||||
```bash
|
||||
# Start all services (gateway, scheduler, 2 workers, Redis)
|
||||
docker compose up -d
|
||||
|
||||
# Scale workers
|
||||
docker compose up -d --scale worker=4
|
||||
|
||||
# With monitoring (Prometheus + Grafana)
|
||||
docker compose --profile monitoring up -d
|
||||
|
||||
# Build images
|
||||
make docker
|
||||
```
|
||||
|
||||
The Dockerfile uses multi-stage builds with separate targets per binary (`worker`, `gateway`, `scheduler`). Each runs as non-root user `fhe`.
|
||||
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
make build # Build all 3 binaries to bin/
|
||||
make test # go test -v -race ./...
|
||||
make test-coverage # Coverage report to coverage.html
|
||||
make fmt # gofumpt
|
||||
make lint # golangci-lint
|
||||
make dev # Start Redis + all 3 services locally
|
||||
make stop # Kill all local services
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"no server key provided, using mock key"** -- The worker starts without real FHE computation capability. Pass `-server-key /path/to/server.key` with a valid TFHE server key generated by `github.com/luxfi/fhe`.
|
||||
|
||||
**Redis connection refused** -- Ensure Redis is running on the configured address. Default is `localhost:6379`. Use `-redis` flag or start via `docker run -d -p 6379:6379 redis:alpine`.
|
||||
|
||||
**Worker not processing jobs** -- Check that gateway and worker use the same `-queue` name (default: `default`). Verify Redis connectivity from both processes. Check scheduler `/workers` endpoint to confirm worker registration.
|
||||
|
||||
**Ciphertext not found errors** -- Gateway and workers must share the same storage path (`-storage`). In Docker Compose, the `ciphertext-storage` volume is shared between gateway and worker containers.
|
||||
|
||||
**NATS queue Get() returns error** -- The NATS implementation does not support `Get()` by design. Use Redis queue or add a separate KV store for job lookup.
|
||||
|
||||
**docker-compose.yml naming** -- The repo still uses `docker-compose.yml` filename. For new deployments, rename to `compose.yml` per project conventions.
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux/lux-fhe.md` -- Core Go FHE library (`github.com/luxfi/fhe`) that powers the worker's TFHE operations
|
||||
- `lux/lux-fhevm.md` -- Rust-based FHEVM coprocessor (alternative architecture using Zama tfhe crate)
|
||||
- `lux/lux-evm.md` -- EVM with FHE precompiles that emit the events this coprocessor consumes
|
||||
- `lux/lux-lattice.md` -- Underlying lattice cryptography primitives
|
||||
|
||||
---
|
||||
|
||||
Last Updated: 2026-03-13
|
||||
@@ -0,0 +1,193 @@
|
||||
---
|
||||
title: "Lux Genesis"
|
||||
description: "## When to use"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
`github.com/luxfi/genesis` is a Go module that provides canonical genesis configurations for all Lux blockchain networks. It contains embedded genesis JSON for mainnet, testnet, devnet, and local/custom networks, along with CLI tools for generating, extracting, and validating genesis state. The package supports the primary C-Chain (EVM), P-Chain (Platform), X-Chain (Exchange), and subnet chains (Zoo, SPC, Hanzo, Pars).
|
||||
|
||||
## When to use
|
||||
|
||||
- Bootstrapping a new Lux node with correct genesis state
|
||||
- Generating custom genesis configurations for local or private networks
|
||||
- Extracting or migrating genesis data between network environments
|
||||
- Validating node keys against expected genesis allocations
|
||||
- Looking up chain IDs, network IDs, or treasury addresses for any Lux network
|
||||
- Building tooling that needs to programmatically load genesis JSON by network ID
|
||||
|
||||
## Hard requirements
|
||||
|
||||
- Go 1.26.1+
|
||||
- Module path: `github.com/luxfi/genesis`
|
||||
- All dependencies use `github.com/luxfi/*` packages exclusively -- never `go-ethereum` or `ava-labs`
|
||||
- Go package versions stay at v1.x.x (no v2+ bumps)
|
||||
- Never expose or commit private keys, mnemonics, or EWOQ keys
|
||||
- Treasury address `0x9011E888251AB053B7bD1cdB598Db4f9DEd94714` is canonical across all networks
|
||||
|
||||
## Quick reference
|
||||
|
||||
### Build and test
|
||||
|
||||
```bash
|
||||
cd ~/work/lux/genesis
|
||||
make build # Produces bin/genesis
|
||||
make test # go test -v -race ./...
|
||||
make test-coverage
|
||||
make lint # golangci-lint
|
||||
make tidy # go mod tidy
|
||||
```
|
||||
|
||||
### CLI usage
|
||||
|
||||
```bash
|
||||
# Generate local genesis from keys
|
||||
genesis -network local -output primary.json
|
||||
|
||||
# Generate mainnet genesis preserving original C-Chain
|
||||
genesis -network mainnet -cchain mainnet/cchain.json -output primary.json
|
||||
|
||||
# Generate from BIP39 mnemonic with 5 validators
|
||||
LUX_MNEMONIC="your mnemonic" genesis -validators 5 -output primary.json
|
||||
|
||||
# Load from existing genesis directory
|
||||
genesis -genesis-dir ./mainnet -output primary.json
|
||||
```
|
||||
|
||||
### CLI flags
|
||||
|
||||
| Flag | Default | Description |
|
||||
|------|---------|-------------|
|
||||
| `-network` | `local` | Network: mainnet, testnet, local, custom |
|
||||
| `-network-id` | 0 | Explicit network ID (overrides -network) |
|
||||
| `-keys-dir` | `~/.lux/keys` | Directory containing node keys |
|
||||
| `-genesis-dir` | - | Directory containing genesis component files |
|
||||
| `-output` | stdout | Output file path |
|
||||
| `-allocation` | 1000000000 | Allocation per validator in nLUX |
|
||||
| `-validators` | 5 | Number of validators for mnemonic-based genesis |
|
||||
| `-cchain` | - | Path to existing C-Chain genesis to preserve |
|
||||
| `-format` | `pretty` | Output format: json, pretty |
|
||||
|
||||
### Environment variables
|
||||
|
||||
| Variable | Purpose |
|
||||
|----------|---------|
|
||||
| `LUX_KEYS_DIR` | Directory containing node keys |
|
||||
| `LUX_MNEMONIC` | BIP39 mnemonic for key derivation |
|
||||
| `LUX_PRIVATE_KEY` | Single private key (hex) |
|
||||
| `LUX_GENESIS_DIR` | Directory containing genesis component files |
|
||||
| `LUX_NETWORK_ID` | Network ID |
|
||||
| `LUX_PCHAIN_ALLOCS` | JSON string of P-Chain allocations |
|
||||
| `LUX_PCHAIN_ALLOCS_FILE` | Path to P-Chain allocations JSON file |
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Network IDs and Chain IDs
|
||||
|
||||
| Network | Network ID | C-Chain ID | Purpose |
|
||||
|---------|------------|------------|---------|
|
||||
| Mainnet | 1 | 96369 | Production network |
|
||||
| Testnet | 2 | 96368 | Public test network |
|
||||
| Devnet | 3 | 96370 | Fast iteration dev network |
|
||||
| Local/Custom | 1337 | 1337 | Local development |
|
||||
|
||||
### Subnet chains
|
||||
|
||||
| Chain | Mainnet ID | Testnet ID | Devnet ID | Treasury |
|
||||
|-------|-----------|-----------|-----------|----------|
|
||||
| Zoo | 200200 | 200201 | 200202 | 2T ZOO |
|
||||
| SPC | 36911 | 36910 | 36912 | 1B SPC |
|
||||
| Hanzo | 36963 | 36962 | 36964 | 1B AI |
|
||||
| Pars | 494949 | 494950 | 494951 | 2T PARS |
|
||||
|
||||
### Directory structure
|
||||
|
||||
```
|
||||
genesis/
|
||||
cmd/
|
||||
genesis/ # Main genesis generator CLI
|
||||
checkkeys/ # Validate node keys against genesis
|
||||
extract-genesis/ # Extract genesis from running node
|
||||
find-validators/ # Find validators in genesis state
|
||||
regenerate-genesis/ # Regenerate genesis from components
|
||||
pkg/genesis/ # Core library
|
||||
allocations.go # Token allocation logic
|
||||
blockchain.go # Blockchain genesis types
|
||||
chain_mapping.go # Chain ID mapping
|
||||
config.go # Genesis config builder
|
||||
keys.go # Key derivation and loading
|
||||
network.go # Network parameter resolution
|
||||
params.go # Network parameters
|
||||
types.go # Core types
|
||||
validator_keys.go # Validator key management
|
||||
configs/ # Embedded genesis JSON (22 network configs)
|
||||
mainnet/ # C-Chain, P-Chain, network, bootstrappers
|
||||
testnet/ # C-Chain, genesis
|
||||
devnet/ # C-Chain
|
||||
local/ # C-Chain, genesis
|
||||
zoo-mainnet/ # Zoo subnet genesis
|
||||
spc-mainnet/ # SPC subnet genesis
|
||||
hanzo-mainnet/ # Hanzo subnet genesis
|
||||
pars-mainnet/ # Pars subnet genesis
|
||||
... # testnet/devnet variants for each subnet
|
||||
builder/ # Genesis builder with tests
|
||||
chains/ # Subnet chain definitions (ai, spc, zoo)
|
||||
mainnet/ # Production genesis exports
|
||||
testnet/ # Test network genesis exports
|
||||
keys/mainnet/ # Mainnet key material
|
||||
```
|
||||
|
||||
### Key dependencies
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `github.com/luxfi/constants` | v1.4.4 | Network ID constants |
|
||||
| `github.com/luxfi/ids` | v1.2.9 | Node/chain identifiers |
|
||||
| `github.com/luxfi/crypto` | v1.17.40 | Cryptographic primitives |
|
||||
| `github.com/luxfi/vm` | v1.0.33 | Virtual machine types |
|
||||
| `github.com/luxfi/utxo` | v0.2.4 | UTXO model types |
|
||||
| `github.com/luxfi/go-bip32` | v1.0.2 | HD key derivation |
|
||||
| `github.com/luxfi/go-bip39` | v1.1.2 | Mnemonic generation |
|
||||
| `github.com/luxfi/tls` | v1.0.3 | TLS certificate handling |
|
||||
|
||||
### Dynamic P-Chain allocations
|
||||
|
||||
The `configs` package supports runtime P-Chain allocation overrides without modifying embedded genesis. Three methods, checked in order:
|
||||
|
||||
1. `LUX_PCHAIN_ALLOCS` environment variable (JSON string)
|
||||
2. `LUX_PCHAIN_ALLOCS_FILE` environment variable (path to JSON file)
|
||||
3. `~/.lux/genesis/{network}/pchain.json` file system override
|
||||
|
||||
C-Chain genesis remains embedded and immutable.
|
||||
|
||||
### Programmatic usage
|
||||
|
||||
```go
|
||||
import "github.com/luxfi/genesis/configs"
|
||||
|
||||
// Load genesis for mainnet
|
||||
genesisBytes, err := configs.GetGenesis(configs.MainnetID)
|
||||
|
||||
// Load with custom P-Chain allocations
|
||||
allocs := []genesis.AllocationJSON{...}
|
||||
genesisBytes, err := configs.GetGenesisWithAllocations(configs.MainnetID, allocs)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **"failed to build config"**: Ensure one of `LUX_KEYS_DIR`, `LUX_MNEMONIC`, or `LUX_PRIVATE_KEY` is set, or that `~/.lux/keys/` exists with valid key files.
|
||||
- **Wrong chain ID after genesis**: Check whether `-network-id` was passed explicitly. If not, the network name resolves to the P-Chain network ID (1, 2, 3, 1337), not the C-Chain ID (96369, etc.).
|
||||
- **Hanzo testnet chain ID mismatch**: Hanzo testnet was deployed with chain ID 36964 (devnet ID) due to a deployment error. The intended ID is 36962. Code and configs reflect the actual deployed value.
|
||||
- **C-Chain genesis not matching**: Use `-cchain` flag to preserve the original C-Chain genesis from an existing file rather than generating a new one.
|
||||
- **Module not found**: Run `go mod tidy` and ensure Go 1.26.1+ is installed.
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux-staking` -- TLS certificate parsing and signature verification for node identity
|
||||
- `lux-kms` -- Secret management for node keys and validator credentials
|
||||
- `lux-node` -- The node implementation that consumes these genesis configs
|
||||
- `lux-cli` -- CLI tool that uses genesis configs for network management
|
||||
|
||||
---
|
||||
|
||||
Last Updated: 2026-03-13
|
||||
@@ -0,0 +1,244 @@
|
||||
---
|
||||
title: "Lux KMS"
|
||||
description: "## When to use"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
`github.com/luxfi/kms` is an open-source, enterprise-grade Key Management Service providing centralized secret management, cryptographic key operations, PKI (Private Certificate Authority), SSH certificate signing, and HSM integration. The platform consists of a Node.js/Fastify backend, a React/TypeScript frontend, a Go-based Kubernetes operator, and Helm charts for deployment. It is the secret management backbone for the Lux ecosystem, deployed at `kms.hanzo.ai` (Hanzo's hosted instance) and self-hostable via Docker or Kubernetes.
|
||||
|
||||
## When to use
|
||||
|
||||
- Managing application secrets (API keys, database credentials, tokens) across environments
|
||||
- Encrypting and decrypting data with AES-128/256-GCM or RSA-2048/4096
|
||||
- Signing and verifying data with RSA-PSS, RSA-PKCS1v15, or ECDSA
|
||||
- Generating and rotating cryptographic keys (symmetric and asymmetric)
|
||||
- Issuing and managing X.509 certificates via internal PKI
|
||||
- Issuing signed SSH certificates for infrastructure access
|
||||
- Syncing secrets to external platforms (GitHub Actions, Vercel, AWS, Terraform, Ansible)
|
||||
- Deploying a Kubernetes operator to inject secrets into workloads
|
||||
- Scanning codebases for leaked secrets (140+ secret types)
|
||||
|
||||
## Hard requirements
|
||||
|
||||
- Node.js 20+ for backend and frontend development
|
||||
- Go 1.26+ for the Kubernetes operator (`k8-operator/`)
|
||||
- PostgreSQL for primary storage, Redis for caching
|
||||
- Docker and Docker Compose for local development and production deployment
|
||||
- Root encryption key must be configured via `ENCRYPTION_KEY` environment variable or HSM
|
||||
- All secrets encrypted at rest with AES-GCM; TLS 1.3 for transit
|
||||
- RBAC enforced on all key and secret operations
|
||||
- Audit logging enabled for all cryptographic operations
|
||||
|
||||
## Quick reference
|
||||
|
||||
### Local development
|
||||
|
||||
```bash
|
||||
cd ~/work/lux/kms
|
||||
|
||||
# Full stack via Docker Compose
|
||||
cp .env.example .env
|
||||
docker compose -f docker-compose.dev.yml up
|
||||
|
||||
# Backend only
|
||||
cd backend && npm install && npm run dev
|
||||
|
||||
# Frontend only
|
||||
cd frontend && npm install && npm run dev
|
||||
|
||||
# Run tests
|
||||
npm test
|
||||
npm run test:e2e
|
||||
```
|
||||
|
||||
### Production deployment
|
||||
|
||||
```bash
|
||||
# Docker standalone
|
||||
docker build -f Dockerfile.standalone-kms -t lux-kms .
|
||||
docker compose -f docker-compose.prod.yml up -d
|
||||
|
||||
# Kubernetes via Helm
|
||||
helm install lux-kms ./helm-charts/kms-standalone-postgres
|
||||
|
||||
# Kubernetes operator
|
||||
cd k8-operator && make deploy
|
||||
```
|
||||
|
||||
### Secret scanning
|
||||
|
||||
```bash
|
||||
# Scan git history for leaked secrets
|
||||
kms scan --verbose
|
||||
|
||||
# Install pre-commit hook
|
||||
kms scan install --pre-commit-hook
|
||||
```
|
||||
|
||||
### Database migrations
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
npm run migration:create -- create_new_table
|
||||
npm run migration:up
|
||||
npm run migration:down
|
||||
```
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Frontend (React 18 / Vite / TypeScript)
|
||||
|
|
||||
Backend API (Node.js / Fastify / TypeScript)
|
||||
|
|
||||
+-- KMS Service Layer (key generation, crypto ops, access control, audit)
|
||||
|
|
||||
+-- PostgreSQL (primary storage) + Redis (cache)
|
||||
|
|
||||
+-- HSM Provider (PKCS#11: AWS CloudHSM, Thales Luna, SoftHSM)
|
||||
```
|
||||
|
||||
### Key hierarchy
|
||||
|
||||
1. **Root Key** -- Protected by HSM or `ENCRYPTION_KEY` env var
|
||||
2. **Organization KEK** -- Key Encryption Key per organization
|
||||
3. **Project KEK** -- Project-specific encryption key
|
||||
4. **Data Keys** -- Actual encryption/signing keys for application use
|
||||
|
||||
### Supported algorithms
|
||||
|
||||
| Category | Algorithms |
|
||||
|----------|-----------|
|
||||
| Symmetric encryption | AES-128-GCM, AES-256-GCM |
|
||||
| Asymmetric encryption | RSA-2048, RSA-4096, ECC-P256, ECC-P384 |
|
||||
| Signing | RSA-PSS (SHA-256/384/512), RSA-PKCS1v15, ECDSA |
|
||||
| Key derivation | PBKDF2, Argon2 |
|
||||
| FHE (planned) | TFHE-Binary, TFHE-Integer |
|
||||
|
||||
### API endpoints
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|--------|------|---------|
|
||||
| POST | `/api/v1/kms/keys` | Create new key |
|
||||
| GET | `/api/v1/kms/keys` | List keys |
|
||||
| GET | `/api/v1/kms/keys/{keyId}` | Get key metadata |
|
||||
| PATCH | `/api/v1/kms/keys/{keyId}` | Update key |
|
||||
| DELETE | `/api/v1/kms/keys/{keyId}` | Delete key |
|
||||
| POST | `/api/v1/kms/keys/{keyId}/encrypt` | Encrypt data |
|
||||
| POST | `/api/v1/kms/keys/{keyId}/decrypt` | Decrypt data |
|
||||
| POST | `/api/v1/kms/keys/{keyId}/sign` | Sign data |
|
||||
| POST | `/api/v1/kms/keys/{keyId}/verify` | Verify signature |
|
||||
| POST | `/api/v1/kms/keys/{keyId}/generate-data-key` | Generate data encryption key |
|
||||
| POST | `/api/v1/kmip` | KMIP 2.0+ protocol endpoint |
|
||||
|
||||
### Directory structure
|
||||
|
||||
```
|
||||
kms/
|
||||
backend/
|
||||
src/
|
||||
services/
|
||||
kms/ # Core KMS service
|
||||
kms-service.ts # Main key operation logic (40K lines)
|
||||
kms-types.ts # Type definitions
|
||||
kms-fns.ts # Utility functions
|
||||
kms-key-dal.ts # Database access layer for keys
|
||||
kms-root-config-dal.ts # Root key configuration
|
||||
internal-kms-dal.ts # Internal KMS operations
|
||||
auth/ # Authentication service
|
||||
certificate/ # X.509 certificate management
|
||||
certificate-authority/ # Private CA
|
||||
identity/ # Machine identity management
|
||||
identity-aws-auth/ # AWS IAM auth
|
||||
identity-gcp-auth/ # GCP auth
|
||||
identity-azure-auth/ # Azure auth
|
||||
identity-kubernetes-auth/ # K8s auth
|
||||
cmek/ # Customer-managed encryption keys
|
||||
... # 70+ service directories
|
||||
frontend/ # React 18 / Vite / Tailwind / Radix UI / i18next (12 languages)
|
||||
k8-operator/ # Go-based Kubernetes operator
|
||||
go.mod # github.com/luxfi/kms/k8-operator (Go 1.26)
|
||||
main.go # Operator entrypoint
|
||||
controllers/ # Reconciliation controllers
|
||||
api/ # CRD API types
|
||||
internal/ # Internal operator logic
|
||||
helm-charts/
|
||||
kms-standalone-postgres/ # Full KMS + PostgreSQL chart
|
||||
kms-gateway/ # Gateway-only chart
|
||||
secrets-operator/ # Secrets operator chart
|
||||
docs/ # Mintlify documentation (MDX)
|
||||
docker-swarm/ # Swarm deployment configs
|
||||
examples/ # Integration examples
|
||||
sink/ # Event sink for audit log streaming
|
||||
```
|
||||
|
||||
### Kubernetes operator
|
||||
|
||||
The K8s operator (`k8-operator/`) is a Go application using controller-runtime. Key dependencies:
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `github.com/luxfi/kms-go` | v0.5.98 | KMS Go SDK |
|
||||
| `sigs.k8s.io/controller-runtime` | v0.14.4 | K8s controller framework |
|
||||
| `k8s.io/api` | v0.26.1 | K8s API types |
|
||||
| `k8s.io/client-go` | v0.26.1 | K8s client |
|
||||
|
||||
It reconciles `KMSSecret` CRDs to sync secrets from the KMS platform into Kubernetes Secrets, with automatic reloading of deployments when secrets change.
|
||||
|
||||
### Authentication methods
|
||||
|
||||
- Universal Auth (API key + secret)
|
||||
- Kubernetes Service Account Auth
|
||||
- AWS IAM Auth
|
||||
- GCP IAM Auth
|
||||
- Azure AD Auth
|
||||
- OIDC Auth
|
||||
- JWT Auth
|
||||
- LDAP Auth
|
||||
- Alibaba Cloud Auth
|
||||
|
||||
### Environment variables (core)
|
||||
|
||||
| Variable | Required | Purpose |
|
||||
|----------|----------|---------|
|
||||
| `ENCRYPTION_KEY` | Yes | 16-byte hex key for platform encryption |
|
||||
| `AUTH_SECRET` | Yes | JWT signing secret |
|
||||
| `DB_CONNECTION_URI` | Yes | PostgreSQL connection string |
|
||||
| `REDIS_URL` | Yes | Redis connection string |
|
||||
| `SITE_URL` | Yes | Base URL for the KMS frontend |
|
||||
| `SMTP_HOST` | No | Mail server for notifications |
|
||||
| `HSM_ENABLED` | No | Enable PKCS#11 HSM backend |
|
||||
| `HSM_LIBRARY_PATH` | No | Path to PKCS#11 shared library |
|
||||
| `HSM_SLOT` | No | HSM slot number |
|
||||
| `HSM_PIN` | No | HSM PIN/password |
|
||||
|
||||
### Monitoring
|
||||
|
||||
| Endpoint | Purpose |
|
||||
|----------|---------|
|
||||
| `/health` | Health check |
|
||||
| `/readiness` | Readiness probe |
|
||||
| `/metrics` | Prometheus metrics |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **HSM connection failed**: Verify `HSM_LIBRARY_PATH` points to the correct PKCS#11 `.so` file, check slot number with `softhsm2-util --show-slots`, and confirm PIN is correct.
|
||||
- **Key generation failed**: Check that the root encryption key is configured (`ENCRYPTION_KEY` env var) and that PostgreSQL is reachable.
|
||||
- **"KMSSecret not syncing"**: Ensure the K8s operator is running, the `KMSSecret` CRD references valid project/environment paths, and the operator's Universal Auth credentials are active.
|
||||
- **Frontend won't start**: Run `npm install` in the `frontend/` directory. Requires Node.js 20+. Check that `SITE_URL` is set correctly.
|
||||
- **Database migration errors**: Run `npm run migration:up` from the `backend/` directory. For fresh installs, ensure the PostgreSQL database exists and `DB_CONNECTION_URI` is correct.
|
||||
- **Port conflicts**: Default ports are 8080 (backend API) and 3000 (frontend dev). Check with `lsof -i :8080`.
|
||||
- **Docker build fails**: Use `Dockerfile.standalone-kms` for production. Ensure `--platform linux/amd64` for K8s deployments.
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux-genesis` -- Genesis configurations that reference KMS for secure key storage
|
||||
- `lux-staking` -- Staking certificates whose private keys should be stored in KMS
|
||||
- `lux-node` -- Node deployments that pull secrets from KMS via the K8s operator
|
||||
|
||||
---
|
||||
|
||||
Last Updated: 2026-03-13
|
||||
@@ -0,0 +1,195 @@
|
||||
---
|
||||
title: "Lux Ledger - Hardware Wallet Integration"
|
||||
description: "## When to use"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Lux Ledger (`github.com/luxfi/ledger`) is a **Go client library** for interacting with the Lux app on Ledger hardware wallets (Nano S, Nano X, Nano S Plus, Stax, Flex). It implements the APDU protocol over USB HID for key derivation (BIP32/BIP44), address generation, transaction signing, hash signing, and multi-signature support. The repository also contains the **Rust source for the Ledger app itself** in `/rust/`.
|
||||
|
||||
## When to use
|
||||
|
||||
- Integrating Ledger hardware wallet signing into Go applications
|
||||
- Implementing the `keychain.Ledger` interface for wallet/signing flows
|
||||
- Building or modifying the Ledger app firmware (Rust, in `/rust/`)
|
||||
- Debugging Ledger device communication issues (APDU protocol)
|
||||
- Adding new Ledger device model support
|
||||
|
||||
## Hard requirements
|
||||
|
||||
1. ALWAYS use `github.com/luxfi/*` packages -- NEVER `go-ethereum` or `ava-labs`
|
||||
2. NEVER bump Go package versions above v1.x.x
|
||||
3. Tests require a **physical Ledger device** with the Lux app installed and unlocked
|
||||
4. HRP (human-readable part) must be <= 83 characters
|
||||
5. Build tags: `ledger_mock` and `ledger_zemu` exclude real HID transport
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Module | `github.com/luxfi/ledger` |
|
||||
| Go | 1.26.1 |
|
||||
| License | Apache 2.0 |
|
||||
| Package Type | Library (no binary) |
|
||||
| Default HRP | `lux` |
|
||||
| Derivation Path | `m/44'/60'/0'` (Ethereum-compatible) |
|
||||
| Default Chain ID | `2q9e4r6Mu3U68nU1fYjgbR6JvwrRx36CohpAX5UQxse55x1Q5` |
|
||||
| Ledger App CLA | `0x80` |
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Entry Points
|
||||
|
||||
```go
|
||||
// Recommended: implements keychain.Ledger interface
|
||||
ledger, err := ledger.NewLedger()
|
||||
defer ledger.Disconnect()
|
||||
|
||||
// Or directly
|
||||
app, err := ledger.FindLedgerLuxApp()
|
||||
defer app.Close()
|
||||
```
|
||||
|
||||
### keychain.Ledger Interface
|
||||
|
||||
The library implements the `keychain.Ledger` interface for seamless wallet integration:
|
||||
|
||||
| Method | Signature | Purpose |
|
||||
|--------|-----------|---------|
|
||||
| `Address` | `(hrp string, index uint32) (ids.ShortID, error)` | Get address at derivation index |
|
||||
| `GetAddresses` | `(indices []uint32) ([]ids.ShortID, error)` | Get multiple addresses |
|
||||
| `Sign` | `(message []byte, index uint32) ([]byte, error)` | Sign raw message |
|
||||
| `SignHash` | `(hash []byte, index uint32) ([]byte, error)` | Sign pre-hashed data (32 bytes) |
|
||||
| `SignTransaction` | `(hash []byte, indices []uint32) ([][]byte, error)` | Multi-address transaction signing |
|
||||
| `Disconnect` | `() error` | Close device connection |
|
||||
|
||||
### Low-Level API
|
||||
|
||||
| Method | Purpose |
|
||||
|--------|---------|
|
||||
| `GetVersion()` | Get app version (major.minor.patch) |
|
||||
| `GetPubKey(path, show, hrp, chainID)` | Get public key, hash, and bech32 address |
|
||||
| `SignFull(pathPrefix, signerPaths, message, changePaths)` | Sign full transaction with multiple paths |
|
||||
| `SignHashFull(pathPrefix, signerPaths, hash)` | Sign a 32-byte hash with multiple paths |
|
||||
| `VerifyMultipleSignatures(resp, hash, rootPath, paths, hrp, chainID)` | Verify all signatures from a multi-sign |
|
||||
|
||||
### APDU Instructions
|
||||
|
||||
| Instruction | Code | Purpose |
|
||||
|-------------|------|---------|
|
||||
| `INS_GET_VERSION` | `0x00` | Get app version |
|
||||
| `INS_WALLET_ID` | `0x01` | Get wallet identifier |
|
||||
| `INS_GET_ADDR` | `0x02` | Get address/pubkey |
|
||||
| `INS_GET_EXTENDED_PUBLIC_KEY` | `0x03` | Get extended public key |
|
||||
| `INS_SIGN_HASH` | `0x04` | Sign a hash |
|
||||
| `INS_SIGN` | `0x05` | Sign a transaction |
|
||||
| `INS_SIGN_MSG` | `0x06` | Sign a message |
|
||||
|
||||
### Supported Devices
|
||||
|
||||
| Device | Product ID (high byte) | Interface |
|
||||
|--------|----------------------|-----------|
|
||||
| Ledger Nano S | `0x10` | 0 |
|
||||
| Ledger Nano X | `0x40` | 0 |
|
||||
| Ledger Nano S Plus | `0x50` | 0 |
|
||||
| Ledger Stax | `0x60` | 0 |
|
||||
| Ledger Flex | `0x70` | 0 |
|
||||
|
||||
All use USB HID with vendor ID `0x2c97`, channel `0x0101`, packet size 64 bytes.
|
||||
|
||||
### Signature Verification
|
||||
|
||||
Uses `btcec/v2` secp256k1 ECDSA verification with malleable signature rejection:
|
||||
|
||||
```go
|
||||
// 64-byte [R || S] format
|
||||
ok := ledger.VerifySignature(pubkey, hash, signature)
|
||||
```
|
||||
|
||||
### Key Dependencies
|
||||
|
||||
```
|
||||
github.com/luxfi/[email protected] -- USB HID device access
|
||||
github.com/luxfi/[email protected] -- Lux ID types (ShortID)
|
||||
github.com/luxfi/[email protected] -- Cryptographic primitives
|
||||
github.com/btcsuite/btcd/btcec/v2 -- secp256k1 ECDSA
|
||||
github.com/mr-tron/base58 -- Base58 encoding (chain IDs)
|
||||
go.uber.org/zap -- Structured logging
|
||||
github.com/stretchr/testify -- Test assertions
|
||||
```
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
ledger/
|
||||
lux.go -- Main API: FindLedgerLuxApp, keychain.Ledger impl
|
||||
types.go -- LedgerLux, VersionInfo, ResponseSign, ResponseAddr
|
||||
common.go -- Path/HRP/ChainID serialization, signature helpers
|
||||
hid.go -- USB HID transport (LedgerAdmin, LedgerDevice)
|
||||
apdu.go -- APDU command wrapping/unwrapping
|
||||
logger.go -- zap-based logger (LEDGER_LOG_LEVEL env)
|
||||
lux_test.go -- Integration tests (require physical device)
|
||||
common_test.go -- Unit tests (path serialization, version check)
|
||||
rust/ -- Ledger app source (Rust workspace)
|
||||
app/ -- Main Ledger app crate
|
||||
app-derive/ -- Derive macros
|
||||
Cargo.toml -- Workspace config (resolver v2)
|
||||
deps/ -- Build dependencies
|
||||
hfuzz/ -- Fuzz testing
|
||||
zemu/ -- Zemu device emulator tests
|
||||
docs/ -- Documentation
|
||||
go-docs/ -- Go-specific documentation
|
||||
```
|
||||
|
||||
### Rust Ledger App
|
||||
|
||||
The `/rust/` directory contains the Ledger app firmware:
|
||||
|
||||
- Workspace with `app` and `app-derive` crates
|
||||
- Build profiles: dev (abort on panic), release (size-optimized, `opt-level = "z"`)
|
||||
- Testing via Zemu emulator and Miri
|
||||
- Fuzz testing via `cargo-hfuzz`
|
||||
- Build: `make` (requires BOLOS SDK or Docker via `deps/dockerized_build.mk`)
|
||||
|
||||
## Build and Test
|
||||
|
||||
```bash
|
||||
# Go library (unit tests, no device needed)
|
||||
go test -short ./...
|
||||
|
||||
# Go library (full tests, requires Ledger device)
|
||||
go test -v ./...
|
||||
|
||||
# Rust Ledger app
|
||||
cd rust && cargo build
|
||||
cd rust && cargo test
|
||||
|
||||
# Zemu emulator tests
|
||||
cd rust/zemu && yarn test
|
||||
|
||||
# Fuzz testing
|
||||
cd rust/hfuzz && cargo hfuzz run apdu
|
||||
```
|
||||
|
||||
### Logging
|
||||
|
||||
Set `LEDGER_LOG_LEVEL` environment variable: `debug`, `info` (default), `warn`, `error`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| `device not found` | No Ledger connected or locked | Connect device, unlock, open Lux app |
|
||||
| `CLA not supported` | Lux app not open on device | Open the Lux app on the Ledger |
|
||||
| `APDU commands < 5` | Malformed command | Check message construction |
|
||||
| `timeout reading from device` | Device busy or unresponsive | Disconnect and reconnect, 20s timeout |
|
||||
| `wrong hash size` | Hash not 32 bytes | Pass SHA-256 digest (32 bytes) |
|
||||
| `invalid path` | Bad BIP32 path format | Use format `m/44'/60'/0'/0/0` |
|
||||
| `read channel closed` | USB connection dropped | Reconnect device |
|
||||
| `Version required error` | App version too old | Update Ledger app firmware |
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux/lux-wallet.md` -- Wallet implementation (uses ledger as signing backend)
|
||||
- `lux/lux-hsm.md` -- HSM integration (software signing alternative)
|
||||
- `lux/lux-crypto.md` -- Cryptographic primitives
|
||||
@@ -0,0 +1,228 @@
|
||||
---
|
||||
title: "Lux Plugin Manager (LPM)"
|
||||
description: "Status: Alpha."
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
LPM is a command-line tool for managing virtual machine binaries and subnet definitions for the Lux Network. It tracks plugin repositories (git-based), downloads and builds VM binaries, installs them to the Lux node plugin directory, and can join subnets by installing all required VMs. The default plugin source is `luxfi/plugins-core`, but users can add custom repositories.
|
||||
|
||||
**Status**: Alpha.
|
||||
|
||||
## When to use
|
||||
|
||||
- Installing VM plugins for the Lux node
|
||||
- Joining subnets that require specific VM binaries
|
||||
- Managing custom plugin repositories for VM distribution
|
||||
- Upgrading installed VM binaries to latest versions
|
||||
- Building and linking local VM source code to the plugin directory
|
||||
|
||||
## Hard requirements
|
||||
|
||||
1. ALWAYS use `github.com/luxfi/*` packages -- NEVER `go-ethereum` or `ava-labs`
|
||||
2. Go packages stay at v1.x.x (module is `github.com/luxfi/lpm`, no version suffix)
|
||||
3. NEVER use EWOQ keys
|
||||
4. Requires a running Lux node for VM hot-reload via admin API (optional; works offline too)
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Repo | `github.com/luxfi/lpm` |
|
||||
| Module | `github.com/luxfi/lpm` |
|
||||
| Go version | 1.26.1 |
|
||||
| Binary | `build/lpm` |
|
||||
| Default plugin repo | `luxfi/plugins-core` (master branch) |
|
||||
| Plugin dir | `~/.lux/plugins` |
|
||||
| LPM state dir | `~/.lpm` |
|
||||
| License | See LICENSE |
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Directory structure
|
||||
|
||||
```
|
||||
luxfi/lpm/
|
||||
main/
|
||||
main.go -- Entry point
|
||||
cmd/
|
||||
root.go -- Root cobra command, global flags, config init
|
||||
install.go -- install-vm command
|
||||
install_github.go -- install-github (from GitHub releases)
|
||||
install_gitlab.go -- install-gitlab (from GitLab releases)
|
||||
install_url.go -- install-url (from arbitrary URL)
|
||||
install_source.go -- install-source (build from local source)
|
||||
uninstall.go -- uninstall-vm command
|
||||
join_subnet.go -- join-subnet command
|
||||
link.go -- link command (symlink local binary)
|
||||
update.go -- update command (fetch latest definitions)
|
||||
upgrade.go -- upgrade command (rebuild/reinstall VMs)
|
||||
add_repository.go -- add-repository command
|
||||
remove_repository.go -- remove-repository command
|
||||
list_repositories.go -- list-repositories command
|
||||
lpm/
|
||||
(core LPM logic)
|
||||
workflow/
|
||||
workflow.go -- Workflow orchestration
|
||||
install.go -- Install workflow
|
||||
installer.go -- Installer interface + impls
|
||||
uninstall.go -- Uninstall workflow
|
||||
upgrade_vm.go -- Upgrade workflow
|
||||
add_repository.go -- Repository management
|
||||
link.go -- Local binary linking
|
||||
executor.go -- Command execution
|
||||
config/ -- Configuration types
|
||||
constant/
|
||||
constants.go -- AppName, CoreAlias, CoreURL, CoreBranch
|
||||
types/ -- Shared type definitions
|
||||
state/ -- Persistent state management
|
||||
engine/ -- Plugin engine
|
||||
git/ -- Git operations (go-git)
|
||||
url/ -- URL handling
|
||||
checksum/ -- Checksum verification
|
||||
admin/ -- Lux node admin API client
|
||||
internal/
|
||||
vmid/ -- VM ID utilities
|
||||
pkg/ -- Shared utilities
|
||||
util/ -- Helper functions
|
||||
scripts/
|
||||
build.sh -- Build script
|
||||
lint.sh -- Linter script
|
||||
```
|
||||
|
||||
### Key dependencies
|
||||
|
||||
```
|
||||
github.com/luxfi/codec v1.1.3 -- Serialization
|
||||
github.com/luxfi/ids v1.2.9 -- ID types (VM IDs, subnet IDs)
|
||||
github.com/luxfi/sdk v1.16.46 -- Lux SDK (admin API client)
|
||||
github.com/luxfi/crypto v1.17.40 -- Crypto primitives (indirect)
|
||||
github.com/go-git/go-git v5.16.4 -- Git repository operations
|
||||
github.com/spf13/cobra v1.10.2 -- CLI framework
|
||||
github.com/spf13/viper v1.21.0 -- Configuration
|
||||
github.com/spf13/afero v1.15.0 -- Filesystem abstraction (testable)
|
||||
```
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
# Repository management
|
||||
lpm add-repository --alias luxfi/core --url https://github.com/luxfi/plugins-core.git --branch master
|
||||
lpm remove-repository --alias luxfi/core
|
||||
lpm list-repositories
|
||||
lpm update # Fetch latest definitions from all repos
|
||||
|
||||
# VM installation
|
||||
lpm install-vm --vm spacesvm # Install by alias (partial or fully qualified)
|
||||
lpm install-vm --vm luxfi/core:spacesvm # Fully qualified (repo:vm)
|
||||
lpm uninstall-vm --vm spacesvm # Remove VM binary from plugin dir
|
||||
|
||||
# Alternative install sources
|
||||
lpm install-github --repo org/repo --release v1.0.0
|
||||
lpm install-gitlab --repo org/repo --release v1.0.0
|
||||
lpm install-url --url https://example.com/vm.tar.gz
|
||||
lpm install-source --path /path/to/source
|
||||
|
||||
# Subnet operations
|
||||
lpm join-subnet --subnet spaces # Install all VMs required by a subnet
|
||||
|
||||
# Upgrade
|
||||
lpm upgrade # Upgrade all installed VMs
|
||||
lpm upgrade --vm spacesvm # Upgrade specific VM
|
||||
|
||||
# Link local build
|
||||
lpm link --vm myvm --path /path/to/binary
|
||||
```
|
||||
|
||||
### Global flags
|
||||
|
||||
```
|
||||
--config-file Path to LPM config file
|
||||
--lpm-path LPM artifacts directory (default: ~/.lpm)
|
||||
--plugin-path Lux plugin directory (default: ~/.lux/plugins)
|
||||
--credentials-file Path to credentials file (for private repos)
|
||||
--admin-api-endpoint Lux admin API (default: 127.0.0.1:9650/ext/admin)
|
||||
```
|
||||
|
||||
### Credentials file format (for private repos)
|
||||
|
||||
```yaml
|
||||
username: your-username
|
||||
password: <github-personal-access-token>
|
||||
```
|
||||
|
||||
### Build and test
|
||||
|
||||
```bash
|
||||
# Build
|
||||
make build # Output: build/lpm
|
||||
./scripts/build.sh # Alternative
|
||||
|
||||
# Test
|
||||
make test # go test -v -race -coverprofile=coverage.out ./...
|
||||
make test-short # Short tests only
|
||||
make coverage # HTML coverage report
|
||||
|
||||
# Lint
|
||||
make lint # Runs scripts/lint.sh
|
||||
|
||||
# Release (multi-platform)
|
||||
make release # linux/amd64, linux/arm64, darwin/amd64, darwin/arm64, windows/amd64
|
||||
|
||||
# Install
|
||||
make install # Copy to $GOPATH/bin
|
||||
|
||||
# Docker
|
||||
make docker # Build luxfi/lpm:VERSION image
|
||||
|
||||
# CI
|
||||
make ci # deps, verify, check (fmt, lint, test)
|
||||
```
|
||||
|
||||
### Install from release binary
|
||||
|
||||
```bash
|
||||
curl -sSfL https://raw.githubusercontent.com/luxfi/pm/master/scripts/install.sh | sh -s
|
||||
cd bin && export PATH=$PWD:$PATH
|
||||
```
|
||||
|
||||
### Constants (constant/constants.go)
|
||||
|
||||
| Constant | Value |
|
||||
|----------|-------|
|
||||
| AppName | `lpm` |
|
||||
| CoreAlias | `luxfi/plugins-core` |
|
||||
| CoreURL | `https://github.com/luxfi/plugins-core.git` |
|
||||
| CoreBranch | `master` |
|
||||
| QualifiedNameDelimiter | `:` |
|
||||
| AliasDelimiter | `/` |
|
||||
| DefaultNetwork | `testnet` |
|
||||
|
||||
### VM naming
|
||||
|
||||
- Partial alias: `spacesvm` (searches all tracked repos)
|
||||
- Fully qualified: `luxfi/core:spacesvm` (org/repo:vm)
|
||||
- When multiple repos have the same VM alias, you must use the fully qualified name
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| `Node was offline` after install | Lux node not running | VMs will be loaded on next node start |
|
||||
| Permission errors on Mac | Downloaded binary from GitHub UI | Use curl install script instead |
|
||||
| Multiple matches for VM alias | Same VM name in multiple repos | Use fully qualified name: `repo:vm` |
|
||||
| Private repo auth failure | Missing or invalid credentials | Use `--credentials-file` with valid PAT |
|
||||
| Build script fails | Missing Go toolchain | Install Go 1.26+ |
|
||||
| Plugin directory not found | Custom node install path | Set `--plugin-path` to correct location |
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux/lux-node.md` -- Lux node (loads plugins from the plugin directory)
|
||||
- `lux/lux-plugins.md` -- Plugin repository format and definitions
|
||||
- `lux/lux-vm.md` -- Virtual machine interface and development
|
||||
- `lux/lux-cli.md` -- Lux CLI (alternative node management tool)
|
||||
- `lux/lux-sdk.md` -- Lux SDK (admin API used by LPM)
|
||||
|
||||
---
|
||||
|
||||
Last Updated: 2026-03-13
|
||||
@@ -0,0 +1,148 @@
|
||||
---
|
||||
title: "Lux Staking"
|
||||
description: "## When to use"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
`github.com/luxfi/staking` is a focused Go package that handles TLS certificate parsing, public key extraction, and signature verification for the Lux blockchain staking layer. It provides a minimal, performance-oriented alternative to Go's `crypto/x509` package, parsing only the fields needed for node identity validation: the public key and raw certificate bytes. Validators on the Lux network use TLS certificates as their identity, and this package is the foundation for verifying those identities.
|
||||
|
||||
## When to use
|
||||
|
||||
- Parsing staking TLS certificates from Lux validator nodes
|
||||
- Verifying signatures from validators (RSA-PKCS1v15 or ECDSA with SHA-256)
|
||||
- Loading TLS key pairs from PEM files or byte slices for node startup
|
||||
- Validating RSA public key well-formedness (modulus size, exponent)
|
||||
- Working with node identity at the certificate level without full x509 overhead
|
||||
|
||||
## Hard requirements
|
||||
|
||||
- Go 1.26.1+
|
||||
- Module path: `github.com/luxfi/staking`
|
||||
- All dependencies use `github.com/luxfi/*` packages exclusively -- never `go-ethereum` or `ava-labs`
|
||||
- Go package versions stay at v1.x.x (no v2+ bumps)
|
||||
- RSA keys must be 2048-bit or 4096-bit with public exponent 65537
|
||||
- ECDSA keys must use P-256 curve
|
||||
- Certificates must not exceed 2 KiB (`MaxCertificateLen = 2 * constants.KiB`)
|
||||
|
||||
## Quick reference
|
||||
|
||||
### Build and test
|
||||
|
||||
```bash
|
||||
cd ~/work/lux/staking
|
||||
go build ./...
|
||||
go test -v ./...
|
||||
go test -race ./...
|
||||
```
|
||||
|
||||
### Package API
|
||||
|
||||
```go
|
||||
import "github.com/luxfi/staking"
|
||||
|
||||
// Parse a raw ASN.1 certificate (extracts public key only)
|
||||
cert, err := staking.ParseCertificate(rawBytes)
|
||||
|
||||
// Load TLS certificate from PEM files
|
||||
tlsCert, err := staking.LoadTLSCertFromFiles(keyPath, certPath)
|
||||
|
||||
// Load TLS certificate from PEM byte slices
|
||||
tlsCert, err := staking.LoadTLSCertFromBytes(keyBytes, certBytes)
|
||||
|
||||
// Verify a signature against a certificate
|
||||
err := staking.CheckSignature(cert, message, signature)
|
||||
|
||||
// Validate RSA public key constraints
|
||||
err := staking.ValidateRSAPublicKeyIsWellFormed(rsaPubKey)
|
||||
```
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Certificate type
|
||||
|
||||
The `Certificate` type is an alias for `ids.Certificate` from `github.com/luxfi/ids`. It holds:
|
||||
- `Raw []byte` -- the original ASN.1 DER-encoded certificate
|
||||
- `PublicKey crypto.PublicKey` -- the extracted RSA or ECDSA public key
|
||||
|
||||
This is intentionally minimal. Unlike `x509.Certificate`, it does not parse validity periods, extensions, subject names, or issuer chains. Lux staking certificates are self-signed and identified solely by their public key.
|
||||
|
||||
### Supported algorithms
|
||||
|
||||
| Algorithm | Key Size | Use Case |
|
||||
|-----------|----------|----------|
|
||||
| RSA-PKCS1v15 + SHA-256 | 2048 or 4096 bit | Signature verification |
|
||||
| ECDSA + SHA-256 | P-256 | Signature verification |
|
||||
|
||||
### RSA key validation rules
|
||||
|
||||
The `ValidateRSAPublicKeyIsWellFormed` function enforces:
|
||||
1. Modulus must be positive
|
||||
2. Modulus bit length must be exactly 2048 or 4096
|
||||
3. Modulus must be odd (not even)
|
||||
4. Public exponent must be exactly 65537
|
||||
|
||||
### ASN.1 parsing
|
||||
|
||||
The package defines OIDs for the two supported public key algorithms:
|
||||
- RSA: `1.2.840.113549.1.1.1` (`oidPublicKeyRSA`)
|
||||
- ECDSA: `1.2.840.10045.2.1` (`oidPublicKeyECDSA`)
|
||||
|
||||
Parsing uses `golang.org/x/crypto/cryptobyte` for efficient ASN.1 traversal without full x509 overhead.
|
||||
|
||||
### File structure
|
||||
|
||||
```
|
||||
staking/
|
||||
asn1.go # OID definitions, SHA-256 init check
|
||||
certificate.go # Certificate type alias (ids.Certificate)
|
||||
parse.go # ParseCertificate, parsePublicKey, ValidateRSAPublicKeyIsWellFormed
|
||||
tls.go # LoadTLSCertFromBytes, LoadTLSCertFromFiles
|
||||
verify.go # CheckSignature (RSA + ECDSA verification)
|
||||
go.mod # Module definition
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `github.com/luxfi/constants` | v1.4.2 | MaxCertificateLen size constant |
|
||||
| `github.com/luxfi/ids` | v1.2.9 | Certificate type definition |
|
||||
| `golang.org/x/crypto` | v0.47.0 | ASN.1 cryptobyte parsing |
|
||||
|
||||
### Error types
|
||||
|
||||
All errors are package-level variables with the `staking:` prefix:
|
||||
|
||||
| Error | Cause |
|
||||
|-------|-------|
|
||||
| `ErrCertificateTooLarge` | Certificate exceeds 2 KiB |
|
||||
| `ErrMalformedCertificate` | Invalid ASN.1 SEQUENCE at top level |
|
||||
| `ErrMalformedTBSCertificate` | Invalid TBS certificate structure |
|
||||
| `ErrMalformedSPKI` | Invalid Subject Public Key Info |
|
||||
| `ErrInvalidRSAPublicKey` | Nil RSA public key |
|
||||
| `ErrUnsupportedRSAModulusBitLen` | RSA modulus not 2048 or 4096 bits |
|
||||
| `ErrRSAModulusIsEven` | RSA modulus is even (invalid) |
|
||||
| `ErrUnsupportedRSAPublicExponent` | RSA exponent is not 65537 |
|
||||
| `ErrFailedUnmarshallingEllipticCurvePoint` | Invalid ECDSA P-256 point |
|
||||
| `ErrUnknownPublicKeyAlgorithm` | Neither RSA nor ECDSA |
|
||||
| `ErrUnsupportedAlgorithm` | CheckSignature: unknown key type |
|
||||
| `ErrECDSAVerificationFailure` | ECDSA signature did not verify |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **"certificate length is greater than 2048"**: The raw DER certificate exceeds 2 KiB. Lux staking certificates should be minimal self-signed certs. Check if extra extensions or long subject names are inflating the size.
|
||||
- **"unsupported RSA modulus bitlen"**: Only 2048-bit and 4096-bit RSA keys are accepted. Generate a new key pair with the correct size.
|
||||
- **"unknown public key algorithm"**: The certificate uses an algorithm other than RSA or ECDSA P-256. Lux only supports these two for staking identity.
|
||||
- **"ECDSA verification failure"**: The message or signature bytes do not match. Ensure the message is the exact bytes that were signed, not a hex-encoded or base64-encoded representation.
|
||||
- **LoadTLSCertFromFiles argument order**: The function takes `(keyPath, certPath)`, not `(certPath, keyPath)`. This matches the convention where the private key comes first.
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux-genesis` -- Genesis configuration that establishes initial validators and their certificates
|
||||
- `lux-kms` -- Key management for generating and storing staking key pairs
|
||||
- `lux-node` -- The node implementation that uses staking certificates for P2P identity
|
||||
|
||||
---
|
||||
|
||||
Last Updated: 2026-03-13
|
||||
@@ -0,0 +1,206 @@
|
||||
---
|
||||
title: "Lux ZapDB - Embeddable Key-Value Store"
|
||||
description: "ZapDB serves as the low-level storage engine across the Lux blockchain stack and is the backend for the Hanzo ORM's ZAP driver."
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
ZapDB is a fork of BadgerDB (dgraph-io/badger) published as `github.com/luxfi/zapdb/v4`. It is an embeddable, persistent, high-performance key-value database written in pure Go. ZapDB uses an LSM tree with a separate value log (WiscKey design) to achieve high read and write throughput simultaneously, with concurrent ACID transactions using serializable snapshot isolation (SSI).
|
||||
|
||||
ZapDB serves as the low-level storage engine across the Lux blockchain stack and is the backend for the Hanzo ORM's ZAP driver.
|
||||
|
||||
## When to use
|
||||
|
||||
- Embedding a persistent key-value store in Go services on the Lux stack
|
||||
- Storing blockchain state, transaction indices, or consensus metadata
|
||||
- Building applications that need ACID transactions with SSI guarantees
|
||||
- Needing an SSD-optimized pure Go database (no CGo required)
|
||||
- Using the Hanzo ORM with the ZAP backend driver
|
||||
|
||||
## Hard requirements
|
||||
|
||||
1. ALWAYS import `github.com/luxfi/zapdb/v4` -- NEVER `github.com/dgraph-io/badger`
|
||||
2. Go packages stay at v4.x.x; v4.0.0 and v4.3.0 are retracted
|
||||
3. The Go package name is `badger` (import path ends with zapdb/v4, but package is `badger`)
|
||||
4. Close the DB with `db.Close()` to prevent data corruption
|
||||
5. All mutations must happen inside transactions (`db.Update` or `db.NewTransaction`)
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Repo | `github.com/luxfi/zapdb` |
|
||||
| Module | `github.com/luxfi/zapdb/v4` |
|
||||
| Go version | 1.26.1 |
|
||||
| Package name | `badger` |
|
||||
| License | Apache 2.0 |
|
||||
| Retracted | v4.0.0, v4.3.0 |
|
||||
| CLI binary | `badger` (in `badger/` subdir) |
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Directory structure
|
||||
|
||||
```
|
||||
luxfi/zapdb/
|
||||
db.go -- DB type: Open, Close, transaction management
|
||||
txn.go -- Transaction: Get, Set, Delete, Commit, Discard
|
||||
iterator.go -- Iterator: prefix scan, seek, key-value access
|
||||
batch.go -- WriteBatch for bulk writes
|
||||
stream.go -- Concurrent key-range streaming
|
||||
merge.go -- MergeOperator for read-modify-write
|
||||
backup.go -- Protobuf backup/restore
|
||||
errors.go -- All exported error sentinels
|
||||
options.go -- Options with builder pattern (WithX methods)
|
||||
compaction.go -- LSM compaction logic
|
||||
value.go -- Value log management
|
||||
levels.go -- LSM level management
|
||||
memtable.go -- In-memory table (skiplist-backed)
|
||||
key_registry.go -- Encryption key rotation
|
||||
options/ -- CompressionType, ChecksumVerificationMode
|
||||
table/ -- SSTable implementation
|
||||
skl/ -- Skiplist (memtable backend)
|
||||
pb/ -- Protobuf definitions
|
||||
fb/ -- FlatBuffers definitions
|
||||
y/ -- Internal utilities (WaterMark, Slice)
|
||||
trie/ -- Trie data structure
|
||||
badger/ -- CLI tool
|
||||
main.go -- CLI entry point (jemalloc, pprof)
|
||||
cmd/ -- CLI subcommands
|
||||
backup.go, restore.go, info.go, flatten.go,
|
||||
bench.go, stream.go, rotate.go, bank.go
|
||||
```
|
||||
|
||||
### Basic usage
|
||||
|
||||
```go
|
||||
import badger "github.com/luxfi/zapdb/v4"
|
||||
|
||||
// Open database
|
||||
opts := badger.DefaultOptions("/tmp/zapdb")
|
||||
db, err := badger.Open(opts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
// Write
|
||||
err = db.Update(func(txn *badger.Txn) error {
|
||||
return txn.Set([]byte("key"), []byte("value"))
|
||||
})
|
||||
|
||||
// Read
|
||||
err = db.View(func(txn *badger.Txn) error {
|
||||
item, err := txn.Get([]byte("key"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return item.Value(func(val []byte) error {
|
||||
fmt.Printf("value: %s\n", val)
|
||||
return nil
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```go
|
||||
// Default: good for most use cases
|
||||
opts := badger.DefaultOptions("/path/to/data")
|
||||
|
||||
// LSM-only mode (values collocated with keys, less disk usage)
|
||||
opts := badger.LSMOnlyOptions("/path/to/data")
|
||||
|
||||
// In-memory mode (no persistence)
|
||||
opts := badger.DefaultOptions("").WithInMemory(true)
|
||||
|
||||
// Common tuning
|
||||
opts = opts.
|
||||
WithSyncWrites(true). // Durability on hard reboot
|
||||
WithCompression(options.ZSTD). // ZSTD compression
|
||||
WithBlockCacheSize(512 << 20). // 512MB block cache
|
||||
WithNumCompactors(4). // Compaction workers
|
||||
WithEncryptionKey(key). // AES encryption (16/24/32 bytes)
|
||||
WithReadOnly(true) // Read-only mode
|
||||
```
|
||||
|
||||
### Key features
|
||||
|
||||
| Feature | Details |
|
||||
|---------|---------|
|
||||
| Design | LSM tree + value log (WiscKey paper) |
|
||||
| Transactions | ACID with SSI, concurrent read-write |
|
||||
| Compression | Snappy (default), ZSTD, None |
|
||||
| Encryption | AES-128/192/256 with key rotation |
|
||||
| TTL | Per-key expiration via `txn.SetEntry` |
|
||||
| Versioning | Multi-version (configurable versions per key) |
|
||||
| Streaming | Concurrent iteration over key ranges |
|
||||
| Merge ops | Read-modify-write via MergeOperator |
|
||||
| Backup | Protobuf-encoded incremental backup/restore |
|
||||
| Pure Go | No CGo dependency |
|
||||
|
||||
### Error sentinels
|
||||
|
||||
Key errors to handle (from `errors.go`):
|
||||
|
||||
- `ErrKeyNotFound` -- Key does not exist
|
||||
- `ErrConflict` -- Transaction conflict, retry
|
||||
- `ErrTxnTooBig` -- Too many writes in one transaction
|
||||
- `ErrReadOnlyTxn` -- Write attempted in read-only txn
|
||||
- `ErrDiscardedTxn` -- Reusing a discarded transaction
|
||||
- `ErrDBClosed` -- Operation after Close
|
||||
- `ErrEmptyKey` -- Empty key not allowed
|
||||
- `ErrBlockedWrites` -- Writes blocked during DropAll
|
||||
|
||||
### CLI tool
|
||||
|
||||
```bash
|
||||
cd badger && go install .
|
||||
|
||||
badger backup --dir /path/to/db # Create backup
|
||||
badger restore --dir /path/to/db # Restore from backup
|
||||
badger info --dir /path/to/db # Show DB info
|
||||
badger flatten --dir /path/to/db # Flatten LSM tree
|
||||
badger stream --dir /path/to/db # Stream keys
|
||||
badger rotate --dir /path/to/db # Rotate encryption key
|
||||
badger bench read --dir /path/to/db # Read benchmark
|
||||
badger bench write --dir /path/to/db # Write benchmark
|
||||
```
|
||||
|
||||
### Build and test
|
||||
|
||||
```bash
|
||||
# Build
|
||||
go build ./...
|
||||
|
||||
# Run tests (requires jemalloc for full test suite on some platforms)
|
||||
make test # Runs test.sh
|
||||
go test ./... # Without jemalloc
|
||||
|
||||
# Build CLI
|
||||
make badger # Compiles badger binary (downloads jemalloc if needed)
|
||||
|
||||
# Dependencies
|
||||
make dependency # Install build deps (apt-based)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| `ErrConflict` on commit | Read-write conflict in concurrent txns | Retry the transaction |
|
||||
| `Value log GC didn't result in cleanup` | No reclaimable space | Normal; GC only runs when beneficial |
|
||||
| Slow reads | Block cache too small | Increase `WithBlockCacheSize` |
|
||||
| High disk usage | Value log not GC'd | Call `db.RunValueLogGC(0.5)` periodically |
|
||||
| `ErrTruncateNeeded` | Corrupt value log after crash | Open with truncate option or restore from backup |
|
||||
| Data directory locked | Another process has DB open | Close other instance or use `WithBypassLockGuard` (unsafe) |
|
||||
| Memory usage too high | Large memtables or cache | Reduce `WithMemTableSize` and `WithBlockCacheSize` |
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `lux/lux-database.md` -- Database layer overview for the Lux stack
|
||||
- `lux/lux-node.md` -- Lux node (uses ZapDB for state storage)
|
||||
|
||||
---
|
||||
|
||||
Last Updated: 2026-03-13
|
||||
@@ -1,25 +1,33 @@
|
||||
{
|
||||
"title": "Lux Skills Reference",
|
||||
"description": "Comprehensive reference for all 60 Lux skills",
|
||||
"description": "Comprehensive reference for all 72+ Lux skills",
|
||||
"icon": "BookOpen",
|
||||
"pages": [
|
||||
"index",
|
||||
"---Core Blockchain---",
|
||||
"lux-node",
|
||||
"lux-evm",
|
||||
"lux-coreth",
|
||||
"lux-consensus",
|
||||
"lux-dex",
|
||||
"lux-vm",
|
||||
"lux-spacesvm",
|
||||
"lux-session",
|
||||
"lux-p2p",
|
||||
"lux-genesis",
|
||||
"lux-staking",
|
||||
"---Cryptography & Security---",
|
||||
"lux-crypto",
|
||||
"lux-fhe",
|
||||
"lux-fhevm",
|
||||
"lux-fhe-coprocessor",
|
||||
"lux-lattice",
|
||||
"lux-corona",
|
||||
"lux-safe",
|
||||
"lux-precompile",
|
||||
"lux-mpc",
|
||||
"lux-hsm",
|
||||
"lux-kms",
|
||||
"---Threshold Signatures---",
|
||||
"lux-threshold",
|
||||
"lux-teleport",
|
||||
@@ -29,6 +37,8 @@
|
||||
"lux-netrunner",
|
||||
"lux-build",
|
||||
"lux-plugins",
|
||||
"lux-lpm",
|
||||
"lux-ledger",
|
||||
"---Wallets---",
|
||||
"lux-wallet",
|
||||
"lux-wwallet",
|
||||
@@ -37,6 +47,7 @@
|
||||
"lux-finance",
|
||||
"---DeFi & Markets---",
|
||||
"lux-exchange",
|
||||
"lux-exchange-api",
|
||||
"lux-market",
|
||||
"lux-liquid",
|
||||
"lux-oracle",
|
||||
@@ -49,6 +60,7 @@
|
||||
"---GPU & Acceleration---",
|
||||
"lux-gpu",
|
||||
"lux-accel",
|
||||
"lux-cpp",
|
||||
"---Infrastructure & Ops---",
|
||||
"lux-universe",
|
||||
"lux-stack",
|
||||
@@ -56,6 +68,7 @@
|
||||
"lux-operator",
|
||||
"lux-monitoring",
|
||||
"lux-database",
|
||||
"lux-zapdb",
|
||||
"lux-migrate",
|
||||
"---AI & Compute---",
|
||||
"lux-ai",
|
||||
|
||||
Reference in New Issue
Block a user