zeekay
a485f34a14
- All source files, configs, docs, CI renamed - Binary: replicate (not litestream) - Config: replicate.yml (litestream.yml as fallback) - Env: REPLICATE_* (LITESTREAM_* as fallback) - Socket: /var/run/replicate.sock - Image: ghcr.io/hanzoai/replicate - Module: github.com/hanzoai/replicate - Package name: replicate (Go), replicate-vfs (npm/pip/gem) - VFS: registered as "replicate", pragmas replicate_* - C exports: ReplicateVFSRegister, GoReplicate* - Windows service: Replicate - Systemd unit: replicate.service - Upstream attribution preserved (README, upgrade test scripts)
2.2 KiB
Analyze LTX file issues in Replicate. This command helps diagnose problems with LTX files, including corruption, missing files, and consistency issues.
First, understand the context:
- What error messages are being reported?
- Which storage backend is being used?
- Are there any eventual consistency issues?
Then perform the analysis:
-
Check LTX file structure: Look for corrupted headers, invalid page indices, or checksum mismatches in the LTX files.
-
Verify file continuity: Ensure there are no gaps in the TXID sequence that could prevent restoration.
-
Check compaction issues: Look for problems during compaction that might corrupt files, especially with eventually consistent storage.
-
Analyze page sequences: Verify that page numbers are sequential and the lock page at 1GB is properly skipped.
-
Review storage backend behavior: Check if the storage backend has eventual consistency that might cause partial reads during compaction.
Key files to examine:
db.go: WAL monitoring and LTX generationreplica_client.go: Storage interfacestore.go: Compaction logic- Backend-specific client in
s3/,gs/, etc.
Common issues to look for:
- "nonsequential page numbers" errors (corrupted compaction)
- "EOF" errors (partial file reads)
- Missing TXID ranges (failed uploads)
- Lock page at 0x40000000 not being skipped
Use the testing harness to reproduce:
./bin/replicate-test validate -source-db test.db -replica-url [URL]
Recovery Options
When analysis reveals corrupted or missing LTX files, two recovery mechanisms are available:
Manual recovery with replicate reset:
replicate reset /path/to/database.db
Clears local LTX state from the metadata directory. The database file is not modified.
Next sync creates a fresh snapshot. See cmd/replicate/reset.go for implementation.
Automatic recovery with auto-recover config:
dbs:
- path: /path/to/database.db
replicas:
- url: s3://bucket/path
auto-recover: true
When enabled, Replicate automatically resets local state when LTX errors are detected
during sync. Disabled by default. See replica.go (auto-recover logic) for implementation.