zeekay
a671dee75a
A version written into a manifest is written into every manifest that wants it, which is how eslint came to exist in a hundred and four forms across this estate and how a package came to resolve twice at run time. The catalog moves the number out of the manifest: a manifest names the package, the workspace names the version. Four are held below their latest because the newer release cannot be used, and the reason is recorded rather than the version alone: @noble/hashes 2 renamed every subpath the web3 stack imports, @splinetool/runtime 2 depends on a package that was never published, @dynamic-labs 5.4.1 the same, and react-resizable-panels 3 points its types condition at the bundle.
Utility Scripts
Utility scripts for Replicate testing and distribution.
Overview
This directory contains utility scripts for post-test analysis and packaging. All long-running soak tests have been migrated to Go integration tests in tests/integration/.
Note: For all soak tests (2-8 hours), see the Go-based test suite in tests/integration/. The bash soak tests have been migrated to Go for better maintainability and cross-platform support
Prerequisites
go build -o bin/replicate ./cmd/replicate
go build -o bin/replicate-test ./cmd/replicate-test
Available Scripts
analyze-test-results.sh
Post-test analysis tool for examining overnight test results.
./scripts/analyze-test-results.sh /tmp/replicate-overnight-<timestamp>
Analyzes:
- Test duration and timeline
- Compaction statistics
- Checkpoint frequency
- Error analysis and categorization
- Performance metrics
- Database growth patterns
- Replica file statistics
- Final validation results
Output:
- Analysis report written to
<test-dir>/analysis-report.txt - Console summary with key findings
- Error categorization and counts
- Performance statistics
What it Reports:
- Total test duration
- Number of compactions by interval
- Checkpoint count and frequency
- Error types and severity
- Database size growth
- WAL file patterns
- Replica size and file counts
- Success/failure summary
Use Cases:
- Post-overnight-test analysis
- Comparing test runs
- Identifying performance trends
- Debugging test failures
- Documenting test results
setup-homebrew-tap.sh
Homebrew tap setup script for packaging and distribution.
./scripts/setup-homebrew-tap.sh
Purpose: Automates Homebrew tap setup for Replicate distribution. Not a test script per se, but part of the release process.
Usage
Analyzing Test Results
ls /tmp/replicate-overnight-* -dt | head -1
./scripts/analyze-test-results.sh $(ls /tmp/replicate-overnight-* -dt | head -1)
Test Duration Guide
| Duration | Use Case | Test Type | Expected Results |
|---|---|---|---|
| 5 minutes | CI/CD smoke test | Go integration tests | Basic functionality |
| 30 minutes | Short integration | Go integration tests | Pattern detection |
| 2-8 hours | Soak testing | Go soak tests (local only) | Full validation |
Note: All soak tests are now Go-based in
tests/integration/. See tests/integration/README.md for details on running comprehensive, self-hosted S3, and overnight S3 soak tests.
Monitoring and Debugging
Real-time Monitoring
All tests create timestamped directories in /tmp/:
LATEST=$(ls /tmp/replicate-* -dt | head -1)
tail -f $LATEST/logs/monitor.log
tail -f $LATEST/logs/replicate.log
Key Metrics to Watch
Database Growth:
- Should grow steadily
- WAL file size should cycle (grow, checkpoint, reset)
Replica Statistics:
- Snapshot count should increase over time
- LTX file count should grow then stabilize (compaction)
- Replica size should be similar to database size
Operations:
- Compactions should occur at scheduled intervals
- Checkpoints should happen regularly
- Sync operations should complete successfully
Errors:
- Should be minimal or zero
- Transient errors OK if recovered
- Persistent errors indicate issues
Common Issues
Test Fails to Start
Check binaries:
ls -la bin/replicate bin/replicate-test
Rebuild if needed:
go build -o bin/replicate ./cmd/replicate
go build -o bin/replicate-test ./cmd/replicate-test
S3 Test Fails
Verify credentials:
aws s3 ls s3://$S3_BUCKET/
Check environment variables:
echo $AWS_ACCESS_KEY_ID
echo $AWS_SECRET_ACCESS_KEY
echo $S3_BUCKET
High Error Counts
Check log for error details:
grep -i error /tmp/replicate-*/logs/replicate.log | head -20
Validation Fails
Compare databases manually:
sqlite3 /tmp/replicate-*/test.db "SELECT COUNT(*) FROM test_data"
sqlite3 /tmp/replicate-*/restored.db "SELECT COUNT(*) FROM test_data"
Stopping Tests Early
Go tests can be interrupted with Ctrl+C. They will cleanup gracefully via defer statements.
Test Artifacts
All tests create timestamped directories with comprehensive artifacts:
/tmp/replicate-overnight-<timestamp>/
├── logs/
│ ├── replicate.log # Replicate replication log
│ ├── load.log # Load generator log
│ ├── monitor.log # Real-time monitoring log
│ ├── populate.log # Initial population log
│ └── validate.log # Final validation log
├── test.db # Source database
├── test.db-wal # Write-ahead log
├── test.db-shm # Shared memory file
├── replica/ # Replica directory (file tests)
│ └── ltx/ # LTX files
└── restored.db # Restored database for validation
Integration with Go Tests
These utility scripts complement the Go integration test suite:
Test Locations:
tests/integration/→ All integration and soak tests (Go-based)cmd/replicate-test/scripts/→ Scenario and debugging tests (bash, being phased out)scripts/→ Utilities only (this directory)
Testing Workflow:
- Run quick integration tests during development
- Run full integration test suite before major changes
- Run soak tests (2-8h) locally before releases:
TestComprehensiveSoak,TestSelfHostedS3Soak,TestOvernightS3Soak - Analyze results with
analyze-test-results.sh
Related Documentation
- Go Integration Tests - Complete Go-based test suite including soak tests
- replicate-test CLI Tool - Testing harness documentation
- Scenario Test Scripts - Focused test scenarios
- S3 Retention Testing - S3-specific testing