mirror of
https://github.com/hanzoai/replicate.git
synced 2026-10-30 18:23:20 +00:00
zeekayandGitHub
5f0d4bb8d5
* fix(replicate): fail-closed age encryption (no plaintext LTX to S3)
Replicate silently wrote plaintext LTX1 to the destination whenever an age
recipient was unset -- fleet-wide plaintext SQLite backups. This makes
encryption fail-closed: when a replica is told it must encrypt, the absence
of a recipient REFUSES the write (ErrEncryptionRequired) instead of
downgrading to plaintext.
- Replica.RequireEncryption + guards in Start(), WriteLTXFile(), and Compactor
.Compact() -- every write path funnels through a fail-closed check.
- DB.init() fails closed at the earliest point (before the "no db file yet"
early return) so a misconfigured deployment surfaces the error on its first
Sync, loudly, every interval -- never silently no-ops until data appears.
- DB.Snapshot() now routes through Replica.WriteLTXFile (was raw Client), so
snapshots are age-encrypted like L0 sync; previously a plaintext bypass.
- internal.ExtractLTXTimestamp: sniff the age v1 intro so encrypted objects
fall back to write time instead of every backend failing to parse an LTX
header out of ciphertext. All 8 replica clients use it.
- AutoReplicate / CLI: RequireEncryption defaults on; opt out only via explicit
REPLICATE_ALLOW_PLAINTEXT / age.allow-plaintext for non-sensitive dev targets.
Tests: replica_failclosed_test.go (unit: write/start/compact fail closed;
encrypt+round-trip with recipient) and replicate_e2e_encrypt_test.go (full DB
stack: every destination object is age ciphertext across L0 sync AND the
snapshot path; live-writer restore round-trip; DB sync fails closed).
Re-applies the fix from #6 onto a fresh branch off main (#6's branch shares no
git history with main and cannot merge). age bumped 1.4.0->1.6.0 to match the
platform (cloud) and resolve main's broken [email protected] go.sum checksum; go mod
tidy prunes the orphaned prometheus indirect deps left by the metric migration.
* test(replicate): define testutil shim so the package test binary compiles
The luxfi/metric migration (commit 7c8ef84) dropped prometheus but left
db_internal_test.go referencing an undefined `testutil`, so `go test ./` did
not compile at all on main. This adds the minimal in-tree ToFloat64 shim
(luxfi/metric Counter/Gauge expose Get() float64) and drops the now-unused
metric import, unblocking the package test binary.
NOTE: this only fixes COMPILATION. The metric value-assertion tests
(TestDB_*Metrics) remain pre-existing RED because luxfi/metric's
WithLabelValues/With are non-memoizing (proven: vec.WithLabelValues("x").Set(5)
then vec.WithLabelValues("x").Get() == 0). That is an orthogonal, incomplete
metric-migration defect (also affects runtime aggregation at db.go checkpoint
Inc) and must be completed in a dedicated metric PR -- not folded into the
fail-closed security fix. Identical failures occur with or without encryption.
* docs(deps): justify luxfi/age v1.4.0->v1.6.0 bump (forced by broken main go.sum)
* fix(ci): unpoison luxfi/zap + run root metric tests under -tags metrics
Two pre-existing main breakages surfaced by CI (not by the fail-closed
change):
1. luxfi/zap v0.8.1's tag was re-published upstream, so its go.sum hash
no longer verifies (SECURITY ERROR on every job). Bump to the latest
patch in-minor, v0.8.12 (sumdb-verified) — same remedy as the age bump,
no lazy major.
2. luxfi/metric defaults to a NO-OP registry (//go:build !metrics); the
real in-process registry is behind `metrics`. `go test .` ran without
it, so every WithLabelValues returned a throwaway gauge that reads 0 —
the 8 TestDB_*Metrics failures. Root cause is the build tag (not
'non-memoizing' as first guessed, nor a prod defect): the DB's cached
gauge holds the value; only re-derivations under the no-op build miss.
Run the root suite with -tags metrics; passes 8/8 (27s, full package).
* revert(ci): drop -tags metrics test kludge
Prod builds cmd/replicate without -tags metrics (Dockerfile), so the
metric registry is no-op in production regardless; adding the tag only to
CI tests validated a mode prod never runs. The 8 TestDB_*Metrics failures
are a pre-existing main issue (luxfi/metric's no-op default), tracked
separately — not papered over with a test-only build tag. The essential
fix in this PR stands: luxfi/zap v0.8.1->v0.8.12 (re-published-tag go.sum
poison). One way, no kludge.
14 lines
638 B
Go
14 lines
638 B
Go
package replicate
|
|
|
|
// testutil shims the single prometheus/client_golang/prometheus/testutil helper
|
|
// (ToFloat64) still used by the internal metric assertions after the
|
|
// prometheus/client_golang → luxfi/metric migration (commit 8521ab5) dropped the
|
|
// prometheus dependency. luxfi/metric's Counter and Gauge both expose Get()
|
|
// float64, so reading a single-series metric value needs no external package.
|
|
var testutil = metricTestutil{}
|
|
|
|
type metricTestutil struct{}
|
|
|
|
// ToFloat64 returns the current value of a single-series counter or gauge.
|
|
func (metricTestutil) ToFloat64(m interface{ Get() float64 }) float64 { return m.Get() }
|