Files
zeekayandHanzo Dev e0c8182042 store: shards may keep their rows on a shared SQL server
TASKSD_STORE=sql keeps every shard's rows in one table, tasks_kv, on the
hanzoai/sql server TASKSD_SQL_ADDR names (its ZAP listener), so a process running
the engine keeps nothing on disk and any process can take its place. A shard is
(principal, ns), the same value it always was; its rows are a fileKV in its own
SQLite file or an sqlKV in the table, behind one rows interface. Keys are bytea
compared as bytes, so a scan walks keys in exactly a shard file's order whatever
the server's collation; each value is sealed (AES-256-GCM) under the key a shard
file of the same tenant and namespace derives from the master, bound to its shard
and key, so the server holds only ciphertext. A shared store refuses to open
without a master key.

Copy moves every shard from one manager to another a page at a time, leaves the
destination holding the source's rows and no others, and proves each shard by
count and a SHA-256 over its rows in key order. SQLite stays the default; nothing
changes for a deployment that sets nothing.

Proved against a hanzoai/sql test server: scans equal a file shard's for UTF-8,
DEL and prefix-boundary keys; the server holds no plaintext; a crashed engine
recovers a workflow mid-activity from the table into a fresh data directory, the
activity running exactly once.

Co-authored-by: Hanzo Dev <[email protected]>
2026-10-03 19:03:47 -07:00

95 lines
3.6 KiB
AMPL

module github.com/hanzoai/tasks
go 1.27.1
retract (
v1.30.0
v1.26.1 // Contains retractions only.
v1.26.0 // Published accidentally.
)
require (
github.com/golang-jwt/jwt/v5 v5.3.1
github.com/hanzoai/authz v1.10.29
github.com/hanzoai/cek v0.2.1
github.com/hanzoai/namespace v1.2.0
github.com/hanzoai/sqlite v0.5.0
github.com/luxfi/consensus v1.36.3
github.com/luxfi/log v1.5.0
github.com/luxfi/zap v1.2.7
github.com/robfig/cron/v3 v3.0.1
github.com/valyala/fasthttp v1.70.0
github.com/zap-proto/http v0.3.5
github.com/zap-proto/zip v1.37.3
golang.org/x/time v0.15.0
)
require (
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/andybalholm/brotli v1.2.1 // indirect
github.com/cenkalti/backoff v2.2.1+incompatible // indirect
github.com/cloudflare/circl v1.6.3 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/gofiber/schema v1.7.1 // indirect
github.com/gofiber/utils/v2 v2.0.4 // indirect
github.com/google/uuid v1.6.1-0.20241114170450-2d3c2a9cc518 // indirect
github.com/gorilla/rpc v1.2.1 // indirect
github.com/grandcat/zeroconf v1.0.0 // indirect
github.com/hanzoai/csqlite v0.1.0 // indirect
github.com/hanzoai/sqlcipher v0.1.1 // indirect
github.com/holiman/uint256 v1.3.2 // indirect
github.com/klauspost/compress v1.18.6 // indirect
github.com/luxfi/accel v1.2.4 // indirect
github.com/luxfi/bft v0.1.5 // indirect
github.com/luxfi/cache v1.3.1 // indirect
github.com/luxfi/compress v0.1.1 // indirect
github.com/luxfi/concurrent v0.1.1 // indirect
github.com/luxfi/constants v1.6.2 // indirect
github.com/luxfi/container v0.2.1 // indirect
github.com/luxfi/crypto v1.20.2 // indirect
github.com/luxfi/database v1.21.1 // indirect
github.com/luxfi/geth v1.20.1 // indirect
github.com/luxfi/ids v1.3.2 // indirect
github.com/luxfi/math v1.5.1 // indirect
github.com/luxfi/math/big v0.1.0 // indirect
github.com/luxfi/mdns v0.1.1 // indirect
github.com/luxfi/metric v1.10.0 // indirect
github.com/luxfi/mock v0.1.1 // indirect
github.com/luxfi/p2p v1.22.1 // indirect
github.com/luxfi/pq v1.1.0 // indirect
github.com/luxfi/sampler v1.1.0 // indirect
github.com/luxfi/validators v1.3.1 // indirect
github.com/luxfi/version v1.0.1 // indirect
github.com/luxfi/warp v1.24.1 // indirect
github.com/mattn/go-colorable v0.1.15 // indirect
github.com/mattn/go-isatty v0.0.22 // indirect
github.com/miekg/dns v1.1.72 // indirect
github.com/mr-tron/base58 v1.3.0 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/philhofer/fwd v1.2.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/supranational/blst v0.3.16 // indirect
github.com/tinylib/msgp v1.6.4 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/zap-proto/fiber/v3 v3.2.1 // indirect
github.com/zap-proto/go v1.8.3 // indirect
github.com/zap-proto/mcp v1.0.5 // indirect
go.uber.org/mock v0.6.0 // indirect
golang.org/x/crypto v0.54.0 // indirect
golang.org/x/exp v0.0.0-20260529124908-c761662dc8c9 // indirect
golang.org/x/mod v0.37.0 // indirect
golang.org/x/net v0.56.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.40.0 // indirect
golang.org/x/tools v0.47.0 // indirect
gonum.org/v1/gonum v0.17.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
modernc.org/libc v1.72.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.51.0 // indirect
)