zeekay 56077de671 feat(tasks): v3.9.0 — distributed engine: per-NS SQLite shards + PQ consensus + sticky routing ...

Architecture
  One SQLite shard per (org, namespace) under <DataDir>/<org>/<ns>.db
  with WAL mode, single writer, 5s busy timeout. Replaces the prior
  shared memdb/zapdb store. Shard manager opens/caches/idle-evicts;
  prefix scans, namespace isolation, and cross-restart durability all
  proven by tests.

Replication (post-quantum, native)
  New pkg/tasks/replication/ exposes a Replicator interface with two
  drivers:
    local  — single-node passthrough (default for embedded/dev)
    quasar — wraps github.com/luxfi/consensus NewPQ (BLS + Ringtail
             dual-threshold). On every WAL-affecting op the shard
             builds a Frame, calls Propose, awaits Decision, applies
             on Accept. Followers receive accepted frames via the
             same Subscribe path so every node converges.

Sticky task-queue routing
  New pkg/tasks/routing/ — rendezvous-style smallest-hash leader
  election keyed by SHA-256(shard|taskQueue|seed). OnChange fan-out
  re-hashes on cluster membership change. Dispatcher consults Router
  before accepting subscriptions and forwards non-leader requests.

Namespace migration
  New pkg/tasks/migration/ — lock → checkpoint → copy → unlock with
  a job lifecycle (pending|locking|copying|replaying|released|failed).
  CLI: `tasksd migrate --org <id> --namespace <name> --to <node>`.
  HTTP: POST /v1/tasks/namespaces/{ns}/migrate.

New env / flags
  --data / TASKSD_DATA_DIR        shard root (./tasks-data)
  --replicator / TASKSD_REPLICATOR local | quasar (default local)
  --node-id / TASKSD_NODE_ID      stable id (required for quasar)
  --validators / TASKSD_VALIDATORS CSV host:port
  --validator-key / TASKSD_VALIDATOR_KEY (reserved for PQ key path)
  TASKSD_STORE                    sqlite | memory (default sqlite)

New routes
  GET  /v1/tasks/cluster
  GET  /v1/tasks/cluster/health
  POST /v1/tasks/namespaces/{ns}/migrate

UI bundle (synced)
  Cluster page with status / validators / shards / leader-map.
  NamespaceDetail "Migrate to node" button gated by
  canWriteNamespace + cluster.enabled. MigrationDialog enforces
  typed-name confirmation. Sidebar Cluster nav hidden when running
  single-node.

Tests
  +33 backend tests (store: 11, replication: 10, routing: 5,
  migration: 3, cluster routes: 2, store factory: 2). All packages
  PASS with -race; go vet clean.
  +13 UI tests (effectiveHealth: 6, canSubmitMigration: 4,
  useCluster: 3). 169 total UI tests, 1 pre-existing skip.

Migration v3.8.x → v3.9.0
  Fresh-data startup. zapdb backend dropped. Operators drain old
  cluster, snapshot for archival, deploy v3.9.0 with new --data
  directory. To enable HA, scale to 3+ replicas with --replicator
  quasar and matching --node-id / --validators.

2026-04-30 00:09:39 -07:00

..

main.go

feat(tasks): v3.9.0 — distributed engine: per-NS SQLite shards + PQ consensus + sticky routing

2026-04-30 00:09:39 -07:00