5 Commits
Author SHA1 Message Date
zeekay e1dd80f342 base serves /v1, and these four said /api
The mount prefix has been /v1 since apis/base.go took BASE_API_PREFIX.
The auth comments, the compose examples, the dev gateway routes and the
smoke script still named PocketBase's prefix — the smoke script actually
called it, so every request in it 404d.
2026-08-23 14:52:41 -07:00
zeekayandHanzo Dev c08a07d026 fix(sqlite): one driver — route all SQLite onto hanzoai/sqlite, drop modernc
Docker / test (push) Waiting to run
Docker / docker (push) Blocked by required conditions
gitleaks / scan (push) Waiting to run
release / build (push) Waiting to run
Base double-registered the database/sql "sqlite" driver when embedded in a CGO
consumer that needs SQLCipher (commerce's per-tenant money DBs): hanzoai/sqlite
registers "sqlite"→mattn under cgo while base's own `_ "modernc.org/sqlite"`
blank imports register "sqlite"→modernc → `panic: sql: Register called twice for
driver sqlite` at init. Fix: base imports ONLY github.com/hanzoai/sqlite (the
canonical dual-backend driver) and never modernc directly. `go mod why
modernc.org/sqlite` now resolves ONLY transitively via hanzoai/sqlite.

- core/db_connect.go + store/multitenant.go: open via
  sqlite.PragmaDSN(path, sqlite.DefaultPragmas) — the ONE pragma set encoded in
  the ACTIVE backend's DSN syntax, so busy_timeout+WAL actually apply under both
  modernc (!cgo) and mattn (cgo). Kills the duplicated modernc-only DSN string.
- core/base_network.go: resolve the raw driver conn via
  sqlite.CommitHookRegisterer (build-tagged; bridges mattn func() int / modernc
  func() int32 to one CommitHookFn). network/* stays backend-agnostic — its own
  HookRegisterer + test fakes are unchanged.
- hack/pitr-restore.go, network/pitr_test.go, tools/search/provider_test.go:
  blank-import hanzoai/sqlite instead of modernc.
- Removed modernc_versions_check.go + its OnBootstrap binding: the modernc/libc
  pairing is hanzoai/sqlite's concern now, not base's (base no longer pins it).
- go.mod: require hanzoai/sqlite v0.2.1; modernc now // indirect; replace
  mattn/go-sqlite3 => v1.14.47 defeats the v2.0.3+incompatible poison (beego)
  under cgo. Encrypted-at-rest (SQLCipher via mattn under cgo) is untouched.

Verified: CGO=0 binary builds; ./core/... ./network/... ./store/...
./tools/search/... tests GREEN; CGO=1 (commerce mode, -tags cloud) compiles;
zero direct modernc imports remain.

Co-authored-by: Hanzo Dev <[email protected]>
2026-07-03 21:13:48 -07:00
zeekay 3fa9b026a3 merge: feat/compose-network 2026-06-01 16:06:03 -07:00
zeekay 2f97904e71 feat(network): archive-only validator — S3/GCS cold storage + PITR
Phase 1 of the Base Network archive tier: witness-only validator
appends quasar-finalised WAL frames to object storage as per-shard
.lbn segments. PITR replay recovers SQLite state at any txseq.

Scope:
- network/archive.go         Archive interface + URL dispatch
                             (s3://…|gs://…|off)
- network/segment.go         .lbn encoder/decoder. Magic "LBN1",
                             CRC32-IEEE footer. Forwards-compat via
                             magic bump on any format change.
- network/archive_writer.go  Goroutine-per-shard buffer with
                             size+time flush, exponential-backoff
                             retry against RetryDeadline, and
                             never-drop re-buffering on timeout.
- network/archive_s3.go      minio-go/v7 backend. Works against AWS
                             S3 and any MinIO-protocol endpoint —
                             hanzoai/s3 self-hosted is one such.
                             Static creds from AWS_ACCESS_KEY_ID /
                             SECRET, else IAM/IRSA via
                             credentials.NewIAM().
- network/archive_gcs.go     cloud.google.com/go/storage backend.
                             ADC by default; Workload Identity in
                             k8s, GOOGLE_APPLICATION_CREDENTIALS
                             for local dev.
- hack/pitr-restore.go       CLI replay tool. Usage:
                               pitr-restore --archive gs://…/svc \
                                 --shard <id> --to-seq <n> \
                                 --out file.db
                             Writes a SQLite file with one row per
                             frame; Base's real apply path replays
                             into the schema.

Segment layout: "LBN1" | shardIDLen u16 | shardID | startSeq u64 |
frameCount u32 | (frameLen u32 | frameBytes){N} | crc32 footer.
PQ signature on each frame is preserved byte-for-byte — the
archive never re-signs.

Storage key layout per docs/NETWORK.md:
  <svc>/<shard>/<16-digit-seq-prefix>/<20-digit-start-seq>.lbn

Config knobs (defaults): SegmentTargetBytes=8MiB, FlushInterval=10s,
RetryDeadline=5m. lagBytes gauge reflects backlog awaiting flush.

Metrics binding: BindArchiveMetrics(*Metrics) wires archive callbacks
onto the core Metrics.ArchiveLagBytes gauge + ApplyErrors counter.
Archive stays decoupled from prometheus/client_golang.

Tests: 12 archive-specific tests pass alongside the 20 network tests
from Phase 1 (32/32 total). Integration tests (`//go:build integration`)
cover 1000-frame S3 round-trip against local MinIO and PITR CLI
restore of 100 frames at cut-off seq 50 producing a SQLite file
with exactly 50 rows.
2026-04-22 13:41:51 -07:00
zeekay b475a4963e feat(compose): base-network local HA dev stack + smoke
compose.network.yml brings up three ghcr.io/hanzoai/base:dev pods in a
quasar quorum (N=3, k=2) plus a gateway using the base-network upstream
type (shipped in hanzo/gateway). Ports 18080 (gateway), 18090-18092
(direct pod access). Kill any one base-* to exercise quorum failure
tolerance.

hack/gateway.network.json configures the gateway side of the stack:
three endpoints (GET/POST/DELETE on /api/{path}) backed by the
base-network upstream with shard_key_source=header:X-User-Id so the
dev loop doesn't require IAM/Casdoor.

hack/smoke-compose.sh is the end-to-end check: spin up, write a row
through the gateway, kill base-b, read it back, tear down. Exits 0 on
success; exit 2 documents the "Agent #1 /-/base/members not yet in
image" condition so the script is runnable pre-merge.
2026-04-22 13:36:09 -07:00