cloud holds no crypto: one derived key, no wrapping
cloud/cek is deleted and github.com/hanzoai/cek v0.2.0 is the whole of
encryption at rest. A database's key is DERIVED from the deployment's master
and the namespace that owns it — HKDF(master, "hanzo/cek/v1/" + ns + "/" +
subsystem) — so it is not generated, not wrapped, not stored and not rotated
in place.
What that deletes, and why each one was a hazard rather than a feature:
- the per-file DEK and its .dek sidecar. Key material beside a file is
key material that can go missing, and it did: the sidecar had to be
framed into every durable snapshot so a successor could open what it
restored, and a successor that received the database without it had an
unreadable store.
- rewrap, and the self-heal retry in OrgDB that called it. A derivation
that must be migrated is a derivation that can be half-migrated; that is
what took the git plane and the mirror engine down in production, and
with them every deploy.
- cek.Global / cek.Org / cek.Principal. There was cloud's name for an
entity (namespace) and cek's name for the same entity (Principal), with
nsPrincipal translating between them. Now the namespace IS what the key
is derived from, so a file and its key cannot name different things.
- cek.Exists and its sidecar probe. A store is a file; asking the
filesystem is os.Stat, at the one call site that asks.
- replication.go, 84 lines of unwired design commentary. Replication is
hanzoai/replicate over hanzoai/vfs.
Callers pass a DIRECTORY and a SUBSYSTEM NAME, never a path — cek renders
the path from the namespace itself. Three hand-rolled org→slug encoders go
with that: finance's orgPattern, treasury's tenantSlug and team's seg were
each a second answer to "which file holds this tenant's data", and the
treasury one needed a reserved slug to keep a tenant out of the house fund.
The system namespace is a different KIND, so no tenant string can render to
it however it is spelled.
New, and small:
basedb.Open is the ONE opener: cek plus the directory the file lives in.
cek does not create it, and on the pure-Go codec the database is written
back at CLOSE — so a missing parent does not fail the open, it loses the
data at the end. Stated once, beside the open, instead of in ~50 stores.
internal/devmaster keys a test binary. cek reads no environment, so a
process with no KMS mints its own master; one blank import per test
package says so, replacing seventeen near-identical TestMains that set
CLOUD_KMS_MASTER_KEY_REF for a reader that no longer exists.
Two consequences worth naming. A store that is OPEN has no file yet on the
pure-Go codec, so OrgStore.Has is the union of the open set and the disk,
and Each and Stored both go through it. And apps/iam never closed its
*sql.DB at all (orm's AdaptSQLDB borrows the handle; its Close is a
documented no-op), which on that codec means the identity store was never
written back — it now has a Shutdown, wired like every other subsystem's.
Databases written under the old wrapping will not open under this
derivation. That is expected: there is no migration, no fallback and no
version probe, because a second derivation tried on failure is exactly what
made the old binding unenforceable.
Also fixes six test-only KMS fakes that never gained DeleteSecret and two
missing imports in apps/kms — pre-existing at origin/main, and the reason
eight packages could not be test-verified at all.
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
This commit is contained in:
+1
-3
@@ -218,8 +218,7 @@ RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
|
||||
RUN --mount=type=cache,id=cloud-gomod-v4,target=/go/pkg/mod,sharing=locked \
|
||||
--mount=type=cache,id=cloud-gobuild-v4,target=/root/.cache/go-build,sharing=locked \
|
||||
CGO_ENABLED=0 go build \
|
||||
-ldflags="-s -w -X github.com/hanzoai/cloud.Version=${VERSION}" -o /cloud ./cmd/cloud && \
|
||||
CGO_ENABLED=0 go build -ldflags="-s -w" -o /cek-rewrap ./cmd/cek-rewrap
|
||||
-ldflags="-s -w -X github.com/hanzoai/cloud.Version=${VERSION}" -o /cloud ./cmd/cloud
|
||||
# The functional smoke prober (plugin/smoke) — a stdlib-only static binary shipped
|
||||
# alongside the host so the release gate can `docker exec` it against the freshly-
|
||||
# built image (and any deployment can be smoked via `docker run --entrypoint /smoke`).
|
||||
@@ -312,7 +311,6 @@ COPY --from=build /usr/share/zoneinfo /usr/share/zoneinfo
|
||||
COPY --from=build /etc/passwd /etc/passwd
|
||||
COPY --from=build /etc/group /etc/group
|
||||
COPY --from=build /cloud /cloud
|
||||
COPY --from=build /cek-rewrap /cek-rewrap
|
||||
COPY --from=build /smoke /smoke
|
||||
# The per-app plugin binaries, landing beside /cloud because that is where the host
|
||||
# looks: manifest.App.Plugin resolves dir(os.Executable())+"/<name>". Copying the
|
||||
|
||||
@@ -180,8 +180,7 @@ tidy` is stable. Do NOT commit a `go.work` here — it would flip the Dockerfile
|
||||
its `-mod=readonly` download step.
|
||||
|
||||
Test modes: `make test` is pure-Go (`CGO_ENABLED=0`). Encrypted-at-rest OrgDB
|
||||
tests (`cek`, `CLOUD_KMS_MASTER_KEY_REF` set) REQUIRE `CGO_ENABLED=1` +
|
||||
libsqlcipher (`cek/cek.go` refuses to encrypt in pure-Go); those run only in the
|
||||
tests REQUIRE `CGO_ENABLED=1` + libsqlcipher; those run only in the
|
||||
Dockerfile's dedicated `-tags libsqlite3` CGO stage, and fail under `make test`
|
||||
by design (kms, flags, x402, plugin/kmsreseal, finance). Bundle-embed
|
||||
tests (apps/tasks/ui) need `make deploy-ui` first (real bundle is gitignored).
|
||||
@@ -210,8 +209,7 @@ STILL DIVERGENT, and a decision for the owner rather than a patch: `go-unit`
|
||||
declares no `CLOUD_KMS_MASTER_KEY_REF`, while `make test` injects a dev key
|
||||
(`TEST_ENV`) precisely so the suite has one dev posture instead of a copy per
|
||||
package — and `hanzoai/ci` exports only GIT_TOKEN / S3 / registry creds into the
|
||||
step, never that key. So every cek-backed package still fails there for want of
|
||||
it (measured: `apps/code`, 6 tests, "cek: CLOUD_KMS_MASTER_KEY_REF is required").
|
||||
step, never that key.
|
||||
The fix is one of two shapes and both are policy: give CI the dev key, or route
|
||||
the step through the Makefile so the posture is declared once. Copying the key
|
||||
into `hanzo.yml` would make it two declarations, which is the drift above again.
|
||||
@@ -461,7 +459,7 @@ for the credentials of the app it is.**
|
||||
*service* credentials. The pod is the data-plane boundary; the app is the
|
||||
credential boundary.
|
||||
|
||||
Ordering is load-bearing: `cek` memoizes the master key on first use, so
|
||||
Ordering is load-bearing: a store opened before the master is installed fails, so
|
||||
`credz.Boot` runs before the first store opens (top of `Serve`, and again at the
|
||||
top of `BuildDeps` for callers that skip `Serve` — it is `sync.Once`). Installing
|
||||
a key any later loses to the cached "no key" while the log claims success, which
|
||||
@@ -636,9 +634,11 @@ package under `apps/<name>` that obeys these seams — nothing more.
|
||||
DataDir). These strings are directory names on live volumes and keys in live
|
||||
buckets, so a second implementation of them does not fail — it opens an empty
|
||||
database beside a real one. cloud keeps only the DOOR: `OrgNamespace` /
|
||||
`MustOrgNamespace` / `PlatformNamespace` in orgns.go, which is the one file
|
||||
allowed to build a namespace (`TestOnlyOrgnsBuildsANamespace` enforces it),
|
||||
plus `nsPrincipal`, which is a cek key derivation and not a name.
|
||||
`MustOrgNamespace` in orgns.go, the one file allowed to fold a value into an
|
||||
ENTITY's name (`TestOnlyOrgnsBuildsANamespace` enforces it). `namespace.System()`
|
||||
is outside that argument rather than an exception to it — it takes no input, so
|
||||
nothing can be folded into it, and a platform store says so where it opens. The
|
||||
key derivation is cek's, from that same name.
|
||||
|
||||
## Zero-downtime HA for per-org stores (rolling-upgrade safe)
|
||||
|
||||
@@ -3017,7 +3017,7 @@ create a single table in the shipped image. `terms` is keyed
|
||||
in every build lane. Verify any SQLite module against the production lane
|
||||
(`-tags "libsqlite3 sqlite_fts5"` + `-lsqlcipher`) before designing on it.
|
||||
|
||||
The store is `{DataDir}/index.db`, and a rename must carry the WHOLE family: cek
|
||||
The store is the `index` subsystem, and a rename must carry the WHOLE family: cek
|
||||
keeps the wrapped data key beside it as `<path>.dek`, so moving the `.db` alone
|
||||
strands the key and every document becomes undecryptable — data loss that presents
|
||||
as an empty index.
|
||||
@@ -3772,7 +3772,7 @@ address.
|
||||
stores open a concurrent read pool AND a serialized write pool on the same file, which
|
||||
needs the LIVE libsqlcipher codec — the pure-Go codec envelope is single-writer and
|
||||
cannot serve that shape. So `commerceMasterKey` gates on `sqlitedrv.CodecLinked()`, the
|
||||
same predicate `cek.EnsureDevKey` uses:
|
||||
the same predicate:
|
||||
|
||||
- codec linked (the image: `CGO_ENABLED=1 -tags "libsqlite3 sqlite_fts5"`) ⇒ inject
|
||||
cloud's master key; commerce encrypts, and its own `resolveMasterKey` still fails
|
||||
@@ -3812,49 +3812,38 @@ another's ledger. The specs are `describe.configure({mode:'serial'})` — not by
|
||||
preference, but because they all move the same balance and the suite is otherwise
|
||||
`fullyParallel`.
|
||||
|
||||
## Encryption at rest: cek is the gate, and it binds the owner
|
||||
## Encryption at rest: one key per database, derived
|
||||
|
||||
`cek.Open(principal, path)` is the ONE encryption-at-rest gate. The principal comes
|
||||
first because it is a question the caller must answer, not one it can forget:
|
||||
`cek.Global` for a platform store, `cek.Org(slug)` for a tenant's. `cek.User(id)` exists
|
||||
for the per-user partition, which has no store yet.
|
||||
`github.com/hanzoai/cek` is the whole of it, and cloud owns none of it:
|
||||
|
||||
If you add a store, open it through cek. A store inside the envelope has a `.dek`
|
||||
sidecar beside it; one outside does not, and this is the check worth running on any data
|
||||
dir — note it looks for the SIDECAR, because on a pure-Go build the codec envelope keys
|
||||
the file out of band and the database may not exist at that path at all:
|
||||
cek.SetMaster(k) // once, at boot, from KMS (credz does this)
|
||||
cek.Open(ns, subsystem, dir) // everywhere else
|
||||
|
||||
find $DATA_DIR -name '*.db' -printf '%P\n' | while read -r r; do
|
||||
printf '%-40s dek=%s\n' "$r" "$([ -f "$DATA_DIR/$r.dek" ] && echo yes || echo NO)"; done
|
||||
The key is DERIVED — `HKDF(master, "hanzo/cek/v1/" + ns + "/" + subsystem)` — so it
|
||||
is not generated, not wrapped, not stored and not rotated in place. There is no
|
||||
unwrap step, no rewrap step, no per-file key material to lose, no sidecar beside the
|
||||
database and no migration path to maintain. A database is born encrypted or it does
|
||||
not exist. Losing the master loses the data, which is the property you want from
|
||||
encryption at rest and the reason the master lives in KMS.
|
||||
|
||||
**The derivation.** The KEK binds (owner, file) and never the path, so a store survives a
|
||||
move but not a change of owner:
|
||||
`namespace` decides WHERE, from the same two values: `{dir}/orgs/{slug}/{sub}.db`
|
||||
for an org, `{dir}/orgs/_platform/{sub}.db` for the deployment's own. A caller
|
||||
therefore passes a DIRECTORY and a NAME, never a path — the file and its key cannot
|
||||
name different things.
|
||||
|
||||
tenant: KEK = HKDF(master, lp("org") || lp(slug || "/" || hex(fileID)))
|
||||
platform: KEK = HKDF(master, lp("global") || lp(hex(fileID)))
|
||||
**Open through `basedb.Open`, not `cek.Open`.** It is cek plus the one thing a file
|
||||
needs that a key does not: the directory it lives in. On the pure-Go codec the
|
||||
database is written back at CLOSE, so a missing parent does not fail the open — it
|
||||
loses the data at the end. `cloud.OrgDB` is `basedb.Open` plus the single-writer +
|
||||
WAL pragmas every per-org store shares.
|
||||
|
||||
The platform form is byte-identical to what every store on disk was written under, which
|
||||
`TestGlobalDerivationIsUnchanged` asserts against an independently written reference — so
|
||||
the platform fleet cannot be silently orphaned. Only tenant stores gained an owner.
|
||||
**A test binary keys itself.** cek reads no environment; a process with no KMS mints
|
||||
its own master. `import _ "github.com/hanzoai/cloud/internal/devmaster"` in one
|
||||
_test.go of the package says so, once, instead of a TestMain per package.
|
||||
|
||||
**Migration is an operation, not a fallback.** `cek.Rebind(from, to, path)` rewraps one
|
||||
sidecar; `cek.RebindOrgs(dataDir, platformSlug)` is the walk over `{DataDir}/orgs`. It
|
||||
rewrites no database page and never opens the file, so it is safe on a store too large to
|
||||
copy and a failure cannot corrupt data. Already-bound reports `ErrNotBound` and counts as
|
||||
skipped, so the walk converges rather than pretending to be a transaction; a sidecar that
|
||||
unwraps under NEITHER principal is a real error, because an operator must not read
|
||||
corruption as success.
|
||||
|
||||
There is deliberately no legacy path inside `Open`. A second derivation tried on failure
|
||||
would mean every open silently accepts two answers forever — which is exactly what made
|
||||
the old binding unenforceable.
|
||||
|
||||
⚠️ **Deploy order.** A volume written before the binding must be rebound before its
|
||||
tenants can open their stores. Run `RebindOrgs` against the data dir, then start.
|
||||
|
||||
**IAM's store is `iam/global.db`** and opens through cek like everything else. It is named
|
||||
for its principal partition, not for a version — it previously opened through
|
||||
`iamserver.OpenSQLite`, which has no key to give it, and sat in plaintext.
|
||||
**A store that is open has no file yet** on the pure-Go codec, so "does this org
|
||||
have a store" is the union of the open set and the disk — `OrgStore.Has`, which
|
||||
`Each` and `Stored` both go through.
|
||||
|
||||
**Still outside the envelope:** `tasks/<org>/<namespace>.db`. `hanzoai/tasks`'s
|
||||
`EmbedConfig` has no key field, so that is an upstream change.
|
||||
|
||||
@@ -11,13 +11,12 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
auditstore "github.com/hanzoai/cloud/audit"
|
||||
"github.com/hanzoai/cloud/apps/admin/core"
|
||||
auditstore "github.com/hanzoai/cloud/audit"
|
||||
luxlog "github.com/luxfi/log"
|
||||
fiber "github.com/zap-proto/fiber/v3"
|
||||
"github.com/zap-proto/zip"
|
||||
@@ -27,8 +26,7 @@ import (
|
||||
// returns the store + a request helper. Only the audit routes are mounted here.
|
||||
func mountWithStore(t *testing.T) (*auditstore.Recorder, func(method, path string, hdr map[string]string) (*http.Response, []byte)) {
|
||||
t.Helper()
|
||||
path := filepath.Join(t.TempDir(), "audit.db")
|
||||
rec, err := auditstore.Open(path, nil)
|
||||
rec, err := auditstore.Open(t.TempDir(), "audit", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("audit.Open: %v", err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package audit
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -308,7 +308,7 @@ func TestCustomerDetail_RealAndNoSecretLeak(t *testing.T) {
|
||||
// tamper-evident audit trail with a before/after.
|
||||
func TestGrantCredit_DepositLandsAndAudited(t *testing.T) {
|
||||
f := newCockpitFakes(t)
|
||||
rec, err := audit.Open(":memory:", nil)
|
||||
rec, err := audit.Open(t.TempDir(), "audit", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("audit open: %v", err)
|
||||
}
|
||||
@@ -426,7 +426,7 @@ func TestGrantCredit_NilAuditStoreFailsClosed(t *testing.T) {
|
||||
// nonce forwards no key (the additive default).
|
||||
func TestGrantCredit_IdempotencyKeyForwarded(t *testing.T) {
|
||||
f := newCockpitFakes(t)
|
||||
rec, err := audit.Open(":memory:", nil)
|
||||
rec, err := audit.Open(t.TempDir(), "audit", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("audit open: %v", err)
|
||||
}
|
||||
@@ -476,7 +476,7 @@ func TestGrantCredit_IdempotencyKeyForwarded(t *testing.T) {
|
||||
// it — the customer's status reflects the change on a re-list.
|
||||
func TestSuspendReactivate_ForbidsUsersAndAudits(t *testing.T) {
|
||||
f := newCockpitFakes(t)
|
||||
rec, _ := audit.Open(":memory:", nil)
|
||||
rec, _ := audit.Open(t.TempDir(), "audit", nil)
|
||||
defer rec.Close()
|
||||
f.service.State.AuditStore = rec
|
||||
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package admin
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -0,0 +1,5 @@
|
||||
package admission
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -37,7 +37,7 @@ import (
|
||||
// (org, project) the flag engine uses for its platform switches, so the registry and
|
||||
// the waitlist.<svc> switches co-locate. One waitlist.db for the deployment.
|
||||
const (
|
||||
// A REAL org namespace named "platform", not cloud.PlatformNamespace(). The
|
||||
// A REAL org namespace named "platform", not namespace.System(). The
|
||||
// system namespace is the right name for the deployment's own partition and
|
||||
// would make it unsquattable by a tenant who registers that org, but it
|
||||
// renders to a different file, and moving a live store is a migration rather
|
||||
|
||||
+1
-6
@@ -40,8 +40,6 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -93,10 +91,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("ads.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("ads.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "ads.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("ads.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -3,13 +3,16 @@ package ads
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
)
|
||||
|
||||
func testStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStore(filepath.Join(t.TempDir(), "ads.db"))
|
||||
s, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
+12
-10
@@ -6,9 +6,11 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
// cek opens the store encrypted at rest (migrate-on-open + shred).
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// The ONE "sqlite" driver, kept registered for cek's no-key plaintext fallback.
|
||||
// basedb is the ONE opener: the database is born encrypted under the key cek
|
||||
// derives from the process master and this namespace.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
// The ONE "sqlite" driver.
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -20,18 +22,18 @@ var (
|
||||
errConflict = errors.New("ads: already exists")
|
||||
)
|
||||
|
||||
// Store is the ads database. ONE SQLite file ({DataDir}/ads.db) holds every
|
||||
// org's records; tenant isolation is the `org` column, enforced on EVERY query.
|
||||
// This mirrors clients/crm exactly (the ONE storage pattern). MaxOpenConns(1)
|
||||
// serializes writes against the single-writer file.
|
||||
// Store is the ads database. ONE SQLite file — the system namespace's "ads" —
|
||||
// holds every org's records; tenant isolation is the `org` column, enforced on
|
||||
// EVERY query. This mirrors clients/crm exactly (the ONE storage pattern).
|
||||
// MaxOpenConns(1) serializes writes against the single-writer file.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "ads", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open ads store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -56,7 +56,6 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -288,10 +287,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("affiliates.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("affiliates.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "affiliates.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("affiliates.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -13,6 +13,9 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
luxlog "github.com/luxfi/log"
|
||||
fiber "github.com/zap-proto/fiber/v3"
|
||||
"github.com/zap-proto/zip"
|
||||
@@ -77,7 +80,7 @@ func (f *fakeCommerce) depositCount() int {
|
||||
// commerce, returning the app, the service, and the fake for assertions.
|
||||
func mount(t *testing.T) (*zip.App, *cloud.Service[state], *fakeCommerce) {
|
||||
t.Helper()
|
||||
store, err := openStore(t.TempDir() + "/affiliates.db")
|
||||
store, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -5,18 +5,20 @@ package affiliates
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
)
|
||||
|
||||
// A store whose affiliate_referrals table predates the referrer_org column must
|
||||
// migrate cleanly: the referrer_org index is created AFTER ADD COLUMN, so it no
|
||||
// longer fails with "no such column: referrer_org" (the v1.800.1 boot crash).
|
||||
func TestMigrateFromPreReferrerOrgSchema(t *testing.T) {
|
||||
path := t.TempDir() + "/old.db"
|
||||
dir := t.TempDir()
|
||||
|
||||
// 1) Stand up the OLD schema: affiliate_referrals WITHOUT referrer_org, and an
|
||||
// 1) Stand up the OLD schema in the SAME database openStore opens, so step 2
|
||||
// migrates this file: affiliate_referrals WITHOUT referrer_org, and an
|
||||
// affiliates row so the backfill has something to resolve.
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
db, err := basedb.Open(namespace.System(), "affiliates", dir)
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
@@ -40,7 +42,7 @@ INSERT INTO affiliate_referrals (id,affiliate_id,referred_org,code,created_at) V
|
||||
_ = db.Close()
|
||||
|
||||
// 2) openStore runs migrate() on the existing old DB — this crashed v1.800.1.
|
||||
s, err := openStore(path)
|
||||
s, err := openStore(dir)
|
||||
if err != nil {
|
||||
t.Fatalf("migrate from old schema: %v", err)
|
||||
}
|
||||
|
||||
@@ -9,9 +9,10 @@ import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
// The ONE Hanzo SQLite driver (registers "sqlite" under both build tags).
|
||||
// basedb is the ONE opener; the ONE Hanzo SQLite driver registers "sqlite".
|
||||
// Mirrors clients/referrals / clients/crm — one storage pattern.
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -179,10 +180,10 @@ type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "affiliates", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open affiliates store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -292,9 +292,6 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("agents.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("agents.Mount: data dir: %w", err)
|
||||
}
|
||||
// The typed-op registry lives on the App: it is what makes each op a document
|
||||
// operation, an MCP tool, a CLI command and an SDK method rather than only a
|
||||
// route. A Router that cannot reach it must fail the mount rather than serve
|
||||
@@ -327,8 +324,8 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
tasks: disabledTaskController{},
|
||||
},
|
||||
}
|
||||
// Split a pre-existing fleet-wide agents.db into per-org files BEFORE a route
|
||||
// exists to read them, and fail the mount if it cannot be done: an empty
|
||||
// Split a pre-existing fleet-wide agents database into per-org files BEFORE a
|
||||
// route exists to read them, and fail the mount if it cannot be done: an empty
|
||||
// registry served over live rows is the one outcome worse than not booting.
|
||||
if err := fanOutLegacy(context.Background(), deps.DataDir, &s.State); err != nil {
|
||||
_ = s.State.stores.CloseAll()
|
||||
|
||||
@@ -3,7 +3,6 @@ package agents
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
@@ -12,7 +11,7 @@ import (
|
||||
|
||||
func testStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStoreAt(filepath.Join(t.TempDir(), "agents.db"))
|
||||
s, err := openStoreAt(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -107,7 +107,7 @@ func TestNoUpstreamNameOnTheWire(t *testing.T) {
|
||||
// are rewritten in place on store open, the rewrite is idempotent, and the
|
||||
// pre-migration value is retained so the change can be reversed.
|
||||
func TestMigrateModelRewritesStoredRows(t *testing.T) {
|
||||
dir := t.TempDir() + "/agents.db"
|
||||
dir := t.TempDir()
|
||||
ctx := context.Background()
|
||||
|
||||
// A store holding exactly what the live registry held.
|
||||
|
||||
+25
-20
@@ -4,19 +4,20 @@ import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"os"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
)
|
||||
|
||||
// legacy.go carries the pre-split registry forward. Until this change every
|
||||
// org's agents, runs, sessions, events, targets and claim keys lived in ONE
|
||||
// {DataDir}/agents.db, isolated by an org column. They now live one file per org
|
||||
// under {DataDir}/orgs/{slug}/agents.db, so a deployment that already has the
|
||||
// single file has to be fanned out before it serves — otherwise a live plane
|
||||
// (hanzo link registers sessions into it right now) reads as empty on the first
|
||||
// boot after the upgrade, which is indistinguishable from data loss to everyone
|
||||
// looking at it.
|
||||
// database — the SYSTEM namespace's "agents" — isolated by an org column. They
|
||||
// now live one per org, the SAME subsystem under each org's own namespace, so a
|
||||
// deployment that already has the single file has to be fanned out before it
|
||||
// serves — otherwise a live plane (hanzo link registers sessions into it right
|
||||
// now) reads as empty on the first boot after the upgrade, which is
|
||||
// indistinguishable from data loss to everyone looking at it.
|
||||
//
|
||||
// The fan-out runs ONCE, on mount, BEFORE any route is registered. It is
|
||||
// FAIL-SECURE: an error fails the mount rather than serving an empty registry
|
||||
@@ -29,24 +30,28 @@ import (
|
||||
// saves the work.
|
||||
//
|
||||
// The legacy file is READ and then left exactly where it is. It is not renamed
|
||||
// and not deleted: its cek sidecar pairing is cek's business and not this
|
||||
// package's to rearrange, and keeping the bytes is what makes the upgrade
|
||||
// reversible on the day someone needs it to be.
|
||||
// and not deleted: keeping the bytes is what makes the upgrade reversible on the
|
||||
// day someone needs it to be.
|
||||
|
||||
// legacyDBName is the single pre-split file, directly under DataDir. The per-org
|
||||
// files live under {DataDir}/orgs/, so the two can never collide.
|
||||
const legacyDBName = "agents.db"
|
||||
// legacySubsystem names the single pre-split database, held in the SYSTEM
|
||||
// namespace. The per-org files are this same subsystem under each org's own
|
||||
// namespace, and the platform partition is a name no org slug can render, so the
|
||||
// two can never collide.
|
||||
const legacySubsystem = "agents"
|
||||
|
||||
// fanOutLegacy copies every org's rows out of a pre-split {dataDir}/agents.db
|
||||
// into that org's own database, once. A deployment with no legacy file (a fresh
|
||||
// install, or one already fanned out) does nothing and returns nil.
|
||||
// fanOutLegacy copies every org's rows out of the pre-split platform-wide agents
|
||||
// database into that org's own database, once. A deployment with no legacy file
|
||||
// (a fresh install, or one already fanned out) does nothing and returns nil.
|
||||
func fanOutLegacy(ctx context.Context, dataDir string, st *state) error {
|
||||
path := filepath.Join(dataDir, legacyDBName)
|
||||
if !cek.Exists(path) {
|
||||
path, err := namespace.Path(dataDir, namespace.System(), legacySubsystem)
|
||||
if err != nil {
|
||||
return fmt.Errorf("legacy store path: %w", err)
|
||||
}
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
return nil // fresh install: nothing was ever written to the shared file
|
||||
}
|
||||
// The legacy file was always opened under the platform key, never an org's.
|
||||
raw, err := cek.Open(cek.Global, path)
|
||||
raw, err := basedb.Open(namespace.System(), legacySubsystem, dataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("open legacy store: %w", err)
|
||||
}
|
||||
|
||||
@@ -3,20 +3,18 @@ package agents
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
)
|
||||
|
||||
// seedLegacy writes one org's worth of every kind of row into a pre-split
|
||||
// agents.db at dir/agents.db, exactly as the shipped single-file store wrote
|
||||
// them, and returns the path.
|
||||
func seedLegacy(t *testing.T, dir string, orgs ...string) string {
|
||||
// seedLegacy writes one org's worth of every kind of row into the pre-split
|
||||
// platform-wide agents database under dir, exactly as the shipped single-file
|
||||
// store wrote them.
|
||||
func seedLegacy(t *testing.T, dir string, orgs ...string) {
|
||||
t.Helper()
|
||||
path := filepath.Join(dir, legacyDBName)
|
||||
st, err := openStoreAt(path)
|
||||
st, err := openStoreAt(dir)
|
||||
if err != nil {
|
||||
t.Fatalf("open legacy: %v", err)
|
||||
}
|
||||
@@ -56,7 +54,6 @@ func seedLegacy(t *testing.T, dir string, orgs ...string) string {
|
||||
if err := st.Close(); err != nil {
|
||||
t.Fatalf("close legacy: %v", err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
// TestFanOutLegacyCarriesEveryOrgForward is the upgrade guard. The agents plane
|
||||
@@ -129,7 +126,7 @@ func TestFanOutLegacyCarriesEveryOrgForward(t *testing.T) {
|
||||
// already has.
|
||||
func TestFanOutLegacyIsIdempotent(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := seedLegacy(t, dir, "acme")
|
||||
seedLegacy(t, dir, "acme")
|
||||
ctx := context.Background()
|
||||
|
||||
st := &state{stores: cloud.NewOrgStore[*Store](cloud.Base{DataDir: dir}, "agents", openStore)}
|
||||
@@ -154,7 +151,7 @@ func TestFanOutLegacyIsIdempotent(t *testing.T) {
|
||||
t.Fatalf("fan-out 2: %v", err)
|
||||
}
|
||||
// Drop the marker so the copy itself is exercised again, not just skipped.
|
||||
raw := rawAt(t, path)
|
||||
raw := rawAt(t, dir)
|
||||
if _, err := raw.Exec(`DELETE FROM agent_fanout`); err != nil {
|
||||
t.Fatalf("clear marker: %v", err)
|
||||
}
|
||||
|
||||
@@ -1,25 +1,5 @@
|
||||
package agents
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
sqlitedrv "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// TestMain makes this package's store-backed tests runnable on either build, the
|
||||
// same way the root package and clients/research already do it.
|
||||
//
|
||||
// Every test here opens an agents.db, and cek REFUSES to open a store without a
|
||||
// master key on an encryption-capable build — which, since hanzoai/sqlite v0.3,
|
||||
// includes the pure-Go build. Without this the WHOLE package fails to run, which
|
||||
// is exactly how the brand guard could go missing unnoticed.
|
||||
//
|
||||
// Supply a throwaway dev key ONLY when the build can encrypt AND the environment
|
||||
// did not already provide one, so a real injected key is never overridden.
|
||||
func TestMain(m *testing.M) {
|
||||
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
|
||||
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=") // 32 zero bytes, dev-only
|
||||
}
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a test
|
||||
// process has no KMS.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
|
||||
@@ -31,8 +31,8 @@ func TestMigrateOverLegacyAgentsTable(t *testing.T) {
|
||||
migratetest.Case{
|
||||
Name: "agents",
|
||||
LegacyDDL: legacyAgentsDDL,
|
||||
Open: func(path string) (io.Closer, error) {
|
||||
st, err := openStoreAt(path)
|
||||
Open: func(dir string) (io.Closer, error) {
|
||||
st, err := openStoreAt(dir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -87,10 +87,12 @@ CREATE TABLE agent_sessions (
|
||||
// schema (the v1.800.1-class regression the release smoke guards against).
|
||||
func TestMigrateOverLegacySessionsTable(t *testing.T) {
|
||||
migratetest.Case{
|
||||
Name: "sessions",
|
||||
// The subsystem, not the table: both cases seed a legacy epoch of the SAME
|
||||
// agents database, which is the one Open then migrates.
|
||||
Name: "agents",
|
||||
LegacyDDL: legacySessionsDDL,
|
||||
Open: func(path string) (io.Closer, error) {
|
||||
st, err := openStoreAt(path)
|
||||
Open: func(dir string) (io.Closer, error) {
|
||||
st, err := openStoreAt(dir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -19,7 +18,7 @@ import (
|
||||
|
||||
func testSessionStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStoreAt(filepath.Join(t.TempDir(), "agents.db"))
|
||||
s, err := openStoreAt(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -2,11 +2,11 @@ package agents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
)
|
||||
|
||||
// TestLifecycleFieldsRoundTrip: the four bot-lifecycle columns persist and read
|
||||
@@ -100,17 +100,16 @@ func TestListLongRunning(t *testing.T) {
|
||||
// (no new columns) is migrated forward on open, existing rows survive with the
|
||||
// column defaults, and re-opening (re-running migrate) is a clean no-op.
|
||||
func TestMigrationIdempotentOnLegacyDB(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "legacy.db")
|
||||
dir := t.TempDir()
|
||||
|
||||
// Hand-build the legacy schema + a legacy row, exactly as the pre-lifecycle
|
||||
// migrate() would have, then close.
|
||||
//
|
||||
// Fabricated through cek, like every real store: what is under test is THIS
|
||||
// package's migrate() over a legacy schema, not cek's storage format. Writing
|
||||
// the fixture with a bare sql.Open would leave a plaintext file, and converting
|
||||
// one is a production operation that requires the live libsqlcipher codec — so
|
||||
// the fixture, not the code under test, would fail the build the suite runs on.
|
||||
legacy, err := cek.Open(cek.Global, path)
|
||||
// Fabricated in the SAME database openStoreAt then opens, and opened the same
|
||||
// way every real store is: what is under test is THIS package's migrate() over
|
||||
// a legacy schema, not the storage format. A bare sql.Open would leave a
|
||||
// plaintext file the keyed opener cannot read at all.
|
||||
legacy, err := basedb.Open(namespace.System(), legacySubsystem, dir)
|
||||
if err != nil {
|
||||
t.Fatalf("open legacy: %v", err)
|
||||
}
|
||||
@@ -134,7 +133,7 @@ CREATE TABLE agents (
|
||||
// Open through the real store TWICE — the first migrates, the second proves
|
||||
// idempotency (no error re-adding existing columns).
|
||||
for i := 0; i < 2; i++ {
|
||||
st, err := openStoreAt(path)
|
||||
st, err := openStoreAt(dir)
|
||||
if err != nil {
|
||||
t.Fatalf("open #%d migrate failed: %v", i, err)
|
||||
}
|
||||
|
||||
@@ -5,15 +5,19 @@ import (
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
"github.com/hanzoai/namespace"
|
||||
)
|
||||
|
||||
// openStoreAt opens an agents store at an explicit path, standing in for what
|
||||
// cloud.OrgDB does for a real per-org file: open through cek, then hand the
|
||||
// *sql.DB to openStore for migration. Only the migration tests care WHERE the
|
||||
// openStoreAt opens the agents store under an explicit dir, standing in for what
|
||||
// cloud.OrgDB does for a real per-org file: open the named database, then hand
|
||||
// the *sql.DB to openStore for migration. Only the migration tests care WHERE the
|
||||
// file is; everything else wants testStore.
|
||||
func openStoreAt(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStoreAt(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), legacySubsystem, dir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -26,13 +30,13 @@ func openStoreAt(path string) (*Store, error) {
|
||||
return st, nil
|
||||
}
|
||||
|
||||
// rawAt opens the file behind a store path without the agents schema, for tests
|
||||
// that plant a legacy table or read one back.
|
||||
func rawAt(t *testing.T, path string) *sql.DB {
|
||||
// rawAt opens the database behind a store dir without the agents schema, for
|
||||
// tests that plant a legacy table or read one back.
|
||||
func rawAt(t *testing.T, dir string) *sql.DB {
|
||||
t.Helper()
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
db, err := basedb.Open(namespace.System(), legacySubsystem, dir)
|
||||
if err != nil {
|
||||
t.Fatalf("open raw %s: %v", path, err)
|
||||
t.Fatalf("open raw %s: %v", dir, err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
return db
|
||||
@@ -40,7 +44,7 @@ func rawAt(t *testing.T, path string) *sql.DB {
|
||||
|
||||
// testStores is the per-org store set Mount builds, over a throwaway data dir.
|
||||
// A test that reaches storage through it exercises the REAL resolution path
|
||||
// (org → namespace.Sanitize → file → cek), not a hand-placed handle, so an isolation
|
||||
// (org → namespace → file → key), not a hand-placed handle, so an isolation
|
||||
// assertion is a statement about the shipped code.
|
||||
func testStores(t *testing.T) *cloud.OrgStore[*Store] {
|
||||
t.Helper()
|
||||
|
||||
@@ -14,11 +14,11 @@ import (
|
||||
"github.com/zap-proto/zip"
|
||||
)
|
||||
|
||||
// newStore opens an in-memory audit recorder (single connection keeps :memory:
|
||||
// alive across queries) and seeds it with records across two orgs.
|
||||
// newStore opens a throwaway audit recorder and seeds it with records across
|
||||
// two orgs.
|
||||
func newStore(t *testing.T) *audit.Recorder {
|
||||
t.Helper()
|
||||
rec, err := audit.Open(":memory:", nil)
|
||||
rec, err := audit.Open(t.TempDir(), "audit", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("audit.Open: %v", err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package auditlog
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -66,7 +66,6 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -167,10 +166,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if zapp == nil {
|
||||
return fmt.Errorf("authors.Mount: router is not a zip app, so the typed ops have no registry")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("authors.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "authors.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("authors.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -7,14 +7,15 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strconv"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
sqlitedrv "github.com/hanzoai/sqlite"
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
luxlog "github.com/luxfi/log"
|
||||
fiber "github.com/zap-proto/fiber/v3"
|
||||
"github.com/zap-proto/zip"
|
||||
@@ -137,23 +138,11 @@ func (g *fakeGitHub) fetchFile(_ context.Context, _, owner, repo, branch, path s
|
||||
return g.files[owner+"/"+repo+"@"+branch+"/"+path], nil
|
||||
}
|
||||
|
||||
// TestMain makes this store-backed suite build-tag agnostic, exactly as the root
|
||||
// package's main_test.go does: an encryption-capable build refuses to open the data
|
||||
// plane without a master key, a pure-Go build refuses a key at all. Supply a throwaway
|
||||
// dev key ONLY when the build can encrypt and the environment did not already provide
|
||||
// one, so CI's real key is never overridden.
|
||||
func TestMain(m *testing.M) {
|
||||
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
|
||||
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=") // 32 zero bytes, dev-only
|
||||
}
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
// mount builds an authors app backed by a fresh store + injected fakes, returning the
|
||||
// app, the service, and the fakes for assertions.
|
||||
func mount(t *testing.T) (*zip.App, *cloud.Service[state], *fakeCommerce, *fakeGitHub) {
|
||||
t.Helper()
|
||||
store, err := openStore(t.TempDir() + "/authors.db")
|
||||
store, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -10,9 +10,10 @@ import (
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
// The ONE Hanzo SQLite driver (registers "sqlite" under both build tags).
|
||||
// basedb is the ONE opener; the ONE Hanzo SQLite driver registers "sqlite".
|
||||
// Mirrors clients/affiliates / clients/referrals — one storage pattern.
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -173,10 +174,10 @@ type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "authors", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open authors store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -47,7 +47,6 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
@@ -131,10 +130,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("automations.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("automations.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "automations.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("automations.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -93,12 +93,12 @@ func derivedTools(app *zip.App) []string {
|
||||
return names
|
||||
}
|
||||
|
||||
// newAppWithAudit mounts the subsystem with a REAL in-memory audit recorder so a
|
||||
// test can read the tamper-evident trail back and assert outcomes (LOW-1) and
|
||||
// newAppWithAudit mounts the subsystem with a REAL audit recorder so a test can
|
||||
// read the tamper-evident trail back and assert outcomes (LOW-1) and
|
||||
// exactly-once run bookkeeping (MED-1). Returns the recorder for querying.
|
||||
func newAppWithAudit(t *testing.T) (*zip.App, *audit.Recorder) {
|
||||
t.Helper()
|
||||
rec, err := audit.Open(":memory:", nil)
|
||||
rec, err := audit.Open(t.TempDir(), "audit", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("audit.Open: %v", err)
|
||||
}
|
||||
@@ -173,7 +173,7 @@ func reqRaw(t *testing.T, app *zip.App, path, org string, raw string) httpResult
|
||||
|
||||
func testStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStore(t.TempDir() + "/automations.db")
|
||||
s, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -1,22 +1,5 @@
|
||||
package automations
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
sqlitedrv "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// TestMain makes the automations suite build-tag agnostic (mirrors the sibling
|
||||
// clients/sync + clients/git harness). On an encryption-capable (cgo) build cek
|
||||
// REFUSES to open a store without a master key; on a pure-Go build a key is itself
|
||||
// refused. So supply a throwaway dev key ONLY when the build can encrypt AND the
|
||||
// environment did not already provide one (CI may inject the real key) — then every
|
||||
// store opens (encrypted on cgo, plaintext on pure-Go) without ever overriding a
|
||||
// provided key. Resolved once per process, order-independent.
|
||||
func TestMain(m *testing.M) {
|
||||
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
|
||||
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=") // 32 zero bytes, dev-only
|
||||
}
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a test
|
||||
// process has no KMS.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
|
||||
+15
-12
@@ -8,11 +8,13 @@ import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
// github.com/hanzoai/sqlite is the ONE Hanzo SQLite driver: it registers the
|
||||
// "sqlite" database/sql name under both build tags (cgo → mattn+SQLCipher,
|
||||
// encrypted at rest; !cgo → pure-Go modernc). Blank import registers it. This
|
||||
// mirrors clients/crm exactly — the ONE storage pattern.
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// basedb is the ONE opener: it renders this subsystem's path from the
|
||||
// namespace and opens it under the key cek derives for that name.
|
||||
// github.com/hanzoai/sqlite is the ONE Hanzo SQLite driver; the blank import
|
||||
// registers the "sqlite" database/sql name. Mirrors clients/crm exactly — the
|
||||
// ONE storage pattern.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -23,18 +25,19 @@ var (
|
||||
errBadRef = errors.New("automations: referenced record not found in org")
|
||||
)
|
||||
|
||||
// Store is the automations database. ONE SQLite file ({DataDir}/automations.db)
|
||||
// holds every org's flows, versions, and runs; tenant isolation is the `org`
|
||||
// column, physical on EVERY uniqueness + lookup index (each leads with org).
|
||||
// MaxOpenConns(1) serializes writes against the single-writer WAL file.
|
||||
// Store is the automations database. ONE SQLite file — the system namespace's
|
||||
// "automations" — holds every org's flows, versions, and runs; tenant isolation
|
||||
// is the `org` column, physical on EVERY uniqueness + lookup index (each leads
|
||||
// with org). MaxOpenConns(1) serializes writes against the single-writer WAL
|
||||
// file.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "automations", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open automations store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package billing
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -60,7 +60,6 @@ func (f *countingCommerce) count() int {
|
||||
// in the fleet lands in) for the duration of the test, funded with cents for org.
|
||||
func publishLedger(t *testing.T, org string, cents int64) finance.Client {
|
||||
t.Helper()
|
||||
t.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=") // 32 zero bytes, dev-only
|
||||
fin := finance.New(t.TempDir())
|
||||
if cents > 0 {
|
||||
if _, err := fin.Deposit(context.Background(), types.DepositInput{
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package books
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -61,8 +61,6 @@ import (
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
@@ -120,10 +118,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if zapp == nil {
|
||||
return fmt.Errorf("campaign.Mount: router is not a zip app, so the typed ops have no registry")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("campaign.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "campaign.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("campaign.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
+12
-10
@@ -7,9 +7,11 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
// cek opens the store encrypted at rest (migrate-on-open + shred).
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// The ONE "sqlite" driver, kept registered for cek's no-key plaintext fallback.
|
||||
// basedb is the ONE opener: the database is born encrypted under the key cek
|
||||
// derives from the process master and this namespace.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
// The ONE "sqlite" driver.
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -76,18 +78,18 @@ type Campaign struct {
|
||||
UpdatedAt int64 `json:"updatedAt"`
|
||||
}
|
||||
|
||||
// Store is the campaign database. ONE SQLite file ({DataDir}/campaign.db) holds
|
||||
// every org's records; tenant isolation is the `org` column, enforced on EVERY
|
||||
// query. Mirrors clients/ads exactly (the ONE storage pattern). MaxOpenConns(1)
|
||||
// serializes writes against the single-writer file.
|
||||
// Store is the campaign database. ONE SQLite file — the system namespace's
|
||||
// "campaign" — holds every org's records; tenant isolation is the `org` column,
|
||||
// enforced on EVERY query. Mirrors clients/ads exactly (the ONE storage
|
||||
// pattern). MaxOpenConns(1) serializes writes against the single-writer file.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "campaign", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open campaign store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -3,13 +3,16 @@ package campaign
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
)
|
||||
|
||||
func testStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStore(filepath.Join(t.TempDir(), "campaign.db"))
|
||||
s, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package captable
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -6,25 +6,15 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
"github.com/hanzoai/cloud/apps/index"
|
||||
sqlitedrv "github.com/hanzoai/sqlite"
|
||||
luxlog "github.com/luxfi/log"
|
||||
"github.com/zap-proto/zip"
|
||||
)
|
||||
|
||||
// Same build-tag-agnostic harness as the index suite it borrows the store from.
|
||||
func TestMain(m *testing.M) {
|
||||
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
|
||||
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
|
||||
}
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
// mount brings up the index (the store) and the catalog (the lens) on one app,
|
||||
// exactly as apps.go orders them.
|
||||
func mount(t *testing.T) *zip.App {
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package catalog
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -12,8 +12,6 @@ package channels
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync/atomic"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
@@ -41,10 +39,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("channels.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("channels.Mount: data dir: %w", err)
|
||||
}
|
||||
st, err := openStore(filepath.Join(deps.DataDir, "channels.db"))
|
||||
st, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("channels.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -2,9 +2,12 @@ package channels
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
)
|
||||
|
||||
// policy_test.go is the pure-store proof of the gate engine and pairing state
|
||||
@@ -16,7 +19,7 @@ const t0 int64 = 1_700_000_000
|
||||
|
||||
func newStore(t *testing.T) *store {
|
||||
t.Helper()
|
||||
st, err := openStore(filepath.Join(t.TempDir(), "channels.db"))
|
||||
st, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
+12
-9
@@ -6,7 +6,10 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// basedb is the ONE opener: it renders this subsystem's path from the
|
||||
// namespace and opens it under the key cek derives for that name.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
// github.com/hanzoai/sqlite is the ONE Hanzo SQLite driver (registers the
|
||||
// "sqlite" database/sql name under both build tags). Blank import registers
|
||||
// the driver — same as apps/integrations.
|
||||
@@ -26,19 +29,19 @@ const (
|
||||
sendKeepSec = 48 * 3600
|
||||
)
|
||||
|
||||
// store is the channels database. ONE SQLite file ({DataDir}/channels.db)
|
||||
// holds every org's policy, pairing, allowlist, inbox, send-idempotency, and
|
||||
// route rows; tenancy is the org column — org leads every PK. No secrets in
|
||||
// any row (pairing codes are capability strings a sender must present; they
|
||||
// are stored, never logged).
|
||||
// store is the channels database. ONE SQLite file — the system namespace's
|
||||
// "channels" — holds every org's policy, pairing, allowlist, inbox,
|
||||
// send-idempotency, and route rows; tenancy is the org column — org leads every
|
||||
// PK. No secrets in any row (pairing codes are capability strings a sender must
|
||||
// present; they are stored, never logged).
|
||||
type store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "channels", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open channels store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package cms
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
+16
-7
@@ -5,11 +5,14 @@ import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud"
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
"github.com/hanzoai/namespace"
|
||||
|
||||
"github.com/zap-proto/zip"
|
||||
@@ -215,12 +218,18 @@ func TestPerOrgIsolationHTTP(t *testing.T) {
|
||||
t.Fatalf("storeFor %s: %v", org, err)
|
||||
}
|
||||
}
|
||||
// Close first: on the pure-Go codec a store still OPEN has not written its
|
||||
// database file yet, so the file only lands at Close.
|
||||
if err := s.stores.CloseAll(); err != nil {
|
||||
t.Fatalf("CloseAll: %v", err)
|
||||
}
|
||||
for _, org := range []string{"orgA", "orgB"} {
|
||||
p := filepath.Join(s.dataDir, "orgs", namespace.Sanitize(org), "code.db")
|
||||
// cek.Exists, not os.Stat: a store still OPEN has not materialized its
|
||||
// database file on the pure-Go codec — only its sidecar is on disk.
|
||||
if !cek.Exists(p) {
|
||||
t.Fatalf("expected per-org store at %s", p)
|
||||
p, err := namespace.Path(s.dataDir, cloud.MustOrgNamespace(org, ""), "code")
|
||||
if err != nil {
|
||||
t.Fatalf("namespace.Path %s: %v", org, err)
|
||||
}
|
||||
if _, err := os.Stat(p); err != nil {
|
||||
t.Fatalf("expected per-org store at %s: %v", p, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package commerce
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -1,24 +0,0 @@
|
||||
// Copyright © 2026 Hanzo AI. MIT License.
|
||||
|
||||
package commerce
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
sqlitedrv "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// TestMain makes this package's store-backed tests (the finance-ledger money proofs in
|
||||
// starter_test.go) build-tag agnostic, exactly as the root package's TestMain does. On
|
||||
// an encryption-capable (cgo) build cek REFUSES to open a store without a master key;
|
||||
// on a pure-Go build a key is itself refused. So supply a throwaway dev key ONLY when
|
||||
// the build can encrypt AND the environment did not already provide one (CI may inject
|
||||
// the real key) — never overriding a provided key. Resolved once per process,
|
||||
// order-independent.
|
||||
func TestMain(m *testing.M) {
|
||||
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
|
||||
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=") // 32 zero bytes, dev-only
|
||||
}
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
@@ -133,9 +133,9 @@ var Prefixes = []string{
|
||||
// boot at all, and the whole money plane is 503 in every local dev build and every
|
||||
// `go test`. That is the state this function exists to end.
|
||||
//
|
||||
// CodecLinked is the SAME predicate commerce's own resolveMasterKey gates on, and
|
||||
// the same one cek.EnsureDevKey uses for cloud's stores, so asking it here keeps ONE
|
||||
// posture decision across the process rather than three that can disagree:
|
||||
// CodecLinked is the SAME predicate commerce's own resolveMasterKey gates on, so
|
||||
// asking it here keeps ONE posture decision across the process rather than two that
|
||||
// can disagree:
|
||||
//
|
||||
// - codec linked (the production image: CGO_ENABLED=1 -tags libsqlite3) → inject.
|
||||
// commerce encrypts, and its resolveMasterKey still fails closed if the key is
|
||||
|
||||
@@ -8,7 +8,6 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -146,10 +145,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if zapp == nil {
|
||||
return fmt.Errorf("company.Mount: router is not a zip app, so the typed ops have no registry")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("company.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "company.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("company.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -3,34 +3,22 @@ package company
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
luxlog "github.com/luxfi/log"
|
||||
fiber "github.com/zap-proto/fiber/v3"
|
||||
"github.com/zap-proto/zip"
|
||||
)
|
||||
|
||||
// TestMain seeds a random cek master key so the encrypted-at-rest store opens on an
|
||||
// encryption-capable test build (mirrors clients/compliance and clients/legal). On a
|
||||
// pure-Go build cek ignores it and uses the plaintext dev path.
|
||||
func TestMain(m *testing.M) {
|
||||
k := make([]byte, 32)
|
||||
if _, err := rand.Read(k); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
cek.SetMasterKey(k)
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
// testTimeout is a generous per-request ceiling for the in-memory fiber test
|
||||
// harness. fiber's default (1s) is too tight: under the race detector or an
|
||||
// occasional GC / cold-sqlite pause a fast handler can momentarily exceed it,
|
||||
|
||||
+13
-10
@@ -7,29 +7,32 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
// cek opens the store encrypted at rest (migrate-on-open + shred).
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// The ONE "sqlite" driver, kept registered for cek's no-key plaintext fallback.
|
||||
// basedb is the ONE opener: the database is born encrypted under the key cek
|
||||
// derives from the process master and this namespace.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
// The ONE "sqlite" driver.
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// errNotFound is returned when an org has no formation yet. Handlers map it to 404.
|
||||
var errNotFound = errors.New("company: formation not found")
|
||||
|
||||
// Store persists formations. ONE SQLite file ({DataDir}/company.db) holds every
|
||||
// org's formation; tenant isolation is the `org` primary key, enforced on EVERY
|
||||
// query. There is at most one formation per org (an org forms one company through
|
||||
// this flow), so the aggregate is stored as a single row: the machine-relevant
|
||||
// Store persists formations. ONE SQLite file — the system namespace's "company"
|
||||
// — holds every org's formation; tenant isolation is the `org` primary key,
|
||||
// enforced on EVERY query. There is at most one formation per org (an org forms
|
||||
// one company through this flow), so the aggregate is stored as a single row:
|
||||
// the machine-relevant
|
||||
// projection (stage/structure/name) in columns for cheap listing, and the full
|
||||
// Formation as a JSON document in `data`. MaxOpenConns(1) serializes writes.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "company", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open company store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -3,13 +3,12 @@ package company
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func testStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStore(filepath.Join(t.TempDir(), "company.db"))
|
||||
s, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -8,7 +8,6 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -62,9 +61,6 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("compliance.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("compliance.Mount: data dir: %w", err)
|
||||
}
|
||||
provider, err := idv.FromConfig(kmsGetter(deps), os.Getenv)
|
||||
if err != nil {
|
||||
return fmt.Errorf("compliance.Mount: idv provider: %w", err)
|
||||
@@ -77,7 +73,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if err != nil {
|
||||
return fmt.Errorf("compliance.Mount: idv webhook: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "compliance.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("compliance.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/hmac"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
@@ -12,8 +11,6 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -21,7 +18,9 @@ import (
|
||||
"github.com/hanzoai/cloud"
|
||||
"github.com/hanzoai/cloud/apps/idv"
|
||||
"github.com/hanzoai/cloud/audit"
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
luxlog "github.com/luxfi/log"
|
||||
fiber "github.com/zap-proto/fiber/v3"
|
||||
"github.com/zap-proto/zip"
|
||||
@@ -29,18 +28,6 @@ import (
|
||||
|
||||
const testTimeout = 30 * time.Second
|
||||
|
||||
// TestMain seeds a random cek master key so the encrypted-at-rest store opens on an
|
||||
// encryption-capable test build (mirrors clients/integrations, flags, git, venue). On
|
||||
// a pure-Go build cek ignores it and uses the plaintext dev path.
|
||||
func TestMain(m *testing.M) {
|
||||
k := make([]byte, 32)
|
||||
if _, err := rand.Read(k); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
cek.SetMasterKey(k)
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
// fakeProvider is an injectable idv.Provider whose Start/Check statuses are fixed by
|
||||
// the test — so we can drive the seam, including a HOSTILE provider that tries to
|
||||
// report a terminal decision on Start.
|
||||
@@ -59,13 +46,12 @@ func (f fakeProvider) Check(context.Context, string, string) (idv.Result, error)
|
||||
return idv.Result{Ref: "ref_" + f.name, Status: f.checkStatus}, nil
|
||||
}
|
||||
|
||||
// mount brings compliance up on a bare app with a real in-memory audit recorder, and
|
||||
// mount brings compliance up on a bare app with a real audit recorder, and
|
||||
// returns the app + the recorder for assertions. The provider defaults to Manual.
|
||||
func mount(t *testing.T) (*zip.App, *audit.Recorder) {
|
||||
t.Helper()
|
||||
// A unique per-test path (not ":memory:") so each test's cek sidecar is written and
|
||||
// read under this process's key — concurrent test binaries never share a sidecar.
|
||||
rec, err := audit.Open(filepath.Join(t.TempDir(), "audit.db"), nil)
|
||||
// A dir of its own per test, so concurrent tests never share a database.
|
||||
rec, err := audit.Open(t.TempDir(), "audit", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("audit.Open: %v", err)
|
||||
}
|
||||
@@ -175,6 +161,7 @@ func (k fakeKMS) GetSecret(_ context.Context, ref string) ([]byte, error) {
|
||||
return nil, fmt.Errorf("no such secret %q", ref)
|
||||
}
|
||||
func (k fakeKMS) PutSecret(context.Context, string, []byte) error { return nil }
|
||||
func (k fakeKMS) DeleteSecret(context.Context, string) error { return nil }
|
||||
func (k fakeKMS) Sign(context.Context, string, []byte) ([]byte, error) { return nil, nil }
|
||||
|
||||
// mountWithWebhook mounts compliance with a signature-authenticated webhook configured:
|
||||
@@ -182,7 +169,7 @@ func (k fakeKMS) Sign(context.Context, string, []byte) ([]byte, error) { return
|
||||
func mountWithWebhook(t *testing.T, secret string) *zip.App {
|
||||
t.Helper()
|
||||
t.Setenv("CLOUD_IDV_WEBHOOK_KEY_REF", "kms://idv-webhook")
|
||||
rec, err := audit.Open(filepath.Join(t.TempDir(), "audit.db"), nil)
|
||||
rec, err := audit.Open(t.TempDir(), "audit", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("audit.Open: %v", err)
|
||||
}
|
||||
|
||||
@@ -7,7 +7,8 @@ import (
|
||||
"fmt"
|
||||
|
||||
"github.com/hanzoai/cloud/apps/idv"
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -16,18 +17,18 @@ import (
|
||||
// tenant is indistinguishable from one that does not exist (no cross-tenant probe).
|
||||
var errNotFound = errors.New("compliance: not found")
|
||||
|
||||
// Store persists compliance records. ONE SQLite file ({DataDir}/compliance.db),
|
||||
// opened through cek so subject PII (name/email) is ENCRYPTED AT REST. Tenant
|
||||
// isolation is a physical property: `org` is a column on every table and every read
|
||||
// and write is filtered by it. MaxOpenConns(1) serializes writes.
|
||||
// Store persists compliance records. ONE SQLite file — the system namespace's
|
||||
// "compliance" — born encrypted, so subject PII (name/email) is ENCRYPTED AT
|
||||
// REST. Tenant isolation is a physical property: `org` is a column on every table
|
||||
// and every read and write is filtered by it. MaxOpenConns(1) serializes writes.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "compliance", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open compliance store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -1,22 +1,5 @@
|
||||
package content
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
)
|
||||
|
||||
// TestMain seeds a random cek master key so the framework store (opened by
|
||||
// framework.Mount inside these tests) opens on an encryption-capable test build
|
||||
// (mirrors clients/compliance, integrations, flags, git, venue). On a pure-Go build
|
||||
// cek ignores it and uses the plaintext dev path.
|
||||
func TestMain(m *testing.M) {
|
||||
k := make([]byte, 32)
|
||||
if _, err := rand.Read(k); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
cek.SetMasterKey(k)
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a test
|
||||
// process has no KMS.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
|
||||
+1
-6
@@ -52,8 +52,6 @@ import (
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -106,10 +104,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("crm.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("crm.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "crm.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("crm.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -3,13 +3,16 @@ package crm
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
)
|
||||
|
||||
func testStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStore(filepath.Join(t.TempDir(), "crm.db"))
|
||||
s, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
+12
-8
@@ -6,12 +6,15 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
// basedb is the ONE opener: it renders this subsystem's path from the
|
||||
// namespace and opens it under the key cek derives for that name.
|
||||
// github.com/hanzoai/sqlite is the ONE Hanzo SQLite driver: it registers
|
||||
// the "sqlite" database/sql name under both build tags (cgo →
|
||||
// mattn+SQLCipher, encrypted at rest; !cgo → pure-Go modernc). Importing
|
||||
// modernc directly instead would double-register "sqlite" under CGO and
|
||||
// panic at init. Blank import registers the driver.
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -24,18 +27,19 @@ var (
|
||||
errBadRef = errors.New("crm: referenced record not found in org")
|
||||
)
|
||||
|
||||
// Store is the CRM database. ONE SQLite file ({DataDir}/crm.db) holds every
|
||||
// org's records; tenant isolation is the `org` column, enforced on EVERY query.
|
||||
// This mirrors clients/prompts and clients/eval exactly (the ONE storage
|
||||
// pattern). MaxOpenConns(1) serializes writes against the single-writer file.
|
||||
// Store is the CRM database. ONE SQLite file — the system namespace's "crm" —
|
||||
// holds every org's records; tenant isolation is the `org` column, enforced on
|
||||
// EVERY query. This mirrors clients/prompts and clients/eval exactly (the ONE
|
||||
// storage pattern). MaxOpenConns(1) serializes writes against the single-writer
|
||||
// file.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "crm", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open crm store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -12,6 +12,9 @@ import (
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
luxlog "github.com/luxfi/log"
|
||||
"github.com/zap-proto/zip"
|
||||
)
|
||||
|
||||
+6
-10
@@ -10,23 +10,19 @@ package dataroom
|
||||
import (
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
type linkIndex struct{ db *sql.DB }
|
||||
|
||||
func openLinkIndex(dataDir string) (*linkIndex, error) {
|
||||
// Its OWN dir, distinct from NewBase's per-tenant tree ({dataDir}/dataroom/):
|
||||
// this global routing table must never collide with a tenant's DB file.
|
||||
dir := filepath.Join(dataDir, "dataroom_index")
|
||||
if err := os.MkdirAll(dir, 0o700); err != nil {
|
||||
return nil, fmt.Errorf("dataroom: mkdir %s: %w", dir, err)
|
||||
}
|
||||
db, err := cek.Open(cek.Global, filepath.Join(dir, "link_index.db"))
|
||||
// The SYSTEM namespace, distinct from NewBase's per-tenant tree
|
||||
// ({dataDir}/dataroom/): this global routing table belongs to no org, and the
|
||||
// platform partition is one no tenant name can render.
|
||||
db, err := basedb.Open(namespace.System(), "link_index", dataDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("dataroom: open link index: %w", err)
|
||||
}
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -99,10 +98,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if zapp == nil {
|
||||
return fmt.Errorf("destinations.Mount: router is not backed by a *zip.App; typed ops have nowhere to register")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("destinations.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "destinations.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("destinations.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
+13
-10
@@ -8,27 +8,30 @@ import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// The ONE "sqlite" driver, kept registered for cek's no-key plaintext fallback.
|
||||
// basedb is the ONE opener: the database is born encrypted under the key cek
|
||||
// derives from the process master and this namespace.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
// The ONE "sqlite" driver.
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// errNotFound is mapped to 404 by handlers.
|
||||
var errNotFound = errors.New("destinations: not found")
|
||||
|
||||
// Store is the destinations database. ONE SQLite file ({DataDir}/destinations.db)
|
||||
// holds every org's connected destinations; tenant isolation is the `org` column,
|
||||
// enforced on EVERY query (the ads/integrations pattern). The row holds only the
|
||||
// NON-SECRET config (measurement/pixel ids) as JSON — the API secret lives in KMS,
|
||||
// never here. MaxOpenConns(1) serializes writes against the single-writer file.
|
||||
// Store is the destinations database. ONE SQLite file — the system namespace's
|
||||
// "destinations" — holds every org's connected destinations; tenant isolation is
|
||||
// the `org` column, enforced on EVERY query (the ads/integrations pattern). The
|
||||
// row holds only the NON-SECRET config (measurement/pixel ids) as JSON — the API
|
||||
// secret lives in KMS, never here. MaxOpenConns(1) serializes writes against the single-writer file.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "destinations", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open destinations store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -2,13 +2,16 @@ package destinations
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
)
|
||||
|
||||
func testStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStore(filepath.Join(t.TempDir(), "destinations.db"))
|
||||
s, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -12,7 +12,8 @@
|
||||
//
|
||||
// TWO AUTHORITIES, NEVER BRAIDED.
|
||||
// - ENABLEMENT (this store): which products the org has toggled on. The org's
|
||||
// intent. Durable per-org SQLite ({DataDir}/entitlements.db), (org,product) key.
|
||||
// intent. Durable SQLite — the deployment's own "entitlements" — keyed
|
||||
// (org, product).
|
||||
// - ENTITLEMENT (commerce): which products the org's plan/subscription grants.
|
||||
// The billing truth. Read via deps.Commerce.CheckEntitlement at WRITE time.
|
||||
//
|
||||
@@ -34,8 +35,6 @@ import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -94,10 +93,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("entitlements.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("entitlements.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "entitlements.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("entitlements.Mount: open store: %w", err)
|
||||
}
|
||||
|
||||
@@ -3,46 +3,28 @@ package entitlements
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/types"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
|
||||
luxlog "github.com/luxfi/log"
|
||||
"github.com/zap-proto/zip"
|
||||
)
|
||||
|
||||
// The cek data plane fail-closes without a master key on encryption-capable
|
||||
// builds; tests supply one process-wide (SetMasterKey is once-only). Every
|
||||
// openStore caller in this package must take it first, or the store refuses to
|
||||
// open the data plane unencrypted and the whole suite fails before it asserts
|
||||
// anything. Mirrors clients/flags/flags_test.go.
|
||||
var cekOnce sync.Once
|
||||
|
||||
func testMasterKey(t *testing.T) {
|
||||
t.Helper()
|
||||
cekOnce.Do(func() {
|
||||
k := make([]byte, 32)
|
||||
if _, err := rand.Read(k); err != nil {
|
||||
t.Fatalf("rng: %v", err)
|
||||
}
|
||||
cek.SetMasterKey(k)
|
||||
})
|
||||
}
|
||||
|
||||
// openTestStore opens a throwaway entitlements store for one test — the ONE way
|
||||
// this package's tests reach the store, so the master key can never be forgotten
|
||||
// at a new call site.
|
||||
// this package's tests reach the store.
|
||||
func openTestStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
testMasterKey(t)
|
||||
store, err := openStore(t.TempDir() + "/entitlements.db")
|
||||
store, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -6,10 +6,13 @@ import (
|
||||
"fmt"
|
||||
"sort"
|
||||
|
||||
// basedb is the ONE opener: the database is born encrypted under the key cek
|
||||
// derives from the process master and this namespace.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
// github.com/hanzoai/sqlite is the ONE Hanzo SQLite driver (registers the
|
||||
// "sqlite" database/sql name under both cgo and pure-Go build tags). Blank
|
||||
// import registers the driver; importing modernc directly would double-register.
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -34,15 +37,15 @@ import (
|
||||
// the visibility toggle, entitlement is the billing gate.
|
||||
|
||||
// Store is the entitlements metastore over one SQLite file
|
||||
// ({DataDir}/entitlements.db). Org-scoping is the (org, product) key.
|
||||
// (the deployment's own "entitlements"). Org-scoping is the (org, product) key.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "entitlements", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open entitlements store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package erp
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -0,0 +1,5 @@
|
||||
package esign
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -1,25 +0,0 @@
|
||||
package esign
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
sqlitedrv "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// TestMain makes the esign suite build-tag agnostic (the same harness as the
|
||||
// sibling clients/search and clients/sync suites). On an encryption-capable
|
||||
// (cgo) build cek REFUSES to open a store without a master key; on a pure-Go
|
||||
// build a key is itself refused. So supply a throwaway dev key ONLY when the
|
||||
// build can encrypt AND the environment did not already provide one (CI may
|
||||
// inject the real key) — then every per-tenant document store opens (encrypted
|
||||
// on cgo, plaintext on pure-Go) without ever overriding a provided key.
|
||||
//
|
||||
// Without this the signing-flow tests fail at the first document write, since
|
||||
// cek fails closed rather than writing legal documents to disk in the clear.
|
||||
func TestMain(m *testing.M) {
|
||||
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
|
||||
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=") // 32 zero bytes, dev-only
|
||||
}
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
+1
-5
@@ -57,7 +57,6 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -167,10 +166,7 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
if deps.DataDir == "" {
|
||||
return fmt.Errorf("eval.Mount: empty DataDir")
|
||||
}
|
||||
if err := os.MkdirAll(deps.DataDir, 0o755); err != nil {
|
||||
return fmt.Errorf("eval.Mount: data dir: %w", err)
|
||||
}
|
||||
store, err := openStore(filepath.Join(deps.DataDir, "evals.db"))
|
||||
store, err := openStore(deps.DataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("eval.Mount: open metastore: %w", err)
|
||||
}
|
||||
|
||||
@@ -5,11 +5,14 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
|
||||
luxlog "github.com/luxfi/log"
|
||||
"github.com/zap-proto/zip"
|
||||
)
|
||||
@@ -213,7 +216,7 @@ func TestNameValidation(t *testing.T) {
|
||||
|
||||
func testStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := openStore(filepath.Join(t.TempDir(), "evals.db"))
|
||||
s, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
@@ -21,7 +21,7 @@ import (
|
||||
// (no datastore in unit tests) but uses the SAME handlers, store, and validation.
|
||||
func mountApp(t *testing.T) (*zip.App, *service) {
|
||||
t.Helper()
|
||||
store, err := openStore(t.TempDir() + "/evals.db")
|
||||
store, err := openStore(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("openStore: %v", err)
|
||||
}
|
||||
|
||||
+12
-9
@@ -8,12 +8,14 @@ import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
// basedb is the ONE opener: the database is born encrypted under the key cek
|
||||
// derives from the process master and this namespace.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
// github.com/hanzoai/sqlite is the ONE Hanzo SQLite driver: it registers
|
||||
// the "sqlite" database/sql name under both build tags (cgo →
|
||||
// mattn+SQLCipher, encrypted at rest; !cgo → pure-Go modernc). Importing
|
||||
// modernc directly instead would double-register "sqlite" under CGO and
|
||||
// panic at init. Blank import registers the driver.
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// the "sqlite" database/sql name under both build tags. Importing modernc
|
||||
// directly instead would double-register "sqlite" under CGO and panic at
|
||||
// init. Blank import registers the driver.
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
@@ -109,17 +111,18 @@ type DatasetRun struct {
|
||||
UpdatedAt int64
|
||||
}
|
||||
|
||||
// Store is the eval metastore over one SQLite file ({DataDir}/evals.db). Tenancy
|
||||
// Store is the eval metastore over one SQLite file — the deployment's own "evals".
|
||||
// Tenancy
|
||||
// is the org column; MaxOpenConns(1) serializes writes against the file lock
|
||||
// (same discipline as prompts/projects).
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func openStore(path string) (*Store, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
func openStore(dir string) (*Store, error) {
|
||||
db, err := basedb.Open(namespace.System(), "evals", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open evals store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package experiments
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -1,18 +0,0 @@
|
||||
package experiments
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
sqlitedrv "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// TestMain injects a dev master key so the per-org SQLite registry opens under an
|
||||
// encryption-capable build (the repo's shared OrgDB-test setup). 32 dev-only bytes,
|
||||
// temp dirs only.
|
||||
func TestMain(m *testing.M) {
|
||||
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
|
||||
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
|
||||
}
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
package finance
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
+28
-46
@@ -29,14 +29,10 @@
|
||||
package finance
|
||||
|
||||
import (
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
@@ -46,6 +42,7 @@ import (
|
||||
"github.com/hanzoai/cloud/apps/treasury/ledger"
|
||||
"github.com/hanzoai/cloud/apps/treasury/ledger/sqlstore"
|
||||
"github.com/hanzoai/cloud/types"
|
||||
"github.com/hanzoai/namespace"
|
||||
)
|
||||
|
||||
// Client is the in-process inter-subsystem seam cloud's money paths call. It IS cloud's
|
||||
@@ -99,73 +96,58 @@ const (
|
||||
acctWallet = "wallet" // the org pool wallet; a per-user subject is "wallet:<user>"
|
||||
)
|
||||
|
||||
// orgPattern is a SECOND physical name for an org, and it stays that way for
|
||||
// now because this is the money path and unifying it is a file move.
|
||||
//
|
||||
// The pattern is character-for-character namespace's own segment rule, which is
|
||||
// the best evidence the primitive is right. But cloud.OrgNamespace folds the org
|
||||
// through namespace.Sanitize first, and the two disagree on every org that is
|
||||
// not a short clean label: "acme_corp" is accepted verbatim here and becomes
|
||||
// "acme-corp-cca8c7942f8c15a2" there, and a 40-character legal org is kept whole
|
||||
// here and truncated-plus-hashed there. Both write under <dataDir>/orgs/, so
|
||||
// switching the encoder would point the wallet at a different, empty file while
|
||||
// the funded one sat next to it. That is a migration with money in it, not a
|
||||
// rename, and it is not something to do as a side effect of a refactor.
|
||||
//
|
||||
// orgPattern is the safe file-path shape for an org directory — the SAME allowlist
|
||||
// sqlstore's per-tenant opener guards with (a leading alphanumeric then [a-z0-9_-]; no
|
||||
// path separators, no dots, no traversal). An org is used verbatim as a directory name
|
||||
// only when it matches; anything else is refused, so a caller can never place a money
|
||||
// file outside <dataDir>/orgs/ or reach another tenant's file.
|
||||
var orgPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9_-]{0,63}$`)
|
||||
|
||||
// ledgerFinance implements types.FinanceClient over one native ledger file per org.
|
||||
type ledgerFinance struct {
|
||||
dataDir string
|
||||
|
||||
mu sync.Mutex
|
||||
stores map[string]*sqlstore.Store // key: resolved db path (encodes org + test-mode)
|
||||
stores map[ledgerName]*sqlstore.Store
|
||||
}
|
||||
|
||||
// ledgerName is WHICH ledger file: whose it is, and whether it is the sandbox
|
||||
// one. It is the cache key and it is also exactly what the opener takes, so a
|
||||
// hit and a miss can never resolve to different files.
|
||||
type ledgerName struct {
|
||||
ns namespace.Namespace
|
||||
subsystem string
|
||||
}
|
||||
|
||||
// compile-time proof ledgerFinance is the money seam.
|
||||
var _ types.FinanceClient = (*ledgerFinance)(nil)
|
||||
|
||||
// New returns a finance client rooting each org's prepaid wallet file under dataDir
|
||||
// (<dataDir>/orgs/<org>/finance.db, or finance-test.db in sandbox mode). Files open
|
||||
// lazily on first use.
|
||||
// New returns a finance client rooting each org's prepaid wallet ledger under
|
||||
// dataDir, in that org's own namespace ("finance", or "finance-test" in sandbox
|
||||
// mode). Files open lazily on first use.
|
||||
func New(dataDir string) *ledgerFinance {
|
||||
return &ledgerFinance{dataDir: dataDir, stores: make(map[string]*sqlstore.Store)}
|
||||
return &ledgerFinance{dataDir: dataDir, stores: map[ledgerName]*sqlstore.Store{}}
|
||||
}
|
||||
|
||||
// storeFor resolves (opening + caching on first use) the org's ledger file. test picks
|
||||
// the sandbox file so sandbox money never mixes with live. The org is validated against
|
||||
// orgPattern FIRST, so it can never traverse the path or reach another tenant's file.
|
||||
// the sandbox ledger so sandbox money never mixes with live.
|
||||
//
|
||||
// The org becomes a NAMESPACE first — the one injective slugger, which is also what
|
||||
// keys the file — so it can never traverse the path, reach another tenant's ledger,
|
||||
// or fold two distinct orgs onto one wallet.
|
||||
func (f *ledgerFinance) storeFor(org string, test bool) (*sqlstore.Store, error) {
|
||||
org = strings.TrimSpace(org)
|
||||
if !orgPattern.MatchString(org) {
|
||||
return nil, fmt.Errorf("finance: invalid org %q", org)
|
||||
ns, err := namespace.OrgProject(org, "")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("finance: %w", err)
|
||||
}
|
||||
name := "finance.db"
|
||||
name := ledgerName{ns: ns, subsystem: "finance"}
|
||||
if test {
|
||||
name = "finance-test.db"
|
||||
name.subsystem = "finance-test"
|
||||
}
|
||||
dir := filepath.Join(f.dataDir, "orgs", org)
|
||||
path := filepath.Join(dir, name)
|
||||
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if s, ok := f.stores[path]; ok {
|
||||
if s, ok := f.stores[name]; ok {
|
||||
return s, nil
|
||||
}
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return nil, fmt.Errorf("finance: create org dir %q: %w", dir, err)
|
||||
}
|
||||
// This ledger belongs to ONE org, and its key says so (cek-rewrap bound it).
|
||||
s, err := sqlstore.Open(cek.Org(org), path)
|
||||
s, err := sqlstore.Open(name.ns, name.subsystem, f.dataDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("finance: open %q: %w", path, err)
|
||||
return nil, fmt.Errorf("finance: open %s ledger for %s: %w", name.subsystem, name.ns, err)
|
||||
}
|
||||
f.stores[path] = s
|
||||
f.stores[name] = s
|
||||
return s, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -15,7 +15,6 @@ import (
|
||||
// makes the cap enforce on the finance ledger (where the unified binary records
|
||||
// usage) instead of the empty commerce transaction store.
|
||||
func TestSumUsageSince_OrgTotal(t *testing.T) {
|
||||
t.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
|
||||
f := New(t.TempDir())
|
||||
ctx := context.Background()
|
||||
const org = "cap-org"
|
||||
@@ -60,7 +59,6 @@ func TestSumUsageSince_OrgTotal(t *testing.T) {
|
||||
// The usage hook fires after a committed debit — the seam the cap's alert-fire rides,
|
||||
// carrying org + test + scope, WITHOUT finance importing commerce.
|
||||
func TestUsageHook_FiresAfterDebit(t *testing.T) {
|
||||
t.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
|
||||
f := New(t.TempDir())
|
||||
ctx := context.Background()
|
||||
|
||||
@@ -103,7 +101,6 @@ func TestUsageHook_FiresAfterDebit(t *testing.T) {
|
||||
// a page of zeros that summed to zero. The row now carries both; the exact
|
||||
// value is the source, the cents its rendering.
|
||||
func TestListUsage_CarriesTheExactDebit(t *testing.T) {
|
||||
t.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
|
||||
f := New(t.TempDir())
|
||||
ctx := context.Background()
|
||||
const org = "exact-org"
|
||||
|
||||
@@ -16,7 +16,6 @@ import (
|
||||
func BenchmarkListUsage(b *testing.B) {
|
||||
for _, n := range []int{100, 1000, 5000} {
|
||||
b.Run(fmt.Sprintf("entries=%d", n), func(b *testing.B) {
|
||||
b.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
|
||||
f := New(b.TempDir())
|
||||
ctx := context.Background()
|
||||
for i := 0; i < n; i++ {
|
||||
|
||||
@@ -12,7 +12,6 @@ import (
|
||||
// RecordUsage wrote (magnitude + model), most-recent-first, and excludes deposits —
|
||||
// so /v1/billing/usage can answer from the ledger instead of self-dispatching.
|
||||
func TestListUsage(t *testing.T) {
|
||||
t.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
|
||||
f := New(t.TempDir())
|
||||
ctx := context.Background()
|
||||
const org = "acme"
|
||||
|
||||
+1
-1
@@ -73,7 +73,7 @@ const (
|
||||
|
||||
// The reserved store the platform switches evaluate from.
|
||||
//
|
||||
// This is a REAL org namespace named "platform", not cloud.PlatformNamespace().
|
||||
// This is a REAL org namespace named "platform", not namespace.System().
|
||||
// The system namespace is the right name for it and would make it unsquattable
|
||||
// by a tenant who registers that org, but it renders to a different file, and
|
||||
// moving a live store is a migration rather than a rename. Left as it is,
|
||||
|
||||
@@ -1,36 +1,21 @@
|
||||
package flags
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/json"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
)
|
||||
|
||||
// The cek data plane fail-closes without a master key on encryption-capable
|
||||
// builds; tests supply one process-wide (SetMasterKey is once-only).
|
||||
var cekOnce sync.Once
|
||||
|
||||
func testMasterKey(t *testing.T) {
|
||||
t.Helper()
|
||||
cekOnce.Do(func() {
|
||||
k := make([]byte, 32)
|
||||
if _, err := rand.Read(k); err != nil {
|
||||
t.Fatalf("rng: %v", err)
|
||||
}
|
||||
cek.SetMasterKey(k)
|
||||
})
|
||||
}
|
||||
|
||||
// newTestClient builds a Client over a temp-dir store tree and installs it as the
|
||||
// process seam, restoring the prior seam on cleanup.
|
||||
func newTestClient(t *testing.T) *Client {
|
||||
t.Helper()
|
||||
testMasterKey(t)
|
||||
prev := mounted
|
||||
c := &Client{
|
||||
stores: cloud.NewOrgStore[*Store](cloud.Base{DataDir: t.TempDir()}, "flags", openStore),
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package framework
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -41,9 +41,10 @@ import (
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
"github.com/hanzoai/cloud/apps/principal"
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/cloud/openapi"
|
||||
engine "github.com/hanzoai/framework"
|
||||
"github.com/hanzoai/namespace"
|
||||
"github.com/zap-proto/zip"
|
||||
)
|
||||
|
||||
@@ -69,15 +70,18 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
|
||||
}
|
||||
log := deps.Logger.New("subsystem", "framework")
|
||||
|
||||
// cek is CLOUD's storage policy — encrypted at rest under a KMS-held master
|
||||
// key. The engine takes it as an opener rather than importing it, so the
|
||||
// same engine runs unencrypted in a test or a standalone app.
|
||||
// The engine opens the deployment's own DocType stores under deps.DataDir, not a
|
||||
// tenant's, so they key under the platform principal. A per-org DocType store would
|
||||
// come through OrgDB, which names its owner.
|
||||
// Encryption at rest is CLOUD's storage policy, under a KMS-held master key. The
|
||||
// engine takes an opener rather than importing it, so the same engine runs
|
||||
// unencrypted in a test or a standalone app.
|
||||
//
|
||||
// The engine has ONE database and offers its path; cloud names it instead. These
|
||||
// are the DEPLOYMENT's own DocType stores rather than a tenant's, so the system
|
||||
// namespace owns them — and a name is what keys a file here, which a path handed
|
||||
// down from a library cannot be. A per-org DocType store would come through
|
||||
// OrgDB, which names its owner.
|
||||
eng, err := engine.Open(engine.Config{
|
||||
Dir: deps.DataDir,
|
||||
OpenDB: func(path string) (*sql.DB, error) { return cek.Open(cek.Global, path) },
|
||||
OpenDB: func(string) (*sql.DB, error) { return basedb.Open(namespace.System(), "framework", deps.DataDir) },
|
||||
Logger: log,
|
||||
})
|
||||
if err != nil {
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
package framework
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
)
|
||||
|
||||
// TestMain seeds a random cek master key so the encrypted-at-rest store opens on an
|
||||
// encryption-capable test build (mirrors clients/compliance, integrations, flags, git,
|
||||
// venue). On a pure-Go build cek ignores it and uses the plaintext dev path.
|
||||
func TestMain(m *testing.M) {
|
||||
k := make([]byte, 32)
|
||||
if _, err := rand.Read(k); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
cek.SetMasterKey(k)
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
package functions
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
sqlitedrv "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// TestMain makes the functions suite build-tag agnostic (mirrors the sibling
|
||||
// clients/automations harness). On an encryption-capable (cgo) build cek REFUSES
|
||||
// to open a store without a master key; on a pure-Go build a key is itself
|
||||
// refused. So supply a throwaway dev key ONLY when the build can encrypt AND the
|
||||
// environment did not already provide one (CI may inject the real key) — then
|
||||
// every store opens (encrypted on cgo, plaintext on pure-Go) without ever
|
||||
// overriding a provided key. Resolved once per process, order-independent.
|
||||
func TestMain(m *testing.M) {
|
||||
if sqlitedrv.EncryptionAvailable() && os.Getenv("CLOUD_KMS_MASTER_KEY_REF") == "" {
|
||||
_ = os.Setenv("CLOUD_KMS_MASTER_KEY_REF", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=") // 32 zero bytes, dev-only
|
||||
}
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
@@ -3,12 +3,16 @@ package functions
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
|
||||
luxlog "github.com/luxfi/log"
|
||||
"github.com/zap-proto/zip"
|
||||
)
|
||||
@@ -48,13 +52,18 @@ func TestPerOrgStoreFileIsolation(t *testing.T) {
|
||||
t.Fatalf("orgB create (same name): %d %s", code, b)
|
||||
}
|
||||
|
||||
// Close before stat: on the pure-Go codec the database is written back at
|
||||
// CLOSE, so an open store has not yet landed its file. Shutdown is idempotent,
|
||||
// so the Cleanup above still runs harmlessly.
|
||||
if err := Shutdown(); err != nil {
|
||||
t.Fatalf("Shutdown: %v", err)
|
||||
}
|
||||
|
||||
fa := filepath.Join(dir, "orgs", "orga", "functions.db")
|
||||
fb := filepath.Join(dir, "orgs", "orgb", "functions.db")
|
||||
for _, p := range []string{fa, fb} {
|
||||
// cek.Exists, not os.Stat: a store still OPEN has not materialized its
|
||||
// database file on the pure-Go codec — only its sidecar is on disk.
|
||||
if !cek.Exists(p) {
|
||||
t.Fatalf("expected per-org functions store at %s", p)
|
||||
if _, err := os.Stat(p); err != nil {
|
||||
t.Fatalf("expected per-org functions store at %s: %v", p, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package gateway
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
@@ -0,0 +1,5 @@
|
||||
package edge
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master, and a test
|
||||
// process has no KMS to resolve one from.
|
||||
import _ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
+10
-12
@@ -34,14 +34,13 @@ import (
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/hanzoai/cloud/cek" // opens gateway.db encrypted at rest; a leaf pkg, no import cycle.
|
||||
_ "github.com/hanzoai/sqlite" // the ONE "sqlite" driver, for cek's no-key plaintext fallback.
|
||||
"github.com/hanzoai/cloud/basedb" // opens the policy DB encrypted at rest; a leaf pkg, no import cycle.
|
||||
"github.com/hanzoai/namespace"
|
||||
_ "github.com/hanzoai/sqlite" // the ONE "sqlite" driver.
|
||||
)
|
||||
|
||||
// Policy is the edge policy for one scope. Zero-valued fields mean "inherit"
|
||||
@@ -190,19 +189,18 @@ type cacheEntry struct {
|
||||
expiry time.Time
|
||||
}
|
||||
|
||||
// New opens (or creates) {dataDir}/gateway.db and returns a Store layered over the
|
||||
// static boot defaults. On any open/migrate error it logs nothing here (the caller
|
||||
// owns logging) and returns a static-only Store plus the error, so the caller can
|
||||
// wire the edge middleware with a working fallback regardless.
|
||||
// New opens (or creates) the deployment's gateway policy database under dataDir and
|
||||
// returns a Store layered over the static boot defaults. On any open/migrate error it
|
||||
// logs nothing here (the caller owns logging) and returns a static-only Store plus the
|
||||
// error, so the caller can wire the edge middleware with a working fallback regardless.
|
||||
func New(dataDir, adminOrg string, static Policy) (*Store, error) {
|
||||
s := &Store{adminOrg: adminOrg, static: static, cache: map[string]cacheEntry{}}
|
||||
if dataDir == "" {
|
||||
return s, fmt.Errorf("edge: empty dataDir; running static-only")
|
||||
}
|
||||
if err := os.MkdirAll(dataDir, 0o750); err != nil {
|
||||
return s, fmt.Errorf("edge: mkdir %s: %w", dataDir, err)
|
||||
}
|
||||
db, err := cek.Open(cek.Global, filepath.Join(dataDir, "gateway.db"))
|
||||
// The policy table is the DEPLOYMENT's, keyed by org rather than split per org,
|
||||
// so it lives in the system namespace.
|
||||
db, err := basedb.Open(namespace.System(), "gateway", dataDir)
|
||||
if err != nil {
|
||||
return s, fmt.Errorf("edge: open: %w", err)
|
||||
}
|
||||
|
||||
@@ -1,28 +1,16 @@
|
||||
package git
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
)
|
||||
|
||||
// The cek data plane fail-closes without a master key on encryption-capable
|
||||
// builds; supply one process-wide so the per-org git.db opens in local runs
|
||||
// exactly as it does in CI (mirrors clients/flags).
|
||||
func TestMain(m *testing.M) {
|
||||
k := make([]byte, 32)
|
||||
if _, err := rand.Read(k); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
cek.SetMasterKey(k)
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
// TestPublicRepo_AnonymousRead proves the visibility model end to end:
|
||||
// - a repo defaults PRIVATE: anonymous fetch advertisement is a uniform 404
|
||||
// - PATCH {"public":true} (org-authed) flips it
|
||||
|
||||
@@ -3,11 +3,10 @@ package git
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
|
||||
"github.com/hanzoai/cloud"
|
||||
luxlog "github.com/luxfi/log"
|
||||
"github.com/zap-proto/zip"
|
||||
@@ -59,14 +58,19 @@ func TestPerOrgStoreFileIsolation(t *testing.T) {
|
||||
t.Fatalf("orgB create (same name): %d %s", code, b)
|
||||
}
|
||||
|
||||
// Close before stat: on the pure-Go codec the database is written back at
|
||||
// CLOSE, so an open store has not yet landed its file. Shutdown is idempotent,
|
||||
// so mountAppDir's Cleanup still runs harmlessly.
|
||||
if err := Shutdown(); err != nil {
|
||||
t.Fatalf("Shutdown: %v", err)
|
||||
}
|
||||
|
||||
// Two physically distinct per-org DBs exist.
|
||||
fa := filepath.Join(dir, "orgs", "orga", "git.db")
|
||||
fb := filepath.Join(dir, "orgs", "orgb", "git.db")
|
||||
for _, p := range []string{fa, fb} {
|
||||
// cek.Exists, not os.Stat: a store still OPEN has not materialized its
|
||||
// database file on the pure-Go codec — only its sidecar is on disk.
|
||||
if !cek.Exists(p) {
|
||||
t.Fatalf("expected per-org git store at %s", p)
|
||||
if _, err := os.Stat(p); err != nil {
|
||||
t.Fatalf("expected per-org git store at %s: %v", p, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+6
-7
@@ -70,16 +70,15 @@ type BaseRequest struct {
|
||||
// BaseConfig configures a BaseHost.
|
||||
type BaseConfig struct {
|
||||
// Name identifies the subsystem ("captable", "esign", "dataroom"). It names
|
||||
// the goja host AND the per-tenant data subdir ({DataDir}/{Name}/).
|
||||
// the goja host AND the per-tenant database it opens.
|
||||
Name string
|
||||
// Bundle is the self-contained goja bundle exposing globalThis.handle.
|
||||
Bundle []byte
|
||||
// Schema is the per-tenant SQLite DDL, run (idempotently — use
|
||||
// CREATE TABLE IF NOT EXISTS) on every tenant DB when it first opens.
|
||||
Schema string
|
||||
// DataDir is the deployment data root; per-tenant files land at
|
||||
// {DataDir}/{Name}/{TenantSegment(tenant)}.db (injective, traversal-safe
|
||||
// base32 of the raw org bytes — see basestore.go TenantSegment).
|
||||
// DataDir is the deployment data root; per-tenant files land where namespace
|
||||
// renders them, at {DataDir}/orgs/{slug}/{Name}.db.
|
||||
DataDir string
|
||||
// OnOpen is an optional per-tenant seed hook run ONCE after migration (e.g.
|
||||
// captable seeds the tenant's company row). It runs outside the per-request
|
||||
@@ -167,7 +166,7 @@ func (h *BaseHost) Dispatch(ctx context.Context, tenant string, req BaseRequest)
|
||||
}
|
||||
// Tenant-bound object-storage seam (e.g. sign's PDFs). Keys are namespaced to
|
||||
// {Name}/{TenantSegment} inside the bridge, so a bundle can only ever reach its
|
||||
// OWN tenant's blobs — the same injective TenantSegment the per-tenant DB uses.
|
||||
// OWN tenant's blobs.
|
||||
if h.blob != nil {
|
||||
globals["__blob"] = h.blobBridge(ctx, tenant)
|
||||
}
|
||||
@@ -226,8 +225,8 @@ func newBridge(ctx context.Context, q execQuerier) map[string]any {
|
||||
// blobBridge builds the __blob object (put/get over base64) bound to ctx + the
|
||||
// tenant. Keys are namespaced to {name}/{TenantSegment(tenant)}/ so a bundle can
|
||||
// only ever address its OWN tenant's objects — cross-tenant isolation is a host
|
||||
// property, using the same injective encoding the per-tenant DB file uses. The
|
||||
// bundle handles base64 strings only; the binding decodes/encodes at the boundary.
|
||||
// property, using an injective encoding of the raw org bytes. The bundle handles
|
||||
// base64 strings only; the binding decodes/encodes at the boundary.
|
||||
func (h *BaseHost) blobBridge(ctx context.Context, tenant string) map[string]any {
|
||||
prefix := h.name + "/" + TenantSegment(tenant) + "/"
|
||||
return map[string]any{
|
||||
|
||||
+57
-54
@@ -7,41 +7,45 @@ import (
|
||||
"encoding/base32"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
// cek opens each per-tenant file encrypted at rest.
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// The ONE "sqlite" driver, kept registered for cek's no-key plaintext fallback.
|
||||
// cloud.OrgNamespace is the ONE door a tenant string passes through to become
|
||||
// the name of a database.
|
||||
"github.com/hanzoai/cloud"
|
||||
// basedb is the ONE opener: each tenant's database is born encrypted under the
|
||||
// key cek derives from the process master and that tenant's namespace.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
// The ONE "sqlite" driver.
|
||||
_ "github.com/hanzoai/sqlite"
|
||||
)
|
||||
|
||||
// tenantEnc is the ONE tenant→path-segment encoder. Lowercased, unpadded base32
|
||||
// tenantEnc is the ONE tenant→object-key encoder. Lowercased, unpadded base32
|
||||
// (RFC 4648 alphabet a-z2-7) of the RAW org bytes. See TenantSegment.
|
||||
var tenantEnc = base32.StdEncoding.WithPadding(base32.NoPadding)
|
||||
|
||||
// TenantSegment maps a tenant/org key to an INJECTIVE, traversal-safe single path
|
||||
// segment: lowercased unpadded base32 of the RAW org bytes.
|
||||
// TenantSegment maps a tenant/org key to an INJECTIVE, traversal-safe single
|
||||
// OBJECT-STORE key segment: lowercased unpadded base32 of the RAW org bytes.
|
||||
//
|
||||
// This is deliberately NOT a "slug": lowercasing + folding illegal bytes to '_'
|
||||
// (the previous slugify) is NON-injective and collapses DISTINCT owners onto ONE
|
||||
// physical store — a cross-tenant break. principal.Org returns the org VERBATIM
|
||||
// physical prefix — a cross-tenant break. principal.Org returns the org VERBATIM
|
||||
// for exactly that reason (folding "Acme"/"acme" or "a b"/"a_b" into one bucket is
|
||||
// itself a tenant break); this encoder preserves that distinction all the way to
|
||||
// the on-disk name. base32 of the raw bytes is a bijection with its output, so
|
||||
// the stored name. base32 of the raw bytes is a bijection with its output, so
|
||||
// distinct orgs ALWAYS map to distinct segments: "Acme", "acme", "a b" and "a_b"
|
||||
// each land on their OWN file (the previous slugify collapsed all four onto two).
|
||||
// each land on their OWN prefix (the previous slugify collapsed all four onto two).
|
||||
//
|
||||
// The [a-z2-7] output contains no path separator and can never equal "." or ".."
|
||||
// (those need chars outside the alphabet), so a segment can never traverse the
|
||||
// data tree or an object-store key prefix. Empty tenant → "" (callers reject it).
|
||||
// (those need chars outside the alphabet), so a segment can never traverse an
|
||||
// object-store key prefix. Empty tenant → "" (callers reject it).
|
||||
//
|
||||
// It is the SINGLE encoding every gojabase tenant-keyed path uses — the per-tenant
|
||||
// SQLite filename here AND the dataroom object-store key prefix — so a tenant maps
|
||||
// to exactly ONE physical identity everywhere.
|
||||
// It does NOT name the per-tenant DATABASE. That name is namespace's — one
|
||||
// slugger for every file cloud opens — and this encoder is what remains for the
|
||||
// blob half, which has no namespace rendering of its own.
|
||||
func TenantSegment(tenant string) string {
|
||||
if tenant == "" {
|
||||
return ""
|
||||
@@ -61,40 +65,38 @@ const (
|
||||
|
||||
// stores is a hand-rolled hanzoai/orm/db.Namespaces — the same design (bound on
|
||||
// open handles, idle sweep, never evict while pinned, single-flighted opens)
|
||||
// arrived at independently — and it is not collapsed onto it yet, deliberately.
|
||||
//
|
||||
// Two things block it, and neither is about this file being wrong. The keys are
|
||||
// TenantSegment, a base32 of the raw org bytes, which shares no output with
|
||||
// namespace.Sanitize for any tenant at all; every gojabase file and every dataroom
|
||||
// object key is named by it, so adopting the fleet name relocates all of them at
|
||||
// once. And orm/db.Namespaces takes its own string-typed db.Namespace rather
|
||||
// than a namespace.Namespace, so adopting it today would bring a THIRD naming
|
||||
// type into cloud — adding a way rather than removing one. When orm/db takes the
|
||||
// value, this pool is the first thing that should go.
|
||||
// arrived at independently — and it is not collapsed onto it yet, deliberately:
|
||||
// orm/db.Namespaces takes its own string-typed db.Namespace rather than a
|
||||
// namespace.Namespace, so adopting it today would bring a THIRD naming type into
|
||||
// cloud — adding a way rather than removing one. When orm/db takes the value,
|
||||
// this pool is the first thing that should go.
|
||||
//
|
||||
// stores is the per-tenant SQLite manager: ONE database FILE per tenant
|
||||
// ({DataDir}/{name}/{TenantSegment}.db), the "Prod = SQLite per tenant" rule
|
||||
// (HIP-0302). Files open lazily on first use, migrate once (the subsystem's Schema
|
||||
// DDL), run the optional OnOpen seed, and are pooled (LRU-capped, idle-evicted).
|
||||
// Concurrent opens of the same tenant are single-flighted under mu.
|
||||
// ({DataDir}/orgs/{slug}/{name}.db, rendered by namespace like every other file
|
||||
// cloud opens), the "Prod = SQLite per tenant" rule (HIP-0302). Files open lazily
|
||||
// on first use, migrate once (the subsystem's Schema DDL), run the optional OnOpen
|
||||
// seed, and are pooled (LRU-capped, idle-evicted). Concurrent opens of the same
|
||||
// tenant are single-flighted under mu.
|
||||
type stores struct {
|
||||
name string
|
||||
dir string
|
||||
schema string
|
||||
onOpen func(ctx context.Context, tenant string, db *sql.DB) error
|
||||
name string
|
||||
dataDir string
|
||||
schema string
|
||||
onOpen func(ctx context.Context, tenant string, db *sql.DB) error
|
||||
|
||||
maxOpen int
|
||||
idleTTL time.Duration
|
||||
|
||||
mu sync.Mutex
|
||||
m map[string]*entry // segment → entry
|
||||
lru *list.List // *entry, front = most-recently-used, back = LRU
|
||||
mu sync.Mutex
|
||||
// m is keyed by the NAMESPACE, which is what names the file: two spellings
|
||||
// that render one database cannot become two pooled handles on it.
|
||||
m map[namespace.Namespace]*entry
|
||||
lru *list.List // *entry, front = most-recently-used, back = LRU
|
||||
}
|
||||
|
||||
// entry is one pooled tenant DB. inUse counts in-flight dispatches holding it; an
|
||||
// entry is evictable only when inUse==0 (never close a DB mid-transaction).
|
||||
type entry struct {
|
||||
seg string
|
||||
ns namespace.Namespace
|
||||
db *sql.DB
|
||||
inUse int
|
||||
lastUsed time.Time
|
||||
@@ -104,12 +106,12 @@ type entry struct {
|
||||
func newStores(name, dataDir, schema string, onOpen func(context.Context, string, *sql.DB) error) *stores {
|
||||
return &stores{
|
||||
name: name,
|
||||
dir: filepath.Join(dataDir, name),
|
||||
dataDir: dataDir,
|
||||
schema: schema,
|
||||
onOpen: onOpen,
|
||||
maxOpen: envInt("CLOUD_GOJABASE_MAX_DBS", defaultMaxOpen),
|
||||
idleTTL: time.Duration(envInt("CLOUD_GOJABASE_IDLE_TTL_SEC", int(defaultIdleTTL/time.Second))) * time.Second,
|
||||
m: make(map[string]*entry),
|
||||
m: make(map[namespace.Namespace]*entry),
|
||||
lru: list.New(),
|
||||
}
|
||||
}
|
||||
@@ -124,14 +126,17 @@ func (s *stores) acquire(ctx context.Context, tenant string) (*sql.DB, func(), e
|
||||
if strings.TrimSpace(tenant) == "" {
|
||||
return nil, nil, fmt.Errorf("gojabase[%s]: empty tenant", s.name)
|
||||
}
|
||||
seg := TenantSegment(tenant)
|
||||
ns, err := cloud.OrgNamespace(tenant, "")
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("gojabase[%s]: %w", s.name, err)
|
||||
}
|
||||
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
s.sweepIdleLocked(time.Now())
|
||||
|
||||
if e, ok := s.m[seg]; ok {
|
||||
if e, ok := s.m[ns]; ok {
|
||||
e.inUse++
|
||||
e.lastUsed = time.Now()
|
||||
s.lru.MoveToFront(e.el)
|
||||
@@ -142,13 +147,13 @@ func (s *stores) acquire(ctx context.Context, tenant string) (*sql.DB, func(), e
|
||||
// entry is pinned we exceed the cap transiently rather than close a live DB.
|
||||
s.evictToCapLocked()
|
||||
|
||||
db, err := s.openLocked(ctx, tenant, seg)
|
||||
db, err := s.openLocked(ctx, ns, tenant)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
e := &entry{seg: seg, db: db, inUse: 1, lastUsed: time.Now()}
|
||||
e := &entry{ns: ns, db: db, inUse: 1, lastUsed: time.Now()}
|
||||
e.el = s.lru.PushFront(e)
|
||||
s.m[seg] = e
|
||||
s.m[ns] = e
|
||||
return e.db, s.releaser(e), nil
|
||||
}
|
||||
|
||||
@@ -165,15 +170,13 @@ func (s *stores) releaser(e *entry) func() {
|
||||
}
|
||||
|
||||
// openLocked opens+pragma+migrates+seeds a tenant DB. Runs under s.mu so a tenant
|
||||
// is opened exactly once (single-flight). The pragmas/migration/seed use ctx.
|
||||
func (s *stores) openLocked(ctx context.Context, tenant, seg string) (*sql.DB, error) {
|
||||
if err := os.MkdirAll(s.dir, 0o750); err != nil {
|
||||
return nil, fmt.Errorf("gojabase[%s]: mkdir %q: %w", s.name, s.dir, err)
|
||||
}
|
||||
path := filepath.Join(s.dir, seg+".db")
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
// is opened exactly once (single-flight). The pragmas/migration/seed use ctx. It
|
||||
// takes the namespace (which names the file) AND the raw tenant (which the OnOpen
|
||||
// seed is written in terms of).
|
||||
func (s *stores) openLocked(ctx context.Context, ns namespace.Namespace, tenant string) (*sql.DB, error) {
|
||||
db, err := basedb.Open(ns, s.name, s.dataDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("gojabase[%s]: open %q: %w", s.name, path, err)
|
||||
return nil, fmt.Errorf("gojabase[%s]: open %s: %w", s.name, ns, err)
|
||||
}
|
||||
// MaxOpenConns(1) serializes writes against the single-writer file; the
|
||||
// per-request transaction (see Dispatch) then holds that one connection for
|
||||
@@ -248,7 +251,7 @@ func (s *stores) oldestClosableLocked() *entry {
|
||||
func (s *stores) closeEntryLocked(e *entry) {
|
||||
_ = e.db.Close()
|
||||
s.lru.Remove(e.el)
|
||||
delete(s.m, e.seg)
|
||||
delete(s.m, e.ns)
|
||||
}
|
||||
|
||||
// closeAll closes every open tenant DB and clears the pool. Idempotent. Called at
|
||||
@@ -262,7 +265,7 @@ func (s *stores) closeAll() error {
|
||||
firstErr = err
|
||||
}
|
||||
}
|
||||
s.m = make(map[string]*entry)
|
||||
s.m = make(map[namespace.Namespace]*entry)
|
||||
s.lru.Init()
|
||||
return firstErr
|
||||
}
|
||||
|
||||
@@ -9,13 +9,17 @@ import (
|
||||
"strconv"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
// devmaster keys this test binary: cek opens nothing without a master and a
|
||||
// test process has no KMS.
|
||||
_ "github.com/hanzoai/cloud/internal/devmaster"
|
||||
)
|
||||
|
||||
// TestTenantSegmentInjective is the C1 proof: the tenant→filename mapping is
|
||||
// INJECTIVE (distinct orgs never share a file) and TRAVERSAL-SAFE (the segment is
|
||||
// always a single [a-z2-7] path component that can never escape the data subtree).
|
||||
// The previous slugify ToLower+folded — collapsing "Acme"/"acme" and "a b"/"a_b"
|
||||
// onto shared SQLite files, a cross-tenant break. This must never regress.
|
||||
// TestTenantSegmentInjective is the C1 proof: the tenant→object-key mapping is
|
||||
// INJECTIVE (distinct orgs never share a blob prefix) and TRAVERSAL-SAFE (the
|
||||
// segment is always a single [a-z2-7] component that can never escape its
|
||||
// subtree). The previous slugify ToLower+folded — collapsing "Acme"/"acme" and
|
||||
// "a b"/"a_b" onto shared storage, a cross-tenant break. This must never regress.
|
||||
func TestTenantSegmentInjective(t *testing.T) {
|
||||
// An adversarial set that specifically includes the collision pairs the old
|
||||
// slugify folded together, plus path-traversal payloads.
|
||||
@@ -131,10 +135,12 @@ func TestConcurrentMultiTenantDispatch(t *testing.T) {
|
||||
h := newTinyHost(t, nil)
|
||||
ctx := context.Background()
|
||||
|
||||
// Distinct tenants, deliberately including collision-prone pairs the old
|
||||
// slugify folded ("Acme"/"acme", "a b"/"a_b") so isolation is proven, not argued.
|
||||
// Distinct tenants, deliberately including collision-prone pairs a lossy
|
||||
// slugger folds ("Acme"/"acme", "a.b"/"a_b") so isolation is proven, not
|
||||
// argued. An org carrying whitespace is not in the corpus because namespace
|
||||
// refuses one outright — it never reaches a file to collide in.
|
||||
tenants := []string{
|
||||
"Acme", "acme", "a b", "a_b", "globex", "GLOBEX",
|
||||
"Acme", "acme", "a.b", "a_b", "globex", "GLOBEX",
|
||||
"initech", "umbrella", "stark", "wayne", "wonka", "hooli",
|
||||
}
|
||||
const perTenant = 25
|
||||
|
||||
@@ -6,7 +6,10 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/hanzoai/cloud/cek"
|
||||
// basedb is the ONE opener: the database is born encrypted under the key cek
|
||||
// derives from the process master and this namespace.
|
||||
"github.com/hanzoai/cloud/basedb"
|
||||
"github.com/hanzoai/namespace"
|
||||
)
|
||||
|
||||
// blueprint_store.go is the SHARED, platform-scoped store for the brand blueprint —
|
||||
@@ -33,12 +36,14 @@ type BlueprintStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
// openBlueprintStore opens the shared blueprint DB at path (a cek-sealed SQLite file,
|
||||
// the house pattern) and migrates. MaxOpenConns(1) serializes writes.
|
||||
func openBlueprintStore(path string) (*BlueprintStore, error) {
|
||||
db, err := cek.Open(cek.Global, path)
|
||||
// openBlueprintStore opens the shared blueprint DB under dir and migrates. It is
|
||||
// the DEPLOYMENT's own partition — namespace.System takes no input and names no
|
||||
// entity — because a brand blueprint is platform content, not an org's.
|
||||
// MaxOpenConns(1) serializes writes.
|
||||
func openBlueprintStore(dir string) (*BlueprintStore, error) {
|
||||
db, err := basedb.Open(namespace.System(), "guide-blueprint", dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open blueprint store %q: %w", path, err)
|
||||
return nil, fmt.Errorf("open blueprint store: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
for _, pragma := range []string{
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user