mirror of
https://github.com/hanzoai/sql.git
synced 2026-11-15 19:38:30 +00:00
Replaces the shared multi-tenant Postgres (ghcr.io/hanzoai/sql:18) with the
per-tenant, SQLite-backed, provision-on-demand model — the relational analog of
docdb (FerretDB-on-SQLite). Builds under sql/managed/, never touches the
Postgres source tree.
Crux decided by real prisma output (evidence/):
- Option A (pglite pg-wire): the UNMODIFIED dataroom pg schema (62 tables, 18
enums, 11 array cols, @db.* native types) `prisma db push`es and runs a live
enum+text[] CRUD roundtrip → the pg-wire escape hatch for un-rewritable
schemas / external ManagedDatabase tenants.
- Option B/C (SQLite dialect): the same schema is rejected at Prisma validation
(P1012, 11 scalar-list + 6 native-type errors) before any backend — no
pg-wire-over-SQLite proxy can fix it. postlite rejected.
- Primary path stays file:SQLite (esign/Service.persistence): rewrite the schema
(arrays→JSON String) + one-time data ETL. esign already runs this in prod.
Deliverables:
- LLM.md: product spec — existing operator model (Service.persistence, SQL/
ManagedDatabase datastore Kinds), per-tenant layout, provisioning, DNS,
DATABASE_URL handoff (KMSSecret→env, mirrors docdb MONGO_URI), image
ghcr.io/hanzoai/sql-managed (semver only), blockers (DatastoreSpec fsGroup gap,
multi-tenant router, inert credentialsSecret).
- deploy/sql.yaml (path A, SQL Kind, sql-managed image) +
service-persistence.example.yaml (path C, esign pattern).
- etl/enum-array-parser.patch: fixes a real data-loss bug in esign's
backfill-pg-to-sqlite.ts — node-pg returns enum arrays (Role[]) as the raw
string {ADMIN,USER}, aborting the load (User.roles is set for every user).
Registers node-pg's text-array parser for enum-array OIDs. Validated pg→SQLite
on 4554 synthetic rows: full parity + byte-exact type fidelity + esign Prisma
read-back (evidence/etl-pg-to-sqlite-validation.log).
- pgwire/: tested pglite pg-wire prototype (server.mjs, provision.mjs, Dockerfile
FROM ghcr.io/hanzoai/nodejs:v24.18.0) — the Option-A follow-up.
Co-authored-by: Hanzo Dev <[email protected]>
42 lines
1.7 KiB
Diff
42 lines
1.7 KiB
Diff
diff --git a/scripts/backfill-pg-to-sqlite.ts b/scripts/backfill-pg-to-sqlite.ts
|
|
index bee5737f7..08a6be8e5 100644
|
|
--- a/scripts/backfill-pg-to-sqlite.ts
|
|
+++ b/scripts/backfill-pg-to-sqlite.ts
|
|
@@ -41,7 +41,26 @@
|
|
import { existsSync, readFileSync, unlinkSync } from 'node:fs';
|
|
import process from 'node:process';
|
|
import { DatabaseSync } from 'node:sqlite';
|
|
-import { Client } from 'pg';
|
|
+import { Client, types } from 'pg';
|
|
+
|
|
+/**
|
|
+ * node-pg parses built-in `text[]` (_text, OID 1009) into a JS array, but a
|
|
+ * column typed as an ARRAY OF A CUSTOM ENUM (e.g. `Role[]`, `WebhookTriggerEvents[]`)
|
|
+ * has a per-database dynamic OID node-pg has no parser for, so it hands back the
|
|
+ * raw literal `{ADMIN,USER}` as a string. That trips toSqlite's fail-closed
|
|
+ * array guard. Reuse node-pg's own text-array parser for every enum-array OID so
|
|
+ * enum arrays decode to `string[]` exactly like `text[]`.
|
|
+ */
|
|
+const registerEnumArrayParsers = async (pg: Client): Promise<number> => {
|
|
+ const textArrayParser = types.getTypeParser(1009); // _text
|
|
+ const res = await pg.query(
|
|
+ `SELECT arr.oid FROM pg_type arr
|
|
+ JOIN pg_type el ON el.oid = arr.typelem
|
|
+ WHERE arr.typcategory = 'A' AND el.typtype = 'e'`,
|
|
+ );
|
|
+ for (const { oid } of res.rows) types.setTypeParser(oid as number, textArrayParser);
|
|
+ return res.rows.length;
|
|
+};
|
|
|
|
type Arg = { batch: number; fresh: boolean };
|
|
|
|
@@ -166,6 +185,8 @@ async function main() {
|
|
|
|
const pg = new Client({ connectionString: pgUrl });
|
|
await pg.connect();
|
|
+ const enumArrayOids = await registerEnumArrayParsers(pg);
|
|
+ console.log(`[backfill] registered enum-array parsers for ${enumArrayOids} OID(s)`);
|
|
|
|
let totalCopied = 0;
|
|
|