A database with a .dek sidecar is encrypted under a random DEK only the
sidecar holds, so a copy of the file alone is unreadable: a durable
snapshot, a backup or a new pod hydrating one loses it the day the disk
holding the sidecar is lost. Rekey moves such a database to the key its
owner and subsystem derive, then the sidecar goes.
The copy is Convert's: export reads the source under the DEK into a
fresh database under the derived key, and verify holds it row for row
before anything moves. The commit sets the original and its sidecar
aside as a pair, swaps the copy in, and drops the pair only once the
copy reads under its key; recoverRekey reads a stop between any two
steps back from the files. With no sidecar and nothing interrupted it
costs two stats and takes no lock.
Claude-Session: https://claude.ai/code/session_01SCPMGSQsLNF2YtHFL4bwyp
One toolchain across the estate. The tree carried 1.26.0, .1, .3, .4 and .5
at once, so which compiler built a binary depended on which repo it came
from.
Co-authored-by: Hanzo Dev <[email protected]>
hanzoai/ci keeps the reusable at .github/workflows/build.yml — the one path
both planes resolve, since github.com reads a reusable only from there and the
forge reads it too. The v1 tag moved with it, so a caller still naming
.hanzo/workflows/build.yml asks for a blob that is not in the tree:
InsertRun: read hanzoai/ci@v1:.hanzo/workflows/build.yml: object does not exist
The caller itself stays at .hanzo/workflows/cicd.yml on purpose: github.com has
no runner for these labels, so a caller placed there could never be scheduled.
Co-authored-by: Hanzo Dev <[email protected]>
A file's key comes from two joins: an owner and a file id joined into one
identity, and a namespace and a subsystem joined into one info string. Both
joins were described but never stated — the vectors took the pieces already
joined, so they held for any way of joining them.
These state them. Every expected value is a literal from an implementation of
the scheme written separately from this one, in another language, from the
original source: hanzoai/sqlite v0.4.0 and hanzoai/cloud v1.801.360. That
implementation reproduces the vectors already here exactly, which is what makes
it a reference rather than a second opinion.
A sidecar sealed by it is opened here from nothing but a namespace and a file
on disk, so the reader has to arrive at the owner and the file rather than be
handed them. The encoding is read back by a reader written from its definition,
over lengths that run past 128, where the length prefix grows a second byte,
and past 16384, where it grows a third.
The derivation is unchanged.
Co-authored-by: Hanzo Dev <[email protected]>
"wrong master key, or a damaged store" was one sentence covering two faults with
opposite remedies. The first wants the other key and the bytes are intact. The
second wants a restore and the bytes are gone. Nothing on disk told them apart,
so telling them apart meant forensics: reading 18MB for a SQLite header that was
not there, dating a lock file to the minute of the first conversion, and finding
no backup beside it — while an identity store sat unreadable and every relying
party in the estate had no JWKS to fetch.
So the seal now records its key. `<database>.keyid` holds SHA-256 over a
domain-separating label and the DERIVED key, truncated — a commitment, not the
key: it cannot be run backwards, it cannot be replayed as one, and it says
nothing about any other database, because two stores under one master derive
different keys and so carry different ids. On a store that will not read,
diagnose() compares the recorded id with the id of the key in hand and answers
which fault this is. A store sealed before ids existed cannot answer, and says
that rather than inventing a verdict — a guess there aims a restore at the wrong
file.
convert() now finishes through settle() instead of repeating it. Its own check
opened a handle and closed it, which proves nothing: database/sql connects
lazily, so a read has to be forced or the key never meets a page. That is the
same mistake as trusting the header, wearing different clothes. One place decides
a store is good, and that place reads the schema, records the key, and only then
releases the plaintext.
Mutation-proven on three axes, each restored before staging and the file
confirmed byte-identical after: not recording the id fails the test; never
detecting a different key falls back to blaming damage, which is the ambiguity
this removes; and writing the raw key instead of a commitment fails on the
assertion that the id must not contain the key.
Co-authored-by: Hanzo Dev <[email protected]>
The copy was SQLCipher's sqlcipher_export, a SQL function only the C libsqlcipher
defines. So a build without that library could not convert a plaintext database at
all: the store it was pointed at stayed unopenable, and every test covering the
conversion skipped rather than ran.
Nothing else here needed the C engine. github.com/hanzoai/sqlite encrypts on every
build — the pure-Go SQLCipher codec envelope, byte-compatible with the C one — so
the conversion was the last place the dependency survived, and it did not have to.
It is now a logical copy in Go over two ordinary handles: the plaintext source
opened with no key, a fresh encrypted target opened with one. Tables, then rows,
then indexes and triggers and views, so an index is not rebuilt per insert and a
trigger cannot fire on rows the source never had. Columns are named from
table_xinfo rather than SELECT * — a generated column cannot be inserted into, and
naming them also stops a normalised column order from writing values into the
wrong column. The AUTOINCREMENT counters come across from sqlite_sequence, which
takes a delete and an insert because that table carries no unique index for an
ON CONFLICT target to name.
A byte-level encrypt is not available for this: SQLCipher reserves 80 bytes per
page for its IV and tag, a plaintext database reserves none, and no pragma
retrofits that into an existing file.
A virtual table is REFUSED, naming itself. Its shadow tables hold an index layout
rather than rows, so recreating it regenerates bytes no logical copy can promise
and verify would report a difference the data does not explain. The C function did
that from inside the engine.
The suite now runs on the pure-Go build with nothing skipped — 30 tests, including
fidelity over AUTOINCREMENT, WITHOUT ROWID, a GENERATED column and foreign keys —
and passes identically against a real libsqlcipher. A store converted by either
engine reads back under the other, rows, header fields and counters intact.
Co-authored-by: Hanzo Dev <[email protected]>
The shred was keyed on "this file does not have a plaintext SQLite header",
read as "it is the encrypted store, so the copy beside it can go". Four
different files answer that question the same way: an empty one, a truncated
one, one encrypted under another master, and the real store. Three of them
would have destroyed the last readable copy of the identity graph and returned
success — reachable by a commit that finished both renames and then failed to
open, where the next boot shreds the backup without opening anything at all.
So a file has four states now, not two, and the plaintext is released only by
settle, which OPENS the store under its key and READS its schema first. Opening
a handle would not do: database/sql connects lazily, so a wrong key surfaces at
the first read and not before. recoverInterrupted no longer shreds anything —
whether the backup may go is settle's question, and it cannot be answered from
file names.
A store too small to be a database is now an error rather than "already
encrypted". A zero-length file where the store was is what an unclean stop
leaves behind, and treating it as done let Open create a fresh one: a fully
mounted, completely empty identity service.
Three more, each one a real window:
- fsync the copy's CONTENTS before the plaintext stops being the source of
truth. syncDir made the RENAME durable, which is a different promise; the
pages could still be in the page cache when the original was released.
- read wal_checkpoint's RESULT. It reports a refusal in its first column and
still returns success, so an unchecked call cannot be told from one that did
nothing — and the plaintext WAL is deleted a few lines later on the strength
of it having worked.
- bind the key as a PARAMETER. Written into the statement, a parse error
quotes it back, and SQLite would carry the whole key into a log the moment a
build turns off the double-quoted-string fallback that makes KEY "x'...'"
parse at all. Also: an attached target journals in DELETE mode, so -journal
is the file that actually turns up beside it — clean it up and refuse to
commit while it is there.
sqlcipher_export copies the schema and every row — including the internal
AUTOINCREMENT counters — but a fresh database starts its HEADER at zero, so
user_version and application_id were silently reset to 0 by a conversion.
Measured, not reasoned about: fidelity_test.go builds a database holding an
AUTOINCREMENT counter, a WITHOUT ROWID table, a generated column, a foreign
key, an index, a view, a trigger and both header fields, converts it, and
compares the two by asking the database itself. Everything else survives; these
two did not.
For the identity store this was harmless — neither hanzoai/orm nor hanzoai/iam
reads user_version — but Convert is a general primitive, and a store whose
migration framework keys on user_version would have come back convinced it was
looking at version 0.
They are carried explicitly now, and the parity check compares them, so the fix
is held up by the same gate as the rows rather than by this message.
Also refuse to commit a copy that still has a write-ahead log beside it. Only
the main file is renamed, so the copy has to be complete in it; closing the last
connection checkpoints and removes a WAL, and one still present means the close
did not finish. Refusing costs a retry on the next boot, renaming anyway costs
whatever the log still held.
A store whose file predates cek is plaintext at rest, and until now the only
answers were to leave it that way or to point cek at a different file and serve
an empty one. Convert is the third: it makes the existing file encrypted under
the key its namespace and subsystem already derive, without losing a row.
The plaintext stays the source of truth until an atomic rename commits, and the
rename happens only after the copy has reproduced the source schema, per-table
row count and per-table content hash, passed integrity_check, and re-opened
under the same keyed path the application uses. A key that cannot open the
result is therefore caught while the original is still there. The transient
plaintext left by the commit is shredded as soon as the encrypted database
opens.
A derived key is what makes the crash cases readable from the files alone:
there is no per-file key material that could be lost or fall out of step with
the database it belongs to, so an interrupted swap can always be finished.
OpenAt comes with it — Open for a store whose location is settled by a mounted
volume rather than by the namespace. Open is now that function over
namespace.Path, so there is still one derivation and one keyed open.
github.com has no runner for hanzo-build-linux-amd64, so a caller under
.github/workflows is a gate that can never be scheduled. This is the ~7-line
caller on the plane that can (git.hanzo.ai git-runner fleet), pinned @v1.
Co-authored-by: Hanzo Dev <[email protected]>
sqlite v0.5.0 removes DeriveKey, WrapDEK, UnwrapDEK and PrincipalAAD, which is
right — a driver opens a file under a key it is handed and has no business
deriving one. v0.2.2 borrowed exactly those to read a sidecar, so it cannot be
built against v0.5.0, and taking v0.5.0 meant dropping back to v0.2.1: a cloud
that derives a key for files whose key came from crypto/rand.
But a format is not retired by the code that used to write it. The bytes on disk
still have it, in 211 org directories. Key material is this package's concern, so
this package states it: HKDF-SHA256 over the master bound to (principal, file id)
by a length-prefixed info, AES-256-GCM with the version byte leading the AAD.
Golden vectors produced under the original implementation pin every part of it —
info composition, AAD, version byte, nonce split. A silent drift now fails a
test instead of a volume.
Still read-only: no wrap, no mint, no rewrite. The test seals its own fixtures
because production never needs to.
Builds and passes against sqlite v0.5.0 and v0.4.0 alike.
Co-authored-by: Hanzo Dev <[email protected]>
Every database written before derived keys was born with a DEK from crypto/rand,
wrapped under a key derived from its owner and a random file id, and stored in a
sidecar next to it. Those pages are encrypted under that random key and under
nothing else, so no derivation reproduces it — arithmetic, not a policy choice.
Deriving a key for such a file and opening it is what SQLCipher answers at the
first statement, not at open:
sqlite: decrypt orgs/hanzo/kms.db (wrong key or corrupt file):
page 1: sqlcipher: wrong key or corrupted page
The files are there: 211 org directories, each with kms, agents, finance, code,
treasury and the rest. So Open now asks the one question that decides which key
a file has — is there a sidecar next to it. A sidecar means the key is in the
sidecar; no sidecar means the key is derived. The file says which scheme it
belongs to, which is the only thing that can say it.
This is a READ path and nothing more. It mints no sidecar, wraps no key and
rewrites no file, so a database opened through it is untouched and can be
converted later by something whose job that is. New databases are still born
with a derived key and no sidecar, so the set this governs only shrinks, and
this file can be deleted with the last of them.
A damaged sidecar or a wrong master is an error, never a fallthrough to the
derived key: falling through reports a healthy database as corrupt and sends the
reader to the wrong file.
Tests build stores in the real on-disk format using the same public primitives
the old scheme used, for both an org and the system namespace, and fail with the
production error when the read path is removed.
Co-authored-by: Hanzo Dev <[email protected]>
Open picks the path and then did not make it exist, so on the pure-Go codec —
the one CGO_ENABLED=0 builds and the shipped image use — the open SUCCEEDED,
writes were accepted, and Close failed with 'envelope seal: no such file or
directory'. The caller was handed a working database and lost every write in
it, at the moment it thought it was committing them.
A function that decides where a file goes owns making that place exist. The
doc already claimed Open creates the database; now it does.
Pinned by a test that writes, closes, reopens and reads back — it fails with
exactly that lost-writes error when the MkdirAll is removed.
Co-authored-by: Hanzo Dev <[email protected]>
There is one master key per process — injected at boot from KMS, used by every
database that process opens. Passing it to each Open would not make it less
global, only harder to see, and it would put a key parameter on all 80 call
sites in cloud for no property gained.
SetMaster installs it; Open takes the namespace, the subsystem and the data
directory. Without a master, Open fails: a service that comes up in plaintext
because a key was missing has failed silently at the only thing this does.
SetDevMaster covers a process with no KMS and is honest about the consequence —
a new random master cannot open the previous run's files, so nothing survives
the process, by construction.
Co-authored-by: Hanzo Dev <[email protected]>
A key derived from the master and the namespace: not generated, not wrapped,
not stored, not rotated in place. That removes the unwrap step, the rewrap
step, the per-file key material, and the plaintext-to-encrypted migration —
every one of which existed to carry old data forward, and none of which is a
property anyone wanted.
A database is born encrypted or it does not exist. The same namespace and
subsystem always derive the same key, so a file reopens after a restart with
nothing persisted beside it, and two databases never share a key because the
subsystem is part of the binding.
Losing the master loses the data. That is what encryption at rest means, and
why the master lives in KMS.
Co-authored-by: Hanzo Dev <[email protected]>
zeekay
