2893 Commits
Author SHA1 Message Date
zeekay cdf0c4c152 A graph's schema is the caller's, and new accounts buy a visitor nothing
The schema a graph of a Base renders named every collection to whoever
asked for it.  It is now as far as the caller reaches: a member sees
the Base whole, a visitor only the collections that admit visitors, and
anyone else only those whose list rule is "".  apis.ReachOf answers the
reach by putting the caller's credential through loadAuthToken, the
door every request passes, so the graph decides nothing of its own.
Each reach keeps its own cached schema, and a root beyond the caller's
reach is no field at all rather than a refusal that says it exists.

A visitor's writes were limited per account, which an account made for
the purpose resets.  They are now also held to fifteen a minute from
one client address and 120 a minute from all visitors in a Base, on
the same limiter.  A write that names no client address is refused.
v1.5.108
2026-10-20 20:41:44 -07:00
zeekay cebf25db06 A collection can admit visitors from other orgs
A Base its host opened for one org refused every token acting in
another, so a reader signed in with Hanzo ID could not comment on a
site whose Base is org hanzo's.  Opening that org to any account would
hand the CRM to anyone, since its rules admit any signed-in caller.

A base collection now admits visitors when it says so, with
"visitors": true.  A visitor reaches only its records and files; every
other route, and every collection that has not opted in, still answers
403.  Its record carries role "visitor", admin false and its own org,
so `author = @request.auth.id` holds a row to its writer.  Any other
collection's rules read a visitor as a guest, so a relation, expand or
filter from a visitors' collection carries it no further.  Its writes
are held to ten a minute per Base, keyed by subject, whatever the
deployment limits.  A superuser is never a visitor.
v1.5.107
2026-10-20 18:10:41 -07:00
zeekay da4803a259 Build on Go 1.27.1, and give each test IAM an address of its own
zip v1.37.43 needs Go 1.27.1, which the 1.26.8 builder image refused,
so v1.5.105 published no image.

TestOrgAdminSchemaAuthority answered a member 401 on the runner: the
JWKS cache keys a document by its address, and a closed test server's
port is soon another test's, so a token was checked against the last
IAM's keys. tests.IAM serves its keys under a path of its own.
v1.5.106
2026-10-20 08:10:16 -07:00
zeekay 1683110b1c A saved view can be its maker's alone
D12: a person's own views are CRM views rows with visibility private,
which only their maker lists, reads, changes or deletes, an org admin
included; an org view (visibility org, or empty for rows made before)
is every member's to read and its maker's or an org admin's to change.
IAM preferences keep only which view a person opens first.

A Base that installed the CRM earlier holds the old rules, and grow
never touched a rule. A grow document may now name the rules its own
replace (`replaces`): a collection still holding one of those exactly
takes the new rule, and a rule an org wrote is kept, so the next
install tightens the views of every Base that has them.

tests.Router compiles an app's routes for a test that asks several
requests of one app; the graph tests use it.
v1.5.105
2026-10-20 07:29:55 -07:00
zeekay ab9da76ea8 A Base's records are a typed GraphQL schema, answered through its own router
graph.New(app, router) renders the Base's collections as GraphQL and
answers every field with a request on the Base's own router, made with
the caller's headers, so the list and view rules, tenancy, rate limits
and expand's view checks are the REST path's and there is no second
authorization. zip's engine executes it: a host mounts g.Schema and
g.Resolve with zip.MountGraphFor, so zip is bumped to v1.37.43.

- A type per base and view collection, named from the collection
  (people: BasePerson, person, people, peopleGroupBy, createPerson,
  updatePerson, upsertPerson, deletePerson), with its fields, relations
  as objects, the records pointing at it as connections
  (opportunities_via_company) and its history.
- A filter input per collection whose operators per field type are the
  where clause's: each is one leaf of core.CompileWhere, so and, or,
  not, @me, relative dates, back-relations and inSegment mean what they
  mean there, every value bound.
- orderBy, cursor connections paged through Base's pages, a totalCount
  asked only when selected, and aggregate and GroupBy from the
  aggregate route.
- Bounds: 100 a page, four relations deep refused before any request,
  50 requests a root field, and zip's 10,000 resolutions.

graph_test drives it through zip against a CRM Base: every filter type,
paging both ways, the figures, the writes as REST reads them back, the
refusals in REST's own words (a rule, a superuser-only collection,
another org's token), the budget, and type names on every object.
tests.IAM signs IAM tokens for any test, and apis' helper uses it.
v1.5.104
2026-10-19 09:31:00 -07:00
zeekay a9d2f1ff46 An aggregate reads no private email, and history no removed field
Red's review. The aggregate route reads columns directly, so it gated
only hidden fields: grouping an auth collection by email, or counting
its unique emails, answered addresses the records keep private, which
the list masks and the filter resolver will not compare. core.EmailGated
is now the one rule both ask: an auth collection's email, which a caller
who is not a superuser reads only where emailVisibility is set. The
aggregate route refuses it as it refuses a hidden field.
TestAPrivateEmailIsNotAFigure is red's proof inverted: the group and the
count are refused, and fail without the change.

A record's history showed the changes of a field the collection had
since removed. An org admin who removes a field means its values to be
gone from view, so a caller who is not a superuser no longer reads
them; _history keeps the row for a superuser.

The composite uniqueness check is a read before the write, which
PostgreSQL can race; that is now said where it is done and in LLM.md,
with the fix (a unique expression index from the dialect).
2026-10-19 09:24:43 -07:00
zeekay 6a3066b08e Test apps open a migrated fixture, so core and apis fit the runner
tests/data lacked twelve migrations, and every test app ran them before
its test began: about four seconds an app, which put core and apis past
go test's ten minutes on the runner, and past thirty once the timeout
was raised. The fixture now holds them (go run ./tests/fixture writes
it), and TestTheFixtureIsMigrated fails, naming the migration, while
one is missing. On one host core went from 267 s to 26 s, apis from
432 s to 23 s, and the whole suite runs in 51 s.

The delete tests gave a background goroutine 100 ms to remove a
record's files, which a slow runner did not always meet. BaseApp
counts those deletions and WaitFileDeletes waits for them: the tests
wait on the count instead of the clock, and ResetBootstrapState waits
too, so a Base that stops no longer drops a deletion in flight.

TestSQLRun expected a CREATE TABLE to report one affected row. SQLite
answers a statement that writes no rows with the connection's last
count, and that row was the last migration's _migrations insert; the
handler now reads 0, the number it changed.

go test runs with its default timeout again, and crm_test.go is gofmt'd.
2026-10-19 09:08:39 -07:00
zeekay 79189973b1 Give core and apis the time the runner takes to test them 2026-10-19 02:33:00 -07:00
zeekay b29ce6e89e A saved user's org authority is refused with any modifier, and hidden from its export v1.5.103 2026-10-18 22:33:37 -07:00
zeekay 1575be07b1 A bound filter value stays a value, and org authority holds only in the org's Base
Filter params were pasted in one at a time, so a value spelling another
param's placeholder was rewritten inside its own quotes and the second
value's quotes closed the first one's string: a bound value could add
conditions to the filter. They are now substituted in one pass over the
filter as written.

An org admin administers, and a member reads the schema of, only a Base
opened for its org (StoreKeyTenant): a Base deployed for no org belongs to
whoever deployed it. The org plugin names the org of every Base it opens.

A saved user's record never carries org authority: @request.auth.admin and
@request.auth.role answer only for the record a verified token mints, and
the two fields are system and hidden.

The emptiness figures of an aggregate are columns of its one query, so a
grouped request counts the groups it shows and runs one query however
many figures it asks for.

A stream the host routed to its org's Base is served by that live Base,
not the instance its grant was minted on.
2026-10-18 22:21:27 -07:00
zeekay b5183f8c5e A multi-valued field can be added on PostgreSQL, and composites read as text
Adding a field the collection did not have ran it through the single to
multiple conversion, which read the fresh JSON column as text. SQLite let
that pass; PostgreSQL refused it (`COALESCE(jsonb, '')`), so no select,
relation or file field with several values could be added to an existing
collection there, and the CRM module could not install. A new field has
no old shape to convert, so it is skipped.

A composite value is read as text like a JSON field, so comparing a whole
value with null is IS NULL and a LIKE compares text on both engines.

The engine corpus installs the CRM and filters its values on both.
2026-10-18 22:08:03 -07:00
zeekay 5592800432 docs: the caller's org role in rules, and who changes an org's schema 2026-10-18 21:41:16 -07:00
zeekay 9c2179c28b LLM.md: schema by org admins, composite fields, where, history and modules 2026-10-18 21:40:51 -07:00
zeekay 7419d13323 A member reads the org's schema, and a grow keeps an index it already has
A client draws a record by its schema, a field's label and a select
value's colour, so every member of the org reads the org's own
collections. Changing them stays an admin's, and Base's own tables stay
hidden from a member.

A module that indexes a column the org already indexed under another
name no longer fails the install: the index is there.
2026-10-18 21:29:25 -07:00
zeekay 3fdb4d5d7d An org admin changes its own Base's schema
The collection routes admit an org admin in its own org's Base as well as a
superuser. An org admin creates, changes, deletes, truncates and imports
the org's ordinary base collections only, judged as each is and as the
request would leave it: auth, view and system collections, Base's
underscore tables, deleteMissing and token-secret rotation stay a
superuser's. Every org-admin schema change is logged.

The superuser-mirror test probed /v1/collections, which an org admin now
reaches; it probes /v1/settings.
2026-10-18 21:25:32 -07:00
zeekay dd46954b01 A record's history, written in the record's own transaction
A base collection that turns History on gets a _history row for every
create, update and delete: the collection and record ids, the action, the
writer (an actor field stamped from the save's context) and each changed
field's from and to. The row and the record commit or roll back together.
id, autodate, actor and hidden fields are left out, a value over 4 KiB is
recorded only as changed, and one writer's updates to one record within ten
minutes fold into one entry.

GET /v1/collections/{collection}/records/{id}/history reads it, newest
first, for whoever the record's view rule admits.
2026-10-18 21:25:32 -07:00
zeekay 060de407b9 Filters written as data, and the figures a list shows
A where clause is a filter as JSON: and, or and not over leaves of a field
path, an operator and a value. CompileWhere checks the path against the
collection (relations, back-relations, composite parts), admits the
operators the field's type takes, binds every value, and writes Base's own
filter language, so it runs under the same resolver and rules as a filter
written by hand. A negation is pushed down to the leaves, each of which
knows its opposite, an empty value included. Relative dates are computed
in the leaf's time zone and week, @me is the caller, and inSegment
expands a saved segment the caller may read.

The records list takes ?where= beside ?filter=, after the list rule.

GET /collections/{c}/aggregate answers count, sum, avg, min, max,
countUnique, countTrue/False and the empty counts and percentages, once
or per value of one field, over the ids the list rule, filter and where
select, so a record joined twice is counted once.
2026-10-18 21:18:59 -07:00
zeekay 7b5f02793e The CRM is a module an org installs into its own Base
A module is a named import document, and installing one is a grow:
core.GrowCollections creates what is missing and adds the fields, select
values and indexes an existing collection lacks, in one transaction,
changing nothing that exists. A relation names its collection by name.
Installing twice changes nothing.

modules/crm carries the thirteen collections: people, companies,
opportunities, candidates, applications, lists, list_members, segments,
activities (one row per source id), notes, tasks, attachments and the
org's shared views. POST /v1/modules/{name} installs, GET /v1/modules
lists, both for a superuser or the org's admin, who can never reach a
collection only a superuser manages this way.

The _view* system collections are retired. Nothing read them, and team
views are the ordinary views collection now.
2026-10-18 21:02:29 -07:00
zeekay 4dd1d9686c The admin UI knows the new field types 2026-10-18 20:58:38 -07:00
zeekay 98826c880c A host can read which org a stream grant opens without spending it 2026-10-18 20:46:30 -07:00
zeekay f223d28dd0 One predicate for an org admin's schema authority 2026-10-18 20:42:37 -07:00
zeekay 23cf129160 Field types for people and money, and the caller's org authority in rules
Seven field types a CRM record needs, each one JSON column whose parts a
filter, a sort or an aggregate reaches by name: currency (amountMicros,
currencyCode), emails, phones, links, address, fullName and actor, plus
rating. Empty is NULL. Emails, phones and links normalize what is typed and
can keep their primary part unique. An actor is stamped by the server from
the writer the save's context states; nothing a request sends sets it.

Every field carries a label, description, icon and position, a select's
values carry a label, colour and position, and a base collection carries a
label, plural, icon, description and an opt-in history flag.

Editor HTML is sanitized before it is written.

A rule reads the caller's authority in the org it acts in:
@request.auth.admin and @request.auth.role, off the token. An IAM-mirrored
auth record is never saved, so its fields are now read off the record
rather than joined against a row that is not there, which also makes
@request.auth.org_id match for the first time. A host that opens one Base
per org names it, and a token acting in another org is refused there.
2026-10-18 20:41:15 -07:00
z f65539d790 Grant hanzoai/ci the run's ID token [skip ci] 2026-09-30 07:16:40 -07:00
zeekay 6d4aff6caa Run the Postgres gate without a container
The runner has the docker CLI and no daemon, and sudo cannot work in its
sandbox, so docker run postgres:18 failed and held every release. The
gate now runs a pinned, self-contained PostgreSQL 18 build as the
runner's user.

Claude-Session: https://claude.ai/code/session_011i2AG9nwpiKUCVxAdDzYuU
2026-09-29 18:44:45 -07:00
zeekay 938941ebaf LLM.md: outbound calls are /v1 too, and the guard says so
Claude-Session: https://claude.ai/code/session_011i2AG9nwpiKUCVxAdDzYuU
2026-09-29 18:39:20 -07:00
zeekay ce8010176b Guard the whole module and its clients against /api
The route-prefix test read one package. It now walks every Go file in
the module and the TypeScript in ui-react and sdk.

Claude-Session: https://claude.ai/code/session_011i2AG9nwpiKUCVxAdDzYuU
2026-09-29 18:36:47 -07:00
zeekay 63208d31b5 Default the private-store client to /v1
Base serves /private under /v1; the old '/api' default reached nothing.

Claude-Session: https://claude.ai/code/session_011i2AG9nwpiKUCVxAdDzYuU
2026-09-29 18:36:44 -07:00
zeekay ea6861f355 Call Hanzo Tasks at /v1/tasks
The HTTP transport posted to /api/v1/namespaces/default/..., which the
tasks server has never served (404), in snake_case bodies it does not
decode. It now speaks the server's /v1/tasks/namespaces/{ns} routes and
camelCase shapes, and a test drives it against the server's own handler.

Claude-Session: https://claude.ai/code/session_011i2AG9nwpiKUCVxAdDzYuU
2026-09-29 18:36:42 -07:00
zeekay 1e995dfa15 The listener test copies a name out of the receive buffer before the next message reuses it 2026-09-29 11:31:49 -07:00
zeekay 4506925bff Mirror a superuser only for the admin org's own people
hanzoai/authz v1.10.42 decides Sudo on the person's own org, the first
entry of orgs, so a brand org's user holding an admin-org membership no
longer mirrors onto _superusers, and that membership no longer selects
another tenant's Base or the admin org's.

The fixtures that stood for a SuperAdmin now carry admin first, and
TestIAMSuperuserMirror, TestOperatorMayReachATenant and
TestOperatorReachesTheOrgItNames each refuse the brand member beside
the operator they admit.

Claude-Session: https://claude.ai/code/session_01HuMDfgFssPUptYUzgZ6Cp8
2026-09-27 19:34:54 -07:00
zeekayandHanzo Dev 9b603572e1 LLM.md is the one committed guide; CLAUDE.md, AGENTS.md and GEMINI.md are local links to it
Co-authored-by: Hanzo Dev <[email protected]>
2026-09-24 15:42:07 -07:00
zeekayandHanzo Dev 86f5a2a6e8 Assert what the file holds, not the encoder's bytes
The fixtures are png.Encode and compress/flate output, and Go 1.27 writes
both differently: the PNG went from 73 bytes to 77 and the zip from 561 to
549. Every release since the toolchain moved has failed on that, with
nothing in Base having changed.

Co-authored-by: Hanzo Dev <[email protected]>
v1.5.102
2026-09-22 22:42:16 -07:00
zeekayandHanzo Dev 0e01a3c442 Forward the request target a caller actually named
An adapted net/http handler was handed a request with an empty RequestURI, so
anything that routes or logs on the target saw "/" and answered for a path
nobody asked for. The pin carries the fix.

The registrations in the tests say which method they answer with, the same as
the ones they exercise.

Co-authored-by: Hanzo Dev <[email protected]>
v1.5.101
2026-09-17 06:00:54 -07:00
zeekayandHanzo Dev a37196d301 Declare operations on the group that serves them
zip's registration is a verb method on the group now, so an operation reads as
the group answering it rather than a function taking the group, and a route that
is not a typed operation says which method it answers with. The package-level
form is gone, along with the Router interface it existed to accept any target
through.

Co-authored-by: Hanzo Dev <[email protected]>
v1.5.100
2026-09-17 05:50:27 -07:00
zeekayandHanzo Dev a94a345909 Build on Go 1.27
1.27 is the released toolchain and the tree already compiles clean on it, so
the go directive names it. Nothing here depends on a 1.27 language feature
yet; this is the floor moving, which is what makes generic methods, the wider
type inference and promoted-field struct literals available to the code that
wants them next.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-16 19:02:47 -07:00
zeekayandHanzo Dev 887f93b7b0 The plugin runtime is its own module
Base is not the only thing that runs plugins. Cloud grew a second goja runtime
— its own shim, its own bundler, its own registry — because the loader and the
runners lived here and there was nothing to import. hanzoai/plugin is that
something, and it implements zip's Module and Loader rather than the private
copies of them that sat beside these runners.

extruntime, gojavm, pyvm and wasmvm move out. jsvm stays, because it is base's
own app scripting rather than a plugin runtime — the two were only ever adjacent.

Their one tie to base was a single call for an unguessable string, and
crypto/rand says that without a dependency.

Co-authored-by: Hanzo Dev <[email protected]>
v1.5.99
2026-09-16 12:42:17 -07:00
zeekayandHanzo Dev 31eb0797ee Document users rows, superuser tokens and the master key
rules.md, versions.md, config.md, threat-model.md and LLM.md say what
the three commits before this one do.

Co-authored-by: Hanzo Dev <[email protected]>
v1.5.98
2026-09-11 13:58:52 -07:00
zeekayandHanzo Dev 10f67d734c Org Bases open under the KMS master key or not at all
The base binary passes IAM_ORGANIZATION as IAMOrg, and plugins/org
reads base/MASTER_KEY_B64 beneath it at Register: 32 bytes, base64.
With KMS configured, Register fails when no org is set, when KMS does
not answer, or when the key is not 32 bytes. No key stored leaves org
Bases unencrypted and logs it; a 404 from the KMS HTTP door reads as a
missing secret.

An encrypted Base needs SQLite to encrypt pages as it writes them, which
only a linked SQLCipher codec does, so Register refuses a master key on a
build without one. Under a key, the connector refuses an org Base file
that is plaintext.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 13:58:52 -07:00
zeekayandHanzo Dev 007a8c8002 Superuser tokens come only from IAM
A migration replaces the _superusers token secrets once, so every
superuser token a Base signed itself stops verifying on the first
start. auth-refresh answers 403 for a superuser token that names no IAM
subject, and answers an IAM token as before.

The test data records the rotation as applied, so its tokens stand for
tokens signed after it.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 13:58:52 -07:00
zeekayandHanzo Dev 3dcab7b84f Only superusers add users rows
A new Base's users collection has a null createRule. On upgrade, a
users collection whose five rules are still the ones the init migration
wrote gets the same; any other rule set stays as the operator left it.

People come from IAM and are mirrored for one request without being
saved, so nothing that signs in creates these rows.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 13:58:51 -07:00
zeekayandHanzo Dev 6f985d3546 Give the inter-service events table its own heading
Co-authored-by: Hanzo Dev <[email protected]>
v1.5.97
2026-09-11 11:50:10 -07:00
zeekayandHanzo Dev 0083b09cf5 Correct the scratch-dir and embedded Tasks notes
hanzoai/sqlite 0.5.10 falls back to the OS temp dir, so the suite now
passes on macOS. The embedded Tasks socket also fails on a data dir's
first start, measured against the pushed binary.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:58:00 -07:00
zeekayandHanzo Dev 2ceb430a12 Document how to run, extend and upgrade Base
The README quickstart, docs/versions.md, hooks.md, rules.md, config.md
and listeners.md, and examples/hooks-hybrid-search, each command run
against the binary built from examples/base. LLM.md is cut to a current
map with test commands and known problems; threat-model, NETWORK,
SECURITY and CONTRIBUTING are corrected to the code.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:52:05 -07:00
zeekayandHanzo Dev 9a96f79bad deps: hanzo.ai/tygoja 0.1.1
0.1.0 declares its module as github.com/hanzoai/tygoja, so this path
cannot require it; 0.1.1 declares hanzo.ai/tygoja again.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:27:31 -07:00
zeekayandHanzo Dev 5161540618 deps: hanzoai authz 1.10.37, orm 0.6.31, replicate 0.9.12, sqlite 0.5.10
cek 0.2.10, dbx 1.17.4, pubsub-go 1.53.0.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:27:31 -07:00
zeekayandHanzo Dev 02b573e90e deps: zap-proto/zip 1.36.47, zap-proto/http 0.3.9
Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:27:31 -07:00
zeekayandHanzo Dev a907735dd8 deps: luxfi consensus 1.36.92, fhe 1.11.2, log 1.6.2, metric 1.11.2
cache 1.3.5, keys 1.4.3.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:27:31 -07:00
zeekayandHanzo Dev c92138e6c0 deps: fexpr 0.6.0, so a {:param} value reaches SQL unchanged
strconv.Quote escapes a placeholder value and fexpr 0.5.0 undid only
the escaped quote, so `escape\"quote` arrived as `escape\\"quote`.
0.6.0 decodes \\ \" \' \n \t \r. The search tests expect the decoded
values and add a newline case.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:27:31 -07:00
zeekayandHanzo Dev fd86d8b94b deps: pgx 5.11.0, wazero 1.12.0, fasthttp 1.74.0, esbuild 0.28.2
goja and go.starlark.net to their newest commits, fsnotify 1.10.1,
mimetype 1.4.15. go-brrr replaces andybalholm/brotli in the graph.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:27:31 -07:00
zeekayandHanzo Dev ad7e7c1c2f ui-react: drop image-size from metro's install
No image-size release fixes GHSA-5p2g-fcmc-qvqq or GHSA-w3rx-r6r6-pgpr
(every version through 2.0.2 is affected). Its one dependent is metro,
React Native's bundler, which arrives with the react-native peer; metro
0.87.0 still depends on it. Vite aliases react-native to react-native-web,
so metro never runs here, and the bundle built from this lockfile is
byte-identical to the committed dist.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 10:27:31 -07:00