datastore/docs/en/operations/external-authenticators/iam.md

description, slug, title

description	slug	title
Authenticate every caller with a Hanzo IAM access token and confine each organization to its own rows	/operations/external-authenticators/iam	Hanzo IAM

Hanzo IAM

The IAM user directory authenticates callers with a Hanzo IAM access token. There are no stored users and no database passwords: a caller presents a token, Datastore verifies it against the issuer's published keys, and the session's identity, roles and organization come from the token.

The contract

Credential. A Hanzo IAM access token, a JWT issued by the configured issuer (default https://hanzo.id) for the configured audience (default datastore). A client obtains one with the resource=datastore parameter (RFC 8707) on its token request, e.g. the client_credentials grant.

• Native protocol: the client's JWT login (datastore client --jwt <token>; drivers send the user JWT AUTHENTICATION with the token as the password).
• HTTP: Authorization: Bearer <token>. A bearer token next to user/password query parameters or X-Datastore-User/X-Datastore-Key headers is refused, never silently ignored.

Validation. Every token is checked for:

• signature, with a key from the issuer's JWKS. The keys come from OIDC discovery (<issuer>/.well-known/openid-configuration → jwks_uri); the discovery document must name the same issuer, and jwks_uri must be https (plain http only when the issuer itself is http). The keys are cached for an hour; a token naming an unknown kid triggers a refetch, at most once every 30 seconds after the previous attempt ended. While the set is refreshed, a key already held keeps verifying: one login refreshes and the others do not wait on the issuer. If the issuer is unreachable, the keys already held stay in service.
• algorithm: RS256, ES256 (P-256) or EdDSA (Ed25519). The key named by kid decides the one algorithm it verifies and the header must agree. none, HS* and everything else is refused. RSA keys under 2048 bits, keys marked for another use, and a kid published twice are ignored.
• iss equal to the issuer; aud containing the audience; exp present and not past; nbf and iat not in the future. Each time check allows 60 seconds of clock skew.
• sub present, printable, at most 256 bytes; tokenType, when present, is access-token; a header with crit is refused.

A native connection ends with its token: every query checks the token's exp.

Identity. The user is ephemeral, held in memory and named by sub (<org>/<name> for a person, <appOwner>/<app> for a program). It has no roles of its own. A token whose sub names a user of another directory (e.g. users.xml) is refused.

The claims are read exactly as hanzoai/authz publishes them, and the two readers are held to one statement of them: authz's testdata/claims.json, which authz's own tests and this server's gtests both run (it is embedded from the contrib/authz submodule).

• Organization is the bearer's HOME org: the first entry of the signed orgs membership set, which IAM writes home-first from the user record (authz.Claims.Home). A token with no membership set, an empty one, or one whose first entry names no org has no organization. A malformed orgs (not an array of objects with string org/role) is refused.
• owner decides nothing. IAM stamps the org of the APPLICATION a token was minted through into it, so reading it would let a person choose their tenant, and reach SuperAdmin, by choosing an application.
• Program: IAM signed type: application (a client_credentials or workload token). Its sub is the IAM application row it was minted for, <owner>/<name>, and its azp the client that holds it. Its organization is a membership IAM granted its application, if any; never its owner.
• Person: not a program, and a member of at least one org (!authz.Claims.Machine).
• SuperAdmin: a person whose home org is exactly admin, in a token signed with the admin org's own key (superadmin_kid, default cert-admin). Never a program, never by an admin membership next to another home org, never by owner.

The organization is bound to the session and read in SQL as currentOrg(): Nullable(String), NULL without one and for every login that is not an IAM token. So a row policy

CREATE ROW POLICY tenant ON event.* USING org = currentOrg() TO tenant;

shows each caller its own organization's rows, and a caller without an organization no row at all (org = NULL matches nothing, including rows whose org is '').

Roles. Granted per session, from the token, by the rules below. Roles and row policies are ordinary SQL objects; the directory only decides which roles a token holds. A role a rule names that does not exist is not granted (and is logged). Two tokens of one subject never share authorization: roles and organization belong to the session, not the user, a named HTTP session (session_id) opened by one token refuses a token carrying other roles or another organization, a query cache entry is readable only under the roles and organization that wrote it (shared entries included), and asynchronous inserts are batched and flushed per session authorization. SET ROLE cannot drop a role the token grants.

Grants. A role holding a grant option (a schema's owner) could otherwise hand rights to anyone. GRANT checks the user's grantees, not the session's, so the directory sets them on the ephemeral user: the roles named in grantees that exist, and nobody else. SuperAdmin's grants may name anyone. A subject's grantees follow its latest token, and every user's follow the roles that exist: a migration that creates its roles and then grants to them in one session works, because a role is a grantee of the logins before it from the moment it exists.

Configuration

One section in config.d. Nothing in it is secret.

<clickhouse>
    <user_directories>
        <iam>
            <issuer>https://hanzo.id</issuer>
            <audience>datastore</audience>
            <superadmin_kid>cert-admin</superadmin_kid>

            <!-- Granted to every identity. -->
            <roles>
                <tenant/>
            </roles>

            <!-- The roles a GRANT made under an IAM identity may name. -->
            <grantees>
                <reader/>
                <writer/>
                <sink/>
            </grantees>

            <rules>
                <!-- A program: a client_credentials or workload token (type = "application")
                     minted for this application row (sub) and held by its client (azp). -->
                <rule>
                    <application>admin/hanzo-event</application>
                    <role>sink</role>
                </rule>
                <!-- A person: not a program, and a member of at least one org. -->
                <rule>
                    <person/>
                    <role>reader</role>
                </rule>
                <!-- SuperAdmin: a person whose home org is admin, signed with superadmin_kid. -->
                <rule>
                    <superadmin/>
                    <role>superadmin</role>
                </rule>
            </rules>
        </iam>
    </user_directories>
</clickhouse>

Element	Meaning
issuer	OIDC issuer, an absolute https URL. Default https://hanzo.id.
audience	Required member of aud. Default datastore.
superadmin_kid	The admin org's signing key; SuperAdmin holds only for a token it signed. Default cert-admin.
roles	Roles every identity holds.
grantees	Roles a GRANT made under an IAM identity may name (User::grantees); SuperAdmin's may name anyone. Default: none.
rules/rule	One matcher and one role. Matchers: <application>owner/name</application> (a program token whose sub is that application row and whose azp is its name), <person/> (a person, as above), <superadmin/> (SuperAdmin, as above). Rules add up; a rule with no matcher, two matchers, no role, or an application that is not owner/name is a configuration error.

A program rule names an application ROW, never a bare client id. IAM keys a row by (owner, name) and lets only an org's own admins, or the platform for the reserved admin org, create one; a client id is first-come across every org. A tenant that registers the client id hanzo-event before the platform does gets tokens with sub evilcorp/hanzo-event, which match no rule. Requiring azp too refuses the row's token in the hands of another client (a delegated or exchanged token).

The Hanzo image ships this section as /etc/datastore-server/config.d/datastore-iam.xml (docker/server/datastore-iam.xml). A binary without the IAM directory refuses to start on it, so it ships only with one that has it. Its rules name the application rows IAM provisions for the platform's processes (hanzoai/universe charts/app/values/hanzo/iam-provision.yaml converges each as the row admin/<org>-<app> with client id <org>-<app>):

Application row	Process	Roles
admin/hanzo-event	the event app	sink
admin/hanzo-o11y	the telemetry plane, owner of the event schema	writer, schema
admin/hanzo-cloud	the rest of the cloud host	platform
any person		reader
SuperAdmin		superadmin

No process reads the event plane across orgs: each reads as the org whose token it holds.

Every identity also holds tenant. The gtests and the integration test load this file itself, not a copy of it.

Roles and row policies for Hanzo

Three roles ship with the image as users-config roles (docker/server/datastore-iam-roles.xml, installed in users.d), because something has to exist before the first migration can run:

• superadmin: GRANT ALL ON *.* WITH GRANT OPTION.
• schema: what a schema's owning service needs to run its own migrations as its own IAM application, with no person, no SuperAdmin and no password. DDL and row policies on event.* and attic.*, SELECT and INSERT on event.* with grant option (so it can give the event-plane roles their rights and nothing outside event; grantees limit whom to reader, writer and sink), CREATE ROLE on exactly tenant, reader, writer and sink, and reads of the system tables its guards query. The migration runner keeps its ledger and lock in the schema's own database (event.migration, event.migration_lock_*), so they need nothing more. No CREATE DICTIONARY: a dictionary's source logs in as a user it names itself (by default the loopback default), so a dictionary in event would read outside it.
• platform: the cloud host's own hanzo database (usage ledgers, rollups, risk features, the audit mirror): DDL, SELECT with grant option, INSERT, and row policies there, so the host confines its own org-keyed tables the way schema confines the event plane; reads of system.tables, system.columns and system.disks. Nothing in event.

Both create tables only with the engines their migrations use: MergeTree, ReplacingMergeTree, AggregatingMergeTree, SummingMergeTree and Null (GRANT TABLE ENGINE ON <engine>). The image's datastore-iam.xml sets access_control_improvements.table_engines_require_grant to true; without it every engine that does not read an external source is every creator's, including Merge, Buffer, Alias and the Replicated* engines. A users.xml user without <grants>, and anyone holding ALL, keeps every engine; a SQL-created user that creates tables needs GRANT TABLE ENGINE ON <engine>.

The event plane's own roles and policies are hanzoai/o11y deploy/datastore/migrations/0005_iam_roles.sql, run by the o11y application under schema:

CREATE ROLE IF NOT EXISTS tenant;
CREATE ROLE IF NOT EXISTS reader;
CREATE ROLE IF NOT EXISTS writer;
CREATE ROLE IF NOT EXISTS sink;

CREATE ROW POLICY OR REPLACE tenant ON event.* USING org = currentOrg() AS PERMISSIVE TO tenant;
CREATE ROW POLICY OR REPLACE superadmin ON event.* USING 1 AS PERMISSIVE TO superadmin;
CREATE ROW POLICY OR REPLACE schema ON event.* USING 1 AS PERMISSIVE TO schema;

GRANT SELECT ON event.* TO reader;
GRANT INSERT ON event.* TO writer;
REVOKE INSERT ON event.fact FROM writer;
GRANT INSERT ON event.fact TO sink;

The order is the safety: roles, then the policies that confine them, then the grants. A run that stops anywhere leaves a reader unable to read, never able to read every organization, because users_without_row_policies_can_read_rows lets a role with SELECT and no policy read everything.

The confinement is on tenant, the role every identity holds, so a role granted SELECT later is confined without anyone extending the policy. Permissive policies are OR-ed: schema sees every row of its own plane, because a migration that backfills would otherwise read none and copy nothing. Nobody else reads across organizations. Row policies filter reads only; sink and writer are the platform's ingest and insert rows for every organization, sink alone into event.fact, and neither reads.

The host's own tables

Cloud reads some tables of its hanzo database AS a person's organization (the person's own token, exchanged for the datastore audience), so the row policy confines those reads whatever their SQL says. The host declares that confinement itself, as platform, for each org-keyed table it reads that way, with the table's own organization column. Table-level policies, never ON hanzo.* for tenant: hanzo also holds tables without an organization column, and a database-level tenant policy would fail every read of them. Policies first, grants last, for the reason above.

CREATE ROW POLICY OR REPLACE tenant ON hanzo.cloud_usage USING organization = currentOrg() AS PERMISSIVE TO tenant;
CREATE ROW POLICY OR REPLACE tenant ON hanzo.account_usage USING org = currentOrg() AS PERMISSIVE TO tenant;
CREATE ROW POLICY OR REPLACE platform ON hanzo.* USING 1 AS PERMISSIVE TO platform;
GRANT SELECT ON hanzo.cloud_usage TO reader;
GRANT SELECT ON hanzo.account_usage TO reader;

Limits

• A distributed query reaches other replicas as the initiating user; IAM users exist only on the replica that verified the token, so such queries fail rather than run unconfined.
• An IAM identity's roles belong to its session, so a view it defines with SQL SECURITY DEFINER runs as <sub>:definer, a stored user that cannot log in and holds no roles: reading such a view, or inserting into the source of such a materialized view, is refused. That is the default for a materialized view (default_materialized_view_sql_security), so an IAM identity must not create one without naming a stored definer that holds the view's rights. Materialized views that already exist keep the definer that created them; re-point them to a stored definer before removing that user.
• The ephemeral users stay in memory until restart (one small entry per distinct sub).
• A table's SETTINGS disk = disk(type = web | s3 | ...) defines a disk that no grant governs, so any role that may create a table (schema, platform) can make the server fetch from a URL it chooses. remote_url_allow_hosts bounds where; the image does not set it.
• A schema's owner is trusted with its plane: schema may write any event table (a migration backfills), rewrite the plane's row policies (whom a policy applies to is not limited), and grant reader, writer and sink anything on event; platform the same for hanzo. Confinement protects tenants from each other and from a read that forgot its predicate, not from the plane's owner.
• A login that is not an IAM token (a users.xml user with a password) holds none of these roles, and with access_control_improvements.users_without_row_policies_can_read_rows at its default (true) it is not confined by them either. Remove such users once every caller presents a token, then set that setting to false, so a table whose policies name nobody a caller holds shows that caller nothing.