refactor(openapi): collapse 3 capability registries to ONE source (capabilities.yaml) (#28)

Three hand-maintained capability registries drifted (8 vs 9 vs hardcoded) — a
one-and-one-way violation. Collapse to ONE machine-readable source, everything
else derived.

Before: 3 sources
  - capabilities.yaml     (structured manifest, read by nothing)
  - CAPABILITIES.md       (10KB hand-written index, wrongly called canonical)
  - merge.py GROUPS dict  (hardcoded, "mirrors CAPABILITIES.md by hand" — the
                           only one that actually drove x-tagGroups)

After: 1 source + 2 derivations
  - capabilities.yaml is the ONE source of truth
  - merge.py READS it (GROUPS dict deleted) → hanzo.yaml x-tagGroups (domain
    titles + Core) AND generates CAPABILITIES.md (now GENERATED — DO NOT EDIT)

Build invariant enforced (fail loud, was only promised): every present
<svc>/openapi.yaml dir maps to exactly one entry across domains∪core (orphan /
unlisted / double-listed → sys.exit); collapsed name must have no spec dir;
internal excluded from master + x-tagGroups.

Reconciled capabilities.yaml so the build PASSES without losing capability,
grounded in `ls */openapi.yaml`:
  - app → applications, do → platform (grouped, not folded; review notes kept)
  - world → applications (dir exists, was unlisted)
  - s3 → data, observe → observability (dirs exist → moved out of `collapsed`,
    which requires no dir; fold-later intent preserved in `review`)
  - team/wallets/treasury: no spec dir → dropped from domains, kept in `pending`

hanzo.yaml aggregation unchanged in substance: 1793 paths / 999 schemas / 150
responses / 39 params / 68 tags — zero added, zero removed. Only x-tagGroups
names change (8 → 9 groups, using domain titles).

Also: gitignore __pycache__/ (was polluting the tree).

.gitignore

@@ -0,0 +1,2 @@
+__pycache__/
+*.pyc

CAPABILITIES.md

@@ -1,171 +1,173 @@
-# Hanzo Capability Manifest
+<!-- GENERATED by merge.py from capabilities.yaml — DO NOT EDIT. Edit capabilities.yaml and run `python3 merge.py`. -->
 
-The **canonical index** of every public Hanzo capability: one canonical name per
-capability, grouped into eight categories, each with its `/v1/<name>` route
-prefix and a one-line description. This file is authoritative — the OpenAPI
-per-service specs, the cloud binary's mounted subsystems, and the console's
-API client all reconcile to these names and groups. One name, one way.
+# Hanzo Capability Manifest — V8 · Open Edition
 
-**Rules**
+GENERATED from `capabilities.yaml`, the ONE registry of Hanzo capabilities. Do NOT edit this file by hand — edit `capabilities.yaml` and run `python3 merge.py`. `merge.py` reads the same registry to emit `hanzo.yaml`'s `x-tagGroups`, so this index and the master never drift.
 
-- One canonical name per capability. No `svc` suffixes, no synonyms.
-- Every route is `/v1/<name>/<resource>` (the `ai` inference surface is the one
-  documented exception — see below).
-- Categories are fixed: Identity, AI, Messaging, Observability, Commerce,
-  Platform, Applications, Core.
-- Internal infrastructure is **not** a public capability and never appears in the
-  public spec (see "Internal" at the end).
-- Backing column: the cloud subsystem id (`clients/<id>`) that mounts it, or the
-  standalone deployment (`hanzoai/<svc>`) when it is not fused into the cloud
-  binary. `(was /v1/x)` marks a canonical rename still cutting over.
+One canonical name per capability = one `/v1/<name>` route = one `<name>/openapi.yaml`. A ✓ marks a name with a present spec dir (live in `hanzo.yaml`); *pending* marks a listed name whose spec is not yet authored.
 
----
+## Identity & Trust
 
-## Identity
+*who you are · what you may touch · where secrets live*
 
-Who you are, what you may do, and the secrets/keys behind it.
+| Capability | Route | Spec |
+|---|---|---|
+| `iam` | `/v1/iam` | ✓ |
+| `authz` | `/v1/authz` | ✓ |
+| `kms` | `/v1/kms` | ✓ |
+| `security` | `/v1/security` | ✓ |
+| `did` | `/v1/did` | ✓ |
+| `zt` | `/v1/zt` | ✓ |
+| `guard` | `/v1/guard` | ✓ |
 
-| Capability | Route | What it does | Backing |
-|---|---|---|---|
-| `auth` | `/oauth/*`, `/.well-known/*`, `/login/oauth/*` | OAuth2 / OIDC login + token issuance (the identity **authentication** surface). | `clients/iamsvc` |
-| `iam` | `/v1/iam` | Identity authority — orgs, users, apps, SCIM, adapters, OIDC discovery. | `clients/iamsvc` |
-| `authz` | `/v1/authz` | Fine-grained per-org access-control policies + `check`. | `hanzoai/authz` |
-| `security` | `/v1/security` | Org security posture + security-event surface. | cloud |
-| `audit` | `/v1/audit` | Org-scoped, tamper-evident, hash-chained audit trail. | `clients/auditlog` |
-| `wallets` | `/v1/wallets` | Custody / accounts / keys / sign (KMS single-sig + luxfi/mpc ring). | `clients/wallets` |
-| `kms` | `/v1/kms` | Per-org, zero-knowledge secret store (embedded luxfi/kms). | `clients/kmssvc` |
-| `zero-trust` | `/v1/zt` | Overlay networks, service mesh, edge nodes (OpenZiti fabric). | `clients/zt` |
+## Intelligence
 
-## AI
+*models · agents · evaluation — the mind of the cloud*
 
-Inference, agents, evaluation, and the DocType foundation.
+| Capability | Route | Spec |
+|---|---|---|
+| `ai` | `/v1/ai` | ✓ |
+| `ml` | `/v1/ml` | ✓ |
+| `engine` | `/v1/engine` | ✓ |
+| `agents` | `/v1/agents` | ✓ |
+| `evals` | `/v1/evals` | ✓ |
+| `prompts` | `/v1/prompts` | ✓ |
+| `functions` | `/v1/functions` | ✓ |
+| `exec` | `/v1/exec` | ✓ |
+| `graph` | `/v1/graph` | ✓ |
 
-| Capability | Route | What it does | Backing |
-|---|---|---|---|
-| `ai` | `/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`, `/v1/models`, `/v1/rerank`, `/v1/messages` | OpenAI- and Claude-compatible inference. **Top-level** — the path is the SDK compatibility contract, so it is NOT under `/v1/ai`. | `hanzoai/ai` |
-| `agents` | `/v1/agents` | Agent definitions + the live agent-session control plane. | `clients/agents` |
-| `models` | `/v1/models`⁽ᵃ⁾ | Per-org deployed model serving (KServe InferenceServices). `(was /v1/ml)` | `clients/ml` |
-| `evals` | `/v1/evals` | Evaluation scores, datasets, evaluators, runs. | `clients/eval` |
-| `prompts` | `/v1/prompts` | Versioned prompt library. | `clients/prompts` |
-| `functions` | `/v1/functions` | Serverless functions. | `clients/functions` |
-| `exec` | `/v1/exec` | Code-Interpreter sandbox (exec / upload / download / files). | `clients/exec` |
-| `framework` | `/v1/framework` | Metadata-driven DocType engine — the foundation CMS/ERP/CRM/Help build on. | `clients/framework` |
-| `graph` | `/v1/graph` | Chain data — indexers + on-chain price/data oracles. | `clients/graph` |
+## Data
 
-⁽ᵃ⁾ Deployed-model serving lives on the org-namespaced `/v1/models`; the top-level
-OpenAI-compat `/v1/models` LIST belongs to `ai`. Same path, disjoint scope — the
-serving control-plane sub-paths (`/v1/models/:name/predict`) are unambiguous.
+*the stores every capability writes to and reads from*
 
-## Messaging
+| Capability | Route | Spec |
+|---|---|---|
+| `storage` | `/v1/storage` | pending |
+| `s3` | `/v1/s3` | ✓ |
+| `base` | `/v1/base` | ✓ |
+| `db` | `/v1/db` | ✓ |
+| `kv` | `/v1/kv` | ✓ |
+| `vector` | `/v1/vector` | ✓ |
 
-Move events, jobs, and notifications.
+## Streams
 
-| Capability | Route | What it does | Backing |
-|---|---|---|---|
-| `pubsub` | `/v1/pubsub` | Embedded NATS + JetStream publish / subscribe. | `clients/pubsub` |
-| `kafka` | `/v1/kafka` | Kafka-wire publish / consume adaptor. | `clients/kafka` |
-| `tasks` | `/v1/tasks` | Durable task / workflow engine. | `clients/tasksvc` |
-| `notify` | `/v1/notify` | Notification delivery. | `clients/notify` |
+*messaging · durable tasks · async orchestration*
+
+| Capability | Route | Spec |
+|---|---|---|
+| `pubsub` | `/v1/pubsub` | ✓ |
+| `mq` | `/v1/mq` | ✓ |
+| `stream` | `/v1/stream` | ✓ |
+| `tasks` | `/v1/tasks` | ✓ |
+| `notify` | `/v1/notify` | ✓ |
+| `automations` | `/v1/automations` | ✓ |
+| `flow` | `/v1/flow` | ✓ |
+| `auto` | `/v1/auto` | ✓ |
 
 ## Observability
 
-See what the platform and your org are doing.
+*see everything — telemetry · analytics · usage*
 
-| Capability | Route | What it does | Backing |
-|---|---|---|---|
-| `o11y` | `/v1/o11y` | Logs / traces / metrics + alert rules (SigNoz runtime). | `hanzoai/o11y` |
-| `metrics` | `/v1/metrics` | Process / runtime metrics. | `hanzoai/metrics` |
-| `observe` | `/v1/o11y/{logs,metrics,status}`, `/v1/settings` | Org-scoped o11y read plane + console product-detail data plane. | `clients/observe` |
-| `analytics` | `/v1/analytics` | Per-org warehouse analytics (LLM usage + web/commerce lenses). | `clients/analytics` |
-| `usage` | `/v1/usage` | Cost roll-up (spend by category) + LLM usage totals. | `clients/usage` |
-| `tracker` | `/v1/tracker` | Native issue tracker (projects + issues). | `clients/tracker` |
+| Capability | Route | Spec |
+|---|---|---|
+| `o11y` | `/v1/o11y` | ✓ |
+| `observe` | `/v1/observe` | ✓ |
+| `analytics` | `/v1/analytics` | ✓ |
 
 ## Commerce
 
-Money, plans, and growth loops.
+*the economy — meter · price · bill · reward*
 
-| Capability | Route | What it does | Backing |
-|---|---|---|---|
-| `billing` | `/v1/billing` | Org-scoped billing reads (usage / balance). | `clients/billing` |
-| `pricing` | `/v1/pricing` | Pricing catalog + feature-enablement registry. | `clients/pricing` |
-| `plans` | `/v1/plans` | Subscription plans. | `clients/plan` |
-| `treasury` | `/v1/finance` | Double-entry reserve fund + finance ledger of record. | `clients/treasury` |
-| `referrals` | `/v1/referrals` | Viral-loop referral credit. | `clients/referrals` |
-| `affiliates` | `/v1/affiliates` | Partner-commission loop. | `clients/affiliates` |
-| `licensing` | `/v1/licensing` | Engine license issuance / validation. | `hanzoai/licensing` |
-| `crm` | `/v1/crm` | Native CRM (companies / contacts / opportunities). | `clients/crm` |
-| `product` | `/v1/product` | Product catalog / registry. | `clients/product` |
+| Capability | Route | Spec |
+|---|---|---|
+| `commerce` | `/v1/commerce` | ✓ |
+| `billing` | `/v1/billing` | ✓ |
+| `pricing` | `/v1/pricing` | ✓ |
+| `plan` | `/v1/plan` | ✓ |
+| `product` | `/v1/product` | ✓ |
+| `crm` | `/v1/crm` | ✓ |
+| `referrals` | `/v1/referrals` | ✓ |
+| `affiliates` | `/v1/affiliates` | ✓ |
+| `licensing` | `/v1/licensing` | pending |
 
 ## Platform
 
-The substrate: provision resources, deploy apps, route the edge.
+*the cloud fabric — deploy · provision · route · host*
 
-| Capability | Route | What it does | Backing |
-|---|---|---|---|
-| `platform` | `/v1/platform` | Per-org container-app PaaS (projects / apps / deploy). | `clients/platform` |
-| `paas` | `/v1/paas` | Admin fleet deploy board (operator Service CRs). | `clients/paas` |
-| `provisioning` | `/v1/{sql,vector,datastore,kv,search,docdb,storage,vpcs,load-balancers}` | Managed data resources + DO-native VPCs / load-balancers. | `clients/provisioning`, `clients/do` |
-| `gateway` | `api.hanzo.ai` | API gateway + LLM router (edge; routes to this binary). | `hanzoai/gateway` |
-| `ingress` | `/v1/ingress` | Runtime edge — routes / TLS / ACME / middlewares. | `clients/ingress` |
-| `visor` | `/v1/visor`, `/v1/{machines,gpus,clusters}` | Compute inventory — machines / GPUs / clusters. | `clients/visor` |
-| `fleet` | `/v1/visor/clusters` | BYO-cluster registry (kubeconfig fabric under visor). | `clients/fleet` |
-| `base` | `/v1/collections` | Embedded Hanzo Base — collections / records store. | `hanzoai/base` |
+| Capability | Route | Spec |
+|---|---|---|
+| `cloud` | `/v1/cloud` | ✓ |
+| `platform` | `/v1/platform` | ✓ |
+| `paas` | `/v1/paas` | ✓ |
+| `provisioning` | `/v1/provisioning` | ✓ |
+| `gateway` | `/v1/gateway` | ✓ |
+| `visor` | `/v1/visor` | ✓ |
+| `edge` | `/v1/edge` | ✓ |
+| `dns` | `/v1/dns` | ✓ |
+| `registry` | `/v1/registry` | ✓ |
+| `nexus` | `/v1/nexus` | ✓ |
+| `do` | `/v1/do` | ✓ |
 
 ## Applications
 
-The products users open.
+*the user-facing surfaces built on all of the above*
 
-| Capability | Route | What it does | Backing |
-|---|---|---|---|
-| `console` | `/v1/console` | Cloud API-key mint/revoke + org onboarding. | `clients/console` |
-| `team` | `/v1/team` | Team workspace read-plane + bots-as-members. | `clients/team` |
-| `projects` | `/v1/projects` | Buildable / deployable projects (artifact/git → S3 → live URL). | `clients/projects` |
-| `git` | `/v1/git` | Git repositories surface. | `clients/git` |
-| `templates` | `/v1/templates` | Starter-kit gallery. | `clients/templates` |
-| `integrations` | `/v1/integrations` | Provider-agnostic OAuth connector framework. | `clients/integrations` |
-| `bot` | `/v1/bot` | Bot gateway (reverse proxy → bot-gateway). | `clients/bot` |
-| `world` | `/v1/world` | News data plane (GDELT + allowlisted RSS). | `clients/world` |
-| `knowledge` | `/v1/knowledge`⁽ᵇ⁾ | Knowledge base + unified AI memory (wiki tree + vector). `(was /v1/kb)` | `clients/kb` |
-| `search` | `/v1/search`⁽ᵇ⁾ | Web meta-search + scrape (SearXNG + Crawl4AI). `(was /v1/websearch)` | `clients/websearch` |
-| `storage` | `/v1/storage`⁽ᵇ⁾ | Org object-storage file manager (buckets / objects). `(was /v1/s3)` | `clients/s3` |
-| `authors` | `/v1/authors` | OSS-author royalty loop. | `clients/authors` |
-| `automations` | `/v1/automations` | Connectors + flows engine (706-connector catalog). | `clients/automations` |
-| `sites` | (Host-based) | Published-site edge — served by Host; its control plane is `/v1/projects`. | `clients/sites` |
-| `cms` | (on `framework`) | Content-types as DocTypes. No bespoke REST — documents live on `/v1/framework`. | `clients/cms` |
-| `erp` | (on `framework`) | ERPNext DocTypes + business hooks. No bespoke REST — on `/v1/framework`. | `clients/erp` |
-| `help` | (on `framework`) | Helpdesk DocTypes. No bespoke REST — documents on `/v1/framework`. | `clients/help` |
-
-⁽ᵇ⁾ Canonical rename in progress (kb→knowledge, websearch→search, s3→storage);
-the old route is preserved as an alias during cutover.
+| Capability | Route | Spec |
+|---|---|---|
+| `console` | `/v1/console` | ✓ |
+| `projects` | `/v1/projects` | ✓ |
+| `app` | `/v1/app` | ✓ |
+| `git` | `/v1/git` | ✓ |
+| `templates` | `/v1/templates` | ✓ |
+| `integrations` | `/v1/integrations` | ✓ |
+| `bot` | `/v1/bot` | ✓ |
+| `chat` | `/v1/chat` | ✓ |
+| `kb` | `/v1/kb` | ✓ |
+| `search` | `/v1/search` | ✓ |
+| `websearch` | `/v1/websearch` | ✓ |
+| `world` | `/v1/world` | ✓ |
+| `authors` | `/v1/authors` | ✓ |
+| `sites` | `/v1/sites` | pending |
+| `tracker` | `/v1/tracker` | ✓ |
+| `framework` | `/v1/framework` | ✓ |
 
 ## Core
 
-Platform-wide primitives.
+| Capability | Route | Spec |
+|---|---|---|
+| `admin` | `/v1/admin` | ✓ |
+| `plugin` | `/v1/plugin` | ✓ |
 
-| Capability | Route | What it does | Backing |
-|---|---|---|---|
-| `admin` | `/v1/admin` | Platform SuperAdmin cross-tenant control plane (the god view). | `clients/admin` |
-| `plugins` | `/v1/plugins` | Runtime WASM / proxy plugins. | `clients/plugin` |
+## Collapsed
 
----
+Resolved duplicates / consolidations — removed, no spec dir. The capability now lives where noted.
+
+| Was | Now lives in |
+|---|---|
+| `eval` | evals |
+| `operative` | bot + visor |
+| `plans` | plan |
+| `plugins` | plugin |
+
+## Pending
+
+Live surfaces missing a spec — author these to complete the set.
+
+`team`, `wallets`, `treasury`, `usage`, `fleet`
+
+## Under review
+
+Braids to verify before folding — grouped today (a spec dir exists), consolidation is a separate verified decision.
+
+- **app_vs_projects** — app (hanzo.app builds/deploys, grouped→applications) vs projects (buildable/deployable sites) — likely ONE concern; fold app→projects?
+- **do_is_a_provider** — do (DigitalOcean VPCs/LBs, grouped→platform) is a PROVIDER, not a capability — fold into provisioning/platform?
+- **s3_vs_storage** — s3 (/v1/s3/*, grouped→data) is the present object store; storage is the canonical rename target (no dir yet) — fold s3→storage once the dir is renamed?
+- **observe_vs_o11y** — observe (/v1/o11y/{logs,metrics,status} + /v1/settings, grouped→observability) is an o11y read-plane facet + a console settings facet — fold observe→o11y (+ console) once the routes migrate?
+- **workflow_cluster** — auto (Hanzo Auto engine) · automations (connector pieces) · flow — three workflow surfaces; decomplect to one?
+- **retrieval_cluster** — search vs kb (semantic) vs vector (store) — overlapping retrieval; is search distinct from kb?
 
 ## Internal — NOT public capabilities
 
-These are trust boundaries and runtime hosts with **no public API surface**. They
-must never appear in the public spec, the capability manifest, or a client's
-known-heads list.
+Trust boundaries and runtime hosts with no public API surface. Excluded from the master, `x-tagGroups`, SDKs and docs.
 
-| Internal | Why it is not a capability |
-|---|---|
-| `principal` | Tenant-resolution trust boundary — a middleware seam, not a mountable surface. |
-| `goja` | JavaScript runtime host for base-embedded node services — an execution host. |
-| `mpc` | Threshold sealing / signing ring — reached only via `wallets`, never directly. |
-| `controlplane` | Build-tagged consensus simulation — off in production builds. |
-
----
-
-*Reconciliation note.* The per-service specs under `<service>/openapi.yaml` are
-the historical, hand-authored set and are being reconciled to the capability
-names above. Where a spec's directory still uses a legacy name (`ml`, `s3`,
-`zt`, `mq`, …), the canonical name in this manifest is the target. The master
-`hanzo.yaml` groups every present tag into these categories via `x-tagGroups`.
+`principal`, `goja`, `mpc`, `controlplane`, `metrics`, `audit`

LLM.md

@@ -6,16 +6,26 @@ references.
 
 ## Layout
 
-- `CAPABILITIES.md` — the CANONICAL capability index: one name per
-  capability, grouped into the eight categories, each with its `/v1/<name>`
-  prefix. Authoritative — every spec, the cloud binary, and the console
-  reconcile to it.
-- `hanzo.yaml` — unified master, aggregated + grouped by `merge.py`.
+- `capabilities.yaml` — the ONE canonical registry: every capability name, in
+  exactly one domain (or `core`), plus `internal` / `collapsed` / `pending` /
+  `review`. This is the single source of truth — `merge.py` READS it. Edit
+  this, then run `python3 merge.py`.
+- `CAPABILITIES.md` — GENERATED from `capabilities.yaml` by `merge.py`. A
+  derived human index; never hand-edit (it carries a `GENERATED — DO NOT EDIT`
+  header).
+- `hanzo.yaml` — unified master, aggregated from the per-service specs and
+  grouped (`x-tagGroups`) by `merge.py` straight from `capabilities.yaml`.
 - `<service>/openapi.yaml` — one self-contained spec per service.
 - `shared/` — shared schemas usable by individual specs in their `components`.
 - `README.md` — service index and usage.
 - `CHANGELOG.md` — v1 lock-in entry.
 
+`merge.py` enforces one-and-one-way as a build invariant: every present
+`<service>/openapi.yaml` dir MUST map to exactly one entry across
+`domains ∪ core` in `capabilities.yaml` (orphan / unlisted / double-listed →
+build fails); a `collapsed` name must have NO spec dir; `internal` services are
+excluded from the master and `x-tagGroups`.
+
 ## Conventions
 
 - Routing: every route is `/v1/<service>/<resource>`.
@@ -25,9 +35,9 @@ references.
 - No cross-file `$ref`. Each spec is self-contained.
 - No `deprecated: true`. Forward-only.
 - No `/api/` prefix anywhere.
-- Master grouping: `merge.py` emits `x-tagGroups` (the eight canonical
-  categories from `CAPABILITIES.md`); every present spec must belong to
-  exactly one group or the merge fails.
+- Master grouping: `merge.py` emits `x-tagGroups` from the domains in
+  `capabilities.yaml` (their `title`) plus a `Core` group; every present spec
+  must belong to exactly one group or the merge fails.
 
 ## Validate
 
@@ -42,8 +52,12 @@ python3 -c "import yaml, glob; [yaml.safe_load(open(s)) for s in glob.glob('*/op
 3. Add components in the spec's own `components.schemas`. No `$ref` to
    other service yamls.
 4. Examples must come from real responses.
-5. Update `CHANGELOG.md` with a dated entry under the v1.0.0 heading.
-6. Run `python3 merge.py` (regenerates `hanzo.yaml`) and commit both.
+5. If you add or remove a `<service>/openapi.yaml` dir, add/remove its name in
+   `capabilities.yaml` (exactly one domain, or `core`) — the merge FAILS
+   otherwise.
+6. Update `CHANGELOG.md` with a dated entry under the v1.0.0 heading.
+7. Run `python3 merge.py` (regenerates `hanzo.yaml` AND `CAPABILITIES.md`) and
+   commit all three.
 
 ## SDK generation — the ONE way (Stainless RETIRED, 2026-07)
 

capabilities.yaml

@@ -1,12 +1,22 @@
 # capabilities.yaml — the ONE registry of Hanzo public capabilities.
 #
+# This file is the single source of truth. merge.py READS it (there is no
+# hand-kept GROUPS dict anymore) to build hanzo.yaml's x-tagGroups AND to
+# generate CAPABILITIES.md. CAPABILITIES.md is a DERIVED artifact — never edit it
+# by hand; edit this file and run `python3 merge.py`.
+#
 # One and one way, by construction:
-#   • every openapi/<name>/openapi.yaml MUST appear here exactly once, in exactly
-#     one domain — merge.py FAILS the build on an orphan spec, an unlisted spec,
-#     or a name in two domains.  Orthogonality is a build invariant, not a hope.
+#   • every openapi/<name>/openapi.yaml MUST appear here exactly once, across
+#     domains ∪ core — merge.py FAILS the build on an orphan spec (dir with no
+#     entry), an unlisted spec, or a name in two entries. Orthogonality is a
+#     build invariant, not a hope.
+#   • a `collapsed` name MUST have NO spec dir (build fails if a dir reappears).
+#   • `internal` names are EXCLUDED from the master, x-tagGroups, SDKs and docs.
+#   • `pending`/`review`/domain names WITHOUT a dir are fine (not-yet-authored or
+#     future-fold); only a spec dir on disk is load-bearing for the invariant.
 #   • the route IS the identity: one capability = one name = one /v1/<name> = one
 #     <name>/openapi.yaml.  No svc suffix, no singular/plural twins, no facade.
-#   • merge.py emits x-tagGroups from `domains` so Redoc/Scalar render the eight
+#   • merge.py emits x-tagGroups from `domains`+`core` so Redoc/Scalar render the
 #     movements automatically.  Docs + SDKs derive from hanzo.yaml — never hand-kept.
 #
 # version: the API path stays /v1 (the compatibility contract, immutable).  The
@@ -29,7 +39,7 @@ domains:
   - id: data
     title: "Data"
     role: "the stores every capability writes to and reads from"
-    services: [storage, base, db, kv, vector]
+    services: [storage, s3, base, db, kv, vector]
 
   - id: streams
     title: "Streams"
@@ -39,22 +49,22 @@ domains:
   - id: observability
     title: "Observability"
     role: "see everything — telemetry · analytics · usage"
-    services: [o11y, analytics]
+    services: [o11y, observe, analytics]
 
   - id: commerce
     title: "Commerce"
     role: "the economy — meter · price · bill · reward"
-    services: [commerce, billing, pricing, plan, product, crm, referrals, affiliates, licensing, treasury, wallets]
+    services: [commerce, billing, pricing, plan, product, crm, referrals, affiliates, licensing]
 
   - id: platform
     title: "Platform"
     role: "the cloud fabric — deploy · provision · route · host"
-    services: [cloud, platform, paas, provisioning, gateway, visor, edge, dns, registry, nexus]
+    services: [cloud, platform, paas, provisioning, gateway, visor, edge, dns, registry, nexus, do]
 
   - id: applications
     title: "Applications"
     role: "the user-facing surfaces built on all of the above"
-    services: [console, projects, git, team, templates, integrations, bot, chat, kb, search, websearch, authors, sites, tracker, framework]
+    services: [console, projects, app, git, templates, integrations, bot, chat, kb, search, websearch, world, authors, sites, tracker, framework]
 
 # Core — governance that owns no single domain.
 core: [admin, plugin]
@@ -66,19 +76,21 @@ internal: [principal, goja, mpc, controlplane, metrics, audit]
 # NO spec dir (they were removed), and records where the capability now lives.
 collapsed:
   eval:      evals                     # pure dup — same /v1/evals
-  observe:   o11y + console/settings   # /v1/o11y facet → o11y; /v1/settings → console
   operative: bot + visor              # computer-using-agent split: bot = the CUA (@hanzo/bot), visor = the VM it runs on
-  s3:        storage                   # renamed: storage is the capability, s3 the implementation
   plans:     plan                      # singular canonical
   plugins:   plugin
 
 # Not-yet-scored — LIVE surfaces missing a spec. Author these to complete the set.
 pending: [team, wallets, treasury, usage, fleet]
 
-# Braids under review — same concern, different names? Verify before merging;
-# do NOT silently consolidate (would lose capability). merge.py warns, not fails.
+# Braids under review — same concern, different names? A spec dir still exists
+# (so it is GROUPED into a domain above and counts for the invariant); folding it
+# away is a separate, verified decision — do NOT silently consolidate here (would
+# lose capability). merge.py groups them; it does not fold them.
 review:
-  app_vs_projects: "app (hanzo.app builds/deploys) vs projects (buildable/deployable sites) — likely ONE concern; fold app→projects?"
-  do_is_a_provider: "do (DigitalOcean VPCs/LBs) is a PROVIDER, not a capability — fold into provisioning/platform?"
+  app_vs_projects: "app (hanzo.app builds/deploys, grouped→applications) vs projects (buildable/deployable sites) — likely ONE concern; fold app→projects?"
+  do_is_a_provider: "do (DigitalOcean VPCs/LBs, grouped→platform) is a PROVIDER, not a capability — fold into provisioning/platform?"
+  s3_vs_storage: "s3 (/v1/s3/*, grouped→data) is the present object store; storage is the canonical rename target (no dir yet) — fold s3→storage once the dir is renamed?"
+  observe_vs_o11y: "observe (/v1/o11y/{logs,metrics,status} + /v1/settings, grouped→observability) is an o11y read-plane facet + a console settings facet — fold observe→o11y (+ console) once the routes migrate?"
   workflow_cluster: "auto (Hanzo Auto engine) · automations (connector pieces) · flow — three workflow surfaces; decomplect to one?"
   retrieval_cluster: "search vs kb (semantic) vs vector (store) — overlapping retrieval; is search distinct from kb?"

hanzo.yaml

@@ -2,8 +2,8 @@ openapi: 3.1.0
 info:
   title: "Hanzo Cloud \u2014 Unified API (V8 \xB7 Open Edition)"
   description: "The single unified OpenAPI surface for ALL Hanzo services, aggregated from the per-service specs. Every route\
-    \ is https://api.hanzo.ai/v1/<service>/*. Grouped by the eight canonical categories (see CAPABILITIES.md). Regenerated\
-    \ by merge.py \u2014 edit the per-service <svc>/openapi.yaml, not this file."
+    \ is https://api.hanzo.ai/v1/<service>/*. Grouped by the canonical domains in capabilities.yaml (the ONE registry). Regenerated\
+    \ by merge.py \u2014 edit the per-service <svc>/openapi.yaml + capabilities.yaml, not this file."
   version: 8.0.0
   contact:
     name: Hanzo AI
@@ -152,90 +152,92 @@ tags:
 - name: zt
   description: Hanzo ZT API
 x-tagGroups:
-- name: Identity
+- name: Identity & Trust
   tags:
   - iam
   - authz
-  - security
   - kms
+  - security
+  - did
   - zt
   - guard
-  - did
-- name: AI
+- name: Intelligence
   tags:
   - ai
-  - agents
   - ml
+  - engine
+  - agents
   - evals
   - prompts
   - functions
   - exec
-  - framework
   - graph
-  - engine
-- name: Messaging
+- name: Data
+  tags:
+  - s3
+  - base
+  - db
+  - kv
+  - vector
+- name: Streams
   tags:
   - pubsub
   - mq
   - stream
   - tasks
   - notify
+  - automations
+  - flow
+  - auto
 - name: Observability
   tags:
   - o11y
   - observe
   - analytics
-  - tracker
 - name: Commerce
   tags:
+  - commerce
   - billing
   - pricing
   - plan
+  - product
+  - crm
   - referrals
   - affiliates
-  - crm
-  - product
-  - commerce
 - name: Platform
   tags:
+  - cloud
   - platform
   - paas
   - provisioning
   - gateway
   - visor
-  - base
-  - do
   - edge
   - dns
   - registry
   - nexus
-  - db
-  - kv
-  - vector
+  - do
 - name: Applications
   tags:
   - console
   - projects
+  - app
   - git
   - templates
   - integrations
   - bot
+  - chat
   - kb
   - search
   - websearch
-  - s3
-  - authors
-  - automations
-  - chat
-  - auto
-  - app
-  - flow
   - world
+  - authors
+  - tracker
+  - framework
 - name: Core
   tags:
   - admin
   - plugin
-  - cloud
 security:
 - bearerAuth: []
 paths:

merge.py

@@ -1,68 +1,90 @@
 #!/usr/bin/env python3
-"""Regenerate the unified master `hanzo.yaml` from every per-service spec.
-
-The master is the ONE unified OpenAPI surface for all Hanzo services. It is a
-pure aggregation — edit a per-service `<svc>/openapi.yaml`, then run this:
+"""Regenerate the unified master `hanzo.yaml` AND `CAPABILITIES.md` from the ONE
+source of truth, `capabilities.yaml`, plus every per-service spec.
 
     python3 merge.py
 
-Every service is exposed at `https://api.hanzo.ai/v1/<service>/*`, so paths are
-already namespaced and never collide. Component schemas/responses/parameters ARE
-namespaced here (`<svc>_<Name>`) with their `$ref`s rewritten to match, so two
-services can both define `Error`/`User`/etc. without clobbering each other.
+There is ONE registry: `capabilities.yaml`. This script READS it — there is no
+hand-kept category map here. From it we derive TWO artifacts:
 
-The master is grouped by the eight canonical categories via `x-tagGroups` (the
-Redocly navigation extension). The category taxonomy is the SAME one documented
-in `CAPABILITIES.md` — that manifest is authoritative; `GROUPS` here mirrors it
-for the present per-service specs.
+  • `hanzo.yaml`  — the unified OpenAPI surface. A pure aggregation of every
+    `<svc>/openapi.yaml`; every service is exposed at
+    `https://api.hanzo.ai/v1/<svc>/*`, so paths never collide, and component
+    schemas/responses/parameters are namespaced `<svc>_<Name>` (with `$ref`s
+    rewritten) so two services can both define `Error`/`User` without clobbering.
+    Grouped by `x-tagGroups` — one group per `capabilities.yaml` domain (using
+    the domain `title`) plus a `Core` group.
+
+  • `CAPABILITIES.md` — a human index. GENERATED; never edit by hand.
+
+Build invariant (fail loud — orthogonality is enforced, not hoped):
+  • every present `<svc>/openapi.yaml` dir maps to EXACTLY ONE entry across
+    `domains ∪ core` (orphan / unlisted / double-listed → sys.exit);
+  • a `collapsed` name MUST have NO spec dir (a reappeared dir → sys.exit);
+  • `internal` services are EXCLUDED from the master + x-tagGroups (their dirs,
+    if any, are skipped);
+  • `pending` / `review` / domain names WITHOUT a dir are fine (not-yet-authored
+    or future-fold candidates) — only a dir on disk is load-bearing.
 """
 import os
 import sys
 import yaml
 
 ROOT = os.path.dirname(os.path.abspath(__file__))
-
-# Canonical category -> the per-service spec dirs that belong to it. Mirrors the
-# capability categories in CAPABILITIES.md; legacy/duplicate spec dirs (e.g. ml,
-# s3, mq, auto) are grouped under the SAME category as their canonical name so the
-# master stays navigable while the specs are reconciled. Every present
-# `<svc>/openapi.yaml` MUST appear in exactly one group; `check_groups` fails the
-# build otherwise so the grouping never drifts.
-GROUPS = {
-    "Identity": ["iam", "authz", "security", "kms", "zt", "guard", "did"],
-    "AI": ["ai", "agents", "ml", "eval", "evals", "prompts", "functions",
-           "exec", "framework", "graph", "engine"],
-    "Messaging": ["pubsub", "mq", "stream", "tasks", "notify"],
-    "Observability": ["o11y", "observe", "analytics", "tracker"],
-    "Commerce": ["billing", "pricing", "plan", "referrals", "affiliates",
-                 "crm", "product", "commerce"],
-    "Platform": ["platform", "paas", "provisioning", "gateway", "visor", "base",
-                 "do", "edge", "dns", "registry", "nexus", "operative", "db",
-                 "kv", "vector"],
-    "Applications": ["console", "projects", "git", "templates", "integrations",
-                     "bot", "kb", "search", "websearch", "s3", "authors",
-                     "automations", "chat", "auto", "app", "flow", "world"],
-    "Core": ["admin", "plugin", "cloud"],
-}
+CAPABILITIES = os.path.join(ROOT, "capabilities.yaml")
 
 
-def group_of(svc):
-    for name, members in GROUPS.items():
-        if svc in members:
-            return name
-    return None
+def load_registry():
+    """The ONE source of truth. Returns (categories, internal, collapsed, cap).
+
+    `categories` is an ordered list of (group_name, [services]) — one per domain
+    (keyed by the domain `title`) then `Core`. This is the ONLY place the
+    category taxonomy comes from; nothing here is hand-maintained.
+    """
+    cap = yaml.safe_load(open(CAPABILITIES))
+    categories = [(d["title"], list(d["services"])) for d in cap["domains"]]
+    categories.append(("Core", list(cap.get("core") or [])))
+    internal = set(cap.get("internal") or [])
+    collapsed = set(cap.get("collapsed") or {})
+    return categories, internal, collapsed, cap
 
 
-def check_groups(services):
-    """Every present service must be grouped exactly once; fail loud on drift."""
-    grouped = [s for members in GROUPS.values() for s in members]
-    dupes = {s for s in grouped if grouped.count(s) > 1}
+def spec_dirs():
+    """Every present `<svc>/openapi.yaml` dir (the ground truth)."""
+    return sorted(
+        d for d in os.listdir(ROOT)
+        if os.path.isdir(os.path.join(ROOT, d))
+        and os.path.isfile(os.path.join(ROOT, d, "openapi.yaml"))
+        and d != "shared"
+    )
+
+
+def check_invariant(present, categories, internal, collapsed):
+    """Fail loud unless the registry and the spec dirs on disk agree exactly."""
+    grouped_list = [s for _, members in categories for s in members]
+    grouped = set(grouped_list)
+
+    dupes = sorted({s for s in grouped_list if grouped_list.count(s) > 1})
     if dupes:
-        sys.exit(f"merge: service(s) in >1 group: {sorted(dupes)}")
-    ungrouped = [s for s in services if group_of(s) is None]
-    if ungrouped:
-        sys.exit(f"merge: service(s) not in any GROUPS category "
-                 f"(add to GROUPS + CAPABILITIES.md): {ungrouped}")
+        sys.exit(f"merge: name(s) listed in >1 domain/core in capabilities.yaml: {dupes}")
+
+    # A collapsed name is a REMOVED capability — its dir must be gone.
+    collapsed_with_dir = sorted(collapsed & set(present))
+    if collapsed_with_dir:
+        sys.exit(f"merge: `collapsed` name(s) still have a spec dir "
+                 f"(group them in a domain or delete the dir): {collapsed_with_dir}")
+
+    # Internal dirs are excluded from the master; they must NOT be grouped.
+    internal_grouped = sorted(internal & grouped)
+    if internal_grouped:
+        sys.exit(f"merge: `internal` name(s) also listed in a domain/core "
+                 f"(internal is excluded from the master): {internal_grouped}")
+
+    # Every present (non-internal) spec dir must be grouped exactly once.
+    orphans = sorted(s for s in present if s not in grouped and s not in internal)
+    if orphans:
+        sys.exit(f"merge: spec dir(s) present but not in any domain/core of "
+                 f"capabilities.yaml (add them): {orphans}")
 
 
 def prefix(node, svc):
@@ -83,17 +105,14 @@ def prefix(node, svc):
     return node
 
 
-def main():
-    services = sorted(
-        d for d in os.listdir(ROOT)
-        if os.path.isdir(os.path.join(ROOT, d))
-        and os.path.isfile(os.path.join(ROOT, d, "openapi.yaml"))
-        and d != "shared"
-    )
-    check_groups(services)
+def build_master(present, categories, internal):
+    """Aggregate every present per-service spec into the unified hanzo.yaml dict.
+
+    Internal services are excluded (their dirs, if any, are skipped)."""
+    included = [s for s in present if s not in internal]
 
     paths, schemas, responses, params, secschemes, tags = {}, {}, {}, {}, {}, []
-    for svc in services:
+    for svc in included:
         spec = yaml.safe_load(open(os.path.join(ROOT, svc, "openapi.yaml")))
         comps = spec.get("components", {}) or {}
         for p, item in (spec.get("paths", {}) or {}).items():
@@ -108,12 +127,13 @@ def main():
             secschemes.setdefault(n, x)
         tags.append({"name": svc, "description": spec.get("info", {}).get("title", svc)})
 
-    # x-tagGroups: canonical category → its present service tags, in category order.
+    # x-tagGroups: one group per capabilities.yaml category, present tags only,
+    # in registry order.
     tag_groups = []
-    for name in GROUPS:
-        members = [s for s in GROUPS[name] if s in services]
-        if members:
-            tag_groups.append({"name": name, "tags": members})
+    for name, members in categories:
+        member_tags = [s for s in members if s in included]
+        if member_tags:
+            tag_groups.append({"name": name, "tags": member_tags})
 
     components = {
         "securitySchemes": secschemes or {"bearerAuth": {"type": "http", "scheme": "bearer"}},
@@ -124,16 +144,17 @@ def main():
     if params:
         components["parameters"] = dict(sorted(params.items()))
 
-    out = {
+    master = {
         "openapi": "3.1.0",
         "info": {
             "title": "Hanzo Cloud — Unified API (V8 · Open Edition)",
             "description": (
                 "The single unified OpenAPI surface for ALL Hanzo services, "
                 "aggregated from the per-service specs. Every route is "
-                "https://api.hanzo.ai/v1/<service>/*. Grouped by the eight "
-                "canonical categories (see CAPABILITIES.md). Regenerated by "
-                "merge.py — edit the per-service <svc>/openapi.yaml, not this file."
+                "https://api.hanzo.ai/v1/<service>/*. Grouped by the canonical "
+                "domains in capabilities.yaml (the ONE registry). Regenerated by "
+                "merge.py — edit the per-service <svc>/openapi.yaml + "
+                "capabilities.yaml, not this file."
             ),
             "version": "8.0.0",
             "contact": {"name": "Hanzo AI", "url": "https://hanzo.ai", "email": "[email protected]"},
@@ -146,11 +167,114 @@ def main():
         "paths": dict(sorted(paths.items())),
         "components": components,
     }
-    yaml.dump(out, open(os.path.join(ROOT, "hanzo.yaml"), "w"),
+    return master, len(included), len(paths), len(schemas), len(responses), len(params), len(tag_groups)
+
+
+def render_capabilities_md(present, categories, cap):
+    """Render CAPABILITIES.md FROM capabilities.yaml — a DERIVED artifact.
+
+    A present spec dir gets a live `/v1/<name>` route; a listed name with no dir
+    yet is shown as `pending` (not-yet-authored)."""
+    present = set(present)
+    lines = []
+    lines.append("<!-- GENERATED by merge.py from capabilities.yaml — DO NOT EDIT. "
+                 "Edit capabilities.yaml and run `python3 merge.py`. -->")
+    lines.append("")
+    lines.append(f"# Hanzo Capability Manifest — {cap.get('edition', '')}".rstrip())
+    lines.append("")
+    lines.append(
+        "GENERATED from `capabilities.yaml`, the ONE registry of Hanzo "
+        "capabilities. Do NOT edit this file by hand — edit `capabilities.yaml` "
+        "and run `python3 merge.py`. `merge.py` reads the same registry to emit "
+        "`hanzo.yaml`'s `x-tagGroups`, so this index and the master never drift."
+    )
+    lines.append("")
+    lines.append(
+        "One canonical name per capability = one `/v1/<name>` route = one "
+        "`<name>/openapi.yaml`. A ✓ marks a name with a present spec dir "
+        "(live in `hanzo.yaml`); *pending* marks a listed name whose spec is "
+        "not yet authored."
+    )
+    lines.append("")
+
+    for name, members in categories:
+        dom = next((d for d in cap["domains"] if d["title"] == name), None)
+        lines.append(f"## {name}")
+        lines.append("")
+        if dom and dom.get("role"):
+            lines.append(f"*{dom['role']}*")
+            lines.append("")
+        lines.append("| Capability | Route | Spec |")
+        lines.append("|---|---|---|")
+        for svc in members:
+            if svc in present:
+                lines.append(f"| `{svc}` | `/v1/{svc}` | ✓ |")
+            else:
+                lines.append(f"| `{svc}` | `/v1/{svc}` | pending |")
+        lines.append("")
+
+    collapsed = cap.get("collapsed") or {}
+    if collapsed:
+        lines.append("## Collapsed")
+        lines.append("")
+        lines.append("Resolved duplicates / consolidations — removed, no spec dir. "
+                     "The capability now lives where noted.")
+        lines.append("")
+        lines.append("| Was | Now lives in |")
+        lines.append("|---|---|")
+        for k, v in collapsed.items():
+            lines.append(f"| `{k}` | {v} |")
+        lines.append("")
+
+    pending = cap.get("pending") or []
+    if pending:
+        lines.append("## Pending")
+        lines.append("")
+        lines.append("Live surfaces missing a spec — author these to complete the set.")
+        lines.append("")
+        lines.append(", ".join(f"`{p}`" for p in pending))
+        lines.append("")
+
+    review = cap.get("review") or {}
+    if review:
+        lines.append("## Under review")
+        lines.append("")
+        lines.append("Braids to verify before folding — grouped today (a spec dir "
+                     "exists), consolidation is a separate verified decision.")
+        lines.append("")
+        for k, v in review.items():
+            lines.append(f"- **{k}** — {v}")
+        lines.append("")
+
+    internal = cap.get("internal") or []
+    if internal:
+        lines.append("## Internal — NOT public capabilities")
+        lines.append("")
+        lines.append("Trust boundaries and runtime hosts with no public API surface. "
+                     "Excluded from the master, `x-tagGroups`, SDKs and docs.")
+        lines.append("")
+        lines.append(", ".join(f"`{s}`" for s in internal))
+        lines.append("")
+
+    return "\n".join(lines).rstrip() + "\n"
+
+
+def main():
+    categories, internal, collapsed, cap = load_registry()
+    present = spec_dirs()
+    check_invariant(present, categories, internal, collapsed)
+
+    master, n_svc, n_paths, n_schemas, n_resp, n_params, n_groups = build_master(
+        present, categories, internal)
+    yaml.dump(master, open(os.path.join(ROOT, "hanzo.yaml"), "w"),
               default_flow_style=False, sort_keys=False, width=120)
-    print(f"merged {len(services)} services → {len(paths)} paths, "
-          f"{len(schemas)} schemas, {len(responses)} responses, {len(params)} params, "
-          f"{len(tag_groups)} categories")
+
+    md = render_capabilities_md(present, categories, cap)
+    open(os.path.join(ROOT, "CAPABILITIES.md"), "w").write(md)
+
+    print(f"merged {n_svc} services → {n_paths} paths, {n_schemas} schemas, "
+          f"{n_resp} responses, {n_params} params, {n_groups} categories")
+    print(f"generated CAPABILITIES.md ({n_groups} categories) from capabilities.yaml")
     return 0