flows: name the six canonical example flows, and gate them
Every Hanzo SDK is supposed to ship the SAME six examples — hello, chat, money,
store, agent, tools. Until now that was six repos independently remembering to
agree, and nothing checked. Nothing could: the flows existed in no manifest, in
no repo, in any language. The TypeScript and Python sets are the first two, and
writing them is what made the gap obvious.
flows.yaml is data only, like sdks.yaml beside it: per flow, the operationIds in
hanzo.yaml in call order, plus why THAT route and not a neighbour. Those choices
are the part worth writing down —
hello ai_getAccount, not bot_authMe (bot-service scoped) and not the IAM OIDC
userinfo endpoint, which is on a different ORIGIN; using it would force
every example set to know two base URLs, and IAM answers 200 text/html
for an unregistered path, so a typo there is silent breakage, not a 404.
tools automations_mcp, not tasks_tasksMcp: only the former has typed
request/response schemas, so `method` is a checked enum in every
generated client instead of a free string.
store the data plane (/v1/kv/keys/*), not the provisioning plane (/v1/kv,
which creates stores) — the two are easy to confuse by name alone.
test_flows.py asserts every id named here still exists in hanzo.yaml. Verified it
fails on a typo, not just passes on the truth:
FAIL test_every_operation_exists_in_the_spec: flows.yaml names operations
hanzo.yaml does not have: [('hello', 'ai_getAccount_TYPO')]
So deleting a route breaks ONE gate here rather than six SDKs' examples quietly
and separately.
Also gives this repo a `test:` block, which it did not have. 25 tests lived here
and not one had ever gated a commit — and that is not academic: hanzo.yaml spent
this entire period unable to generate a client in ANY language, because 35
/v1/platform operations carried no `responses` and openapi-generator refuses the
whole document over it. Nothing here noticed, because nothing here ever tried.
hanzo.yaml is the single input to seven language SDKs; a defect merged here is a
defect merged into all seven at once.
25 passed in 19.99s
Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
+82
@@ -0,0 +1,82 @@
|
||||
# The canonical example flows — data only, like sdks.yaml beside it.
|
||||
#
|
||||
# Every Hanzo SDK ships the SAME six examples under `examples/<flow>/`, in this
|
||||
# order, under these names. A reader who knows one language's set can then find
|
||||
# their way around another's without re-learning what the repo chose to
|
||||
# demonstrate. This file is what makes "the same six" a fact rather than six
|
||||
# repos independently remembering to agree.
|
||||
#
|
||||
# `operations` are operationIds in hanzo.yaml, in call order. That is the part a
|
||||
# language cannot fudge: test_flows.py asserts every id here exists in the
|
||||
# merged spec, so deleting or renaming a route upstream breaks THIS gate — one
|
||||
# place — instead of breaking six SDKs quietly and separately.
|
||||
#
|
||||
# Nothing here is executable and no SDK repo duplicates it. What each example
|
||||
# prints is the language's business; WHICH routes it calls is not.
|
||||
#
|
||||
# Rules an example must hold to, in every language:
|
||||
# • read the key from HANZO_API_KEY;
|
||||
# • default to https://api.hanzo.ai, overridable by HANZO_BASE_URL;
|
||||
# • resolve both in ONE shared place per repo, not once per flow;
|
||||
# • be compiled or imported by that repo's CI gate, so it cannot rot.
|
||||
|
||||
flows:
|
||||
hello:
|
||||
summary: Identity — prove the key works and print who it belongs to.
|
||||
# The AI plane's account read. Deliberately not /v1/bot/auth/me (bot-service
|
||||
# scoped) and not the IAM OIDC userinfo endpoint, which lives on a DIFFERENT
|
||||
# origin — that would force every example set to know two base URLs, and IAM
|
||||
# answers 200 text/html for an unregistered path, so a typo there is silent
|
||||
# breakage rather than a 404.
|
||||
operations: [ai_getAccount]
|
||||
notes: >-
|
||||
Every /v1/ai/* route answers with an envelope where `status` is the
|
||||
OUTCOME, not the HTTP code: a handled failure is HTTP 200 with
|
||||
status=error. This flow is where an SDK teaches that, because it is the
|
||||
first call anyone makes.
|
||||
|
||||
chat:
|
||||
summary: One completion.
|
||||
operations: [ai_createChatCompletion]
|
||||
notes: >-
|
||||
OpenAI-compatible request and response. Non-streaming on purpose —
|
||||
streaming is a different transport (SSE) that the generated clients return
|
||||
as an opaque body, so demonstrating it here would teach the wrong shape.
|
||||
|
||||
money:
|
||||
summary: Balance, then the usage that moved it.
|
||||
operations: [billing_billingBalance, billing_billingUsage]
|
||||
notes: >-
|
||||
Neither takes an org: both derive the tenant server-side from the JWT
|
||||
`owner` claim, so a key can only read its own money. The usage ledger is
|
||||
unbounded by default — an example asks for a window.
|
||||
|
||||
store:
|
||||
summary: KV round-trip — set, read back, delete.
|
||||
operations: [kv_setKey, kv_getKey, kv_deleteKey]
|
||||
notes: >-
|
||||
The delete belongs in the language's finally/defer, so a failed read still
|
||||
cleans up instead of leaving the key for the next run to collide with.
|
||||
This is the data plane (/v1/kv/keys/*), not the provisioning plane
|
||||
(/v1/kv, which creates stores).
|
||||
|
||||
agent:
|
||||
summary: Create an agent, run it, read it back.
|
||||
operations:
|
||||
- cloud_AgentsController.Create
|
||||
- cloud_AgentsController.Run
|
||||
- cloud_AgentsController.Get
|
||||
notes: >-
|
||||
`ref` accepts the public id (agent_...) OR the org-unique name, which is
|
||||
why run and read can both use the name just created without waiting for an
|
||||
id. Names are org-unique, so an example must not hardcode one.
|
||||
|
||||
tools:
|
||||
summary: List the MCP tools this key can reach.
|
||||
operations: [automations_mcp]
|
||||
notes: >-
|
||||
JSON-RPC 2.0; send method=tools/list. Chosen over the other MCP door
|
||||
(tasks_tasksMcp) because this one has typed request/response schemas, so
|
||||
`method` is a checked enum in every generated client rather than a free
|
||||
string. JSON-RPC reports failure INSIDE a 200 as error={code,message}, so
|
||||
an example checks `error` before reading `result`.
|
||||
@@ -11,3 +11,31 @@ images:
|
||||
context: .
|
||||
dockerfile: Dockerfile.skills
|
||||
repo: ghcr.io/hanzoai/agent-skills
|
||||
|
||||
test:
|
||||
- name: spec
|
||||
# This repo had 25 tests and no `test:` block, so none of them had ever
|
||||
# gated a commit. They matter more here than in most repos: hanzo.yaml is
|
||||
# the single input to SEVEN language SDKs, so a defect merged here is a
|
||||
# defect merged into all seven at once, discovered seven separate times.
|
||||
#
|
||||
# It is not hypothetical. hanzo.yaml spent this whole period unable to
|
||||
# generate ANY client in ANY language — 35 /v1/platform operations carried
|
||||
# no `responses`, which OAS 3.x requires and openapi-generator refuses the
|
||||
# entire document over. Nothing in this repo noticed, because nothing in
|
||||
# this repo ever tried.
|
||||
#
|
||||
# test_flows.py is the new one: it asserts every operationId named in
|
||||
# flows.yaml still exists in hanzo.yaml, so deleting a route breaks one gate
|
||||
# here instead of six SDKs' examples quietly and separately.
|
||||
#
|
||||
# uv rather than a bare python3 — the arc runner image promises no
|
||||
# interpreter, let alone one with PyYAML.
|
||||
run: |
|
||||
set -e
|
||||
if command -v uv >/dev/null 2>&1; then UV=uv; else
|
||||
curl -fsSL https://astral.sh/uv/install.sh | sh
|
||||
UV="$HOME/.local/bin/uv"
|
||||
fi
|
||||
"$UV" run --no-project --python 3.12 --with pyyaml --with pytest \
|
||||
python -m pytest test_audit.py test_skills.py test_flows.py -q
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for flows.yaml — the manifest that keeps every SDK's examples the same six.
|
||||
|
||||
Runnable standalone (`python3 test_flows.py`, non-zero exit on failure) AND under
|
||||
pytest, like test_audit.py beside it.
|
||||
|
||||
The manifest is only worth having if it cannot drift from the spec it names. An
|
||||
operationId that no longer exists is the exact failure this catches: without it,
|
||||
deleting a route upstream leaves six SDKs each carrying an example that no longer
|
||||
compiles, discovered six separate times. Here it is one red test in one repo.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
ROOT = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
# The set is closed on purpose. A seventh flow is a fleet-wide decision — every
|
||||
# language owes the same six — so adding one should require editing this line.
|
||||
CANONICAL = ["hello", "chat", "money", "store", "agent", "tools"]
|
||||
|
||||
|
||||
def load():
|
||||
flows = yaml.safe_load(open(os.path.join(ROOT, "flows.yaml")))["flows"]
|
||||
spec = yaml.safe_load(open(os.path.join(ROOT, "hanzo.yaml")))
|
||||
methods = ("get", "post", "put", "patch", "delete", "head", "options")
|
||||
ids = {
|
||||
op["operationId"]
|
||||
for item in spec["paths"].values()
|
||||
for verb, op in item.items()
|
||||
if verb in methods and isinstance(op, dict) and op.get("operationId")
|
||||
}
|
||||
return flows, ids
|
||||
|
||||
|
||||
def test_the_six_are_present_and_only_those():
|
||||
flows, _ = load()
|
||||
assert list(flows) == CANONICAL, f"flows.yaml lists {list(flows)}, expected {CANONICAL}"
|
||||
|
||||
|
||||
def test_every_operation_exists_in_the_spec():
|
||||
flows, ids = load()
|
||||
missing = [
|
||||
(name, op)
|
||||
for name, flow in flows.items()
|
||||
for op in flow["operations"]
|
||||
if op not in ids
|
||||
]
|
||||
assert not missing, "flows.yaml names operations hanzo.yaml does not have: " + repr(missing)
|
||||
|
||||
|
||||
def test_every_flow_says_what_it_does():
|
||||
flows, _ = load()
|
||||
for name, flow in flows.items():
|
||||
assert flow.get("summary"), f"{name} has no summary"
|
||||
assert flow.get("operations"), f"{name} names no operations"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
failures = 0
|
||||
for name, fn in sorted(globals().items()):
|
||||
if name.startswith("test_") and callable(fn):
|
||||
try:
|
||||
fn()
|
||||
print(f"ok {name}")
|
||||
except AssertionError as e:
|
||||
failures += 1
|
||||
print(f"FAIL {name}: {e}")
|
||||
sys.exit(1 if failures else 0)
|
||||
Reference in New Issue
Block a user