key(run) is the sha256 the cloud derives as a run's key: benchmark,
split, system, version, k, temperature, max_tokens, embedder, reader and
the four digests, each as text and NUL-terminated. tests/test_measure.py
holds the vectors hanzo-inc/cloud's apps/research TestKeyVector does.
Run keeps the crate's fields; the key is computed, not carried.
Research.measure(identity, fn, force=False) asks for a complete run under
that key and returns the latest to end without calling fn. A miss, a
partial run or force calls fn, records the run and returns it as read
back. fn may not change an identity field.
Arena.post_run follows the arena: it answers the latest run held per
benchmark, or that nothing was queued, and spends nothing, so confirm,
endpoint and attempts go. Admission becomes Measurements.
Claude-Session: https://claude.ai/code/session_01M8RoubcPN71KgJuZ8LdBUL
A review of 0.2.0 found the client reading identity and scope other
than the server does. The acting org is now the server's: a person's
home is the first of its signed orgs, not owner (the org of the app it
signed in through); a machine token acts in its owner alone; an API key
acts in its own org and project default. A switch the server would not
honour is refused before the request, a write answered under another
project raises, and a bearer token goes over http to this machine only.
A run id is unique only within a project, so get_run reads the client's
project and revise and retract refuse a record from another one; revise
carries the run's grants to the new version, which lands private; and
both refuse a version the server already holds. A claim needs a score,
and post_run needs confirm=True, nothing merely truthy.
capture() pins the contents of untracked files and records unknown, not
clean, when git fails; a folder with SHA256SUMS is checked against it;
listings escape names as sha256sum does; a remote keeps no userinfo. A
torn last WAL line is cut on open. The CLI's public read is --public.
The Python client now reaches what the Rust crate does and the server
serves. Research covers every /v1/research route: record() files
benchmarks, studies, runs and papers in one batch a kind; runs(),
get_run() and compare() read them; grant(), revise() and retract() move
consent and versions; the experiment plane and artifacts as before.
Arena covers /v1/benchmark: catalog, leaderboard, claims (written with
protocol and source), history, compare, presets, and runs, which queue a
paid execution and are refused without confirm=True.
wire holds the crate's Run, Benchmark, Measure, Study, Paper and
Comparison field for field; canonical writes RFC 8785 bytes equal to the
crate's (tests/vectors/rust.json, written by the crate); lineage and wal
are the crate's records, and a WAL the crate wrote verifies here.
capture() reads the harness commit (the diff's SHA-256 when dirty), the
weights' SHA-256, host, CPU, GPU, device, dtype and packages, and
apply() folds them into a run.
runs(org=) sends no credential: the server answers ?org= only to a
caller with no org. An org the token does not belong to is refused
before the request, since the server would act in the home org. The
command is hanzo-research; the native CLI owns `hanzo`.
The research client spoke only the experiments wire. The record
hanzo.ai/benchmarks draws from is /v1/research/{benchmarks,runs,studies,
papers,compare}, and producers posted it by hand (Kai's bench/code
post.py). record() posts each non-empty batch under its own key; runs(),
benchmarks(), studies() and papers() read with the API's own filters,
org= included for another org's public runs; compare() reads two runs
side by side. Tested against a stand-in server and read live.
Claude-Session: https://claude.ai/code/session_01M8RoubcPN71KgJuZ8LdBUL
The default surface was 34 tools, 61 KB, ~15,300 tokens of schema in every
prompt of every agent. The mode system already names what a session should
carry, but only the tools register_all_tools enumerates obeyed it: zsh,
curl, playwright and the helpers (ps, npx, uvx, jq, wget, cdp, open) stayed
on in every mode. A mode now decides every discovered tool, the `hanzo`
mode lists what a coding agent calls every few steps (18 tools, 32 KB,
~8,000 tokens), and the rest is a name away: HANZO_MCP_ENABLED_TOOLS=a,b
or =all, `tool enable`, or `mode`.
hanzo-tools-shell 0.6.6: run_shell kept every byte a foreground command
wrote until the command ended. `yes` for thirty seconds is gigabytes. It
keeps the first 16 MiB per stream and drains the rest.
The tests' session-wide patch of asyncio.create_subprocess_exec handed
every shell command a stream that never reached EOF; the first test to run
one read it in a loop until the machine was out of memory (22 GB, killed).
The patch is now opt-in and its process has exited.
bench/latency.py measures the cold start, the tools/list bill and the common
calls through a real MCP client over stdio. Interleaved rounds, wall and CPU
seconds, p50 over 6 cold starts:
cold wall cold CPU tools bytes ~tokens
installed 0.15.18 9.34 s 10.46 s 30 52,601 13,150
main 0.15.20 1.52 s 1.13 s 34 61,287 15,321
this 0.96 s 1.06 s 18 32,222 8,055
The installed environment's 9 s is its hanzoai 8.5.156, whose package init
imports the whole generated cloud client; hanzo-mcp >= 0.15.20 requires
hanzoai >= 8.5.624, which loads it lazily. Per-call latency is spawn-bound
(zsh 17.6 ms, git status 33 ms, fs.read 2.5 ms at p50) and moves with the
machine's load; no call path changed.
Every mode disables a registered tool it does not list, and llm was
listed nowhere but HEAVY_TOOLS, written when it pulled a 100 MB client.
It is an HTTP call to api.hanzo.ai now, so llm and kai_decide join the
essential set every personality carries, and the llm package maps to the
two tools it ships.
hanzo-mcp 0.15.19, on hanzo-tools 0.3.8 and hanzo-tools-llm 0.2.1.
Co-authored-by: Hanzo Dev <[email protected]>
The llm tool called upstream providers through the litellm fork, read
provider keys, and never reached the MCP surface: its register() was a
no-op and it skipped BaseTool's constructor. Kai had no Python tool.
hanzo-tools-llm now carries two HIP-0300 tools over HanzoCloud:
- llm (default action query): POST /v1/chat/completions with model
enso-auto unless named, "auto" for Enso's pick, max_cost and
max_latency_ms as X-Max-Cost and X-Max-Latency-Ms, returning id and the
model that served; models: GET /v1/models, one row per model with the
catalog's per-million prices, narrowed by family; feedback: POST
/v1/ai/feedback with request_id, signal and a 1-3 rating.
- kai_decide (default action decide): POST /v1/decisions with the
TypeScript tool's schema and shape checks.
The consensus, llm_manage and provider modules wrapped the upstream
client and were never registered, so they go, with hanzo-consensus and
the [full] extra. hanzo-tools-llm 0.2.1.
Co-authored-by: Hanzo Dev <[email protected]>
Enso reads its routing bounds from X-Max-Cost and X-Max-Latency-Ms on
/v1/chat/completions, and HanzoCloud could send no header but its own, so
no tool could bound a routed request. post() and the shared request path
now take extra headers that ride beside the bearer.
BaseTool runs `help` for a call that names no action. A tool with one
obvious act (llm's query, kai_decide's decide) now declares
DEFAULT_ACTION, which the call, the advertised schema's default and its
required list all read, so it answers the same no-action call the
TypeScript and Rust runtimes do. Every other tool keeps `help`.
hanzo-tools 0.3.8.
Co-authored-by: Hanzo Dev <[email protected]>
zeekayandHanzo Dev