mirror of
https://github.com/hanzoai/python-sdk.git
synced 2026-10-26 11:49:36 +00:00
zeekay
7374804405
The default surface was 34 tools, 61 KB, ~15,300 tokens of schema in every
prompt of every agent. The mode system already names what a session should
carry, but only the tools register_all_tools enumerates obeyed it: zsh,
curl, playwright and the helpers (ps, npx, uvx, jq, wget, cdp, open) stayed
on in every mode. A mode now decides every discovered tool, the `hanzo`
mode lists what a coding agent calls every few steps (18 tools, 32 KB,
~8,000 tokens), and the rest is a name away: HANZO_MCP_ENABLED_TOOLS=a,b
or =all, `tool enable`, or `mode`.
hanzo-tools-shell 0.6.6: run_shell kept every byte a foreground command
wrote until the command ended. `yes` for thirty seconds is gigabytes. It
keeps the first 16 MiB per stream and drains the rest.
The tests' session-wide patch of asyncio.create_subprocess_exec handed
every shell command a stream that never reached EOF; the first test to run
one read it in a loop until the machine was out of memory (22 GB, killed).
The patch is now opt-in and its process has exited.
bench/latency.py measures the cold start, the tools/list bill and the common
calls through a real MCP client over stdio. Interleaved rounds, wall and CPU
seconds, p50 over 6 cold starts:
cold wall cold CPU tools bytes ~tokens
installed 0.15.18 9.34 s 10.46 s 30 52,601 13,150
main 0.15.20 1.52 s 1.13 s 34 61,287 15,321
this 0.96 s 1.06 s 18 32,222 8,055
The installed environment's 9 s is its hanzoai 8.5.156, whose package init
imports the whole generated cloud client; hanzo-mcp >= 0.15.20 requires
hanzoai >= 8.5.624, which loads it lazily. Per-call latency is spawn-bound
(zsh 17.6 ms, git status 33 ms, fs.read 2.5 ms at p50) and moves with the
machine's load; no call path changed.
80 lines
1.6 KiB
JSON
80 lines
1.6 KiB
JSON
{
|
|
"A": {
|
|
"cold_ready_s_p50": 9.34,
|
|
"cold_cpu_s_p50": 10.46,
|
|
"tools": 30,
|
|
"bytes": 52601,
|
|
"tokens_est": 13150,
|
|
"n": 6,
|
|
"calls_p50_ms": {
|
|
"fs.read": 3.0,
|
|
"fs.search_text": 12.6,
|
|
"zsh.echo": 69.7,
|
|
"git.status": 99.5,
|
|
"playwright.snapshot": 33.1,
|
|
"playwright.click": 56.6,
|
|
"playwright.fill": 42.2
|
|
},
|
|
"calls_p95_ms": {
|
|
"fs.read": 4.0,
|
|
"fs.search_text": 33.8,
|
|
"zsh.echo": 118.0,
|
|
"git.status": 158.2,
|
|
"playwright.snapshot": 52.5,
|
|
"playwright.click": 85.8,
|
|
"playwright.fill": 87.9
|
|
}
|
|
},
|
|
"B": {
|
|
"cold_ready_s_p50": 1.52,
|
|
"cold_cpu_s_p50": 1.13,
|
|
"tools": 34,
|
|
"bytes": 61287,
|
|
"tokens_est": 15321,
|
|
"n": 6,
|
|
"calls_p50_ms": {
|
|
"fs.read": 3.2,
|
|
"fs.search_text": 21.6,
|
|
"zsh.echo": 36.8,
|
|
"git.status": 74.9,
|
|
"playwright.snapshot": 16.8,
|
|
"playwright.click": 43.0,
|
|
"playwright.fill": 15.2
|
|
},
|
|
"calls_p95_ms": {
|
|
"fs.read": 4.7,
|
|
"fs.search_text": 58.3,
|
|
"zsh.echo": 118.7,
|
|
"git.status": 223.3,
|
|
"playwright.snapshot": 20.7,
|
|
"playwright.click": 51.4,
|
|
"playwright.fill": 51.8
|
|
}
|
|
},
|
|
"C": {
|
|
"cold_ready_s_p50": 0.96,
|
|
"cold_cpu_s_p50": 1.06,
|
|
"tools": 18,
|
|
"bytes": 32222,
|
|
"tokens_est": 8055,
|
|
"n": 6,
|
|
"calls_p50_ms": {
|
|
"fs.read": 2.5,
|
|
"fs.search_text": 9.9,
|
|
"zsh.echo": 17.6,
|
|
"git.status": 33.0,
|
|
"playwright.snapshot": 13.8,
|
|
"playwright.click": 39.4,
|
|
"playwright.fill": 10.6
|
|
},
|
|
"calls_p95_ms": {
|
|
"fs.read": 4.0,
|
|
"fs.search_text": 12.5,
|
|
"zsh.echo": 29.1,
|
|
"git.status": 41.3,
|
|
"playwright.snapshot": 18.7,
|
|
"playwright.click": 51.6,
|
|
"playwright.fill": 16.2
|
|
}
|
|
}
|
|
} |