mirror of
https://github.com/hanzoai/python-sdk.git
synced 2026-10-23 20:23:00 +00:00
chore(hips): remove duplicated HIP docs — canonical is hanzoai/HIPs (#37)
No duplication: the SDK must not carry copies of governance proposals. HIP-0300 duplicated canonical hip-0300; the rest diverge. Canonical home is hanzoai/HIPs (git history preserves these). Forwards-only, no backcompat.
This commit is contained in:
GitHub
@@ -1,466 +0,0 @@
|
||||
# HIP-0300: Unified MCP Tools Architecture
|
||||
|
||||
**Status:** Draft
|
||||
**Author:** Hanzo AI
|
||||
**Created:** January 2025
|
||||
**Updated:** January 2025
|
||||
|
||||
## Abstract
|
||||
|
||||
HIP-0300 defines a unified architecture for Model Context Protocol (MCP) tools, consolidating 52+ individual tools into ~16 orthogonal, composable operators. The design follows Unix philosophy: each tool does one thing well, with clear composition laws.
|
||||
|
||||
## Motivation
|
||||
|
||||
The original hanzo-mcp implementation grew organically to 52+ tools with significant overlap:
|
||||
- Multiple search tools (grep, search, find, ast_search)
|
||||
- Multiple file tools (read, write, edit, cat, head, tail)
|
||||
- Multiple shell tools (bash, zsh, sh, shell, cmd)
|
||||
|
||||
This creates confusion for LLM agents and increases cognitive load. HIP-0300 restructures tools around orthogonal axes with a formal effect lattice.
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
### Core Principles
|
||||
|
||||
1. **Orthogonal Axes**: Tools organized along independent dimensions
|
||||
2. **Composability**: Small operators that combine predictably
|
||||
3. **Effect Tracking**: Every operation has a declared effect level
|
||||
4. **Transform vs Apply**: Pure transforms produce Patches; Apply commits them
|
||||
5. **Minimal Surface Area**: 16 core operators cover all use cases
|
||||
|
||||
### The Three Lattices
|
||||
|
||||
Three lattices constrain all operations:
|
||||
|
||||
```
|
||||
EFFECT LATTICE (purity)
|
||||
├─ PURE (no side effects, referentially transparent)
|
||||
├─ DETERMINISTIC_EFFECT (effects, but reproducible)
|
||||
└─ NONDETERMINISTIC_EFFECT (network, time, randomness)
|
||||
|
||||
REPRESENTATION LATTICE (data granularity)
|
||||
├─ Bytes (raw content)
|
||||
├─ Lines (text with line numbers)
|
||||
├─ AST (parsed structure)
|
||||
├─ Patch (diff/change set)
|
||||
└─ Symbols (semantic references)
|
||||
|
||||
SCOPE LATTICE (operational boundary)
|
||||
├─ Span (byte/char range)
|
||||
├─ Region (line range)
|
||||
├─ File (single file)
|
||||
├─ Tree (directory subtree)
|
||||
└─ Repo (entire repository)
|
||||
```
|
||||
|
||||
## Operator Surface
|
||||
|
||||
### Core Operators (HIP-0300)
|
||||
|
||||
| Tool | Axis | Actions | Effect |
|
||||
|------|------|---------|--------|
|
||||
| `fs` | Bytes + Paths | read, write, edit, search, list, stat, patch, tree, glob | DETERMINISTIC |
|
||||
| `id` | Identity | hash, uri, ref, verify | PURE |
|
||||
| `code` | Symbols + Structure | parse, serialize, symbols, definition, references, transform, summarize | PURE/DETERMINISTIC |
|
||||
| `proc` | Execution | run, bg, signal, status, wait | NONDETERMINISTIC |
|
||||
| `vcs` | History + Diffs | status, diff, commit, log, branch, stash | DETERMINISTIC |
|
||||
| `test` | Validation | check, build, test, detect | NONDETERMINISTIC |
|
||||
| `net` | Network | search, fetch, download, crawl, head | NONDETERMINISTIC |
|
||||
| `plan` | Orchestration | intent, route, compose | PURE |
|
||||
|
||||
### Control Surfaces
|
||||
|
||||
| Tool | Surface | Actions | Effect |
|
||||
|------|---------|---------|--------|
|
||||
| `browser` | Web DOM | navigate, click, type, screenshot, evaluate, etc. | NONDETERMINISTIC |
|
||||
| `computer` | OS Desktop | screenshot, click, type, key, scroll, etc. | NONDETERMINISTIC |
|
||||
|
||||
### Extended Operators
|
||||
|
||||
| Tool | Domain | Actions |
|
||||
|------|--------|---------|
|
||||
| `lsp` | Semantic Stream | diagnostics, code_actions, hover, completion |
|
||||
| `memory` | Knowledge Persistence | read, write, search, create, recall |
|
||||
| `todo` | Task Tracking | list, add, update, remove |
|
||||
| `reasoning` | Cognition | think, critic |
|
||||
| `agent` | Multi-Agent | run, list, status, config |
|
||||
| `llm` | LLM Interface | chat, consensus |
|
||||
|
||||
## Verb Kernel
|
||||
|
||||
The 27-verb kernel with typed signatures:
|
||||
|
||||
### File Operations (fs)
|
||||
```
|
||||
read : Path → Bytes | Text | Lines
|
||||
write : (Path, Content) → {ok, hash}
|
||||
edit : (Path, Patch) → {ok, hash}
|
||||
search : (Pattern, Scope) → [Match]
|
||||
list : Path → [Entry]
|
||||
stat : Path → {size, mtime, mode, hash}
|
||||
patch : (Path, Patch, base_hash?) → {ok, new_hash}
|
||||
tree : (Path, depth?) → TreeNode
|
||||
glob : (Pattern, Path?) → [Path]
|
||||
```
|
||||
|
||||
### Identity Operations (id)
|
||||
```
|
||||
hash : Content → {digest, algo, size}
|
||||
uri : Path → {uri, path, exists}
|
||||
ref : (Path, line?, col?) → {uri, range?, hash?}
|
||||
verify : (Content, Digest) → {match, actual, expected}
|
||||
```
|
||||
|
||||
### Code Operations (code)
|
||||
```
|
||||
parse : (Path | Text, lang?) → AST
|
||||
serialize : AST → Text
|
||||
symbols : (Path | AST, kind?) → [Symbol]
|
||||
definition : (Path, Position) → [Location]
|
||||
references : (Path, Position) → [Location]
|
||||
transform : (Path | Text, kind, params) → Patch # PURE!
|
||||
summarize : (Diff | Log | Report) → {summary, risks, next_actions}
|
||||
```
|
||||
|
||||
### Process Operations (proc)
|
||||
```
|
||||
run : Command → {stdout, stderr, code}
|
||||
bg : Command → ProcessID
|
||||
signal : (ProcessID, Signal) → {ok}
|
||||
status : ProcessID? → [ProcessStatus]
|
||||
wait : (ProcessID, timeout?) → {stdout, stderr, code}
|
||||
```
|
||||
|
||||
### Version Control (vcs)
|
||||
```
|
||||
status : () → {staged, unstaged, untracked}
|
||||
diff : (ref1?, ref2?, paths?) → Diff
|
||||
commit : (message, files?) → {sha, message}
|
||||
log : (n?, since?, until?, path?) → [Commit]
|
||||
branch : (name?, action?) → {current, branches}
|
||||
stash : (action, message?) → {ok}
|
||||
```
|
||||
|
||||
### Validation (test)
|
||||
```
|
||||
check : (Path?, tool?) → {diagnostics, pass} # Lint/typecheck
|
||||
build : (Path?, tool?) → {success, artifacts} # Compilation
|
||||
test : (selector?, tool?) → {passed, failed, summary} # Runtime
|
||||
detect : (Path?) → {test_runner, build_tool, check_tool}
|
||||
```
|
||||
|
||||
### Network (net)
|
||||
```
|
||||
search : (Query, engine?) → [{url, title, snippet}]
|
||||
fetch : (URL, extract_text?) → {text, mime, status, hash}
|
||||
download : (URL, dest?, assets?) → {path, size, mime}
|
||||
crawl : (URL, dest, depth?, limit?) → {pages, count}
|
||||
head : URL → {status, headers, size?, mime?}
|
||||
```
|
||||
|
||||
### Orchestration (plan)
|
||||
```
|
||||
intent : NL → IntentIR
|
||||
route : (IntentIR, Policy?) → Plan
|
||||
compose : Plan → ExecGraph
|
||||
```
|
||||
|
||||
## Effect Annotations
|
||||
|
||||
Every action declares its effect level:
|
||||
|
||||
```python
|
||||
class Effect(Enum):
|
||||
PURE = "pure" # No side effects
|
||||
DETERMINISTIC = "deterministic" # Effects, reproducible
|
||||
NONDETERMINISTIC = "nondeterministic" # Network/time/random
|
||||
|
||||
# Examples:
|
||||
# fs.read → DETERMINISTIC_EFFECT (reads disk)
|
||||
# code.parse → PURE (in-memory transform)
|
||||
# code.transform → PURE (produces Patch value)
|
||||
# fs.patch → DETERMINISTIC_EFFECT (writes disk)
|
||||
# net.fetch → NONDETERMINISTIC_EFFECT (network I/O)
|
||||
```
|
||||
|
||||
## Transform vs Apply
|
||||
|
||||
Critical separation of concerns:
|
||||
|
||||
```
|
||||
Transform (PURE) Apply (EFFECT)
|
||||
───────────────── ────────────────
|
||||
code.transform fs.patch
|
||||
→ Patch value → committed change
|
||||
→ base_hash → requires base_hash precondition
|
||||
→ preview-safe → point of no return
|
||||
|
||||
vcs.diff vcs.commit
|
||||
→ Diff value → committed history
|
||||
```
|
||||
|
||||
### Content-Addressable Edits
|
||||
|
||||
All mutations use content-addressable storage for safety:
|
||||
|
||||
```python
|
||||
# Generate a transform
|
||||
result = await code.call(ctx, action="transform",
|
||||
path="/src/main.py",
|
||||
kind="rename",
|
||||
old_name="foo",
|
||||
new_name="bar")
|
||||
# Returns: {patch: [...], base_hash: "sha256:abc...", new_hash: "sha256:def..."}
|
||||
|
||||
# Apply with precondition
|
||||
result = await fs.call(ctx, action="patch",
|
||||
path="/src/main.py",
|
||||
patch=patch,
|
||||
base_hash="sha256:abc...") # Must match current
|
||||
# Fails if file changed since transform was computed
|
||||
```
|
||||
|
||||
## Intent Routing
|
||||
|
||||
Natural language maps to canonical operator chains:
|
||||
|
||||
### Intent IR Structure
|
||||
```python
|
||||
class IntentIR:
|
||||
category: str # navigate, explain, modify, validate, debug, create
|
||||
action: str # find, understand, rename, test, trace, add
|
||||
target: str # extracted entity/path/concept
|
||||
confidence: float
|
||||
```
|
||||
|
||||
### Canonical Chains
|
||||
|
||||
| Intent Pattern | Canonical Chain |
|
||||
|---------------|-----------------|
|
||||
| "find X" | `fs.search(X)` |
|
||||
| "what is X" | `fs.search(X) → code.summarize` |
|
||||
| "rename X to Y" | `code.references(X) → code.transform(rename) → code.summarize → [policy] → fs.patch → test.run` |
|
||||
| "fix bug in X" | `fs.read(X) → code.parse → code.transform(fix) → [policy] → fs.patch → test.run` |
|
||||
| "add tests for X" | `code.symbols(X) → code.transform(add_tests) → fs.write → test.run` |
|
||||
| "why does X fail" | `test.run(X) → vcs.log → code.summarize` |
|
||||
| "refactor X" | `code.references(X) → code.transform → code.summarize → [policy] → fs.patch → test.run` |
|
||||
|
||||
### Policy Gates
|
||||
|
||||
High-risk operations require explicit approval:
|
||||
- `fs.patch` (file modifications)
|
||||
- `fs.write` (new file creation)
|
||||
- `proc.run` with shell=True
|
||||
- `vcs.commit` / `vcs.push`
|
||||
|
||||
## Validation Loops
|
||||
|
||||
Three distinct validation operations (Vim-inspired):
|
||||
|
||||
### CHECK (`:make` equivalent)
|
||||
- Fast, incremental feedback
|
||||
- Lint, typecheck, format check
|
||||
- Tools: ruff, mypy, eslint, tsc, clippy
|
||||
- Returns: `{diagnostics: [], pass: bool}`
|
||||
|
||||
### BUILD (`:!make` equivalent)
|
||||
- Whole-project compilation
|
||||
- Dependency resolution
|
||||
- Tools: pip, npm, cargo, go build, make
|
||||
- Returns: `{success: bool, artifacts: [], errors: []}`
|
||||
|
||||
### TEST (`:!make test` equivalent)
|
||||
- Runtime behavior validation
|
||||
- Isolated execution environment
|
||||
- Tools: pytest, jest, go test, cargo test
|
||||
- Returns: `{passed: int, failed: int, summary: str}`
|
||||
|
||||
### Auto-Detection
|
||||
|
||||
```python
|
||||
# test.detect() returns:
|
||||
{
|
||||
"test_runner": {"name": "pytest", "cmd": ["pytest", "-v"]},
|
||||
"build_tool": {"name": "pip", "cmd": ["pip", "install", "-e", "."]},
|
||||
"check_tool": {"name": "ruff", "cmd": ["ruff", "check", "."]}
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
### BaseTool Pattern
|
||||
|
||||
```python
|
||||
from hanzo_tools.core import BaseTool, ActionHandler, ToolError
|
||||
|
||||
class FsTool(BaseTool):
|
||||
name: ClassVar[str] = "fs"
|
||||
|
||||
def __init__(self, cwd: str | None = None):
|
||||
super().__init__()
|
||||
self.cwd = cwd or os.getcwd()
|
||||
self._register_actions()
|
||||
|
||||
def _register_actions(self):
|
||||
@self.action("read", "Read file content")
|
||||
async def read(ctx, path: str, encoding: str = "utf-8"):
|
||||
"""Path → Content"""
|
||||
# Implementation
|
||||
return {"text": content, "hash": content_hash(content)}
|
||||
|
||||
@self.action("search", "Search for pattern")
|
||||
async def search(ctx, pattern: str, path: str = ".", **opts):
|
||||
"""(Pattern, Scope) → [Match]"""
|
||||
# Implementation
|
||||
return {"matches": matches, "count": len(matches)}
|
||||
```
|
||||
|
||||
### Unified Response Envelope
|
||||
|
||||
All tools return a consistent envelope:
|
||||
|
||||
```python
|
||||
{
|
||||
"ok": True, # Success indicator
|
||||
"data": {...}, # Action-specific result
|
||||
"error": None, # Or {code, message, details}
|
||||
"meta": { # Optional metadata
|
||||
"duration_ms": 42,
|
||||
"effect": "deterministic"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Entry Points
|
||||
|
||||
Each package exports via entry points:
|
||||
|
||||
```toml
|
||||
# pyproject.toml
|
||||
[project.entry-points."hanzo.tools"]
|
||||
filesystem = "hanzo_tools.filesystem:TOOLS"
|
||||
code = "hanzo_tools.code:TOOLS"
|
||||
plan = "hanzo_tools.plan:TOOLS"
|
||||
test = "hanzo_tools.test:TOOLS"
|
||||
net = "hanzo_tools.net:TOOLS"
|
||||
```
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
pkg/
|
||||
├── hanzo-tools-core/ # BaseTool, IdTool, ToolRegistry
|
||||
│ └── hanzo_tools/core/
|
||||
├── hanzo-tools-filesystem/ # FsTool (read, write, edit, search, etc.)
|
||||
│ └── hanzo_tools/filesystem/
|
||||
├── hanzo-tools-code/ # CodeTool (parse, transform, summarize)
|
||||
│ └── hanzo_tools/code/
|
||||
├── hanzo-tools-shell/ # ProcTool (run, bg, signal, wait)
|
||||
│ └── hanzo_tools/shell/
|
||||
├── hanzo-tools-vcs/ # VcsTool (status, diff, commit, log)
|
||||
│ └── hanzo_tools/vcs/
|
||||
├── hanzo-tools-test/ # TestTool (check, build, test)
|
||||
│ └── hanzo_tools/test/
|
||||
├── hanzo-tools-net/ # NetTool (search, fetch, download, crawl)
|
||||
│ └── hanzo_tools/net/
|
||||
├── hanzo-tools-plan/ # PlanTool (intent, route, compose)
|
||||
│ └── hanzo_tools/plan/
|
||||
├── hanzo-tools-browser/ # BrowserTool (Playwright control)
|
||||
│ └── hanzo_tools/browser/
|
||||
├── hanzo-tools-computer/ # ComputerTool (OS desktop control)
|
||||
│ └── hanzo_tools/computer/
|
||||
└── hanzo-mcp/ # MCP server (discovers tools via entry points)
|
||||
└── hanzo_mcp/
|
||||
```
|
||||
|
||||
## Migration Path
|
||||
|
||||
### Phase 1: Implement Core Operators
|
||||
- [x] `fs` - Filesystem operations (existing hanzo-tools-filesystem)
|
||||
- [x] `id` - Identity operations (hash, uri, ref, verify)
|
||||
- [x] `code` - Code operations (parse, transform, summarize)
|
||||
- [x] `proc` - Process execution (existing hanzo-tools-shell)
|
||||
- [x] `vcs` - Version control (existing hanzo-tools-vcs)
|
||||
- [x] `test` - Validation loops (check, build, test)
|
||||
- [x] `net` - Network operations (search, fetch, download, crawl)
|
||||
- [x] `plan` - Orchestration (intent, route, compose)
|
||||
|
||||
### Phase 2: Control Surfaces
|
||||
- [x] `browser` - Web DOM control (existing hanzo-tools-browser)
|
||||
- [ ] `computer` - OS desktop control (needs update)
|
||||
|
||||
### Phase 3: Extended Operators
|
||||
- [ ] `lsp` - Language server integration
|
||||
- [ ] `dbg` - Debugger control (breakpoint, step, eval)
|
||||
- [ ] `repl` - Interactive evaluation (send, recv, reset)
|
||||
|
||||
### Phase 4: Deprecation
|
||||
- Deprecate individual tools (read_file, search_code, etc.)
|
||||
- Map old names to new unified tools
|
||||
- Remove after 6 months
|
||||
|
||||
## Composition Examples
|
||||
|
||||
### Safe Refactoring
|
||||
```python
|
||||
# 1. Find all references (PURE)
|
||||
refs = await code.call(ctx, action="references", path="src/auth.py", position={"line": 42, "col": 10})
|
||||
|
||||
# 2. Generate patch (PURE)
|
||||
patch = await code.call(ctx, action="transform", path="src/auth.py", kind="rename", old_name="authenticate", new_name="verify_user")
|
||||
|
||||
# 3. Summarize changes (PURE)
|
||||
summary = await code.call(ctx, action="summarize", diff=patch["patch"])
|
||||
|
||||
# 4. [POLICY GATE] - User approves
|
||||
|
||||
# 5. Apply patch (EFFECT)
|
||||
await fs.call(ctx, action="patch", path="src/auth.py", patch=patch["patch"], base_hash=patch["base_hash"])
|
||||
|
||||
# 6. Verify (EFFECT)
|
||||
await test.call(ctx, action="run")
|
||||
```
|
||||
|
||||
### Site Mirroring
|
||||
```python
|
||||
# Crawl site recursively
|
||||
result = await net.call(ctx, action="crawl", url="https://docs.example.com", dest="./mirror", depth=3, limit=100)
|
||||
|
||||
# Returns: {pages: ["/mirror/index.html", ...], count: 47}
|
||||
```
|
||||
|
||||
### Intelligent Search
|
||||
```python
|
||||
# Parse intent
|
||||
intent = await plan.call(ctx, action="intent", nl="find where user authentication happens")
|
||||
|
||||
# Route to canonical chain
|
||||
plan = await plan.call(ctx, action="route", intent_ir=intent)
|
||||
|
||||
# Execute chain
|
||||
for step in plan["nodes"]:
|
||||
if step.get("policy_gate"):
|
||||
# Request approval
|
||||
pass
|
||||
result = await dispatch(step["tool"], step["action"], step["params"])
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Path Traversal**: All path operations validate against allowed directories
|
||||
2. **Command Injection**: `proc.run` sanitizes inputs, requires explicit shell=True
|
||||
3. **Network Access**: `net.*` operations respect proxy settings and rate limits
|
||||
4. **File Modifications**: Require `base_hash` precondition to prevent race conditions
|
||||
5. **Policy Gates**: High-risk operations require explicit approval
|
||||
|
||||
## References
|
||||
|
||||
- [Unix Philosophy](https://en.wikipedia.org/wiki/Unix_philosophy)
|
||||
- [Model Context Protocol](https://github.com/anthropics/mcp)
|
||||
- [Effect Systems](https://en.wikipedia.org/wiki/Effect_system)
|
||||
- [Content-Addressable Storage](https://en.wikipedia.org/wiki/Content-addressable_storage)
|
||||
|
||||
## Changelog
|
||||
|
||||
- **2025-01-22**: Initial draft with operator lattice specification
|
||||
- **2025-01-21**: Created CodeTool, PlanTool, TestTool, NetTool, IdTool packages
|
||||
@@ -1,420 +0,0 @@
|
||||
# HIP-0301: High-Performance Agent Communication Protocol
|
||||
|
||||
**Status:** Draft
|
||||
**Author:** Hanzo AI
|
||||
**Created:** January 2025
|
||||
**Updated:** January 2025
|
||||
|
||||
## Abstract
|
||||
|
||||
HIP-0301 defines a high-performance agent-to-agent communication protocol using Cap'n Proto as the wire format, with MCP JSON-RPC as a fallback. This enables sub-millisecond tool invocation between agents while maintaining backward compatibility with existing MCP infrastructure.
|
||||
|
||||
## Motivation
|
||||
|
||||
Current MCP communication has inherent limitations:
|
||||
|
||||
1. **JSON Serialization Overhead**: Every tool call requires JSON encode/decode (~100-500μs per call)
|
||||
2. **No Zero-Copy**: JSON requires full parsing before use; data copied multiple times
|
||||
3. **Schema Flexibility = Runtime Errors**: JSON's flexibility means errors surface at runtime
|
||||
4. **Consensus Latency**: Multi-agent consensus requires O(n²) messages; JSON overhead compounds
|
||||
|
||||
For AI agent swarms running metastable consensus, the overhead is unacceptable:
|
||||
- 10 agents × 10 rounds × 100μs = 10ms just for serialization
|
||||
- Real-world latencies 10-100x higher due to memory allocation
|
||||
|
||||
Cap'n Proto solves this with:
|
||||
- **Zero-copy reads**: Wire format = memory format (no parsing)
|
||||
- **Incremental reads**: Access fields without deserializing entire message
|
||||
- **Type safety**: Schema-defined contracts, compile-time verification
|
||||
- **RPC built-in**: Native async RPC with promise pipelining
|
||||
|
||||
## Design
|
||||
|
||||
### Transport Hierarchy
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Agent Bus Interface │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────┐ │
|
||||
│ │ Cap'n Proto │ │ MCP JSON-RPC │ │ Unix IPC │ │
|
||||
│ │ (preferred) │ │ (fallback) │ │ (local) │ │
|
||||
│ └────────┬────────┘ └────────┬────────┘ └──────┬──────┘ │
|
||||
│ │ │ │ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐│
|
||||
│ │ Unified Tool Invocation Layer ││
|
||||
│ │ tool.call(action, params) -> Response ││
|
||||
│ └─────────────────────────────────────────────────────────┘│
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### AgentBus Trait
|
||||
|
||||
```rust
|
||||
/// High-performance agent communication bus
|
||||
#[async_trait]
|
||||
pub trait AgentBus: Send + Sync {
|
||||
/// Get bus capabilities
|
||||
fn capabilities(&self) -> BusCapabilities;
|
||||
|
||||
/// Register local agent
|
||||
async fn register(&self, agent_id: &str, handler: Arc<dyn AgentHandler>) -> Result<()>;
|
||||
|
||||
/// Unregister agent
|
||||
async fn unregister(&self, agent_id: &str) -> Result<()>;
|
||||
|
||||
/// Call tool on remote agent (point-to-point)
|
||||
async fn call(
|
||||
&self,
|
||||
from: &str,
|
||||
to: &str,
|
||||
tool: &str,
|
||||
action: &str,
|
||||
params: &[u8], // Pre-serialized (capnp or json)
|
||||
) -> Result<Vec<u8>>;
|
||||
|
||||
/// Broadcast to all agents
|
||||
async fn broadcast(
|
||||
&self,
|
||||
from: &str,
|
||||
tool: &str,
|
||||
action: &str,
|
||||
params: &[u8],
|
||||
) -> Result<Vec<(String, Vec<u8>)>>; // (agent_id, response)
|
||||
|
||||
/// Run consensus protocol
|
||||
async fn consensus(
|
||||
&self,
|
||||
prompt: &str,
|
||||
participants: &[&str],
|
||||
config: ConsensusConfig,
|
||||
) -> Result<ConsensusState>;
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct BusCapabilities {
|
||||
pub supports_capnp: bool,
|
||||
pub supports_mcp: bool,
|
||||
pub supports_unix_ipc: bool,
|
||||
pub max_message_size: usize,
|
||||
pub supports_streaming: bool,
|
||||
}
|
||||
|
||||
pub struct ConsensusConfig {
|
||||
pub rounds: u32,
|
||||
pub k: u32, // Peer sample size
|
||||
pub alpha: f64, // Confidence threshold
|
||||
pub beta_1: f64, // Phase I threshold
|
||||
pub beta_2: f64, // Phase II (finality) threshold
|
||||
pub timeout_ms: u64,
|
||||
}
|
||||
```
|
||||
|
||||
### Cap'n Proto Schema
|
||||
|
||||
```capnp
|
||||
@0xb5e7e8d3c9f2a1b4; # Unique file ID
|
||||
|
||||
# Core message types for agent communication
|
||||
|
||||
struct AgentId {
|
||||
id @0 :Text;
|
||||
nodeId @1 :Text; # For distributed deployment
|
||||
capabilities @2 :List(Text);
|
||||
}
|
||||
|
||||
struct ToolCall {
|
||||
id @0 :UInt64; # Request ID for correlation
|
||||
tool @1 :Text; # Tool name (fs, code, plan, etc.)
|
||||
action @2 :Text; # Action within tool
|
||||
params @3 :Data; # Serialized params (nested capnp or json)
|
||||
effect @4 :Effect; # Declared effect level
|
||||
traceId @5 :Text; # Distributed tracing
|
||||
}
|
||||
|
||||
enum Effect {
|
||||
pure @0;
|
||||
deterministic @1;
|
||||
nondeterministic @2;
|
||||
}
|
||||
|
||||
struct ToolResponse {
|
||||
id @0 :UInt64; # Matching request ID
|
||||
ok @1 :Bool;
|
||||
data @2 :Data; # Serialized result
|
||||
error @3 :ToolError;
|
||||
meta @4 :ResponseMeta;
|
||||
}
|
||||
|
||||
struct ToolError {
|
||||
code @0 :ErrorCode;
|
||||
message @1 :Text;
|
||||
details @2 :Data; # JSON or nested capnp
|
||||
}
|
||||
|
||||
enum ErrorCode {
|
||||
unknownAction @0;
|
||||
invalidParams @1;
|
||||
notFound @2;
|
||||
conflict @3;
|
||||
permissionDenied @4;
|
||||
timeout @5;
|
||||
internalError @6;
|
||||
}
|
||||
|
||||
struct ResponseMeta {
|
||||
tool @0 :Text;
|
||||
version @1 :Text;
|
||||
action @2 :Text;
|
||||
effect @3 :Effect;
|
||||
durationNs @4 :UInt64;
|
||||
traceId @5 :Text;
|
||||
}
|
||||
|
||||
# Consensus protocol messages
|
||||
|
||||
struct ConsensusVote {
|
||||
round @0 :UInt32;
|
||||
fromAgent @1 :Text;
|
||||
vote @2 :Text; # Agent's response/vote
|
||||
confidence @3 :Float64;
|
||||
luminance @4 :Float64; # Response time weight
|
||||
signature @5 :Data; # Optional cryptographic signature
|
||||
}
|
||||
|
||||
struct ConsensusState {
|
||||
round @0 :UInt32;
|
||||
phase @1 :ConsensusPhase;
|
||||
votes @2 :List(ConsensusVote);
|
||||
winner @3 :Text;
|
||||
synthesis @4 :Text;
|
||||
finalized @5 :Bool;
|
||||
confidence @6 :Float64;
|
||||
}
|
||||
|
||||
enum ConsensusPhase {
|
||||
sampling @0; # Phase I: k-peer sampling
|
||||
finality @1; # Phase II: threshold aggregation
|
||||
complete @2;
|
||||
}
|
||||
|
||||
# Agent mesh management
|
||||
|
||||
struct AgentRegistration {
|
||||
agent @0 :AgentId;
|
||||
endpoint @1 :Text; # capnp://host:port or mcp://host:port
|
||||
tools @2 :List(Text); # Available tools
|
||||
publicKey @3 :Data; # For authenticated consensus
|
||||
}
|
||||
|
||||
struct MeshTopology {
|
||||
agents @0 :List(AgentRegistration);
|
||||
gossipInterval @1 :UInt32; # ms between topology updates
|
||||
}
|
||||
|
||||
# RPC Interface
|
||||
|
||||
interface AgentService {
|
||||
# Core tool invocation
|
||||
call @0 (request :ToolCall) -> (response :ToolResponse);
|
||||
|
||||
# Streaming for large responses
|
||||
stream @1 (request :ToolCall) -> (stream :StreamHandle);
|
||||
|
||||
# Consensus participation
|
||||
vote @2 (round :UInt32, prompt :Text, context :Data) -> (vote :ConsensusVote);
|
||||
|
||||
# Mesh management
|
||||
register @3 (registration :AgentRegistration) -> (accepted :Bool);
|
||||
topology @4 () -> (mesh :MeshTopology);
|
||||
|
||||
# Health
|
||||
ping @5 () -> (pong :Bool, latencyNs :UInt64);
|
||||
}
|
||||
|
||||
interface StreamHandle {
|
||||
next @0 () -> (chunk :Data, done :Bool);
|
||||
cancel @1 () -> ();
|
||||
}
|
||||
```
|
||||
|
||||
### Protocol Negotiation
|
||||
|
||||
When agents connect, they negotiate the best available transport:
|
||||
|
||||
```
|
||||
Agent A Agent B
|
||||
│ │
|
||||
│──── HELLO (capabilities) ─────▶│
|
||||
│ │
|
||||
│◀─── HELLO_ACK (selected) ──────│
|
||||
│ │
|
||||
│ Transport: capnp (preferred) │
|
||||
│ or: mcp-json (fallback) │
|
||||
│ or: unix-ipc (local) │
|
||||
│ │
|
||||
│══════ Established ═════════════│
|
||||
```
|
||||
|
||||
### Integration with HIP-0300 Tools
|
||||
|
||||
All HIP-0300 tools gain an `agent` action for consensus-based execution:
|
||||
|
||||
```rust
|
||||
// fs tool with consensus
|
||||
fs_tool.call("agent", json!({
|
||||
"action": "read",
|
||||
"path": "/etc/hosts",
|
||||
"consensus": {
|
||||
"participants": ["claude", "gpt4", "gemini"],
|
||||
"require_agreement": true
|
||||
}
|
||||
}));
|
||||
|
||||
// plan tool routes through consensus automatically
|
||||
plan_tool.call("intent", json!({
|
||||
"nl": "refactor the authentication module",
|
||||
"consensus": true // Uses metastable consensus for routing
|
||||
}));
|
||||
```
|
||||
|
||||
### Metastable Consensus Integration
|
||||
|
||||
The plan tool uses metastable consensus for LLM-based intent parsing:
|
||||
|
||||
```rust
|
||||
impl PlanTool {
|
||||
async fn intent_with_consensus(
|
||||
&self,
|
||||
nl: &str,
|
||||
bus: &dyn AgentBus,
|
||||
) -> Result<IntentIR> {
|
||||
// Get available LLM agents
|
||||
let agents = bus.list_agents_with_capability("llm").await?;
|
||||
|
||||
// Run consensus
|
||||
let state = bus.consensus(
|
||||
&format!("Parse this intent: {}", nl),
|
||||
&agents,
|
||||
ConsensusConfig {
|
||||
rounds: 3,
|
||||
k: 3,
|
||||
alpha: 0.6,
|
||||
beta_1: 0.5,
|
||||
beta_2: 0.8,
|
||||
timeout_ms: 30000,
|
||||
},
|
||||
).await?;
|
||||
|
||||
// Parse winner's response as IntentIR
|
||||
serde_json::from_str(&state.synthesis)
|
||||
.map_err(|e| ToolError::internal(format!("Invalid IR: {}", e)))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
| Metric | JSON-RPC (MCP) | Cap'n Proto | Improvement |
|
||||
|--------|----------------|-------------|-------------|
|
||||
| Serialization | 100-500μs | 0μs (zero-copy) | ∞ |
|
||||
| Deserialization | 50-200μs | 0μs (zero-copy) | ∞ |
|
||||
| Message overhead | ~40% | ~5% | 8x smaller |
|
||||
| Memory allocations | O(n) | O(1) | Constant |
|
||||
| Type safety | Runtime | Compile-time | Earlier errors |
|
||||
| Consensus (10 agents, 10 rounds) | ~50-100ms | ~1-5ms | 10-100x |
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
1. **MCP Fallback**: If Cap'n Proto unavailable, seamlessly falls back to JSON-RPC
|
||||
2. **Schema Evolution**: Cap'n Proto supports adding fields without breaking
|
||||
3. **Hybrid Mesh**: Agents using different transports can still communicate via bridge
|
||||
4. **Tool API Unchanged**: HIP-0300 tool signatures remain identical
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Authentication**: Optional Ed25519 signatures on consensus votes
|
||||
2. **Encryption**: TLS 1.3 for capnp-rpc, with post-quantum option (ML-KEM)
|
||||
3. **Rate Limiting**: Per-agent rate limits on consensus participation
|
||||
4. **Message Size**: Configurable max message size (default 16MB)
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Schema & Core (Week 1-2)
|
||||
- [ ] Cap'n Proto schema definitions
|
||||
- [ ] Rust capnp code generation
|
||||
- [ ] Python pycapnp bindings
|
||||
- [ ] AgentBus trait implementation
|
||||
|
||||
### Phase 2: Transport (Week 3-4)
|
||||
- [ ] Cap'n Proto RPC transport (Rust)
|
||||
- [ ] Cap'n Proto RPC transport (Python)
|
||||
- [ ] MCP JSON-RPC fallback
|
||||
- [ ] Protocol negotiation
|
||||
|
||||
### Phase 3: Consensus Integration (Week 5-6)
|
||||
- [ ] Integrate hanzo-metastable-consensus
|
||||
- [ ] Update plan_tool for consensus routing
|
||||
- [ ] Agent tool with consensus support
|
||||
- [ ] Benchmark and optimize
|
||||
|
||||
### Phase 4: Production (Week 7-8)
|
||||
- [ ] Documentation
|
||||
- [ ] Migration guide
|
||||
- [ ] Performance benchmarks
|
||||
- [ ] Security audit
|
||||
|
||||
## References
|
||||
|
||||
- [Cap'n Proto](https://capnproto.org/) - Zero-copy serialization
|
||||
- [HIP-0300](./HIP-0300.md) - Unified MCP Tools Architecture
|
||||
- [Metastable Consensus](https://github.com/luxfi/consensus) - Hanzo consensus protocol
|
||||
- [MCP Specification](https://modelcontextprotocol.io/) - Model Context Protocol
|
||||
|
||||
## Appendix A: Message Size Comparison
|
||||
|
||||
```
|
||||
Tool Call: fs.read("/etc/hosts")
|
||||
|
||||
JSON-RPC (MCP):
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": 1,
|
||||
"method": "tools/call",
|
||||
"params": {
|
||||
"name": "fs",
|
||||
"arguments": {
|
||||
"action": "read",
|
||||
"path": "/etc/hosts"
|
||||
}
|
||||
}
|
||||
}
|
||||
Total: 156 bytes
|
||||
|
||||
Cap'n Proto:
|
||||
[8-byte header][tool: 2 bytes][action: 4 bytes][path: 11 bytes][padding: 1 byte]
|
||||
Total: 26 bytes (6x smaller)
|
||||
```
|
||||
|
||||
## Appendix B: Consensus Latency Analysis
|
||||
|
||||
For N agents over R rounds with message size M:
|
||||
|
||||
```
|
||||
JSON-RPC: T = N × R × (serialize(M) + deserialize(M) + network(M))
|
||||
≈ N × R × (200μs + 100μs + network)
|
||||
|
||||
Cap'n Proto: T = N × R × network(M') where M' ≈ M/6
|
||||
≈ N × R × network_only
|
||||
|
||||
For N=10, R=10, typical network=1ms:
|
||||
JSON-RPC: 10 × 10 × (300μs + 1ms) = 130ms
|
||||
Cap'n Proto: 10 × 10 × 1ms = 100ms + 0μs serialization = 100ms
|
||||
|
||||
Actual improvement: ~30% in this scenario, but:
|
||||
- Local agents (no network): 100% improvement (0μs vs 30ms)
|
||||
- Large messages: Scales linearly better
|
||||
- High-frequency consensus: Cumulative savings
|
||||
```
|
||||
@@ -1,25 +0,0 @@
|
||||
# Hanzo Improvement Proposals (HIPs)
|
||||
|
||||
Design documents for major architectural decisions in the Hanzo ecosystem.
|
||||
|
||||
## Active HIPs
|
||||
|
||||
| HIP | Title | Status | Description |
|
||||
|-----|-------|--------|-------------|
|
||||
| [HIP-0300](HIP-0300.md) | Unified MCP Tools Architecture | Draft | Consolidates 52+ tools into ~16 orthogonal operators |
|
||||
| [HIP-0301](HIP-0301.md) | High-Performance Agent Communication | Draft | Cap'n Proto for sub-ms agent-to-agent communication |
|
||||
|
||||
## HIP Process
|
||||
|
||||
1. **Draft**: Initial proposal with design rationale
|
||||
2. **Review**: Community feedback and iteration
|
||||
3. **Accepted**: Approved for implementation
|
||||
4. **Final**: Fully implemented and documented
|
||||
|
||||
## Contributing
|
||||
|
||||
To propose a new HIP:
|
||||
|
||||
1. Create a new markdown file: `HIP-XXXX.md`
|
||||
2. Follow the template structure (Abstract, Motivation, Design, Implementation)
|
||||
3. Submit a PR for review
|
||||
Reference in New Issue
Block a user