mirror of
https://github.com/zenlm/engine.git
synced 2026-10-30 15:00:05 +00:00
* Add builtin mcp client * Use async loader * Add headers * Handle sse * More flexible search request * Add tool callbacks with tools, for mcp * Add bearer token support * Add websocket support * Update docs * Add python api * Clippy * Add http api, docs * Tests pass * Make these configs actually work * Add docs * Make mistralrs-mcp * Refactor examples * Update examples * Add defaults * Add defaults * Add defaults * Update docs * Improved docs * Add -y to npx usages * Even better examples
3.7 KiB
3.7 KiB
MCP Transport Types
mistral.rs supports three transport types for connecting to MCP servers, each optimized for different use cases.
HTTP Transport
Best for public APIs, RESTful services, and servers behind load balancers.
Configuration
{
"source": {
"type": "Http",
"url": "https://api.example.com/mcp",
"timeout_secs": 30,
"headers": {
"X-API-Version": "v1",
"User-Agent": "mistral-rs/0.6.0"
}
},
"bearer_token": "your-api-token"
}
Features
- Server-Sent Events (SSE) support for streaming responses
- Custom headers for API versioning or client identification
- Bearer token authentication (added as
Authorization: Bearer <token>) - Configurable timeouts
- Standard HTTP semantics
Example: Hugging Face MCP
McpServerSource::Http {
url: "https://hf.co/mcp".to_string(),
timeout_secs: Some(30),
headers: None,
}
WebSocket Transport
Best for real-time applications, bidirectional communication, and low-latency requirements.
Configuration
{
"source": {
"type": "WebSocket",
"url": "wss://realtime.example.com/mcp",
"timeout_secs": 60,
"headers": {
"Origin": "https://mistral.rs",
"Sec-WebSocket-Protocol": "mcp"
}
},
"bearer_token": "your-websocket-token"
}
Features
- Persistent connections reduce handshake overhead
- Server-initiated notifications
- Lower latency for frequent tool calls
- Automatic reconnection handling
- WebSocket-specific headers support
Example: Real-time Data Feed
McpServerSource::WebSocket {
url: "wss://data.example.com/mcp".to_string(),
timeout_secs: Some(60),
headers: Some(headers),
}
Process Transport
Best for local tools, development servers, and sandboxed environments.
Configuration
{
"source": {
"type": "Process",
"command": "mcp-server-filesystem",
"args": ["--root", "/tmp", "--readonly"],
"work_dir": "/home/user/workspace",
"env": {
"MCP_LOG_LEVEL": "info",
"MCP_TIMEOUT": "30"
}
}
}
Features
- No network overhead
- Process isolation for security
- Direct stdin/stdout communication
- Environment variable configuration
- Working directory control
- No authentication needed (process inherits permissions)
Example: Filesystem Server
McpServerSource::Process {
command: "mcp-server-filesystem".to_string(),
args: vec!["--root".to_string(), "/tmp".to_string()],
work_dir: None,
env: None,
}
Transport Selection Guide
| Use Case | Recommended Transport | Why |
|---|---|---|
| Public APIs | HTTP | Standard auth, caching, load balancing |
| Local tools | Process | No network, process isolation |
| Real-time data | WebSocket | Low latency, server push |
| Corporate proxies | HTTP | Proxy support, standard ports |
| Development | Process | Easy debugging, no network setup |
| Interactive apps | WebSocket | Bidirectional, persistent connection |
Security Considerations
HTTP
- Always use HTTPS in production
- Bearer tokens transmitted with each request
- Consider token rotation strategies
WebSocket
- Use WSS (WebSocket Secure) in production
- Bearer token sent during handshake
- Connection persists with authenticated state
Process
- Inherits user permissions
- Sandboxing via work_dir and env
- No network exposure
Performance Tips
- HTTP: Enable keep-alive, use connection pooling
- WebSocket: Reuse connections, handle reconnection gracefully
- Process: Minimize startup time, use long-running processes
Error Handling
All transports implement automatic retry with exponential backoff:
- Initial retry: 1 second
- Max retry: 60 seconds
- Max attempts: 5
Custom retry behavior can be configured per server.