engine/examples/MCP_QUICK_START.md

MCP Quick Start Guide

🔗 Connect your models to external tools and services using the Model Context Protocol (MCP).

What is MCP? The Model Context Protocol allows your AI models to access external tools like web search, file systems, databases, and APIs automatically during conversations.

Key Benefits:

• 🚀 Zero setup - Tools work automatically once configured
• 🔧 Multi-tool support - Connect to multiple services simultaneously
• 🌐 Universal protocol - Works with any MCP-compatible server
• 🔒 Secure - Built-in authentication and timeout controls

📚 Full Documentation | ⚙️ Configuration Reference

Quick Start

1. Create Configuration File

Create mcp-config.json with a working example using the filesystem server:

{
  "servers": [
    {
      "name": "Filesystem Tools",
      "source": {
        "type": "Process",
        "command": "npx",
        "args": ["@modelcontextprotocol/server-filesystem", "."]
      }
    }
  ],
  "auto_register_tools": true
}

Note: To install the filesystem server, run: npx @modelcontextprotocol/server-filesystem . -y

Alternative Transport Examples (commented out by default):

HTTP Example - Hugging Face MCP Server

{
  "servers": [
    {
      "name": "Hugging Face MCP",
      "source": {
        "type": "Http",
        "url": "https://hf.co/mcp",
        "timeout_secs": 30
      },
      "bearer_token": "hf_xxx",
      "tool_prefix": "hf",
      "enabled": false
    },
    {
      "name": "Filesystem Tools",
      "source": {
        "type": "Process",
        "command": "npx",
        "args": ["@modelcontextprotocol/server-filesystem", "."]
      }
    }
  ],
  "auto_register_tools": true
}

WebSocket Example

{
  "servers": [
    {
      "name": "WebSocket Example",
      "source": {
        "type": "WebSocket",
        "url": "wss://api.example.com/mcp",
        "timeout_secs": 30
      },
      "enabled": false
    },
    {
      "name": "Filesystem Tools",
      "source": {
        "type": "Process",
        "command": "npx",
        "args": ["@modelcontextprotocol/server-filesystem", "."]
      }
    }
  ],
  "auto_register_tools": true
}

2. Start Server with MCP Tools

# Start with MCP configuration
mistralrs-server --mcp-config mcp-config.json --port 1234 run -m Qwen/Qwen3-4B

# Alternative: Use environment variable
export MCP_CONFIG_PATH=mcp-config.json
mistralrs-server --port 1234 run -m Qwen/Qwen3-4B

✅ Server starts with tools automatically loaded!

3. Use Tools Automatically

That's it! Tools work automatically in conversations:

# File operations example
curl -X POST http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-4B",
    "messages": [
      {"role": "user", "content": "List the files in the current directory and create a hello.txt file"}
    ]
  }'

🎉 The model automatically uses tools when needed - no extra steps required!

✅ Verify Tools are Working

curl http://localhost:1234/v1/models

Look for MCP status in the response:

{
  "object": "list",
  "data": [{
    "id": "Qwen/Qwen3-4B",
    "object": "model", 
    "tools_available": true,
    "mcp_tools_count": 3,
    "mcp_servers_connected": 1
  }]
}

✅ tools_available: true means MCP tools are working!

🔧 Quick Verification

Test filesystem server is working:

# This should return "3" or more (filesystem tools available)
curl http://localhost:1234/v1/models | jq '.data[0].mcp_tools_count'

# Check if filesystem server process is running
ps aux | grep server-filesystem

🚀 Popular MCP Servers

Server	Description	Installation	Use Case
Filesystem	File operations	npm i -g @modelcontextprotocol/server-filesystem	Read/write files
Hugging Face	HF API access	Web service at https://hf.co/mcp	Models, datasets, spaces
Postgres	Database	npm i -g @modelcontextprotocol/server-postgres	SQL queries

Links to more servers:

• Brave Search - Web search capabilities
• GitHub - Repository access

Transport Types

Transport	When to Use	Examples
Process	Local tools, npm packages	Most MCP servers
HTTP	REST APIs, cloud services	Custom web services
WebSocket	Real-time streaming	Live data feeds

📋 Ready-to-Use Configurations

Process Example (Default - Filesystem Server)

{
  "servers": [{
    "name": "File Operations",
    "source": {
      "type": "Process",
      "command": "npx",
      "args": ["@modelcontextprotocol/server-filesystem", "."]
    }
  }],
  "auto_register_tools": true
}

HTTP Example (Hugging Face MCP Server)

{
  "servers": [
    {
      "name": "Hugging Face MCP",
      "source": {
        "type": "Http",
        "url": "https://hf.co/mcp",
        "timeout_secs": 30
      },
      "bearer_token": "hf_xxx",
      "tool_prefix": "hf",
      "enabled": false
    },
    {
      "name": "File Operations",
      "source": {
        "type": "Process",
        "command": "npx",
        "args": ["@modelcontextprotocol/server-filesystem", "."]
      }
    }
  ],
  "auto_register_tools": true
}

WebSocket Example

{
  "servers": [
    {
      "name": "WebSocket Example",
      "source": {
        "type": "WebSocket",
        "url": "wss://api.example.com/mcp",
        "timeout_secs": 30
      },
      "enabled": false
    },
    {
      "name": "File Operations",
      "source": {
        "type": "Process",
        "command": "npx",
        "args": ["@modelcontextprotocol/server-filesystem", "."]
      }
    }
  ],
  "auto_register_tools": true
}

Error Handling

The system gracefully handles failures:

• Startup: Invalid configurations are caught and reported with helpful messages
• Runtime: Failed MCP connections are logged as warnings, server continues without MCP
• Tools: Individual tool failures don't crash the server

📚 Next Steps

Ready for more?

• 🔧 Configuration Reference - All available options
• 📖 Full MCP Documentation - Complete guide
• 🛠️ Server Examples - Real-world configurations
• 🚀 Advanced Usage - Multi-server setups

Need help?

• MCP Server Registry - Find more servers
• Troubleshooting - Common issues
• Discord Community - Get support