Route multiple upstreams

Problem: you have more than one MCP server (filesystem, git, databases) and you want one mcpgw endpoint that routes calls to the right backend based on the tool name.

Solution: declare each backend in upstreams[], then add routes[] entries that match by tool prefix, exact name, glob, regex, or explicit name list.

Recipe

default_upstream: filesystem    # catches initialize, tools/list, ping, etc.

upstreams:
  - name: filesystem
    url: http://mcp-fs:9000
  - name: git
    url: http://mcp-git:9001
  - name: db
    url: http://mcp-pg:9002

routes:
  - match: { tool_prefix: "fs_" }
    upstream: filesystem
  - match: { tool_prefix: "git_" }
    upstream: git
  - match: { tool_glob: "db_*" }
    upstream: db

A tools/call for fs_read goes to filesystem. A call for git_log goes to git. A call for an unknown prefix returns HTTP 404 with JSON-RPC error -32601 no_route.

Why default_upstream matters

Spec-compliant MCP clients send several methods at startup that have no tool name:

  • initialize — handshake
  • tools/list — enumeration
  • resources/list, prompts/list — capability discovery
  • ping — keep-alive

These cannot be matched by tool_prefix. Without default_upstream, the client fails at the initialize handshake with 404 no_route and you spend an afternoon debugging “why does Claude Desktop refuse to connect.”

Set default_upstream to whichever backend is canonical for your environment — usually the one whose tools/list you most want clients to see. Multi-upstream tools/list aggregation is on the roadmap; today the gateway routes the entire response from a single upstream.

Pitfalls

  • First match wins. Put more specific matchers before less specific ones (fs_admin_ before fs_).
  • Matchers are case-sensitive. Fs_ does not match fs_.
  • Use one matcher per route. Supported route matchers are tool_prefix, tool_name, tool_glob, tool_regex, and tool_name_in.
  • Upstream URLs must be reachable from inside the mcpgw container. Use the orchestrator’s service DNS (Docker Compose service name, Kubernetes Service, etc.), not localhost.

Verifying

# Watch a tools/list — should hit the default_upstream
curl -s -X POST http://localhost:7332/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' \
  | jq '.result.tools[].name'

# A tools/call should go to the prefix-matched upstream
curl -s -X POST http://localhost:7332/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"git_log","arguments":{}}}'

Span mcp.upstream confirms which backend handled each call.

Previous ← Health endpoints Next Tool-search tradeoffs →