Files
promptfoo--promptfoo/site/docs/integrations/mcp.md
T
wehub-resource-sync 0d3cb498a3
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:24:08 +08:00

248 lines
9.2 KiB
Markdown

---
title: Using MCP (Model Context Protocol) in Promptfoo
description: Enable Model Context Protocol (MCP) integration for enhanced tool use, persistent memory, and agentic workflows across providers
sidebar_label: Model Context Protocol (MCP)
sidebar_position: 20
---
# Using MCP (Model Context Protocol) in Promptfoo
Promptfoo supports the Model Context Protocol (MCP) for advanced tool use, and agentic workflows. MCP allows you to connect your Promptfoo providers to an external MCP server, such as the [modelcontextprotocol/server-memory](https://github.com/modelcontextprotocol/server-memory), to enable tool orchestration, and more.
:::note MCP SDK dependency
Promptfoo's general MCP integration uses the optional `@modelcontextprotocol/sdk` runtime package. Standard npm installs include optional dependencies, but installs that omit them need:
```bash
npm install @modelcontextprotocol/sdk
```
:::
## Basic Configuration
To enable MCP for a provider, add the `mcp` block to your provider's `config` in your `promptfooconfig.yaml`:
```yaml title="promptfooconfig.yaml"
description: Testing MCP memory server integration with Google AI Studio
providers:
- id: google:gemini-2.0-flash
config:
mcp:
enabled: true
server:
command: npx
args: ['-y', '@modelcontextprotocol/server-memory']
name: memory
```
### MCP Config Options
- `enabled`: Set to `true` to enable MCP for this provider.
- `timeout`: (Optional) Request timeout in milliseconds for MCP tool calls. Defaults to 60000 (60 seconds). Set higher for long-running tools.
- `resetTimeoutOnProgress`: (Optional) Reset timeout when progress notifications are received. Useful for long-running operations. Default: false.
- `maxTotalTimeout`: (Optional) Absolute maximum timeout in milliseconds regardless of progress notifications.
- `pingOnConnect`: (Optional) Ping the server after connecting to verify it's responsive. Default: false.
- `server`: (Optional) Configuration for launching or connecting to an MCP server.
- `command`: The command to launch the MCP server (e.g., `npx`).
- `args`: Arguments to pass to the command (e.g., `['-y', '@modelcontextprotocol/server-memory']`).
- `name`: (Optional) A name for the server instance.
- `url`: URL for connecting to a remote MCP server.
- `headers`: (Optional) Custom HTTP headers to send when connecting to a remote MCP server (only applies to `url`-based connections).
- `auth`: (Optional) Authentication configuration for the server. Can be used to automatically set auth headers for all connection types.
- `type`: Authentication type, either `'bearer'` or `'api_key'`.
- `token`: Token for bearer authentication.
- `api_key`: API key for api_key authentication.
- You can also connect to a remote MCP server by specifying a `url` instead of `command`/`args`.
MCP servers can be run locally or accessed remotely. For development and testing, a local server is often simplest, while production environments may use a centralized remote server.
#### Example: Connecting to a Remote MCP Server
```yaml
providers:
- id: openai:responses:gpt-5.1
config:
apiKey: <your-api-key>
mcp:
enabled: true
server:
url: http://localhost:8000
```
#### Example: Using Custom Headers with a Remote MCP Server
```yaml
providers:
- id: openai:responses:gpt-5.1
config:
apiKey: <your-api-key>
mcp:
enabled: true
server:
url: http://localhost:8000
headers:
X-API-Key: your-custom-api-key
Authorization: Bearer your-token
X-Custom-Header: custom-value
```
This can be useful when:
- The MCP server requires an API key or authentication token
- You need to provide custom identifiers or session information
- The server needs specific headers for configuration or tracking
## Connecting a Single Provider to Multiple MCP Servers
Promptfoo allows a single provider to connect to multiple MCP servers by using the `servers` array in your provider's MCP config. All tools from all connected servers will be available to the provider.
### Example: One Provider, Multiple MCP Servers
```yaml title="promptfooconfig.yaml"
providers:
- id: openai:responses:gpt-5.1
config:
mcp:
enabled: true
servers:
- command: npx
args: ['-y', '@modelcontextprotocol/server-memory']
name: server_a
- url: http://localhost:8001
name: server_b
headers:
X-API-Key: your-api-key
```
- Use the `servers:` array (not just `server:`) to specify multiple MCP servers.
- Each entry can be a local launch or a remote URL (if supported).
- All tools from all servers will be available to the provider.
- You can specify different headers for each server when using URL connections.
- You can also connect to the same server multiple times if needed:
```yaml
providers:
- id: anthropic:claude-sonnet-4-5-20250929
config:
mcp:
enabled: true
servers:
- command: npx
args: ['-y', '@modelcontextprotocol/server-memory']
name: memory
- command: npx
args: ['-y', '@modelcontextprotocol/server-filesystem']
name: filesystem
- command: npx
args: ['-y', '@modelcontextprotocol/server-github']
name: github
```
This configuration connects a single provider to multiple MCP servers, giving it access to memory storage, filesystem operations, and GitHub integration simultaneously.
## Using Multiple MCP Servers
You can configure multiple MCP servers by assigning different MCP server configurations to different providers in your `promptfooconfig.yaml`. Each provider can have its own `mcp.server` block, allowing you to run separate memory/tool servers for different models or use cases.
```yaml title="promptfooconfig.yaml"
description: Using multiple MCP servers
providers:
- id: google:gemini-2.0-flash
config:
mcp:
enabled: true
server:
command: npx
args: ['-y', '@modelcontextprotocol/server-memory']
name: gemini-memory
- id: openai:responses:gpt-5.1
config:
apiKey: <your-api-key>
mcp:
enabled: true
server:
url: http://localhost:8001
name: openai-memory
headers:
X-API-Key: openai-server-api-key
- id: anthropic:claude-sonnet-4-5-20250929
config:
mcp:
enabled: true
server:
url: http://localhost:8002
name: anthropic-memory
headers:
Authorization: Bearer anthropic-server-token
```
In this example:
- The Gemini provider launches a local MCP server using `npx`.
- The OpenAI and Anthropic providers connect to different remote MCP servers running on different ports.
- Each provider can have its own memory, tool set, and context, isolated from the others.
- Custom headers are specified for the remote servers to handle authentication or other requirements.
This setup is useful for testing, benchmarking, or running isolated agentic workflows in parallel.
## Supported Providers
MCP is supported by most major providers in Promptfoo, including:
- Google Gemini (AI Studio, Vertex)
- OpenAI (and compatible providers like Groq, Together, etc.)
- Anthropic
## OpenAI Responses API MCP Integration
In addition to the general MCP integration described above, OpenAI's Responses API has native MCP support that allows direct connection to remote MCP servers without running local MCP servers. This approach is specific to OpenAI's Responses API and offers:
- Direct connection to remote MCP servers (like DeepWiki, Stripe, etc.)
- Built-in approval workflows for data sharing
- Authentication header support for secured MCP servers
- Tool filtering capabilities
For detailed information about using MCP with OpenAI's Responses API, see the [OpenAI Provider MCP documentation](../providers/openai.md#mcp-model-context-protocol-support).
## Tool Schema Compatibility
Promptfoo automatically handles JSON Schema compatibility between MCP servers and LLM providers by removing provider-incompatible metadata fields (like `$schema`) while preserving supported features. Tools with no input parameters work without modification.
## Timeout Configuration
MCP tool calls have a default timeout of 60 seconds. For long-running tools, increase the timeout:
```yaml
providers:
- id: openai:responses:gpt-5.1
config:
mcp:
enabled: true
timeout: 900000 # 15 minutes in milliseconds
server:
url: https://api.example.com/mcp
```
You can also set a global default via environment variable:
```bash
export MCP_REQUEST_TIMEOUT_MS=900000 # 15 minutes
```
Priority: `config.timeout` > `MCP_REQUEST_TIMEOUT_MS` env var > SDK default (60s).
## Troubleshooting
- Ensure your MCP server is running and accessible.
- Check your provider logs for MCP connection errors.
- Verify that your custom headers are correctly formatted if you're having authentication issues.
- If tool calls timeout, increase the `timeout` config option or set `MCP_REQUEST_TIMEOUT_MS`.
## See Also
- [Configuration Reference](../configuration/reference.md)
- [Provider Configuration](../providers/index.md)