wehub-resource-sync
0d3cb498a3
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Waiting to run
Test and Publish Multi-arch Docker Image / test (push) Waiting to run
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Blocked by required conditions
Validate Renovate Config / Validate Renovate Configuration (push) Waiting to run
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
anthropic/mcp (Anthropic Messages + MCP tools)
This example wires Claude up to a Model Context Protocol server through the Anthropic Messages provider. Promptfoo exposes the MCP server's tools to Claude, executes any tool_use blocks the model emits, and feeds the tool_result back into the conversation until Claude produces a final reply.
npx promptfoo@latest init --example anthropic/mcp
cd anthropic/mcp
Setup
export ANTHROPIC_API_KEY=sk-ant-...
npx promptfoo@latest eval
The bundled config uses the public deepwiki MCP server so the example works out of the box without installing anything. To swap in your own server, replace the mcp block:
mcp:
enabled: true
# Local stdio server
server:
command: npx
args: ['-y', '@modelcontextprotocol/server-filesystem', '/tmp/workspace']
# …or remote SSE / streamable HTTP
# servers:
# - name: my-server
# url: https://example.com/mcp
What it demonstrates
- Tool discovery: Tools exposed by the MCP server are forwarded to Claude alongside any inline
toolsyou define. - Multi-round execution: When Claude returns a
tool_useblock matching an MCP tool, promptfoo invokes the tool with the model's arguments and appends atool_resulton the next user turn. The loop continues until Claude returns text — bounded bymax_tool_calls(default8). - Error pass-through: Tool errors come back as
tool_resultblocks withis_error: trueso Claude can recover or report the failure. - Mixed tool handling: Non-MCP tools (regular function-calling tools, or built-ins like
web_search) fall through to the existing tool-use output without being auto-executed.
Caching
Promptfoo's disk response cache is skipped automatically when mcp.enabled is true, because tool results can vary between runs. Use max_tool_calls to bound the per-request cost.
See also
- Anthropic provider docs
- MCP integration guide — full server configuration (auth, timeouts, multi-server)
- examples/openai-mcp — the same pattern via the OpenAI Responses API
- examples/simple-mcp — testing an MCP server directly without an LLM provider