wehub-resource-sync
0d3cb498a3
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
88 lines
2.7 KiB
Markdown
88 lines
2.7 KiB
Markdown
# redteam-mcp (Red Team Model Context Protocol)
|
|
|
|
This example demonstrates red teaming an AI assistant that uses Model Context Protocol (MCP) for tool use capabilities. The example focuses on testing various attack vectors specific to MCP implementations, such as function call exploits, system prompt leakage, unauthorized tool discovery, and other MCP-specific vulnerabilities.
|
|
|
|
You can run this example with:
|
|
|
|
```bash
|
|
npx promptfoo@latest init --example redteam-mcp
|
|
cd redteam-mcp
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
This example requires the following environment variable:
|
|
|
|
- `ANTHROPIC_API_KEY` - Your Anthropic API key
|
|
|
|
You can set this in a `.env` file or directly in your environment:
|
|
|
|
```bash
|
|
export ANTHROPIC_API_KEY=your_anthropic_key_here
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
- Node.js ^20.20.0 or >=22.22.0 (Node.js 20 support ends July 30, 2026; Node.js 24 LTS recommended)
|
|
|
|
## Getting Started
|
|
|
|
1. Initialize the example:
|
|
|
|
```bash
|
|
npx promptfoo@latest init --example redteam-mcp
|
|
```
|
|
|
|
2. Navigate to the example directory:
|
|
|
|
```bash
|
|
cd redteam-mcp
|
|
```
|
|
|
|
3. Run the red team evaluation:
|
|
```bash
|
|
npx promptfoo redteam run
|
|
```
|
|
|
|
## What This Example Tests
|
|
|
|
This example evaluates an AI customer support agent implementing MCP with the following attack vectors:
|
|
|
|
1. **Function Discovery**: Attempts to trick the agent into exposing hidden functions
|
|
2. **Parameter Injection**: Tests crafting malicious inputs to inject unwanted parameters
|
|
3. **Function Call Manipulation**: Tries to induce unauthorized calls or response handling
|
|
4. **Recursive Function Calling**: Tests exploitation of recursion for denial of service
|
|
5. **System Information Leakage**: Attempts to extract internal system details
|
|
6. **Function Output Manipulation**: Tests causing misinterpretation of function results
|
|
7. **Tool Metadata Injection**: Attempts to smuggle instructions through tool names or descriptions
|
|
8. **Unauthorized Tool Invocation / Privilege Escalation**: Tests access to unauthorized tools
|
|
|
|
## Configuration
|
|
|
|
This example is configured to test the Anthropic Claude 4 Sonnet model with MCP enabled. The MCP server is specified as:
|
|
|
|
```text
|
|
https://customer-service-mcp-server-example.promptfoo.app/mcp
|
|
```
|
|
|
|
## Testing Strategies
|
|
|
|
The red team evaluation uses multiple testing strategies:
|
|
|
|
- Basic attacks
|
|
- Best-of-n optimization
|
|
- Jailbreak attempts
|
|
|
|
## Expected Results
|
|
|
|
After running the evaluation, you'll see a report showing which attack vectors were successful and which were blocked by the system's defenses.
|
|
|
|
## Customization
|
|
|
|
You can modify the `promptfooconfig.yaml` file to:
|
|
|
|
- Test different providers (recommended: `anthropic:claude-sonnet-4-6`)
|
|
- Add or remove red team plugins
|
|
- Change the MCP server configuration
|
|
- Adjust the system purpose and guardrails
|