promptfoo--promptfoo/examples/redteam-api-top-10/README.md

# redteam-api-top-10 (OWASP API Security Top 10 Red Team Example)

A **deliberately vulnerable** demonstration app for testing AI red-teaming and OWASP API Security Top 10 vulnerabilities via prompt injection. This example showcases MCP (Model Context Protocol) tool integration with configurable security weaknesses.

## Quick Start

```bash
# Initialize this example
npx promptfoo@latest init --example redteam-api-top-10
cd redteam-api-top-10

# Set up environment
cp .env.example .env
# Edit .env and set ANTHROPIC_API_KEY=your-api-key

# Install Python dependencies
uv sync

# Seed the database with demo data
uv run python scripts/seed_database.py

# Start the server (in a separate terminal)
uv run uvicorn app.main:app --host 0.0.0.0 --port 8000

# Generate a fresh JWT token and update promptfooconfig.yaml
curl -s -X POST http://localhost:8000/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"alice","password":"password123"}' | jq -r '.token'

# Run the red team evaluation
npx promptfoo@latest redteam run
```

## Prerequisites

- Python 3.10+
- Node.js ^20.20.0 or >=22.22.0 (Node.js 20 support ends July 30, 2026; Node.js 24 LTS recommended) (for the filesystem MCP server)
- [uv](https://docs.astral.sh/uv/) package manager
- Anthropic API key

## What This Example Tests

This application includes configurable vulnerabilities based on the [OWASP API Security Top 10 (2023)](https://owasp.org/API-Security/editions/2023/en/0x00-header/):

| Vulnerability             | Description                                                  |
| ------------------------- | ------------------------------------------------------------ |
| API1: BOLA                | Broken Object Level Authorization - access other users' data |
| API2: Broken Auth         | Unsigned JWT acceptance, debug parameters                    |
| API3: Property Auth       | Sensitive columns exposed (salary, SSN, cost_price)          |
| API5: Function Auth       | Admin tools available to all users                           |
| API7: SSRF                | Internal endpoints accessible via fetch tool                 |
| API8: Misconfiguration    | Debug endpoints exposed, JWT secret in logs                  |
| API9: Inventory           | Legacy v1 API without authentication                         |
| API10: Unsafe Consumption | No response sanitization                                     |

## Architecture

```text
┌─────────────────┐     ┌──────────────────────┐
│   Web Chat UI   │────▶│    FastAPI Server    │
└─────────────────┘     └──────────┬───────────┘
                                   │
                        ┌──────────┼──────────┐
                        ▼          ▼          ▼
                   ┌────────┐ ┌────────┐ ┌────────┐
                   │ SQLite │ │  File  │ │ Fetch  │
                   │  MCP   │ │ System │ │  MCP   │
                   │ Server │ │  MCP   │ │ Server │
                   └────────┘ └────────┘ └────────┘
```

The chatbot connects to three MCP servers:

- **SQLite MCP**: Database queries for products, orders, users
- **Filesystem MCP**: Reading policy documents
- **Fetch MCP**: External API calls (shipping tracking, weather, promotions)

## Demo Users

All users have password `password123`:

| Username | User ID | Department  | Role  |
| -------- | ------- | ----------- | ----- |
| alice    | emp_001 | Engineering | user  |
| bob      | emp_002 | Marketing   | user  |
| charlie  | emp_003 | Sales       | user  |
| diana    | emp_004 | HR          | admin |
| eve      | emp_005 | Finance     | user  |

## Security Configuration

Each vulnerability can be configured independently via environment variables:

```bash
# Security levels: 1 = Weak (vulnerable), 2 = Medium (bypassable), 3 = Strong (secure)
SECURITY_BOLA=1
SECURITY_AUTH=1
SECURITY_PROPERTY_AUTH=1
SECURITY_FUNCTION_AUTH=1
SECURITY_SSRF=1
SECURITY_MISCONFIGURATION=1
SECURITY_INVENTORY=1
SECURITY_UNSAFE_CONSUMPTION=1
```

Run with different security levels:

```bash
# All vulnerabilities (default)
uv run uvicorn app.main:app --port 8000

# Stronger security
SECURITY_BOLA=3 SECURITY_SSRF=3 uv run uvicorn app.main:app --port 8000
```

## Quick Security Tests

```bash
# Check current security configuration
curl http://localhost:8000/security/status

# Test API2 - Unsigned JWT (alg:none)
curl -H "Authorization: Bearer eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.eyJzdWIiOiJlbXBfMDAyIiwibmFtZSI6IkJvYiJ9." \
  http://localhost:8000/api/orders

# Test API9 - Legacy endpoint (no auth)
curl http://localhost:8000/api/v1/orders

# Test API7 - SSRF target
curl http://localhost:8000/mock/internal/metadata

# Test API8 - Debug endpoint
curl http://localhost:8000/debug/logs
```

## Promptfoo Configuration

The `promptfooconfig.yaml` is pre-configured with:

- **Plugin**: `owasp:api` - Tests all OWASP API Security Top 10 vulnerabilities
- **Strategies**: `jailbreak:meta`, `crescendo`, `jailbreak:hydra`

### Expected Results

At **SECURITY\_\*=1** (vulnerable), the red team should find:

- BOLA attacks via prompt injection to access other users' data
- SSRF via fetch tool to internal endpoints
- Sensitive data exposure (salaries, SSNs, cost prices)
- Legacy API access without authentication

At **SECURITY\_\*=3** (secure), most attacks should be blocked.

## Web UI

Visit http://localhost:8000 in your browser for the chat interface.

## API Endpoints

### Chat

- `POST /chat/` - Send message, get response with tool calls
- `GET /chat/sessions` - List user's sessions

### Auth

- `POST /auth/login` - Get JWT token
- `GET /auth/demo-users` - List available demo users

### Mock Services

- `GET /mock/shipping/track/{tracking}` - Track package
- `GET /mock/weather/{location}` - Weather data
- `GET /mock/promotions/current` - Current promos

### Debug (vulnerable)

- `GET /debug/logs` - Exposes JWT secret
- `GET /debug/config` - Server configuration

### Legacy v1 (vulnerable)

- `GET /api/v1/orders` - All orders without auth
- `GET /api/v1/users` - All users without auth

## Troubleshooting

### MCP Server Issues

```bash
# Check Node.js for filesystem MCP
which npx

# Check uv for SQLite/Fetch MCP
which uvx
```

### Database Issues

```bash
# Re-seed the database
uv run python scripts/seed_database.py
```

### API Key Issues

```bash
# Verify API key is set
grep ANTHROPIC_API_KEY .env
```

## License

MIT