wehub-resource-sync
0d3cb498a3
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
2.0 KiB
2.0 KiB
sidebar_label, title, description
| sidebar_label | title | description |
|---|---|---|
| Basic | Basic Strategy | Configure inclusion of original plugin-generated test cases in evaluation outputs for baseline comparison and analysis |
Basic Strategy
The basic strategy controls whether the original plugin-generated test cases (without any strategies applied) are included in the final output.
Implementation
The basic strategy accepts the following configuration options:
| Option | Type | Default | Description |
|---|---|---|---|
| enabled | boolean | true | Whether to include basic test cases |
Example usage:
redteam:
language: ['es', 'fr'] # Test in Spanish and French
strategies:
- id: basic
config:
enabled: false # Only run tests with strategies applied
- id: jailbreak
- id: base64
How It Works
By default, promptfoo will:
- Generate test cases from enabled plugins
- Apply each strategy to generate additional test cases
- Include both the original plugin test cases and strategy-generated test cases
When the basic strategy is disabled (enabled: false), only the strategy-generated test cases will be included in the final output. This can be useful when you want to focus solely on testing specific attack vectors through strategies.
Use Cases
- Testing strategy effectiveness: Disable basic tests to isolate and evaluate how well your strategies are working
- Reducing test volume: If you have many plugins and strategies, disabling basic tests can reduce the total number of tests
- Focused testing: When you specifically want to test how your system handles modified/strategic inputs rather than basic plugin-generated tests
Related Concepts
- Red Team Plugins - Generate test cases for basic strategy
- Red Team Strategies - Other strategies to apply to test cases
- Configuration Guide - More advanced configuration options