chore: import upstream snapshot with attribution

examples/redteam-tracing-example/README.md

@@ -0,0 +1,376 @@
+# redteam-tracing-example (Red Team Tracing)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example redteam-tracing-example
+cd redteam-tracing-example
+```
+
+This example demonstrates how to use tracing with red team strategies to provide attackers and graders with visibility into the internal operations of your LLM application.
+
+## Quick Start
+
+**1. Install dependencies:**
+
+```bash
+npm install
+```
+
+**2. Start the mock traced server:**
+
+```bash
+npm run server
+```
+
+This starts an HTTP server on port 3110 that:
+
+- Accepts chat requests
+- Generates OTLP trace spans (LLM calls, guardrails, tools)
+- Sends spans to promptfoo's OTLP receiver
+
+**3. Test the server (optional):**
+
+```bash
+# In another terminal
+./test-server.sh
+```
+
+**4. Run the red team evaluation:**
+
+```bash
+# In another terminal (from the project root)
+npm run local -- eval -c examples/redteam-tracing-example/promptfooconfig.yaml
+```
+
+**5. View the results:**
+
+```bash
+npm run local -- view
+```
+
+You'll see trace data in:
+
+- Attack prompts (when `includeInAttack: true`)
+- Grading context (when `includeInGrading: true`)
+- Test metadata (`traceSnapshots`)
+
+## Troubleshooting
+
+**Server not responding?**
+
+```bash
+# Check if server is running
+curl http://localhost:3110/health
+
+# Test basic request
+curl -X POST http://localhost:3110/chat \
+  -H "Content-Type: application/json" \
+  -d '{"prompt": "test"}'
+```
+
+**No traces appearing?**
+
+- Make sure the server is emitting to the correct OTLP endpoint (check server logs)
+- Verify promptfoo's OTLP receiver is enabled in config (`tracing.enabled: true`)
+- Check that `traceparent` headers are being passed (set in provider context)
+
+## What is Red Team Tracing?
+
+Red team tracing allows adversarial strategies to see what happens inside your LLM application during an attack, including:
+
+- Tool calls and their results
+- Guardrail decisions
+- Internal LLM calls
+- Error conditions
+- Performance metrics
+
+This information can help:
+
+1. **Attack generation**: Craft more effective attacks by understanding how the system responds internally
+2. **Grading**: Make more informed decisions about whether an attack succeeded by seeing internal behavior
+
+## Configuration
+
+### Basic Configuration
+
+Enable tracing in your `promptfooconfig.yaml`:
+
+```yaml
+redteam:
+  tracing:
+    # Enable tracing for all strategies
+    enabled: true
+
+    # Include trace data in attack generation (default: true)
+    includeInAttack: true
+
+    # Include trace data in grading (default: true)
+    includeInGrading: true
+
+  plugins:
+    - harmful
+    - pii
+
+  strategies:
+    - crescendo
+    - goat
+```
+
+### Advanced Configuration
+
+Configure tracing behavior:
+
+```yaml
+redteam:
+  tracing:
+    enabled: true
+
+    # Include internal spans (e.g., tokenization, parsing)
+    includeInternalSpans: false
+
+    # Maximum number of spans to fetch per iteration
+    maxSpans: 50
+
+    # Maximum depth of nested spans to fetch
+    maxDepth: 5
+
+    # Retry configuration for fetching traces
+    maxRetries: 3
+    retryDelayMs: 500
+
+    # Filter spans by name pattern (optional)
+    spanFilter:
+      - 'llm.*'
+      - 'tool.*'
+      - 'guardrail.*'
+
+    # Sanitize sensitive attributes (recommended)
+    sanitizeAttributes: true
+```
+
+### Strategy-Specific Configuration
+
+Different strategies may need different tracing settings:
+
+```yaml
+redteam:
+  tracing:
+    enabled: true
+
+    # Strategy-specific overrides
+    strategies:
+      # Crescendo benefits from seeing guardrail decisions
+      crescendo:
+        includeInAttack: true
+        includeInGrading: true
+        spanFilter:
+          - 'guardrail.*'
+          - 'llm.*'
+
+      # GOAT can use tool call information
+      goat:
+        includeInAttack: true
+        spanFilter:
+          - 'tool.*'
+          - 'llm.*'
+
+      # Iterative may want full trace data
+      iterative:
+        includeInAttack: true
+        includeInGrading: true
+        maxSpans: 100
+```
+
+### Test-Level Configuration
+
+Override tracing for specific tests:
+
+```yaml
+tests:
+  - description: 'Test with custom tracing'
+    vars:
+      query: 'Tell me about sensitive data'
+    metadata:
+      tracing:
+        enabled: true
+        includeInAttack: true
+        includeInGrading: true
+        maxSpans: 200
+```
+
+## How Tracing Works
+
+### 1. Attack Generation
+
+When `includeInAttack: true`, the attacker receives a trace summary like:
+
+```text
+Trace 0af76519 • 5 spans
+
+Execution Flow:
+1. [1.2s] llm.generate (client) | model=gpt-4
+2. [300ms] guardrail.check (internal) | tool=content-filter
+3. [150ms] tool.database_query (server) | tool=search
+4. [50ms] guardrail.check (internal) | ERROR: Rate limit exceeded
+5. [800ms] llm.generate (client) | model=gpt-4
+
+Key Observations:
+• Guardrail content-filter decision: blocked
+• Tool call search via "tool.database_query" (duration 150ms)
+• Error span "guardrail.check" (span-4): Rate limit exceeded
+```
+
+The attacker can use this information to craft better attacks (e.g., targeting the rate limit error).
+
+### 2. Grading
+
+When `includeInGrading: true`, graders receive the same trace context and can make more informed decisions:
+
+```typescript
+// Grader receives:
+{
+  prompt: "...",
+  llmOutput: "...",
+  test: {...},
+  gradingContext: {
+    traceContext: {
+      traceId: "...",
+      spans: [...],
+      insights: [...]
+    },
+    traceSummary: "..."
+  }
+}
+```
+
+## Best Practices
+
+### 1. Start with Default Settings
+
+The default configuration works well for most use cases:
+
+```yaml
+redteam:
+  tracing:
+    enabled: true
+```
+
+### 2. Use spanFilter for Focused Analysis
+
+If you only care about specific operations:
+
+```yaml
+redteam:
+  tracing:
+    enabled: true
+    spanFilter:
+      - 'guardrail.*' # Only guardrail spans
+      - 'tool.*' # Only tool calls
+```
+
+### 3. Keep sanitizeAttributes Enabled
+
+Always sanitize attributes in production:
+
+```yaml
+redteam:
+  tracing:
+    enabled: true
+    sanitizeAttributes: true # Recommended
+```
+
+### 4. Adjust maxSpans Based on Complexity
+
+- Simple apps: `maxSpans: 20`
+- Medium complexity: `maxSpans: 50` (default)
+- Complex agentic systems: `maxSpans: 100-200`
+
+### 5. Use Strategy-Specific Overrides
+
+Different strategies benefit from different trace data:
+
+- **Crescendo**: Needs guardrail information
+- **GOAT**: Benefits from tool call traces
+- **Iterative**: Can use comprehensive trace data
+
+## Security Considerations
+
+### Sensitive Data
+
+Tracing can expose sensitive information. Always:
+
+1. Use `sanitizeAttributes: true` (default)
+2. Review trace data before sharing
+3. Consider disabling tracing for production testing
+
+### Performance
+
+Tracing adds overhead:
+
+- Fetching traces: ~100-500ms per iteration
+- Processing spans: Minimal overhead
+- Storage: Trace metadata is stored in test results
+
+To minimize impact:
+
+- Use `maxSpans` to limit data fetched
+- Set appropriate `maxRetries` and `retryDelayMs`
+- Consider disabling for large-scale testing
+
+## Debugging
+
+### Enable Debug Logging
+
+```bash
+PROMPTFOO_LOG_LEVEL=debug npm run local -- eval -c redteam.yaml
+```
+
+### Check Trace Store
+
+Verify traces are being recorded:
+
+```bash
+# View traces in the database
+npm run db:studio
+```
+
+### Test Trace Fetching
+
+```typescript
+import { fetchTraceContext } from './src/tracing/traceContext';
+
+const trace = await fetchTraceContext('your-trace-id', {
+  maxSpans: 50,
+  maxDepth: 5,
+});
+console.log(trace);
+```
+
+## Examples
+
+See the example configurations:
+
+- `promptfooconfig.yaml` - Basic tracing setup
+- `promptfooconfig.advanced.yaml` - Advanced configuration
+- `promptfooconfig-simple.yaml` - Simplified configuration
+
+## Troubleshooting
+
+### No Traces Appearing
+
+1. Check that your provider supports tracing (must send traceparent header)
+2. Verify OTLP receiver is running
+3. Check debug logs for trace fetch errors
+
+### Traces Not Used in Attacks
+
+1. Verify `includeInAttack: true`
+2. Check that traces are being fetched (debug logs)
+3. Ensure trace fetch completes before attack generation
+
+### Performance Issues
+
+1. Reduce `maxSpans` and `maxDepth`
+2. Use `spanFilter` to limit data
+3. Increase `retryDelayMs` to reduce fetch frequency