chore: import upstream snapshot with attribution

examples/config-python-test-cases/README.md

@@ -0,0 +1,101 @@
+# config-python-test-cases (Python Test Cases with Configuration)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example config-python-test-cases
+cd config-python-test-cases
+```
+
+This example demonstrates how to use Python functions to generate test cases with configurable parameters using the new TestGeneratorConfig feature.
+
+## Overview
+
+Previously, test generators could only be called without parameters:
+
+```yaml
+tests:
+  - file://test_cases.py:generate_simple_tests
+```
+
+Now you can pass configuration objects to customize the test generation:
+
+```yaml
+tests:
+  - path: file://test_cases.py:generate_simple_tests
+    config:
+      languages: [German, Italian]
+```
+
+## Requirements
+
+1. **Python Dependencies**:
+
+   ```bash
+   pip install pandas
+   ```
+
+2. **Environment Variables**:
+   ```bash
+   export OPENAI_API_KEY=your_api_key_here
+   ```
+
+## Usage
+
+Run the evaluation with:
+
+```bash
+promptfoo eval
+```
+
+## Features Demonstrated
+
+### 1. Backward Compatibility
+
+The old format still works:
+
+```yaml
+- file://test_cases.py:generate_simple_tests
+```
+
+### 2. Simple Configuration
+
+Pass configuration to customize test generation:
+
+```yaml
+- path: file://test_cases.py:generate_simple_tests
+  config:
+    languages: [German, Italian]
+```
+
+### 3. Row Limiting
+
+Control how many test cases are generated:
+
+```yaml
+- path: file://test_cases.py:generate_from_csv
+  config:
+    max_rows: 2
+```
+
+## Implementation
+
+The Python functions accept an optional `config` parameter:
+
+```python
+from typing import Optional, Dict, Any
+
+def generate_simple_tests(config: Optional[Dict[str, Any]] = None):
+    languages = ["Spanish", "French"]  # defaults
+
+    if config:
+        languages = config.get("languages", languages)
+
+    # Generate test cases using the configuration...
+```
+
+This enables:
+
+- **Backward compatibility**: Existing generators work unchanged
+- **Flexible configuration**: Pass any parameters as JSON
+- **Reusable functions**: Same function, different configurations

examples/config-python-test-cases/promptfooconfig.yaml

@@ -0,0 +1,23 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Python test generators with configuration
+
+prompts:
+  - 'Translate the following text to {{target_language}}: {{text}}'
+
+providers:
+  - openai:gpt-4.1-mini
+
+# Demonstrate test generators with and without configuration
+tests:
+  # 1. Basic generator with default languages (Spanish, French)
+  - file://test_cases.py:generate_simple_tests
+
+  # 2. Same generator with different languages (German, Italian)
+  - path: file://test_cases.py:generate_simple_tests
+    config:
+      languages: [German, Italian]
+
+  # 3. CSV generator with row limit
+  - path: file://test_cases.py:generate_from_csv
+    config:
+      max_rows: 2

examples/config-python-test-cases/test_cases.py

@@ -0,0 +1,63 @@
+from typing import Any, Dict, Optional
+
+import pandas as pd
+
+
+def generate_simple_tests(config: Optional[Dict[str, Any]] = None):
+    """Generate a simple set of test cases with optional configuration."""
+    # Default values
+    languages = ["Spanish", "French"]
+    phrases = ["Hello", "Thank you"]
+
+    # Override with config if provided
+    if config:
+        languages = config.get("languages", languages)
+        phrases = config.get("phrases", phrases)
+
+    test_cases = []
+    for phrase in phrases:
+        for lang in languages:
+            test_case = {
+                "vars": {"text": phrase, "target_language": lang},
+                "assert": [
+                    {"type": "regex", "value": ".+"}
+                ],  # Just check for non-empty output
+                "description": f"Translate '{phrase}' to {lang}",
+            }
+            test_cases.append(test_case)
+
+    return test_cases
+
+
+def generate_from_csv(config: Optional[Dict[str, Any]] = None):
+    """Generate test cases from CSV data with optional row limit."""
+    # Default test data
+    data = {
+        "source_text": ["Good morning", "Thank you", "Goodbye"],
+        "target_language": ["French", "Spanish", "German"],
+        "expected_translation": ["Bonjour", "Gracias", "Auf Wiedersehen"],
+    }
+
+    # Use custom data if provided in config
+    if config and "data" in config:
+        data = config["data"]
+
+    df = pd.DataFrame(data)
+
+    # Limit rows if specified in config
+    if config and "max_rows" in config:
+        df = df.head(config["max_rows"])
+
+    test_cases = []
+    for _, row in df.iterrows():
+        test_case = {
+            "vars": {
+                "text": row["source_text"],
+                "target_language": row["target_language"],
+            },
+            "assert": [{"type": "contains", "value": row["expected_translation"]}],
+            "description": f"Translate '{row['source_text']}' to {row['target_language']}",
+        }
+        test_cases.append(test_case)
+
+    return test_cases