chore: import upstream snapshot with attribution

2026-07-13 13:24:08 +08:00
commit 0d3cb498a3
5438 changed files with 1316560 additions and 0 deletions
@@ -0,0 +1,61 @@
+# config-csv-metadata (CSV Metadata Example)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example config-csv-metadata
+cd config-csv-metadata
+```
+
+## Usage
+
+This example demonstrates how to use metadata columns in CSV and Excel test files to organize and filter your test cases. You can see more details on how to build test cases from CSV in the [Promptfoo docs](https://www.promptfoo.dev/docs/configuration/test-cases/#csv-format)
+
+### Files
+
+- `promptfooconfig.yaml`: Configuration file with a simple prompt and provider setup
+- `tests.csv`: Test cases with metadata columns for categorization and filtering
+- `tests.xlsx`: Same test cases in Excel format (optional - requires `npm install read-excel-file`)
+
+### Metadata Columns
+
+The CSV/Excel file includes two types of metadata columns:
+
+1. `__metadata:category` - Single value metadata for broad categorization
+2. `__metadata:tags[]` - Array metadata for multiple tags, with comma separation and escape support
+
+### Running the Example
+
+Basic evaluation:
+
+```bash
+promptfoo eval
+```
+
+To use Excel file instead of CSV:
+
+```bash
+# First install the read-excel-file package
+npm install read-excel-file
+
+# Then run with Excel file
+promptfoo eval -c promptfooconfig.yaml -t file://tests.xlsx
+```
+
+Filter by category:
+
+```bash
+promptfoo eval --filter-metadata category=technology
+```
+
+Filter by tags:
+
+```bash
+promptfoo eval --filter-metadata tags=ai
+```
+
+Note that array metadata values (tags) support escaped commas. For example, "global\,warming" is treated as a single tag.
+
+### Expected Output
+
+The example includes basic assertions using the `__expected` column to verify outputs contain relevant keywords.
@@ -0,0 +1,12 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Loading test cases and metadata from CSV files
+
+prompts:
+  - 'Tell me about {{topic}}'
+
+providers:
+  - id: openai:gpt-4
+    config:
+      temperature: 0.7
+
+tests: file://tests.csv
@@ -0,0 +1,4 @@
+topic,__expected1,__expected2,__metadata:category,__metadata:tags[],__config:__expected:threshold,__config:__expected2:threshold
+"machine learning","contains: algorithm","llm-rubric:Does this describe {{topic}} accurately?","technology","ai,data science,algorithms",0.9,0.8
+"quantum physics","contains: quantum","llm-rubric:Does this describe {{topic}} accurately?","science","physics,advanced\,theory,quantum",0.85,0.75
+"climate change","contains: temperature","llm-rubric:Does this describe {{topic}} inaccurately?","environment","science,global\,warming,research",0.7,0.675