chore: import upstream snapshot with attribution

2026-07-13 13:24:08 +08:00
commit 0d3cb498a3
5438 changed files with 1316560 additions and 0 deletions
@@ -0,0 +1,45 @@
+# openai-audio (OpenAI Audio Example)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example openai-audio
+cd openai-audio
+```
+
+This example demonstrates how to use promptfoo to test OpenAI's audio capabilities using audio-capable models. The example focuses on audio input processing (speech-to-text) and audio output generation (speech-to-speech).
+
+## Quick Start
+
+You can initialize this example in a new directory using:
+
+```bash
+npx promptfoo@latest init --example openai-audio
+```
+
+This will create all necessary files and folder structure to get started quickly.
+
+## Setup
+
+1. Set your OpenAI API key as an environment variable:
+
+```bash
+export OPENAI_API_KEY=your-api-key-here
+```
+
+2. The example includes sample audio files in the `assets` directory:
+   - `Armstrong_Small_Step.mp3` - Neil Armstrong's moon landing speech
+   - `Kennedy_berliner.mp3` - JFK's "Ich bin ein Berliner" speech
+
+## Files
+
+- `promptfooconfig.yaml`: Configuration file defining the providers and tests
+- `audio-input.json`: JSON template for the audio input prompt
+
+## Running the Example
+
+From the root directory of promptfoo, run:
+
+```bash
+npx promptfoo eval -c examples/openai-audio/promptfooconfig.yaml
+```
@@ -0,0 +1,18 @@
+[
+  {
+    "role": "user",
+    "content": [
+      {
+        "type": "text",
+        "text": "{{question}}"
+      },
+      {
+        "type": "input_audio",
+        "input_audio": {
+          "data": "{{audio_file}}",
+          "format": "mp3"
+        }
+      }
+    ]
+  }
+]
@@ -0,0 +1,26 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Audio Input and Output Example
+prompts:
+  - file://audio-input.json
+
+providers:
+  - id: openai:chat:gpt-audio-1.5
+    config:
+      modalities: ['text', 'audio']
+  - id: openai:chat:gpt-audio-mini
+    config:
+      modalities: ['text']
+
+tests:
+  - vars:
+      question: What is in this recording?
+      audio_file: file://assets/Armstrong_Small_Step.mp3
+    assert:
+      - type: llm-rubric
+        value: Is a speech by Neil Armstrong about the moon landing
+  - vars:
+      question: Is there a funny double meaning in this JFK speech?
+      audio_file: file://assets/Kennedy_berliner.mp3
+    assert:
+      - type: llm-rubric
+        value: Is a speech by John F. Kennedy in Berlin about being a jelly donut