chore: import upstream snapshot with attribution

examples/provider-elevenlabs/README.md

@@ -0,0 +1,12 @@
+# provider-elevenlabs (ElevenLabs)
+
+Examples for using promptfoo with [ElevenLabs](https://elevenlabs.io/) audio APIs.
+
+## Examples
+
+- [agents](./agents/) - Evaluate conversational AI agents with multi-turn interactions
+- [alignment](./alignment/) - Forced alignment for word-level timestamps in subtitle formats
+- [isolation](./isolation/) - Audio noise removal and background isolation
+- [stt](./stt/) - Speech-to-text transcription with speaker diarization
+- [tts](./tts/) - Text-to-speech with model comparison and streaming
+- [tts-advanced](./tts-advanced/) - Advanced TTS: pronunciation dictionaries, voice design, voice remixing

examples/provider-elevenlabs/agents/README.md

@@ -0,0 +1,141 @@
+# provider-elevenlabs/agents (ElevenLabs Conversational Agents)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example provider-elevenlabs/agents
+cd provider-elevenlabs/agents
+```
+
+Test and evaluate ElevenLabs voice AI agents with multi-turn conversations.
+
+## What this tests
+
+- **Agent conversation quality**: Multi-turn dialogue handling
+- **Evaluation criteria**: Greeting, understanding, accuracy, helpfulness
+- **Simulated user behavior**: Automated conversation testing
+- **Tool usage**: Agent tool calls and responses
+- **Cost and latency** metrics
+
+## Setup
+
+Set your ElevenLabs API key:
+
+```bash
+export ELEVENLABS_API_KEY=your_api_key_here
+```
+
+## Run the example
+
+```bash
+npx promptfoo@latest eval -c ./promptfooconfig.yaml
+```
+
+Or view in the UI:
+
+```bash
+npx promptfoo@latest eval -c ./promptfooconfig.yaml
+npx promptfoo@latest view
+```
+
+## What to look for
+
+1. **Conversation flow**: How well the agent maintains context across turns
+2. **Evaluation scores**: Automated grading on multiple criteria (0-1 scale)
+3. **Tool usage**: When and how the agent calls available tools
+4. **Response quality**: Agent's ability to understand and respond accurately
+5. **Cost tracking**: Per-conversation and per-turn costs
+
+## Conversation formats
+
+This example supports multiple input formats:
+
+### 1. Plain text (treated as first user message)
+
+```yaml
+prompts:
+  - 'Hello, I need help with my order'
+```
+
+### 2. Multi-line with role prefixes
+
+```yaml
+prompts:
+  - |
+    User: Hi, what's the weather like?
+    Agent: I'd be happy to help! Where are you located?
+    User: I'm in San Francisco
+```
+
+### 3. Structured JSON
+
+```yaml
+prompts:
+  - |
+    {
+      "turns": [
+        {"speaker": "user", "message": "Hello"},
+        {"speaker": "agent", "message": "Hi! How can I help?"},
+        {"speaker": "user", "message": "I need support"}
+      ]
+    }
+```
+
+## Agent configuration
+
+Customize the agent behavior:
+
+```yaml
+config:
+  agentConfig:
+    name: Customer Support Agent
+    prompt: You are a helpful, empathetic customer support agent...
+    firstMessage: Hi! I'm here to help. What can I do for you today?
+    language: en
+    voiceId: 21m00Tcm4TlvDq8ikWAM
+    llmModel: gpt-4o
+    temperature: 0.7
+    maxTokens: 500
+```
+
+## Evaluation criteria
+
+Common criteria presets available:
+
+- `greeting` - Professional greeting (weight: 0.8, threshold: 0.8)
+- `understanding` - Accurate intent understanding (weight: 1.0, threshold: 0.9)
+- `accuracy` - Correct information (weight: 1.0, threshold: 0.9)
+- `helpfulness` - Helpful responses (weight: 0.9, threshold: 0.8)
+- `professionalism` - Professional tone (weight: 0.7, threshold: 0.8)
+- `empathy` - Empathetic responses (weight: 0.8, threshold: 0.7)
+- `efficiency` - Concise responses (weight: 0.7, threshold: 0.7)
+- `resolution` - Problem resolution (weight: 1.0, threshold: 0.8)
+
+## Simulated user
+
+Configure the simulated user's behavior:
+
+```yaml
+simulatedUser:
+  prompt: Act as a customer who is frustrated but polite
+  temperature: 0.8
+  responseStyle: casual # concise | verbose | casual | formal
+```
+
+## Available tools
+
+Example tools for agents:
+
+- `get_weather` - Get current weather
+- `search_knowledge_base` - Search documentation
+- `create_ticket` - Create support ticket
+- `send_email` - Send email notification
+- `get_order_status` - Check order status
+- `schedule_callback` - Schedule callback
+- `transfer_agent` - Transfer to human agent
+
+## Learn more
+
+- [ElevenLabs Conversational AI Docs](https://elevenlabs.io/docs/conversational-ai)
+- [Agent Configuration Guide](https://elevenlabs.io/docs/conversational-ai/agents)
+- [Pricing](https://elevenlabs.io/pricing)

examples/provider-elevenlabs/agents/promptfooconfig.yaml

@@ -0,0 +1,197 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: ElevenLabs Agents - Conversational AI testing
+
+prompts:
+  # Simple greeting conversation
+  - 'Hello, I need help with my order'
+
+  # Multi-turn conversation
+  - |
+    User: Hi, what's the weather like today?
+    Agent: I'd be happy to help! Where are you located?
+    User: I'm in San Francisco
+    Agent: Let me check the weather in San Francisco for you.
+    User: Thanks!
+
+  # Customer support scenario
+  - |
+    User: My package hasn't arrived and it's been a week
+    Agent: I'm sorry to hear that. Let me help you track your package. Can you provide your order number?
+    User: It's ORDER-12345
+    Agent: Thank you. Let me look that up for you now.
+    User: Please hurry, I need it urgently
+
+  # Technical support scenario
+  - |
+    User: I'm having trouble logging into my account
+    Agent: I understand how frustrating that can be. Let's get you logged in. Are you getting an error message?
+    User: It says "invalid password" but I'm sure it's correct
+    Agent: Let's try resetting your password. I can send you a reset link.
+    User: Okay, please send it to my email
+
+providers:
+  # Basic agent with evaluation criteria
+  - id: elevenlabs:agents
+    label: Support Agent (GPT-5-mini)
+    config:
+      agentConfig:
+        name: Customer Support Agent
+        prompt: |
+          You are a helpful, empathetic customer support agent. Your goal is to:
+          - Greet customers warmly
+          - Listen carefully to understand their needs
+          - Provide accurate, helpful information
+          - Show empathy and professionalism
+          - Resolve issues efficiently
+        firstMessage: Hi! I'm here to help. What can I do for you today?
+        language: en
+        voiceId: 21m00Tcm4TlvDq8ikWAM
+        llmModel: gpt-5-mini
+        temperature: 0.7
+        maxTokens: 500
+      evaluationCriteria:
+        - name: greeting
+          description: Agent provides a friendly, professional greeting
+          weight: 0.8
+          passingThreshold: 0.8
+        - name: understanding
+          description: Agent accurately understands user intent and questions
+          weight: 1.0
+          passingThreshold: 0.9
+        - name: accuracy
+          description: Agent provides correct and accurate information
+          weight: 1.0
+          passingThreshold: 0.9
+        - name: helpfulness
+          description: Agent provides helpful, actionable responses
+          weight: 0.9
+          passingThreshold: 0.8
+        - name: professionalism
+          description: Agent maintains professional, courteous tone
+          weight: 0.7
+          passingThreshold: 0.8
+      simulatedUser:
+        prompt: Act as a customer who is polite but needs help
+        temperature: 0.7
+        responseStyle: casual
+      maxTurns: 10
+
+  # Agent with different LLM
+  - id: elevenlabs:agents
+    label: Support Agent (Claude)
+    config:
+      agentConfig:
+        name: Customer Support Agent
+        prompt: |
+          You are a helpful, empathetic customer support agent focused on
+          understanding customer needs and providing clear, accurate solutions.
+        firstMessage: Hello! How may I assist you today?
+        language: en
+        voiceId: 2EiwWnXFnvU5JabPnv8n
+        llmModel: claude-sonnet-4-6
+        temperature: 0.7
+        maxTokens: 500
+      evaluationCriteria:
+        - name: greeting
+          description: Agent provides a friendly, professional greeting
+          weight: 0.8
+          passingThreshold: 0.8
+        - name: understanding
+          description: Agent accurately understands user intent
+          weight: 1.0
+          passingThreshold: 0.9
+        - name: helpfulness
+          description: Agent provides helpful responses
+          weight: 0.9
+          passingThreshold: 0.8
+      maxTurns: 10
+
+  # Agent with tools
+  - id: elevenlabs:agents
+    label: Support Agent with Tools (GPT-5-mini)
+    config:
+      agentConfig:
+        name: Customer Support Agent with Tools
+        prompt: |
+          You are a customer support agent with access to tools.
+          Use the available tools when appropriate to help customers:
+          - Check order status
+          - Create support tickets
+          - Send notifications
+        firstMessage: Hi! I can help you with orders, support tickets, and more. What do you need?
+        language: en
+        voiceId: 21m00Tcm4TlvDq8ikWAM
+        llmModel: gpt-5-mini
+        temperature: 0.7
+        tools:
+          - name: get_order_status
+            description: Get the current status of an order
+            parameters:
+              type: object
+              properties:
+                order_id:
+                  type: string
+                  description: The order ID or number
+              required:
+                - order_id
+          - name: create_ticket
+            description: Create a support ticket for the customer
+            parameters:
+              type: object
+              properties:
+                title:
+                  type: string
+                  description: Brief title describing the issue
+                description:
+                  type: string
+                  description: Detailed description of the issue
+                priority:
+                  type: string
+                  enum:
+                    - low
+                    - medium
+                    - high
+                    - urgent
+              required:
+                - title
+                - description
+      evaluationCriteria:
+        - name: understanding
+          description: Agent understands when to use tools
+          weight: 1.0
+          passingThreshold: 0.8
+        - name: tool_usage
+          description: Agent uses tools appropriately
+          weight: 1.0
+          passingThreshold: 0.7
+        - name: helpfulness
+          description: Agent provides helpful responses
+          weight: 0.9
+          passingThreshold: 0.8
+      maxTurns: 10
+
+tests:
+  - description: Conversation completes successfully
+    assert:
+      - type: javascript
+        value: output.includes('conversation_id') || output.includes('history')
+
+  - description: Evaluation criteria are met
+    assert:
+      - type: javascript
+        value: |
+          const result = JSON.parse(output);
+          const analysis = result.analysis;
+          if (!analysis || !analysis.evaluation_criteria_results) return false;
+          const passedCriteria = analysis.evaluation_criteria_results.filter(r => r.passed);
+          return passedCriteria.length >= 2;
+
+  - description: Cost is reasonable
+    assert:
+      - type: cost
+        threshold: 0.50
+
+  - description: Response time is acceptable
+    assert:
+      - type: latency
+        threshold: 30000

examples/provider-elevenlabs/alignment/README.md

@@ -0,0 +1,166 @@
+# provider-elevenlabs/alignment (ElevenLabs Forced Alignment)
+
+Generate time-aligned subtitles (SRT/VTT) from audio and transcripts using ElevenLabs forced alignment.
+
+## Quick Start
+
+```bash
+npx promptfoo@latest init --example provider-elevenlabs/alignment
+cd provider-elevenlabs/alignment
+export ELEVENLABS_API_KEY=your_api_key_here
+npx promptfoo@latest eval
+```
+
+## What this tests
+
+- **Subtitle generation**: Create SRT and VTT subtitle files
+- **Word-level alignment**: Precise timestamp data for each word
+- **Multiple formats**: JSON (raw data), SRT (video players), VTT (web players)
+- **Accuracy**: Verify alignment matches audio timing
+
+## How it works
+
+Forced alignment takes two inputs:
+
+1. **Audio file**: Speech recording (MP3, WAV, etc.)
+2. **Transcript**: Text of what was spoken
+
+It returns precise timestamps showing when each word was spoken, formatted as subtitles.
+
+## Use Cases
+
+- **Video subtitles**: Generate SRT files for video editing software
+- **Web captions**: Create VTT files for HTML5 video players
+- **Karaoke apps**: Word-level timing for synchronized highlighting
+- **Accessibility**: Auto-generate captions for spoken content
+- **Translation sync**: Time-align translations to original audio
+
+## Output Formats
+
+### JSON (Raw alignment data)
+
+```json
+{
+  "alignment": [
+    { "char": "T", "start": 0.0, "end": 0.1 },
+    { "char": "h", "start": 0.1, "end": 0.15 }
+  ],
+  "characters": "That's one small step..."
+}
+```
+
+### SRT (Standard video subtitles)
+
+```text
+1
+00:00:00,000 --> 00:00:02,500
+That's one small step for man
+
+2
+00:00:02,500 --> 00:00:05,000
+one giant leap for mankind
+```
+
+### VTT (WebVTT for web players)
+
+```text
+WEBVTT
+
+1
+00:00:00.000 --> 00:00:02.500
+That's one small step for man
+
+2
+00:00:02.500 --> 00:00:05.000
+one giant leap for mankind
+```
+
+## Configuration
+
+### Basic alignment (JSON output)
+
+```yaml
+providers:
+  - id: elevenlabs:alignment:json
+    label: Alignment (JSON)
+
+tests:
+  - vars:
+      audioFile: path/to/audio.mp3
+      transcript: 'Your transcript text here'
+      format: json
+```
+
+### SRT subtitles
+
+```yaml
+providers:
+  - id: elevenlabs:alignment:srt
+    label: Alignment (SRT Subtitles)
+
+tests:
+  - vars:
+      audioFile: path/to/audio.mp3
+      transcript: 'Your transcript text here'
+      format: srt
+```
+
+### VTT subtitles
+
+```yaml
+providers:
+  - id: elevenlabs:alignment:vtt
+    label: Alignment (VTT Subtitles)
+
+tests:
+  - vars:
+      audioFile: path/to/audio.mp3
+      transcript: 'Your transcript text here'
+      format: vtt
+```
+
+## Testing Assertions
+
+```yaml
+tests:
+  # Verify alignment succeeds
+  - assert:
+      - type: javascript
+        value: output.includes('words') # JSON format
+      - type: not-contains
+        value: error
+
+  # Verify SRT format
+  - assert:
+      - type: javascript
+        value: output.includes('-->') && output.includes('small step')
+```
+
+## Best Practices
+
+1. **Transcript accuracy**: Ensure transcript exactly matches spoken audio
+2. **Include punctuation**: Better subtitle chunking and timing
+3. **Audio quality**: Clear audio produces more accurate timestamps
+4. **Format selection**:
+   - Use SRT for video editing (Premiere, Final Cut, DaVinci)
+   - Use VTT for web players (HTML5 `<video>` tag)
+   - Use JSON for custom processing
+
+## Cost Information
+
+Forced alignment pricing is based on audio duration:
+
+- ~$0.05 per minute of audio
+
+The provider automatically tracks costs in evaluation results.
+
+## Related Examples
+
+- [ElevenLabs STT](../stt/) - Speech-to-text transcription
+- [ElevenLabs Isolation](../isolation/) - Audio noise removal
+
+## Resources
+
+- [ElevenLabs Alignment API Docs](https://elevenlabs.io/docs/api-reference/alignment)
+- [SRT Format Specification](https://en.wikipedia.org/wiki/SubRip)
+- [WebVTT Format Guide](https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API)

examples/provider-elevenlabs/alignment/promptfooconfig.yaml

@@ -0,0 +1,54 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: ElevenLabs Forced Alignment - Subtitle generation
+
+# Alignment uses audio files + transcripts - pass via vars
+prompts:
+  - '{{transcript}}'
+
+providers:
+  # Basic alignment (JSON output)
+  - id: elevenlabs:alignment:json
+    label: Alignment (JSON)
+
+  # SRT subtitle format
+  - id: elevenlabs:alignment:srt
+    label: Alignment (SRT Subtitles)
+
+# Default test configuration
+defaultTest:
+  # All tests will require alignment to complete
+  assert:
+    - type: not-contains
+      value: error
+
+tests:
+  - description: Align Armstrong moon landing speech
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample1.mp3
+      transcript: "That's one small step for man, one giant leap for mankind."
+      format: json
+    assert:
+      - type: javascript
+        value: output.includes('words')
+      - type: not-contains
+        value: error
+
+  - description: Align Armstrong to SRT format
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample1.mp3
+      transcript: "That's one small step for man, one giant leap for mankind."
+      format: srt
+    assert:
+      - type: javascript
+        value: output.includes('-->') && output.includes('small step')
+
+  - description: Align sample2 hello message
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample2.wav
+      transcript: "Hello. What's today's date? Could you please let me know?"
+      format: json
+    assert:
+      - type: javascript
+        value: output.includes('words')
+      - type: not-contains
+        value: error

examples/provider-elevenlabs/isolation/README.md

@@ -0,0 +1,195 @@
+# provider-elevenlabs/isolation (ElevenLabs Audio Isolation)
+
+Remove background noise from audio files to extract clean speech using ElevenLabs audio isolation.
+
+## Quick Start
+
+```bash
+npx promptfoo@latest init --example provider-elevenlabs/isolation
+cd provider-elevenlabs/isolation
+export ELEVENLABS_API_KEY=your_api_key_here
+npx promptfoo@latest eval
+```
+
+## What this tests
+
+- **Noise removal**: Extract clean speech from noisy audio
+- **Audio quality**: Compare original vs isolated audio size/quality
+- **Output formats**: MP3 at different bitrates (128kbps, 192kbps)
+- **Cost tracking**: Monitor per-file processing costs
+
+## How it works
+
+Audio isolation takes a noisy audio file and returns a cleaned version with:
+
+- Background noise removed
+- Speech preserved and enhanced
+- Consistent audio quality
+- Reduced file size (noise-free)
+
+## Use Cases
+
+- **Podcast cleanup**: Remove background noise from recordings
+- **Interview enhancement**: Clean up phone/video call audio
+- **STT preprocessing**: Improve transcription accuracy
+- **Voiceover repair**: Fix audio recorded in noisy environments
+- **Call center QA**: Enhance customer service call recordings
+
+## Supported Formats
+
+**Input formats**: MP3, WAV, FLAC, OGG, M4A, OPUS, WebM
+
+**Output formats**:
+
+- `mp3_44100_128` - Standard quality (128kbps)
+- `mp3_44100_192` - High quality (192kbps)
+- `pcm_44100` - Uncompressed PCM
+
+## Configuration
+
+### Basic isolation (MP3)
+
+```yaml
+providers:
+  - id: elevenlabs:isolation:basic
+    label: Audio Isolation (MP3)
+    config:
+      outputFormat: mp3_44100_128
+
+tests:
+  - vars:
+      audioFile: path/to/noisy-audio.mp3
+```
+
+### High quality output
+
+```yaml
+providers:
+  - id: elevenlabs:isolation:hq
+    label: Audio Isolation (HQ)
+    config:
+      outputFormat: mp3_44100_192
+```
+
+## Testing Assertions
+
+```yaml
+tests:
+  - description: Verify isolation succeeds
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample1.mp3
+    assert:
+      - type: javascript
+        value: output.includes('isolated successfully')
+      - type: not-contains
+        value: error
+      - type: cost
+        threshold: 1.00 # Max $1.00 per file
+```
+
+## What to look for in results
+
+The response includes:
+
+- **Isolated audio**: Base64-encoded cleaned audio file
+- **Original size**: Size of input audio file
+- **Isolated size**: Size of cleaned audio (typically smaller)
+- **Format**: Output audio format (mp3, pcm, etc.)
+- **Latency**: Processing time in milliseconds
+- **Cost**: Estimated processing cost
+
+## Best Practices
+
+1. **Source quality**: Use highest quality source audio available
+2. **Noise type**: Works best with constant background noise (AC, fan, hum)
+3. **Multiple speakers**: Preserves all speech, removes only noise
+4. **Pre-processing**: For music removal, consider manual editing first
+
+## Audio Quality Comparison
+
+Isolation typically provides:
+
+- **Signal-to-Noise Ratio (SNR)**: 15-25 dB improvement
+- **File size reduction**: 10-30% smaller (noise-free)
+- **Speech clarity**: Enhanced intelligibility
+- **Frequency response**: Preserved natural voice tones
+
+## Cost Information
+
+Audio isolation pricing is based on audio duration:
+
+- ~$0.10 per minute of audio
+- Free tier: Varies by plan
+
+The provider automatically tracks costs in evaluation results.
+
+## Common Issues
+
+### "Audio isolated successfully" but quality unchanged
+
+**Possible causes:**
+
+- Source audio already very clean
+- Noise level too low to detect
+- Music/speech mixed with background (try manual editing)
+
+**Solution**: Check original audio quality - isolation works best with noisy recordings.
+
+### Large file processing timeout
+
+**Solution**: Increase timeout in config:
+
+```yaml
+config:
+  timeout: 180000 # 3 minutes
+```
+
+### Unsupported audio format
+
+**Solution**: Convert to supported format using ffmpeg:
+
+```bash
+ffmpeg -i input.video -vn -acodec mp3 output.mp3
+```
+
+## Pipeline Integration
+
+### Isolation → Transcription
+
+```yaml
+# Step 1: Isolate noisy audio
+providers:
+  - id: elevenlabs:isolation
+
+# Step 2: Transcribe cleaned audio
+providers:
+  - id: elevenlabs:stt
+    config:
+      audioFile: '{{previousOutput.audio}}'
+```
+
+### Isolation → Alignment
+
+```yaml
+# Step 1: Clean audio
+providers:
+  - id: elevenlabs:isolation
+
+# Step 2: Generate subtitles from clean audio
+providers:
+  - id: elevenlabs:alignment
+    config:
+      audioFile: '{{previousOutput.audio}}'
+      transcript: 'Your transcript here'
+```
+
+## Related Examples
+
+- [ElevenLabs STT](../stt/) - Transcribe audio to text
+- [ElevenLabs Alignment](../alignment/) - Generate subtitles
+
+## Resources
+
+- [ElevenLabs Audio Isolation Docs](https://elevenlabs.io/docs/api-reference/audio-isolation)
+- [Audio Quality Metrics](https://en.wikipedia.org/wiki/Audio_quality_measurement)
+- [Noise Reduction Techniques](https://en.wikipedia.org/wiki/Noise_reduction)

examples/provider-elevenlabs/isolation/promptfooconfig.yaml

@@ -0,0 +1,50 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: ElevenLabs Audio Isolation - Background noise removal
+
+# Isolation uses audio files as inputs - pass via vars
+prompts:
+  - '{{audioFile}}'
+
+providers:
+  # Basic audio isolation (noise removal)
+  - id: elevenlabs:isolation:basic
+    label: Audio Isolation (MP3)
+    config:
+      outputFormat: mp3_44100_128
+
+  # High quality output
+  - id: elevenlabs:isolation:hq
+    label: Audio Isolation (HQ)
+    config:
+      outputFormat: mp3_44100_192
+
+# Default test configuration
+defaultTest:
+  # All tests will require isolation to complete
+  assert:
+    - type: not-contains
+      value: error
+
+tests:
+  - description: Isolate speech from sample1 (Armstrong)
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample1.mp3
+    assert:
+      - type: javascript
+        value: output.includes('isolated successfully')
+      - type: cost
+        threshold: 1.00
+
+  - description: Isolate speech from sample2 (Hello message)
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample2.wav
+    assert:
+      - type: javascript
+        value: output.includes('isolated successfully')
+
+  - description: Isolate speech from sample3 (Kennedy)
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample3_multiple_speakers.mp3
+    assert:
+      - type: javascript
+        value: output.includes('isolated successfully')

examples/provider-elevenlabs/stt/README.md

@@ -0,0 +1,404 @@
+# provider-elevenlabs/stt (ElevenLabs Speech-to-Text)
+
+This example demonstrates how to use ElevenLabs STT provider for audio transcription testing.
+
+## Quick Start
+
+```bash
+npx promptfoo@latest init --example provider-elevenlabs/stt
+cd provider-elevenlabs/stt
+export ELEVENLABS_API_KEY=your_api_key_here
+npx promptfoo@latest eval
+```
+
+## Features
+
+- **Audio Transcription**: Convert speech to text with high accuracy
+- **Speaker Diarization**: Identify and separate multiple speakers in audio
+- **Word Error Rate (WER)**: Measure transcription accuracy against reference text
+- **Multi-format Support**: MP3, WAV, FLAC, M4A, OGG, OPUS, WebM
+
+## Setup
+
+1. **Set your API key**:
+
+   ```bash
+   export ELEVENLABS_API_KEY=your_api_key_here
+   ```
+
+2. **Prepare audio files**:
+   Create an `audio/` directory with your test audio files:
+
+   ```bash
+   mkdir -p audio
+   # Place your audio files in the audio/ directory
+   ```
+
+3. **Run the evaluation**:
+   ```bash
+   promptfoo eval
+   ```
+
+## Configuration
+
+### Basic Transcription
+
+```yaml
+providers:
+  - id: elevenlabs:stt:basic
+    config:
+      modelId: eleven_speech_to_text_v1
+      language: en # ISO 639-1 language code
+```
+
+### Speaker Diarization
+
+Identify and label different speakers in your audio:
+
+```yaml
+providers:
+  - id: elevenlabs:stt:diarization
+    config:
+      modelId: eleven_speech_to_text_v1
+      diarization: true
+      maxSpeakers: 3 # Optional: hint for expected number of speakers
+```
+
+The response will include speaker segments:
+
+```json
+{
+  "text": "Full transcription...",
+  "diarization": [
+    {
+      "speaker_id": "speaker_0",
+      "text": "Hello, how are you?",
+      "start_time_ms": 0,
+      "end_time_ms": 2500,
+      "confidence": 0.95
+    },
+    {
+      "speaker_id": "speaker_1",
+      "text": "I'm doing well, thanks!",
+      "start_time_ms": 2500,
+      "end_time_ms": 5000,
+      "confidence": 0.92
+    }
+  ]
+}
+```
+
+### Accuracy Testing with WER
+
+Word Error Rate (WER) measures transcription accuracy. Lower is better (0 = perfect).
+
+```yaml
+providers:
+  - id: elevenlabs:stt:accuracy
+    config:
+      modelId: eleven_speech_to_text_v1
+      calculateWER: true
+      referenceText: The quick brown fox jumps over the lazy dog
+```
+
+**WER Formula**: `(Substitutions + Deletions + Insertions) / Total Words`
+
+The response includes detailed WER metrics:
+
+```json
+{
+  "wer": 0.05, // 5% error rate
+  "substitutions": 1,
+  "deletions": 0,
+  "insertions": 0,
+  "correct": 19,
+  "totalWords": 20,
+  "details": {
+    "reference": "the quick brown fox jumps",
+    "hypothesis": "the quick green fox jumps",
+    "alignment": "REF: the quick brown fox jumps\nHYP: the quick green fox jumps\nOPS:           SSSSS"
+  }
+}
+```
+
+**WER Interpretation**:
+
+- **0.00 - 0.05**: Excellent (95%+ accurate)
+- **0.05 - 0.10**: Good (90-95% accurate)
+- **0.10 - 0.20**: Fair (80-90% accurate)
+- **0.20+**: Poor (< 80% accurate)
+
+## Supported Audio Formats
+
+| Format    | Extension  | Notes                      |
+| --------- | ---------- | -------------------------- |
+| MP3       | .mp3       | Widely compatible          |
+| MP4 Audio | .mp4, .m4a | AAC/MPEG-4 audio           |
+| WAV       | .wav       | Uncompressed, high quality |
+| FLAC      | .flac      | Lossless compression       |
+| OGG       | .ogg       | Open format                |
+| Opus      | .opus      | Modern, efficient codec    |
+| WebM      | .webm      | Web-optimized              |
+
+## Audio Input Methods
+
+### Method 1: Config-level
+
+```yaml
+providers:
+  - id: elevenlabs:stt
+    config:
+      audioFile: path/to/audio.mp3
+```
+
+### Method 2: Prompt-level
+
+```yaml
+prompts:
+  - audio/sample1.mp3
+  - audio/sample2.wav
+```
+
+### Method 3: Vars-level
+
+```yaml
+tests:
+  - vars:
+      audioFile: audio/sample.mp3
+```
+
+## Testing Assertions
+
+### Cost Threshold
+
+```yaml
+tests:
+  - assert:
+      - type: cost
+        threshold: 0.05 # Max $0.05 per transcription
+```
+
+### Latency Threshold
+
+```yaml
+tests:
+  - assert:
+      - type: latency
+        threshold: 10000 # Max 10 seconds
+```
+
+### Transcription Quality
+
+```yaml
+tests:
+  - assert:
+      - type: contains
+        value: expected phrase
+
+      - type: not-contains
+        value: incorrect phrase
+```
+
+### WER Threshold
+
+```yaml
+tests:
+  - assert:
+      - type: javascript
+        value: |
+          const wer = context.vars.metadata?.wer?.wer || 1;
+          wer < 0.1  // Less than 10% error
+```
+
+### Speaker Count
+
+```yaml
+tests:
+  - assert:
+      - type: javascript
+        value: |
+          const diarization = context.vars.metadata?.transcription?.diarization || [];
+          const uniqueSpeakers = new Set(diarization.map(s => s.speaker_id));
+          uniqueSpeakers.size === 2  // Expect 2 speakers
+```
+
+## Language Support
+
+ElevenLabs STT supports 30+ languages. Specify using ISO 639-1 codes:
+
+```yaml
+config:
+  language: en # English
+  # language: es  # Spanish
+  # language: fr  # French
+  # language: de  # German
+  # language: it  # Italian
+  # language: pt  # Portuguese
+  # language: ja  # Japanese
+  # language: ko  # Korean
+  # language: zh  # Chinese
+```
+
+**Auto-detection**: Omit `language` to let the API detect the language automatically.
+
+## Cost Information
+
+STT pricing is based on audio duration:
+
+- **Free tier**: 1 hour/month
+- **Paid tiers**: ~$0.10 per minute (~$0.00167 per second)
+
+The provider automatically tracks and reports costs in the evaluation results.
+
+## Advanced Usage
+
+### Batch Transcription
+
+```yaml
+prompts:
+  - audio/batch1.mp3
+  - audio/batch2.mp3
+  - audio/batch3.mp3
+
+providers:
+  - id: elevenlabs:stt
+    config:
+      modelId: eleven_speech_to_text_v1
+
+# Test all files with consistent assertions
+tests:
+  - assert:
+      - type: cost
+        threshold: 0.10
+      - type: latency
+        threshold: 15000
+```
+
+### Multi-language Testing
+
+```yaml
+providers:
+  - id: elevenlabs:stt:english
+    config:
+      language: en
+
+  - id: elevenlabs:stt:spanish
+    config:
+      language: es
+
+  - id: elevenlabs:stt:autodetect
+    config:
+      # No language specified = auto-detect
+
+prompts:
+  - audio/english_sample.mp3
+  - audio/spanish_sample.mp3
+```
+
+### Accuracy Comparison
+
+Compare transcription accuracy across different audio qualities:
+
+```yaml
+prompts:
+  - audio/high_quality_48khz.wav
+  - audio/medium_quality_16khz.mp3
+  - audio/low_quality_8khz.mp3
+
+providers:
+  - id: elevenlabs:stt
+    config:
+      calculateWER: true
+      referenceText: This is the expected transcription text
+
+tests:
+  - description: High quality should have WER < 5%
+    vars:
+      audioFile: audio/high_quality_48khz.wav
+    assert:
+      - type: javascript
+        value: (context.vars.metadata?.wer?.wer || 1) < 0.05
+
+  - description: Medium quality should have WER < 10%
+    vars:
+      audioFile: audio/medium_quality_16khz.mp3
+    assert:
+      - type: javascript
+        value: (context.vars.metadata?.wer?.wer || 1) < 0.10
+```
+
+## Troubleshooting
+
+### API Key Issues
+
+```bash
+# Verify your API key is set
+echo $ELEVENLABS_API_KEY
+
+# Or set it inline
+ELEVENLABS_API_KEY=your_key promptfoo eval
+```
+
+### Audio File Not Found
+
+```text
+Error: Failed to read audio file: ENOENT: no such file or directory
+```
+
+**Solution**: Use absolute paths or paths relative to the config file:
+
+```yaml
+prompts:
+  - /absolute/path/to/audio.mp3
+  - ./relative/path/to/audio.mp3
+```
+
+### Unsupported Format
+
+```text
+Error: Unsupported audio format
+```
+
+**Solution**: Convert your audio to a supported format (MP3, WAV, etc.) using tools like `ffmpeg`:
+
+```bash
+ffmpeg -i input.video -vn -acodec mp3 output.mp3
+```
+
+### High WER on Clear Audio
+
+If you're getting unexpectedly high WER:
+
+1. **Check reference text** - ensure it exactly matches the audio (including punctuation)
+2. **Specify language** - auto-detection may choose the wrong language
+3. **Audio quality** - ensure audio is clear with minimal background noise
+4. **Normalization** - WER calculation normalizes text (lowercase, removes punctuation)
+
+## API Reference
+
+### Config Options
+
+| Option          | Type    | Default                        | Description                    |
+| --------------- | ------- | ------------------------------ | ------------------------------ |
+| `modelId`       | string  | `eleven_speech_to_text_v1`     | STT model to use               |
+| `language`      | string  | auto-detect                    | ISO 639-1 language code        |
+| `diarization`   | boolean | `false`                        | Enable speaker identification  |
+| `maxSpeakers`   | number  | -                              | Expected number of speakers    |
+| `audioFile`     | string  | -                              | Path to audio file             |
+| `audioFormat`   | string  | auto-detect                    | Audio format override          |
+| `referenceText` | string  | -                              | Expected transcription for WER |
+| `calculateWER`  | boolean | `false`                        | Calculate Word Error Rate      |
+| `baseUrl`       | string  | `https://api.elevenlabs.io/v1` | API endpoint                   |
+| `timeout`       | number  | `120000`                       | Request timeout (ms)           |
+| `retries`       | number  | `3`                            | Number of retry attempts       |
+
+## Related Examples
+
+- [ElevenLabs TTS](../tts/) - Text-to-Speech synthesis
+- [ElevenLabs Isolation](../isolation/) - Audio cleanup quality comparison
+
+## Resources
+
+- [ElevenLabs STT Documentation](https://elevenlabs.io/docs/speech-to-text)
+- [Supported Languages](https://elevenlabs.io/languages)
+- [Word Error Rate (WER)](https://en.wikipedia.org/wiki/Word_error_rate)

examples/provider-elevenlabs/stt/audio/.gitkeep

@@ -0,0 +1,15 @@
+# Place your audio files in this directory
+#
+# Supported formats:
+# - MP3 (.mp3)
+# - WAV (.wav)
+# - FLAC (.flac)
+# - M4A (.m4a)
+# - OGG (.ogg)
+# - Opus (.opus)
+# - WebM (.webm)
+#
+# Example files:
+# - sample1.mp3
+# - sample2.wav
+# - sample3_multiple_speakers.mp3

examples/provider-elevenlabs/stt/promptfooconfig.yaml

@@ -0,0 +1,63 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: ElevenLabs Speech-to-Text (STT) - Transcription accuracy testing
+
+# STT uses audio files as inputs - pass via vars
+prompts:
+  - '{{audioFile}}'
+
+providers:
+  # Basic transcription
+  - id: elevenlabs:stt:basic
+    label: Basic Transcription
+    config:
+      modelId: scribe_v1
+      language: en
+
+  # Transcription with speaker diarization
+  - id: elevenlabs:stt:diarization
+    label: With Speaker Diarization
+    config:
+      modelId: scribe_v1
+      language: en
+      diarization: true
+      maxSpeakers: 3
+
+  # Accuracy testing with WER calculation
+  - id: elevenlabs:stt:accuracy
+    label: Accuracy Testing (WER)
+    config:
+      modelId: scribe_v1
+      language: en
+      calculateWER: true
+      referenceText: The quick brown fox jumps over the lazy dog
+
+# Default test configuration
+defaultTest:
+  # All tests will require transcription to complete
+  assert:
+    - type: not-contains
+      value: error
+
+tests:
+  - description: Transcription of sample1
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample1.mp3
+    assert:
+      - type: javascript
+        value: output.length > 10
+      - type: cost
+        threshold: 0.10
+
+  - description: Transcription of sample2
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample2.wav
+    assert:
+      - type: javascript
+        value: output.length > 10
+
+  - description: Diarization test
+    vars:
+      audioFile: examples/provider-elevenlabs/stt/audio/sample3_multiple_speakers.mp3
+    assert:
+      - type: javascript
+        value: output.length > 10

examples/provider-elevenlabs/tts-advanced/README.md

@@ -0,0 +1,512 @@
+# provider-elevenlabs/tts-advanced (ElevenLabs Advanced TTS Features)
+
+This example demonstrates advanced TTS capabilities:
+
+- **Pronunciation Dictionaries** - Custom pronunciation for technical terms
+- **Voice Design** - Generate voices from text descriptions
+- **Voice Remixing** - Modify existing voices (style, pacing, gender, age)
+- **Streaming with Advanced Features** - Combine streaming with pronunciation control
+
+## Quick Start
+
+```bash
+npx promptfoo@latest init --example provider-elevenlabs/tts-advanced
+cd provider-elevenlabs/tts-advanced
+export ELEVENLABS_API_KEY=your_api_key_here
+npx promptfoo@latest eval
+```
+
+## Features Demonstrated
+
+### 1. Pronunciation Dictionaries
+
+Control how technical terms, acronyms, and brand names are pronounced.
+
+**Use Case**: Technical documentation, product demos, brand-specific content
+
+```yaml
+providers:
+  - id: elevenlabs:tts
+    config:
+      pronunciationRules:
+        # Spell out acronyms
+        - word: API
+          pronunciation: A-P-I
+
+        # Custom pronunciation
+        - word: SQL
+          pronunciation: sequel
+
+        # Multi-word terms
+        - word: PostgreSQL
+          pronunciation: post-gres-Q-L
+
+        # Brand names
+        - word: OpenAI
+          pronunciation: open-A-I
+```
+
+**Common Use Cases**:
+
+1. **Technical Content**
+
+   ```yaml
+   pronunciationRules:
+     - word: JavaScript
+       pronunciation: java-script
+     - word: TypeScript
+       pronunciation: type-script
+     - word: Python
+       pronunciation: pie-thon
+     - word: Node.js
+       pronunciation: node-jay-ess
+     - word: GraphQL
+       pronunciation: graph-Q-L
+   ```
+
+2. **Medical/Scientific Terms**
+
+   ```yaml
+   pronunciationRules:
+     - word: COVID-19
+       pronunciation: covid-nineteen
+     - word: mRNA
+       pronunciation: messenger-R-N-A
+     - word: DNA
+       pronunciation: D-N-A
+   ```
+
+3. **Brand Names & Products**
+   ```yaml
+   pronunciationRules:
+     - word: Anthropic
+       pronunciation: an-throw-pick
+     - word: Llama
+       pronunciation: lama
+     - word: ChatGPT
+       pronunciation: chat-G-P-T
+   ```
+
+### 2. Voice Design
+
+Generate custom voices from natural language descriptions.
+
+**Use Case**: Create unique voices for specific content types or brand identities
+
+```yaml
+providers:
+  - id: elevenlabs:tts
+    config:
+      voiceDesign:
+        description: A warm, professional voice with excellent clarity and a slight smile in the tone, perfect for technical documentation
+        gender: female
+        age: middle_aged
+        accent: american
+        accentStrength: 0.5 # 0-2, subtle to strong
+```
+
+**Voice Design Templates**:
+
+#### Professional Voices
+
+```yaml
+# Corporate Presenter
+voiceDesign:
+  description: A confident, authoritative voice with clear articulation, perfect for business presentations
+  gender: male
+  age: middle_aged
+  accent: american
+
+# Educational Instructor
+voiceDesign:
+  description: A warm, patient voice with excellent clarity, ideal for educational content
+  gender: female
+  age: middle_aged
+  accent: british
+```
+
+#### Friendly & Conversational
+
+```yaml
+# Customer Service
+voiceDesign:
+  description: A friendly, approachable voice with a smile in the tone, great for customer interactions
+  gender: female
+  age: young
+  accent: american
+
+# Podcast Host
+voiceDesign:
+  description: A casual, engaging voice with natural conversational flow, perfect for podcasts
+  gender: male
+  age: young
+  accent: australian
+```
+
+#### Narrative & Storytelling
+
+```yaml
+# Audiobook Narrator
+voiceDesign:
+  description: A deep, resonant voice with storytelling quality and emotional range
+  gender: male
+  age: middle_aged
+  accent: british
+
+# Meditation Guide
+voiceDesign:
+  description: A soothing, tranquil voice with calming tones and gentle pacing
+  gender: female
+  age: middle_aged
+  accent: american
+  accentStrength: 0.3
+```
+
+### 3. Voice Remixing
+
+Modify existing voices to change their characteristics.
+
+**Use Case**: Adapt pre-made voices for different contexts or emotions
+
+```yaml
+providers:
+  # Make a voice more energetic
+  - id: elevenlabs:tts:energetic
+    config:
+      voiceId: 21m00Tcm4TlvDq8ikWAM # Rachel
+      voiceRemix:
+        style: energetic
+        pacing: fast
+        promptStrength: medium # low, medium, high, max
+
+  # Make a voice calmer and slower
+  - id: elevenlabs:tts:calm
+    config:
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      voiceRemix:
+        style: calm
+        pacing: slow
+        promptStrength: high
+```
+
+**Remix Parameters**:
+
+| Parameter        | Options                                         | Use Case                      |
+| ---------------- | ----------------------------------------------- | ----------------------------- |
+| `style`          | energetic, calm, professional, casual, dramatic | Match voice to content mood   |
+| `pacing`         | slow, normal, fast                              | Adjust speech speed           |
+| `gender`         | male, female                                    | Change voice gender           |
+| `age`            | young, middle_aged, old                         | Adjust perceived age          |
+| `accent`         | american, british, australian, etc.             | Change accent                 |
+| `promptStrength` | low, medium, high, max                          | How strongly to apply changes |
+
+**Common Remix Scenarios**:
+
+```yaml
+# Sports Commentary (Energetic & Fast)
+voiceRemix:
+  style: energetic
+  pacing: fast
+  promptStrength: max
+
+# ASMR Content (Calm & Slow)
+voiceRemix:
+  style: calm
+  pacing: slow
+  promptStrength: high
+
+# News Anchor (Professional & Measured)
+voiceRemix:
+  style: professional
+  pacing: normal
+  promptStrength: medium
+
+# Storytelling (Dramatic & Expressive)
+voiceRemix:
+  style: dramatic
+  pacing: normal
+  promptStrength: high
+```
+
+## Advanced Combinations
+
+### Streaming + Pronunciation
+
+Combine real-time streaming with custom pronunciation:
+
+```yaml
+providers:
+  - id: elevenlabs:tts
+    config:
+      streaming: true
+      pronunciationRules:
+        - word: API
+          pronunciation: A-P-I
+        - word: WebSocket
+          pronunciation: web-socket
+```
+
+**Benefits**:
+
+- ~75ms first chunk latency
+- Custom pronunciation for technical terms
+- Ideal for live demos and interactive applications
+
+### Voice Design + Pronunciation
+
+Create a custom voice with domain-specific pronunciation:
+
+```yaml
+providers:
+  - id: elevenlabs:tts
+    config:
+      voiceDesign:
+        description: A friendly tech educator with clear pronunciation
+        gender: female
+        age: middle_aged
+      pronunciationRules:
+        - word: Python
+          pronunciation: pie-thon
+        - word: JavaScript
+          pronunciation: java-script
+```
+
+## Cost Optimization
+
+All advanced features use the same character-based pricing as basic TTS:
+
+- ~$0.00002 per character (~$0.02 per 1000 characters)
+- Free tier: 10,000 characters/month
+
+**Cost Tracking**:
+
+```yaml
+tests:
+  - assert:
+      - type: cost
+        threshold: 0.05 # Max $0.05 per test
+```
+
+## Testing Assertions
+
+### Pronunciation Accuracy
+
+```yaml
+tests:
+  - description: Verify tech terms are included
+    vars:
+      expectedTerms:
+        - API
+        - SQL
+        - JavaScript
+    assert:
+      - type: javascript
+        value: |
+          const terms = context.vars.expectedTerms;
+          terms.every(term => output.includes(term))
+```
+
+### Voice Quality Comparison
+
+```yaml
+tests:
+  - description: Compare baseline vs custom pronunciation
+    vars:
+      baseline: '{{providers[0].output}}'
+      custom: '{{providers[1].output}}'
+    assert:
+      - type: javascript
+        value: |
+          // Both should succeed
+          !context.vars.baseline.includes('error') &&
+          !context.vars.custom.includes('error')
+```
+
+### Latency with Advanced Features
+
+```yaml
+tests:
+  - description: Ensure advanced features don't slow generation
+    assert:
+      - type: latency
+        threshold: 8000 # 8 seconds max
+```
+
+## Real-World Use Cases
+
+### 1. Technical Documentation
+
+```yaml
+config:
+  voiceDesign:
+    description: Clear, professional voice for technical content
+    gender: female
+    age: middle_aged
+  pronunciationRules:
+    - word: API
+      pronunciation: A-P-I
+    - word: REST
+      pronunciation: rest
+    - word: GraphQL
+      pronunciation: graph-Q-L
+    - word: WebSocket
+      pronunciation: web-socket
+    - word: JSON
+      pronunciation: jay-sawn
+    - word: YAML
+      pronunciation: yam-mel
+```
+
+### 2. Brand-Specific Content
+
+```yaml
+config:
+  voiceId: your-brand-voice-id
+  voiceRemix:
+    style: professional
+    pacing: normal
+  pronunciationRules:
+    - word: YourProduct
+      pronunciation: your-product
+    - word: YourCompany
+      pronunciation: your-company
+```
+
+### 3. Multi-Language Support
+
+```yaml
+# English with British accent
+providers:
+  - id: elevenlabs:tts:en-gb
+    config:
+      voiceDesign:
+        description: British English speaker
+        accent: british
+        accentStrength: 1.5
+
+  # English with American accent
+  - id: elevenlabs:tts:en-us
+    config:
+      voiceDesign:
+        description: American English speaker
+        accent: american
+        accentStrength: 1.0
+```
+
+### 4. Dynamic Content Adaptation
+
+```yaml
+# Morning news (Energetic)
+providers:
+  - id: elevenlabs:tts:morning
+    config:
+      voiceId: news-anchor-voice
+      voiceRemix:
+        style: energetic
+        pacing: fast
+
+  # Evening news (Calm)
+  - id: elevenlabs:tts:evening
+    config:
+      voiceId: news-anchor-voice
+      voiceRemix:
+        style: calm
+        pacing: normal
+```
+
+## Troubleshooting
+
+### Voice Design Not Working
+
+```text
+Error: Voice design failed
+```
+
+**Solutions**:
+
+1. Ensure description is detailed (minimum 10 characters)
+2. Specify gender and age for better results
+3. Check API quota (voice design uses generation credits)
+
+### Pronunciation Not Applied
+
+```text
+Warning: Pronunciation dictionary not found
+```
+
+**Solutions**:
+
+1. Verify pronunciation rules syntax
+2. Ensure words match exactly (case-sensitive)
+3. Check that you're not using both `pronunciationDictionaryId` and `pronunciationRules`
+
+### Remix Changes Too Subtle
+
+```text
+Issue: Voice sounds the same after remix
+```
+
+**Solutions**:
+
+1. Increase `promptStrength` from medium to high or max
+2. Make more significant parameter changes
+3. Some voices have limited remix range - try a different base voice
+
+## API Reference
+
+### Pronunciation Dictionary Options
+
+| Option                      | Type                  | Description                   |
+| --------------------------- | --------------------- | ----------------------------- |
+| `pronunciationRules`        | `PronunciationRule[]` | Array of pronunciation rules  |
+| `pronunciationDictionaryId` | string                | Use existing dictionary by ID |
+
+**PronunciationRule**:
+
+```typescript
+{
+  word: string;           // Word to customize
+  pronunciation: string;  // Phonetic pronunciation
+  phoneme?: string;       // IPA/CMU phoneme (advanced)
+  alphabet?: 'ipa' | 'cmu';  // Phonetic alphabet
+}
+```
+
+### Voice Design Options
+
+```typescript
+{
+  description: string;    // Natural language description
+  gender?: 'male' | 'female';
+  age?: 'young' | 'middle_aged' | 'old';
+  accent?: string;        // e.g., 'british', 'american'
+  accentStrength?: number;  // 0-2, default 1.0
+  sampleText?: string;    // Optional sample for preview
+}
+```
+
+### Voice Remix Options
+
+```typescript
+{
+  style?: string;         // e.g., 'energetic', 'calm'
+  pacing?: 'slow' | 'normal' | 'fast';
+  gender?: 'male' | 'female';
+  age?: 'young' | 'middle_aged' | 'old';
+  accent?: string;
+  promptStrength?: 'low' | 'medium' | 'high' | 'max';
+}
+```
+
+## Related Examples
+
+- [Basic TTS](../tts/) - Voice comparison and basic features
+- [STT](../stt/) - Speech-to-Text transcription
+- [Streaming TTS](../tts/#streaming) - Real-time voice generation
+
+## Resources
+
+- [ElevenLabs Voice Design Docs](https://elevenlabs.io/docs/voice-design)
+- [Pronunciation Dictionary Guide](https://elevenlabs.io/docs/pronunciation)
+- [Voice Remixing API](https://elevenlabs.io/docs/voice-remix)
+- [Supported Accents](https://elevenlabs.io/voice-library)

examples/provider-elevenlabs/tts-advanced/promptfooconfig.yaml

@@ -0,0 +1,119 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Advanced TTS with voice settings and output formats
+
+prompts:
+  - 'Welcome to ElevenLabs! This advanced example demonstrates various voice settings and output configurations.'
+  - 'The quick brown fox jumps over the lazy dog. Pack my box with five dozen liquor jugs.'
+  - |
+    In a world where technology evolves rapidly, artificial intelligence continues
+    to push the boundaries of what's possible in voice synthesis and natural language processing.
+
+providers:
+  # 1. Baseline with default settings
+  - id: elevenlabs:tts:rachel
+    label: Default Settings (Rachel)
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_128
+
+  # 2. High stability for consistent voice
+  - id: elevenlabs:tts:rachel
+    label: High Stability
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_128
+      voiceSettings:
+        stability: 0.9
+        similarity_boost: 0.75
+        style: 0.0
+        use_speaker_boost: true
+        speed: 1.0
+
+  # 3. Expressive with style
+  - id: elevenlabs:tts:rachel
+    label: Expressive Style
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_128
+      voiceSettings:
+        stability: 0.3
+        similarity_boost: 0.75
+        style: 0.8
+        use_speaker_boost: true
+        speed: 1.0
+
+  # 4. Fast paced
+  - id: elevenlabs:tts:rachel
+    label: Fast Speed
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_128
+      voiceSettings:
+        stability: 0.5
+        similarity_boost: 0.75
+        style: 0.0
+        use_speaker_boost: true
+        speed: 1.2
+
+  # 5. High quality output format
+  - id: elevenlabs:tts:rachel
+    label: High Quality (192kbps)
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_192
+
+  # 6. PCM format for further processing
+  - id: elevenlabs:tts:rachel
+    label: PCM Format
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: pcm_44100
+
+  # 7. Reproducible with seed
+  - id: elevenlabs:tts:rachel
+    label: Reproducible (Seed)
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_128
+      seed: 12345
+
+  # 8. Streaming mode
+  - id: elevenlabs:tts:rachel
+    label: Streaming Mode
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      streaming: true
+      outputFormat: mp3_44100_128
+
+# Default assertions for all tests
+defaultTest:
+  assert:
+    - type: javascript
+      value: output.includes('Generated') && output.includes('characters of speech')
+    - type: cost
+      threshold: 0.10
+
+tests:
+  - description: Voice settings test
+    assert:
+      - type: javascript
+        value: output.includes('Generated') && output.includes('characters of speech')
+      - type: cost
+        threshold: 0.05
+      - type: latency
+        threshold: 10000
+
+  - description: Output format and streaming test
+    assert:
+      - type: javascript
+        value: output.includes('characters')
+      - type: cost
+        threshold: 0.05

examples/provider-elevenlabs/tts/README.md

@@ -0,0 +1,84 @@
+# provider-elevenlabs/tts (ElevenLabs Text-to-Speech)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example provider-elevenlabs/tts
+cd provider-elevenlabs/tts
+```
+
+Test and compare ElevenLabs TTS models and voice settings.
+
+## What this tests
+
+- **Model comparison**: Flash v2.5, Turbo v2.5, Multilingual v2
+- **Streaming vs. non-streaming** performance
+- **Voice quality** across different text inputs
+- **Cost and latency** metrics
+
+## Setup
+
+Set your ElevenLabs API key:
+
+```bash
+export ELEVENLABS_API_KEY=your_api_key_here
+```
+
+## Run the example
+
+```bash
+npx promptfoo@latest eval -c ./promptfooconfig.yaml
+```
+
+Or view in the UI:
+
+```bash
+npx promptfoo@latest eval -c ./promptfooconfig.yaml
+npx promptfoo@latest view
+```
+
+## What to look for
+
+1. **Model differences**: Flash v2.5 has lowest latency (~200ms), Multilingual v2 best quality
+2. **Streaming benefits**: First chunk arrives in ~75ms for real-time feel
+3. **Cost tracking**: ~$0.02 per 1000 characters
+4. **Audio metadata**: Duration, size, format info in response
+
+## Available voices
+
+This example uses Rachel (21m00Tcm4TlvDq8ikWAM). Try other popular voices:
+
+- **Rachel**: Calm, clear female voice (default)
+- **Clyde**: Warm, grounded male voice (2EiwWnXFnvU5JabPnv8n)
+- **Drew**: Well-rounded male voice (29vD33N1CtxCmqQRPOHJ)
+- **Paul**: Casual male voice (5Q0t7uMcjvnagumLfvZi)
+
+## Voice settings
+
+Customize the voice output:
+
+```yaml
+voiceSettings:
+  stability: 0.5 # 0 (more variable) to 1 (more stable)
+  similarity_boost: 0.75 # 0 (low) to 1 (high)
+  style: 0.0 # 0 to 1 (only for v2 models)
+  use_speaker_boost: true # Enhance clarity
+  speed: 1.0 # 0.25 to 4.0
+```
+
+## Output formats
+
+Available formats:
+
+- `mp3_22050_32` - Smallest size, lower quality
+- `mp3_44100_128` - Balanced (default)
+- `mp3_44100_192` - High quality
+- `pcm_16000` - Raw PCM for processing
+- `pcm_44100` - High quality PCM
+- `ulaw_8000` - Phone quality
+
+## Learn more
+
+- [ElevenLabs TTS Documentation](https://elevenlabs.io/docs/api-reference/text-to-speech)
+- [Voice Library](https://elevenlabs.io/voice-library)
+- [Pricing](https://elevenlabs.io/pricing)

examples/provider-elevenlabs/tts/promptfooconfig.yaml

@@ -0,0 +1,65 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: ElevenLabs TTS - Voice comparison across models
+
+prompts:
+  - 'Welcome to ElevenLabs! Our AI voice technology delivers natural-sounding speech in over 30 languages.'
+  - 'The quick brown fox jumps over the lazy dog. How vexingly quick daft zebras jump!'
+  - |
+    In a world where technology evolves at lightning speed,
+    artificial intelligence has become the cornerstone of innovation.
+    From healthcare to entertainment, AI is reshaping every industry.
+
+providers:
+  # Flash v2.5 - Fastest, lowest latency
+  - id: elevenlabs:tts:rachel
+    label: Flash v2.5 (Rachel)
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_128
+      voiceSettings:
+        stability: 0.5
+        similarity_boost: 0.75
+        style: 0.0
+        use_speaker_boost: true
+        speed: 1.0
+
+  # Turbo v2.5 - High quality, fast
+  - id: elevenlabs:tts:rachel
+    label: Turbo v2.5 (Rachel)
+    config:
+      modelId: eleven_turbo_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_128
+
+  # Multilingual v2 - Best for non-English
+  - id: elevenlabs:tts:rachel
+    label: Multilingual v2 (Rachel)
+    config:
+      modelId: eleven_multilingual_v2
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      outputFormat: mp3_44100_128
+
+  # Streaming mode
+  - id: elevenlabs:tts:rachel
+    label: Flash v2.5 Streaming (Rachel)
+    config:
+      modelId: eleven_flash_v2_5
+      voiceId: 21m00Tcm4TlvDq8ikWAM
+      streaming: true
+      outputFormat: mp3_44100_128
+
+tests:
+  - description: Audio generation quality
+    assert:
+      - type: javascript
+        value: output.includes('Generated') && output.includes('characters of speech')
+      - type: cost
+        threshold: 0.01
+      - type: latency
+        threshold: 5000
+
+  - description: Character count and metadata
+    assert:
+      - type: javascript
+        value: output.includes('characters') && output.includes('speech')