chore: import upstream snapshot with attribution
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,166 @@
|
||||
# provider-elevenlabs/alignment (ElevenLabs Forced Alignment)
|
||||
|
||||
Generate time-aligned subtitles (SRT/VTT) from audio and transcripts using ElevenLabs forced alignment.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
npx promptfoo@latest init --example provider-elevenlabs/alignment
|
||||
cd provider-elevenlabs/alignment
|
||||
export ELEVENLABS_API_KEY=your_api_key_here
|
||||
npx promptfoo@latest eval
|
||||
```
|
||||
|
||||
## What this tests
|
||||
|
||||
- **Subtitle generation**: Create SRT and VTT subtitle files
|
||||
- **Word-level alignment**: Precise timestamp data for each word
|
||||
- **Multiple formats**: JSON (raw data), SRT (video players), VTT (web players)
|
||||
- **Accuracy**: Verify alignment matches audio timing
|
||||
|
||||
## How it works
|
||||
|
||||
Forced alignment takes two inputs:
|
||||
|
||||
1. **Audio file**: Speech recording (MP3, WAV, etc.)
|
||||
2. **Transcript**: Text of what was spoken
|
||||
|
||||
It returns precise timestamps showing when each word was spoken, formatted as subtitles.
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **Video subtitles**: Generate SRT files for video editing software
|
||||
- **Web captions**: Create VTT files for HTML5 video players
|
||||
- **Karaoke apps**: Word-level timing for synchronized highlighting
|
||||
- **Accessibility**: Auto-generate captions for spoken content
|
||||
- **Translation sync**: Time-align translations to original audio
|
||||
|
||||
## Output Formats
|
||||
|
||||
### JSON (Raw alignment data)
|
||||
|
||||
```json
|
||||
{
|
||||
"alignment": [
|
||||
{ "char": "T", "start": 0.0, "end": 0.1 },
|
||||
{ "char": "h", "start": 0.1, "end": 0.15 }
|
||||
],
|
||||
"characters": "That's one small step..."
|
||||
}
|
||||
```
|
||||
|
||||
### SRT (Standard video subtitles)
|
||||
|
||||
```text
|
||||
1
|
||||
00:00:00,000 --> 00:00:02,500
|
||||
That's one small step for man
|
||||
|
||||
2
|
||||
00:00:02,500 --> 00:00:05,000
|
||||
one giant leap for mankind
|
||||
```
|
||||
|
||||
### VTT (WebVTT for web players)
|
||||
|
||||
```text
|
||||
WEBVTT
|
||||
|
||||
1
|
||||
00:00:00.000 --> 00:00:02.500
|
||||
That's one small step for man
|
||||
|
||||
2
|
||||
00:00:02.500 --> 00:00:05.000
|
||||
one giant leap for mankind
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Basic alignment (JSON output)
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- id: elevenlabs:alignment:json
|
||||
label: Alignment (JSON)
|
||||
|
||||
tests:
|
||||
- vars:
|
||||
audioFile: path/to/audio.mp3
|
||||
transcript: 'Your transcript text here'
|
||||
format: json
|
||||
```
|
||||
|
||||
### SRT subtitles
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- id: elevenlabs:alignment:srt
|
||||
label: Alignment (SRT Subtitles)
|
||||
|
||||
tests:
|
||||
- vars:
|
||||
audioFile: path/to/audio.mp3
|
||||
transcript: 'Your transcript text here'
|
||||
format: srt
|
||||
```
|
||||
|
||||
### VTT subtitles
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- id: elevenlabs:alignment:vtt
|
||||
label: Alignment (VTT Subtitles)
|
||||
|
||||
tests:
|
||||
- vars:
|
||||
audioFile: path/to/audio.mp3
|
||||
transcript: 'Your transcript text here'
|
||||
format: vtt
|
||||
```
|
||||
|
||||
## Testing Assertions
|
||||
|
||||
```yaml
|
||||
tests:
|
||||
# Verify alignment succeeds
|
||||
- assert:
|
||||
- type: javascript
|
||||
value: output.includes('words') # JSON format
|
||||
- type: not-contains
|
||||
value: error
|
||||
|
||||
# Verify SRT format
|
||||
- assert:
|
||||
- type: javascript
|
||||
value: output.includes('-->') && output.includes('small step')
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Transcript accuracy**: Ensure transcript exactly matches spoken audio
|
||||
2. **Include punctuation**: Better subtitle chunking and timing
|
||||
3. **Audio quality**: Clear audio produces more accurate timestamps
|
||||
4. **Format selection**:
|
||||
- Use SRT for video editing (Premiere, Final Cut, DaVinci)
|
||||
- Use VTT for web players (HTML5 `<video>` tag)
|
||||
- Use JSON for custom processing
|
||||
|
||||
## Cost Information
|
||||
|
||||
Forced alignment pricing is based on audio duration:
|
||||
|
||||
- ~$0.05 per minute of audio
|
||||
|
||||
The provider automatically tracks costs in evaluation results.
|
||||
|
||||
## Related Examples
|
||||
|
||||
- [ElevenLabs STT](../stt/) - Speech-to-text transcription
|
||||
- [ElevenLabs Isolation](../isolation/) - Audio noise removal
|
||||
|
||||
## Resources
|
||||
|
||||
- [ElevenLabs Alignment API Docs](https://elevenlabs.io/docs/api-reference/alignment)
|
||||
- [SRT Format Specification](https://en.wikipedia.org/wiki/SubRip)
|
||||
- [WebVTT Format Guide](https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API)
|
||||
@@ -0,0 +1,54 @@
|
||||
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
|
||||
description: ElevenLabs Forced Alignment - Subtitle generation
|
||||
|
||||
# Alignment uses audio files + transcripts - pass via vars
|
||||
prompts:
|
||||
- '{{transcript}}'
|
||||
|
||||
providers:
|
||||
# Basic alignment (JSON output)
|
||||
- id: elevenlabs:alignment:json
|
||||
label: Alignment (JSON)
|
||||
|
||||
# SRT subtitle format
|
||||
- id: elevenlabs:alignment:srt
|
||||
label: Alignment (SRT Subtitles)
|
||||
|
||||
# Default test configuration
|
||||
defaultTest:
|
||||
# All tests will require alignment to complete
|
||||
assert:
|
||||
- type: not-contains
|
||||
value: error
|
||||
|
||||
tests:
|
||||
- description: Align Armstrong moon landing speech
|
||||
vars:
|
||||
audioFile: examples/provider-elevenlabs/stt/audio/sample1.mp3
|
||||
transcript: "That's one small step for man, one giant leap for mankind."
|
||||
format: json
|
||||
assert:
|
||||
- type: javascript
|
||||
value: output.includes('words')
|
||||
- type: not-contains
|
||||
value: error
|
||||
|
||||
- description: Align Armstrong to SRT format
|
||||
vars:
|
||||
audioFile: examples/provider-elevenlabs/stt/audio/sample1.mp3
|
||||
transcript: "That's one small step for man, one giant leap for mankind."
|
||||
format: srt
|
||||
assert:
|
||||
- type: javascript
|
||||
value: output.includes('-->') && output.includes('small step')
|
||||
|
||||
- description: Align sample2 hello message
|
||||
vars:
|
||||
audioFile: examples/provider-elevenlabs/stt/audio/sample2.wav
|
||||
transcript: "Hello. What's today's date? Could you please let me know?"
|
||||
format: json
|
||||
assert:
|
||||
- type: javascript
|
||||
value: output.includes('words')
|
||||
- type: not-contains
|
||||
value: error
|
||||
Reference in New Issue
Block a user