426e9eeabd
Voice Workbench / headless workbench (mocked backends) (push) Has been cancelled
Voice Workbench / real acoustic lane (nightly, provisioned only) (push) Has been cancelled
ci / test (push) Has been cancelled
ci / lint-and-format (push) Has been cancelled
ci / build (push) Has been cancelled
ci / dev-startup (push) Has been cancelled
gitleaks / gitleaks (push) Has been cancelled
Markdown Links / Relative Markdown Links (push) Has been cancelled
Quality (Extended) / Homepage Build (PR smoke) (push) Has been cancelled
Quality (Extended) / Comment-only diff guard (push) Has been cancelled
Quality (Extended) / Format + Type Safety Ratchet (push) Has been cancelled
Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Has been cancelled
Quality (Extended) / Develop Gate (lint) (push) Has been cancelled
Chat shell gestures / Chat shell gesture + parity e2e (push) Has been cancelled
Cloud Gateway Discord / Test (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx @biomejs/biome check packages/lifeops-bench/src, benchmark-lint) (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests, benchmark-tests) (push) Has been cancelled
Build Agent Image / build-and-push (push) Has been cancelled
Dev Smoke / bun run dev onboarding chat (push) Has been cancelled
Dev Smoke / Vite HMR dependency-level smoke (push) Has been cancelled
Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Has been cancelled
Publish @elizaos/example-code / check_npm (push) Has been cancelled
Publish @elizaos/example-code / publish_npm (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / verify_version (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / publish_npm (push) Has been cancelled
Sandbox Live Smoke / Sandbox live smoke (push) Has been cancelled
Snap Build & Test / Build Snap (amd64) (push) Has been cancelled
Snap Build & Test / Build Snap (arm64) (push) Has been cancelled
Test Packaging / elizaos CLI global-install smoke (node + bun) (push) Has been cancelled
Cloud Gateway Webhook / Test (push) Has been cancelled
Cloud Tests / lint-and-types (push) Has been cancelled
Cloud Tests / unit-tests (push) Has been cancelled
Cloud Tests / integration-tests (push) Has been cancelled
Cloud Tests / e2e-tests (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Deploy Apps Worker (Product 2) / Determine environment (push) Has been cancelled
Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Has been cancelled
Deploy Eliza Provisioning Worker / Determine environment (push) Has been cancelled
Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Has been cancelled
Dev Smoke / Classify changed paths (push) Has been cancelled
supply-chain / sbom (push) Has been cancelled
supply-chain / vulnerability-scan (push) Has been cancelled
Build, Push & Deploy to Phala Cloud / build-and-push (push) Has been cancelled
Test Packaging / Validate Packaging Configs (push) Has been cancelled
Test Packaging / Build & Test PyPI Package (push) Has been cancelled
Test Packaging / PyPI on Python ${{ matrix.python }} (push) Has been cancelled
Test Packaging / Pack & Test JS Tarballs (push) Has been cancelled
UI Fixture E2E / ui-fixture-e2e (push) Has been cancelled
UI Fixture E2E / fixture-e2e (push) Has been cancelled
UI Story Gate / story-gate (push) Has been cancelled
vault-ci / test (macos-latest) (push) Has been cancelled
vault-ci / test (ubuntu-latest) (push) Has been cancelled
vault-ci / test (windows-latest) (push) Has been cancelled
vault-ci / app-core wiring tests (push) Has been cancelled
verify-patches / verify patches/CHECKSUMS.sha256 (push) Has been cancelled
Voice Benchmark Smoke / voice-emotion fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voiceagentbench fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench-quality unit smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench TypeScript unit (no audio) (push) Has been cancelled
Voice Benchmark Smoke / voice bench smoke summary (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Has been cancelled
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Has been cancelled
394 lines
9.8 KiB
Plaintext
394 lines
9.8 KiB
Plaintext
---
|
|
title: Voice & TTS
|
|
description: Text-to-speech and voice cloning on elizaOS Cloud.
|
|
---
|
|
|
|
# Voice & TTS
|
|
|
|
Text-to-speech generation and voice cloning capabilities.
|
|
|
|
<Info>
|
|
**API Key Support:** `/api/v1/voice/*` endpoints support both session-based authentication and API key authentication.
|
|
Legacy `/api/elevenlabs/*` endpoints are session-based only.
|
|
</Info>
|
|
|
|
## Endpoint Patterns
|
|
|
|
Voice APIs are available at two paths:
|
|
|
|
| Pattern | Description | Auth | Use Case |
|
|
|---------|-------------|------|----------|
|
|
| `/api/v1/voice/*` | **Recommended** - Generic, provider-agnostic endpoints | Session or API key | New integrations, programmatic access |
|
|
| `/api/elevenlabs/*` | Legacy endpoints (still supported) | Session only | Existing integrations, backwards compatibility |
|
|
|
|
API key authentication is available only on `/api/v1/voice/*`. Legacy `/api/elevenlabs/*` endpoints require session-based auth and do not accept API keys.
|
|
|
|
---
|
|
|
|
## Text to Speech
|
|
|
|
<div className="api-endpoint">
|
|
<span className="method-badge method-badge-post">POST</span>
|
|
<span className="path">/api/v1/voice/tts</span>
|
|
</div>
|
|
|
|
<Note>
|
|
Legacy path: `/api/elevenlabs/tts` (still supported for backwards compatibility)
|
|
</Note>
|
|
|
|
Convert text to speech audio using premium AI voices.
|
|
|
|
<Tabs>
|
|
<Tab title="cURL">
|
|
|
|
```bash
|
|
curl -X POST "https://elizacloud.ai/api/v1/voice/tts" \
|
|
-H "X-API-Key: YOUR_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"text": "Hello, this is a test of the text to speech system.",
|
|
"voiceId": "21m00Tcm4TlvDq8ikWAM",
|
|
"modelId": "eleven_multilingual_v2"
|
|
}' \
|
|
--output speech.mp3
|
|
```
|
|
</Tab>
|
|
<Tab title="JavaScript">
|
|
|
|
```javascript
|
|
const response = await fetch('https://elizacloud.ai/api/v1/voice/tts', {
|
|
method: 'POST',
|
|
headers: {
|
|
'X-API-Key': 'YOUR_API_KEY',
|
|
'Content-Type': 'application/json',
|
|
},
|
|
body: JSON.stringify({
|
|
text: 'Hello, this is a test.',
|
|
voiceId: '21m00Tcm4TlvDq8ikWAM',
|
|
modelId: 'eleven_multilingual_v2',
|
|
}),
|
|
});
|
|
|
|
const audioBlob = await response.blob();
|
|
const audioUrl = URL.createObjectURL(audioBlob);
|
|
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
|
|
```python
|
|
import requests
|
|
|
|
response = requests.post(
|
|
'https://elizacloud.ai/api/v1/voice/tts',
|
|
headers={
|
|
'X-API-Key': 'YOUR_API_KEY',
|
|
'Content-Type': 'application/json',
|
|
},
|
|
json={
|
|
'text': 'Hello, this is a test.',
|
|
'voiceId': '21m00Tcm4TlvDq8ikWAM',
|
|
'modelId': 'eleven_multilingual_v2',
|
|
}
|
|
)
|
|
|
|
with open('speech.mp3', 'wb') as f:
|
|
f.write(response.content)
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
### Parameters
|
|
|
|
| Parameter | Type | Required | Description |
|
|
| ------------------ | ------ | -------- | ------------------------------------------- |
|
|
| `text` | string | ✓ | Text to convert to speech (max 5000 chars) |
|
|
| `voiceId` | string | ✓ | Voice ID to use (see List Voices) |
|
|
| `modelId` | string | | Model ID. Default: `eleven_multilingual_v2` |
|
|
| `stability` | number | | Voice stability (0-1). Default: 0.5 |
|
|
| `similarity_boost` | number | | Voice similarity (0-1). Default: 0.75 |
|
|
|
|
### Available Models
|
|
|
|
| Model | Languages | Quality | Speed |
|
|
| ------------------------ | --------- | ------- | ------- |
|
|
| `eleven_multilingual_v2` | 29 | Highest | Medium |
|
|
| `eleven_turbo_v2_5` | 32 | High | Fast |
|
|
| `eleven_flash_v2_5` | 32 | High | Fast |
|
|
| `eleven_v3` | Multi | Highest | Medium |
|
|
|
|
<Info>
|
|
Voice pricing is refreshed separately from the text/image/video catalogs. TTS
|
|
is billed per character, STT per decoded audio duration, and voice cloning by
|
|
clone tier.
|
|
</Info>
|
|
|
|
### Response
|
|
|
|
Returns audio data as `audio/mpeg` stream. The `Content-Length` header indicates file size.
|
|
|
|
---
|
|
|
|
## Speech to Text
|
|
|
|
<div className="api-endpoint">
|
|
<span className="method-badge method-badge-post">POST</span>
|
|
<span className="path">/api/v1/voice/stt</span>
|
|
</div>
|
|
|
|
<Note>
|
|
Legacy path: `/api/elevenlabs/stt` (still supported for backwards compatibility)
|
|
</Note>
|
|
|
|
Transcribe audio to text.
|
|
|
|
### Request
|
|
|
|
Upload audio as `multipart/form-data`:
|
|
|
|
```bash
|
|
curl -X POST "https://elizacloud.ai/api/v1/voice/stt" \
|
|
-H "X-API-Key: YOUR_API_KEY" \
|
|
-F "audio=@recording.mp3"
|
|
```
|
|
|
|
### Response
|
|
|
|
Returns JSON with `transcript` and `duration_ms`.
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `transcript` | string | Transcribed text |
|
|
| `duration_ms` | number | Audio duration in milliseconds |
|
|
|
|
```json
|
|
{
|
|
"transcript": "Hello, this is a transcription test.",
|
|
"duration_ms": 3245
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## List Voices
|
|
|
|
<div className="api-endpoint">
|
|
<span className="method-badge method-badge-get">GET</span>
|
|
<span className="path">/api/v1/voice/list</span>
|
|
</div>
|
|
|
|
<Note>
|
|
Legacy path: `/api/elevenlabs/voices/user` (still supported for backwards compatibility)
|
|
</Note>
|
|
|
|
Get your cloned voices with pagination and filtering.
|
|
|
|
### Query Parameters
|
|
|
|
| Parameter | Type | Description |
|
|
|-----------|------|-------------|
|
|
| `includeInactive` | boolean | Include inactive voices (default: false) |
|
|
| `cloneType` | string | Filter by `instant` or `professional` |
|
|
| `limit` | number | Results per page (default: 50, max: 100) |
|
|
| `offset` | number | Pagination offset |
|
|
|
|
### Response
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"voices": [
|
|
{
|
|
"id": "123e4567-e89b-12d3-a456-426614174000",
|
|
"elevenlabsVoiceId": "xyz789",
|
|
"name": "My Custom Voice",
|
|
"description": "A professional voice clone",
|
|
"cloneType": "instant",
|
|
"sampleCount": 3,
|
|
"usageCount": 150,
|
|
"isActive": true,
|
|
"createdAt": "2024-01-15T10:30:00Z"
|
|
}
|
|
],
|
|
"total": 5,
|
|
"limit": 50,
|
|
"offset": 0,
|
|
"hasMore": false
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Clone Voice
|
|
|
|
<div className="api-endpoint">
|
|
<span className="method-badge method-badge-post">POST</span>
|
|
<span className="path">/api/v1/voice/clone</span>
|
|
</div>
|
|
|
|
<Note>
|
|
Legacy path: `/api/elevenlabs/voices/clone` (still supported for backwards compatibility)
|
|
</Note>
|
|
|
|
Create a voice clone from audio samples.
|
|
|
|
### Request
|
|
|
|
Upload audio samples as `multipart/form-data`:
|
|
|
|
```bash
|
|
curl -X POST "https://elizacloud.ai/api/v1/voice/clone" \
|
|
-H "X-API-Key: YOUR_API_KEY" \
|
|
-F "name=My Voice" \
|
|
-F "cloneType=instant" \
|
|
-F "file0=@sample1.mp3" \
|
|
-F "file1=@sample2.mp3"
|
|
```
|
|
|
|
### Response
|
|
|
|
```json
|
|
{
|
|
"id": "123e4567-e89b-12d3-a456-426614174000",
|
|
"name": "My Voice",
|
|
"status": "processing"
|
|
}
|
|
```
|
|
|
|
<Info>
|
|
**Voice Cloning Tips:** - Provide 1-5 minutes of clear audio for best results
|
|
- Use high-quality recordings with minimal background noise - Speaking clearly
|
|
and at a natural pace produces better clones - Multiple samples in different
|
|
contexts improve voice quality
|
|
</Info>
|
|
|
|
---
|
|
|
|
## Get Voice
|
|
|
|
<div className="api-endpoint">
|
|
<span className="method-badge method-badge-get">GET</span>
|
|
<span className="path">/api/v1/voice/{"{id}"}</span>
|
|
</div>
|
|
|
|
<Note>
|
|
Legacy path: `/api/elevenlabs/voices/{id}` (still supported for backwards compatibility)
|
|
</Note>
|
|
|
|
Get details for a specific voice by its internal UUID.
|
|
|
|
```bash
|
|
curl "https://elizacloud.ai/api/v1/voice/123e4567-e89b-12d3-a456-426614174000" \
|
|
-H "X-API-Key: YOUR_API_KEY"
|
|
```
|
|
|
|
---
|
|
|
|
## Update Voice
|
|
|
|
<div className="api-endpoint">
|
|
<span className="method-badge method-badge-patch">PATCH</span>
|
|
<span className="path">/api/v1/voice/{"{id}"}</span>
|
|
</div>
|
|
|
|
Update a voice's metadata.
|
|
|
|
```bash
|
|
curl -X PATCH "https://elizacloud.ai/api/v1/voice/123e4567-e89b-12d3-a456-426614174000" \
|
|
-H "X-API-Key: YOUR_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"name": "Updated Voice Name", "isActive": true}'
|
|
```
|
|
|
|
---
|
|
|
|
## Delete Voice
|
|
|
|
<div className="api-endpoint">
|
|
<span className="method-badge method-badge-delete">DELETE</span>
|
|
<span className="path">/api/v1/voice/{"{id}"}</span>
|
|
</div>
|
|
|
|
<Note>
|
|
Legacy path: `/api/elevenlabs/voices/{id}` (still supported for backwards compatibility)
|
|
</Note>
|
|
|
|
Delete a cloned voice from your account.
|
|
|
|
```bash
|
|
curl -X DELETE "https://elizacloud.ai/api/v1/voice/123e4567-e89b-12d3-a456-426614174000" \
|
|
-H "X-API-Key: YOUR_API_KEY"
|
|
```
|
|
|
|
### Response
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"message": "Voice deleted successfully"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Voice Cloning Jobs
|
|
|
|
<div className="api-endpoint">
|
|
<span className="method-badge method-badge-get">GET</span>
|
|
<span className="path">/api/v1/voice/jobs</span>
|
|
</div>
|
|
|
|
<Note>
|
|
Legacy path: `/api/elevenlabs/voices/jobs` (still supported for backwards compatibility)
|
|
</Note>
|
|
|
|
Check status of active voice cloning jobs.
|
|
|
|
### Response
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"jobs": [
|
|
{
|
|
"id": "job_xyz789",
|
|
"voiceName": "My Voice",
|
|
"jobType": "instant",
|
|
"status": "processing",
|
|
"progress": 50,
|
|
"createdAt": "2024-01-15T10:30:00Z"
|
|
}
|
|
],
|
|
"total": 1
|
|
}
|
|
```
|
|
|
|
### Status Values
|
|
|
|
| Status | Description |
|
|
| ------------ | ------------------------------------ |
|
|
| `pending` | Job is queued |
|
|
| `processing` | Voice clone is being generated |
|
|
| `completed` | Voice is ready to use |
|
|
| `failed` | Cloning failed (check audio quality) |
|
|
|
|
---
|
|
|
|
## Pricing
|
|
|
|
Voice pricing can vary by provider configuration and clone tier. See
|
|
[Billing & Credits](/cloud/billing) and the dashboard/API Explorer for current
|
|
rates.
|
|
|
|
---
|
|
|
|
## Error Handling
|
|
|
|
| Code | Error | Solution |
|
|
| ---- | -------------------- | --------------------------------------- |
|
|
| 400 | Invalid voice ID | Use a valid voice from List Voices |
|
|
| 400 | Text too long | Split text into chunks under 5000 chars |
|
|
| 402 | Insufficient credits | Add credits to your account |
|
|
| 404 | Voice not found | Voice may have been deleted |
|
|
| 429 | Rate limited | Wait and retry |
|