elizaos--eliza/plugins/plugin-edge-tts/AGENTS.md

@elizaos/plugin-edge-tts

Free text-to-speech synthesis for Eliza agents using Microsoft Edge TTS — no API key required.

Purpose / role

Registers a ModelType.TEXT_TO_SPEECH model handler that calls Microsoft's Edge TTS WebSocket service via the node-edge-tts npm package. The plugin auto-enables when ELIZA_CLOUD_PROVISIONED=1 or when config.features.tts is truthy. It is Node-only; the browser export is an unavailable-entry plugin shape that logs a warning.

The plugin is loaded by including @elizaos/plugin-edge-tts in the agent's plugin list, or automatically via the elizaOS auto-enable engine (checks auto-enable.ts).

Plugin surface

This plugin registers no actions, providers, evaluators, routes, or events. It registers one model handler:

• ModelType.TEXT_TO_SPEECH (src/index.ts) — accepts a plain string or an EdgeTTSParams object; returns a Buffer of MP3 (or other configured format) audio. Hard limits: non-empty text, max 5000 characters.

Exported symbols from src/index.ts:

• edgeTTSPlugin — the Plugin object; default export.
• synthesizeEdgeSpeech(text, overrides?) — standalone helper that synthesizes without an AgentRuntime; reads settings from environment only. Used by pre-agent server routes (e.g. onboarding TTS before an agent exists).
• EdgeTTSParams, EdgeTTSSettings — types.
• _test — internal helpers exposed for unit tests (resolveVoice, speedToRate, inferExtension, getEdgeTTSSettings).

Layout

plugins/plugin-edge-tts/
  src/
    index.ts            Main implementation: plugin object, model handler, helpers
    index.node.ts       Node entry — re-exports src/index.ts
    index.browser.ts    Browser-unavailable entry — plugin shape + warning log
  auto-enable.ts        Lightweight auto-enable check (env reads only, no full runtime import)
  index.ts              Barrel — re-exports src/index.ts
  index.node.ts         (root) re-exports src/index.node.ts
  index.browser.ts      (root) re-exports src/index.browser.ts
  __tests__/
    smoke.test.ts       Unit tests (voice mapping, rate conversion, settings validation)
    core-test-mock.ts   Mock AgentRuntime for tests
  build.ts              Bun.build() script (node ESM, browser ESM, node CJS targets)
  vitest.config.ts      Vitest config

Commands

Only scripts defined in this package's package.json:

bun run --cwd plugins/plugin-edge-tts build         # compile dist/ (node + browser)
bun run --cwd plugins/plugin-edge-tts dev           # hot-rebuild via bun --hot build.ts
bun run --cwd plugins/plugin-edge-tts test          # vitest unit suite
bun run --cwd plugins/plugin-edge-tts test:e2e      # live smoke via run-local-plugin-live-smoke.mjs
bun run --cwd plugins/plugin-edge-tts lint          # biome check --write --unsafe
bun run --cwd plugins/plugin-edge-tts lint:check    # biome check (read-only)
bun run --cwd plugins/plugin-edge-tts format        # biome format --write
bun run --cwd plugins/plugin-edge-tts typecheck     # tsgo --noEmit
bun run --cwd plugins/plugin-edge-tts clean         # rm dist .turbo tsconfig.tsbuildinfo

Config / env vars

All variables are optional. Resolution order: runtime.getSetting(key) → process.env[key] → default.

Variable	Default	Description
EDGE_TTS_VOICE	en-US-MichelleNeural	Voice ID; accepts Edge TTS IDs or OpenAI preset names (see below)
EDGE_TTS_LANG	en-US	BCP-47 language code
EDGE_TTS_OUTPUT_FORMAT	audio-24khz-48kbitrate-mono-mp3	Output format string passed to node-edge-tts
EDGE_TTS_RATE	(unset)	Speech rate (e.g. +10%, -5%)
EDGE_TTS_PITCH	(unset)	Pitch (e.g. +5Hz, -10Hz)
EDGE_TTS_VOLUME	(unset)	Volume (e.g. +20%, -10%)
EDGE_TTS_PROXY	(unset)	HTTP proxy URL for the TTS WebSocket connection
EDGE_TTS_TIMEOUT_MS	30000	Request timeout in milliseconds

OpenAI-style voice name aliases resolved in resolveVoice() (src/index.ts): alloy→en-US-GuyNeural, echo→en-US-ChristopherNeural, fable→en-GB-RyanNeural, onyx→en-US-DavisNeural, nova→en-US-JennyNeural, shimmer→en-US-AriaNeural.

Auto-enable triggers (auto-enable.ts, also duplicated inline in the plugin object):

• ELIZA_CLOUD_PROVISIONED=1 in env, OR
• config.features.tts === true (or object with enabled !== false).

How to extend

This plugin has a single responsibility (TTS model handler). The typical extension points are:

Change the model handler behaviour — edit generateSpeech() and the ModelType.TEXT_TO_SPEECH handler in src/index.ts. Both the params object and settings struct are typed; keep them in sync.

Add a new voice preset alias — add an entry to VOICE_PRESETS in src/index.ts (lowercase key → Edge TTS voice ID string).

Add a provider or action to this plugin — add the implementation file under src/, import it in src/index.ts, and add it to the edgeTTSPlugin object's providers or actions array. Follow the root AGENTS.md architecture rules.

Add a test — add a TestSuite entry to the tests array inside edgeTTSPlugin in src/index.ts, or add a vitest file under __tests__/. Use core-test-mock.ts for a minimal IAgentRuntime mock.

Conventions / gotchas

• Node-only. The browser build (src/index.browser.ts) exports a browser-unavailable plugin shape and warning log. Do not add Node.js file system or WebSocket code to the browser entry point.
• Temp file I/O. node-edge-tts writes audio to a temp file (via mkdtempSync); the plugin reads it back and cleans up in a finally block. Cleanup failure is logged at warn/debug level but must not mask the audio result.
• 5000-character limit. Enforced explicitly before calling the TTS service. The upstream service has its own practical limit near this value; errors above it are opaque network failures.
• Type declarations. node-edge-tts ships its own TypeScript declarations in its dist/ folder (edge-tts.d.ts, drm.d.ts). No hand-written type declarations are needed for this package.
• synthesizeEdgeSpeech passes null as the runtime to getEdgeTTSSettings, so it reads only from process.env. Do not call it inside an agent handler where a runtime is available — use runtime.useModel(ModelType.TEXT_TO_SPEECH, ...) instead.
• Triple build targets. build.ts produces dist/node/ (ESM), dist/browser/ (ESM), and dist/cjs/ (CJS) bundles. The exports map in package.json selects the right bundle per environment. Keep index.node.ts and index.browser.ts as thin re-exports; all synthesis logic lives in src/index.ts and the browser boundary lives in src/index.browser.ts.

⛔ NON-NEGOTIABLE — evidence, trajectories & real end-to-end tests

The binding, repo-wide standard is AGENTS.md. Read it. Nothing in this package is done until it is proven done — a reviewer must confirm it works without reading the code, from the artifacts you attach. This applies to every feature, fix, refactor, and chore here. "Tests pass" is not proof; "CI is green" is not proof.

• Record AND read model trajectories. Capture the actual inputs and outputs of the model from a live LLM — not the deterministic proxy, not a mock: the prompt, the providers/context, the raw model output, every tool/action call, and the result. Then open the trajectory and review it by hand. A captured-but-unread trajectory is not evidence (packages/scenario-runner/bin/eliza-scenarios run <scenario> --report <out>).
• Real, full-featured E2E — no larp. Every feature ships detailed end-to-end tests that drive the real path end to end. Not the happy "front door" only: cover error paths, edge/empty/invalid input, concurrency, roles/permissions, and adversarial input. A test that asserts against a mock/stub/fixture standing in for the thing under test does not count. If the real model/device/chain/connector/account is hard to reach, make it reachable — that is the work, not an excuse to mock. If the existing tests here are shallow or mocked, fixing them is part of your change.
• Screenshots + logs at every phase, plus a complete walkthrough video/run-through of the entire feature or view, start to finish (bun run test:e2e:record).
• Manually review every artifact the change touches — never just the green check: client logs (console + network), server logs ([ClassName] …), the model trajectories in and out, before/after full-page screenshots, and the domain artifacts listed below for this package.
• No residuals. No shortcuts. The goal is not "done" — it is everything done. Clear every blocker by the hard path: build the real architecture, stand up the real model/device/service, actually test it. Never leave a TODO, a stub, a stepping-stone, or a "follow-up." When unsure, research thoroughly, weigh the options, and ship the best, highest-effort, production-ready version. Keep going until every possibility is exhausted.

Artifacts → attached inline in the PR (MP4 video, JPG screenshots, logs in <details>); attach each evidence type or explicitly mark it N/A with a reason — never leave it blank. If develop moved and changed behavior, re-capture evidence; stale proof is worse than none.

Capture & manually review for this package — voice / audio:

• Captured audio of the real round-trip (STT in, TTS out) plus the transcript, with a narrated walkthrough of what is happening.
• Latency, barge-in/interruption, and wake-word behavior measured on real audio — across platforms, not Linux-x64-synthetic only (see #9958).
• The model trajectory for any LLM turn inside the loop.
• Failure paths: no mic, silence, noise, overlapping speech, network drop mid-stream.