wehub-resource-sync
426e9eeabd
Benchmark Bridge Tests / benchmark (bunx @biomejs/biome check packages/lifeops-bench/src, benchmark-lint) (push) Waiting to run
Benchmark Bridge Tests / benchmark (bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests, benchmark-tests) (push) Waiting to run
Build Agent Image / build-and-push (push) Waiting to run
Chat shell gestures / Chat shell gesture + parity e2e (push) Waiting to run
ci / test (push) Waiting to run
ci / lint-and-format (push) Waiting to run
ci / build (push) Waiting to run
ci / dev-startup (push) Waiting to run
Cloud Gateway Discord / Test (push) Waiting to run
Cloud Gateway Webhook / Test (push) Waiting to run
Cloud Tests / lint-and-types (push) Waiting to run
Cloud Tests / unit-tests (push) Waiting to run
Cloud Tests / integration-tests (push) Waiting to run
Cloud Tests / e2e-tests (push) Blocked by required conditions
CodeQL Advanced / Analyze (javascript-typescript) (push) Waiting to run
Deploy Apps Worker (Product 2) / Determine environment (push) Waiting to run
Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Blocked by required conditions
Deploy Eliza Provisioning Worker / Determine environment (push) Waiting to run
Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Blocked by required conditions
Dev Smoke / Classify changed paths (push) Waiting to run
Dev Smoke / bun run dev onboarding chat (push) Blocked by required conditions
Dev Smoke / Vite HMR dependency-level smoke (push) Blocked by required conditions
Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Waiting to run
gitleaks / gitleaks (push) Waiting to run
Markdown Links / Relative Markdown Links (push) Waiting to run
Publish @elizaos/example-code / check_npm (push) Waiting to run
Publish @elizaos/example-code / publish_npm (push) Blocked by required conditions
Publish @elizaos/plugin-elizacloud / verify_version (push) Waiting to run
Publish @elizaos/plugin-elizacloud / publish_npm (push) Blocked by required conditions
Quality (Extended) / Homepage Build (PR smoke) (push) Waiting to run
Quality (Extended) / Comment-only diff guard (push) Waiting to run
Quality (Extended) / Format + Type Safety Ratchet (push) Waiting to run
Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Waiting to run
Quality (Extended) / Develop Gate (lint) (push) Waiting to run
Sandbox Live Smoke / Sandbox live smoke (push) Waiting to run
Snap Build & Test / Build Snap (amd64) (push) Waiting to run
Snap Build & Test / Build Snap (arm64) (push) Waiting to run
supply-chain / sbom (push) Waiting to run
supply-chain / vulnerability-scan (push) Waiting to run
Build, Push & Deploy to Phala Cloud / build-and-push (push) Waiting to run
Test Packaging / Validate Packaging Configs (push) Waiting to run
Test Packaging / PyPI on Python ${{ matrix.python }} (push) Waiting to run
Test Packaging / Pack & Test JS Tarballs (push) Waiting to run
Test Packaging / elizaos CLI global-install smoke (node + bun) (push) Waiting to run
UI Fixture E2E / ui-fixture-e2e (push) Waiting to run
UI Fixture E2E / fixture-e2e (push) Waiting to run
UI Story Gate / story-gate (push) Waiting to run
vault-ci / test (macos-latest) (push) Waiting to run
vault-ci / test (ubuntu-latest) (push) Waiting to run
vault-ci / test (windows-latest) (push) Waiting to run
vault-ci / app-core wiring tests (push) Waiting to run
verify-patches / verify patches/CHECKSUMS.sha256 (push) Waiting to run
Voice Benchmark Smoke / voice-emotion fixture smoke (push) Waiting to run
Voice Benchmark Smoke / voiceagentbench fixture smoke (push) Waiting to run
Voice Benchmark Smoke / voicebench-quality unit smoke (push) Waiting to run
Voice Benchmark Smoke / voicebench TypeScript unit (no audio) (push) Waiting to run
Voice Benchmark Smoke / voice bench smoke summary (push) Blocked by required conditions
Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Waiting to run
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Waiting to run
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Waiting to run
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Waiting to run
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Waiting to run
Test Packaging / Build & Test PyPI Package (push) Waiting to run
Voice Workbench / headless workbench (mocked backends) (push) Has been cancelled
Voice Workbench / real acoustic lane (nightly, provisioned only) (push) Has been cancelled
2.7 KiB
2.7 KiB
Voice RTT Benchmark
Provider-agnostic end-to-end latency harness for a conversational voice path:
Deepgram Flux turn detection, Cerebras gemma-4-31b streaming text generation,
and Cartesia Sonic 3.5 streaming speech synthesis.
The harness does not create an elizaOS runtime and does not call production
routes or UI. It measures the client-observable path directly through provider
adapters and emits a PR #15931-compatible trace shape: X-Eliza-Voice-Trace-Id
per turn plus Server-Timing components in the JSON artifact.
Measured Checkpoints
- acoustic/input end
- STT eager end and final turn
- chat admission
- Cerebras preforward
- first text token
- first speakable phrase
- Cartesia request
- first audio frame
- client playout simulation
- interruption and playout silence for the barge-in case
Run
# Deterministic no-key mode. Enforces gates.
bun run --cwd packages/benchmarks/voice-rtt bench:mock
# Write artifacts.
bun run --cwd packages/benchmarks/voice-rtt bench:mock -- --out=./results
# Opt-in live mode. Requires provider keys and corpus PCM files.
DEEPGRAM_API_KEY=... CEREBRAS_API_KEY=... CARTESIA_API_KEY=... \
bun run --cwd packages/benchmarks/voice-rtt bench:live -- --audio-dir=./audio
Live mode expects short.pcm, long.pcm, pause.pcm, and barge-in.pcm in
the supplied audio directory, encoded as 16 kHz signed 16-bit little-endian PCM.
The fixture corpus is fixed in fixtures/corpus.json; committed text is used
for deterministic mock timing and for expected reply lengths, not logged by
default.
Gates
Mock mode enforces:
- EOS to first audio P50
< 1000ms - EOS to first audio P95
< 1500ms - interruption to silence
< 300ms - zero audio frames accepted after interrupt silence
Live mode reports those gates as advisory unless --enforce-live-gates is
supplied.
Privacy
Artifacts redact transcripts and model replies by default. They include only
trace IDs, provider request IDs, lengths, timings, stage attribution, and gate
results. Use --unsafe-transcripts only for a local diagnostic run where the
artifact will not be shared.
Provider Contracts
Adapters implement SttAdapter, LlmAdapter, and TtsAdapter from
src/types.ts. Future OpenRouter or alternate provider adapters should return
the same timestamped contracts rather than changing scoring/reporting code.
The live adapter follows the documented provider APIs:
- Deepgram Flux turn-based audio:
wss://api.deepgram.com/v2/listen - Cerebras chat completions streaming:
POST /v1/chat/completions - Cartesia Text-to-Speech WebSocket:
/tts/websocket
Test
bun run --cwd packages/benchmarks/voice-rtt test
bun run --cwd packages/benchmarks/voice-rtt typecheck