wehub-resource-sync
63 lines
2.6 KiB
Markdown
63 lines
2.6 KiB
Markdown
# Turn detector plugin for LiveKit Agents
|
|
|
|
> ⚠️ **Deprecated.** This plugin is deprecated and will be removed in a future release. Use [`livekit.agents.inference.TurnDetector`](https://docs.livekit.io/agents/build/turns/turn-detector/) instead — it ships with `livekit-agents`, requires no additional install, and replaces both the English and Multilingual text-based models with a unified audio end-of-turn detector.
|
|
|
|
This plugin introduces end-of-turn detection for LiveKit Agents using a custom open-weight model to determine when a user has finished speaking.
|
|
|
|
Traditional voice agents use VAD (voice activity detection) for end-of-turn detection. However, VAD models lack language understanding, often causing false positives where the agent interrupts the user before they finish speaking.
|
|
|
|
By leveraging a language model specifically trained for this task, this plugin offers a more accurate and robust method for detecting end-of-turns.
|
|
|
|
See [https://docs.livekit.io/agents/build/turns/turn-detector/](https://docs.livekit.io/agents/build/turns/turn-detector/) for more information.
|
|
|
|
## Usage
|
|
|
|
The recommended replacement is `TurnDetector`, available from `livekit-agents` directly:
|
|
|
|
```python
|
|
from livekit.agents.inference import TurnDetector
|
|
|
|
session = AgentSession(
|
|
...
|
|
turn_detection=TurnDetector(),
|
|
)
|
|
```
|
|
|
|
### Usage with RealtimeModel
|
|
|
|
`TurnDetector` works with speech-to-speech models such as OpenAI's Realtime API:
|
|
|
|
```python
|
|
session = AgentSession(
|
|
...
|
|
llm=openai.realtime.RealtimeModel(),
|
|
turn_detection=TurnDetector(),
|
|
)
|
|
```
|
|
|
|
## Running your agent
|
|
|
|
This plugin requires model files. Before starting your agent for the first time, or when building Docker images for deployment, run the following command to download the model files:
|
|
|
|
```bash
|
|
python -m livekit.agents download-files
|
|
```
|
|
|
|
## Downloaded model files
|
|
|
|
Model files are downloaded to and loaded from the location specified by the `HF_HUB_CACHE` environment variable. If not set, this defaults to `$HF_HOME/hub` (typically `~/.cache/huggingface/hub`).
|
|
|
|
For offline deployment, download the model files first while connected to the internet, then copy the cache directory to your deployment environment.
|
|
|
|
## Model system requirements
|
|
|
|
The end-of-turn model is optimized to run on CPUs with modest system requirements. It is designed to run on the same server hosting your agents.
|
|
|
|
The model requires <500MB of RAM and runs within a shared inference server, supporting multiple concurrent sessions.
|
|
|
|
## License
|
|
|
|
The plugin source code is licensed under the Apache-2.0 license.
|
|
|
|
The end-of-turn model is licensed under the [LiveKit Model License](https://huggingface.co/livekit/turn-detector/blob/main/LICENSE).
|