chore: import upstream snapshot with attribution

2026-07-13 13:28:58 +08:00
commit ba4be087d5
2316 changed files with 2668701 additions and 0 deletions
@@ -0,0 +1,311 @@
+# NeMo Voice Agent
+
+A fully open-source NVIDIA NeMo Voice Agent example demonstrating a simple way to combine NVIDIA NeMo STT/TTS service and HuggingFace LLM together into a conversational agent. Everything is open-source and deployed locally so you can have your own voice agent. Feel free to explore the code and see how different speech technologies can be integrated with LLMs to create a seamless conversation experience. 
+
+As of now, we only support English input and output, but more languages will be supported in the future.
+
+## 📋 Table of Contents
+- [✨ Key Features](#-key-features)
+- [💡 Upcoming Next](#-upcoming-next)
+- [📅 Latest Updates](#-latest-updates)
+- [🚀 Quick Start](#-quick-start)
+- [📑 Supported Models and Features](#-supported-models-and-features)
+  - [🤖 LLM](#-llm)
+  - [🎤 ASR](#-asr)
+  - [💬 Speaker Diarization](#-speaker-diarization)
+  - [🔉 TTS](#-tts)
+  - [🔄 Turn-taking](#-turn-taking)
+  - [🔧 Tool Calling](#-tool-calling)
+- [📝 Notes \& FAQ](#-notes--faq)
+- [☁️ NVIDIA NIM Services](#️-nvidia-nim-services)
+- [Acknowledgments](#acknowledgments)
+- [Contributing](#contributing)
+
+
+## ✨ Key Features
+
+- Open-source, local deployment, and flexible customization.
+- Allow users to talk to most LLMs from HuggingFace with configurable prompts. 
+- Streaming speech recognition with low latency and end-of-utterance detection.
+- Low latency TTS for fast audio response generation.
+- Speaker diarization up to 4 speakers in different user turns.
+- WebSocket server for easy deployment.
+- Tool calling for LLMs to use external tools and adjust its own behavior.
+
+
+## 💡 Upcoming Next
+- Evaluation tools and benchmarks for voice agents.
+- Better inference latency for Magpie TTS model.
+- Accuracy and robustness ASR model improvements.
+- Combine ASR and speaker diarization model to handle overlapping speech.
+
+
+## 📅 Latest Updates
+- 2026-01-26: Added support for [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) LLM model, and support for [magpie_tts_multilingual_357m](https://huggingface.co/nvidia/magpie_tts_multilingual_357m) TTS model.
+- 2025-12-31: Added examples for [tool calling](#tool-calling), such as changing the speaking speed, switching between male/female voices and British/American accents, and getting the current weather of a city. Diarization model is updated to [nvidia/diar_streaming_sortformer_4spk-v2.1](https://huggingface.co/nvidia/diar_streaming_sortformer_4spk-v2.1) with improved performance.
+- 2025-11-14: Added support for joint ASR and EOU detection with [Parakeet-realtime-eou-120m](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1) model.
+- 2025-10-10: Added support for [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) TTS model.
+- 2025-10-03: Add support for serving LLM with vLLM and auto-switch between vLLM and HuggingFace, add [nvidia/NVIDIA-Nemotron-Nano-9B-v2](https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2) as default LLM.
+- 2025-09-05: First release of NeMo Voice Agent.
+
+
+
+## 🚀 Quick Start
+
+### Hardware requirements
+
+- A computer with at least one GPU. At least 21GB VRAM is recommended for using 9B LLMs, and 13GB VRAM for 4B LLMs.
+- A microphone connected to the computer.
+- A speaker connected to the computer.
+
+### Install dependencies
+
+First, install or update the npm and node.js to the latest version, for example:
+
+```bash
+sudo apt-get update
+sudo apt-get install -y npm nodejs
+```
+
+or:
+
+```bash
+curl -fsSL https://fnm.vercel.app/install | bash
+. ~/.bashrc
+fnm use --install-if-missing 20
+```
+
+Second, create a new conda environment with the dependencies:
+
+```bash
+conda env create -f environment.yaml
+```
+
+Then you can activate the environment via `conda activate nemo-voice`.
+
+### Configure the server
+
+If you want to just try the default server config, you can skip this step.
+
+Edit the `server/server_configs/default.yaml` file to configure the server as needed, for example:
+- Changing the LLM and system prompt you want to use in `llm.model` and `llm.system_prompt`, by either putting a local path to a text file or the whole prompt string. See `server/example_prompts/` for examples to start with. 
+- Configure the LLM parameters, such as temperature, max tokens, etc. You may also need to change the HuggingFace or vLLM server parameters, depending on the LLM you are using. Please refer to the LLM's model page for details on the recommended parameters.
+- If you know whether you want to use vLLM or HuggingFace, you can set `llm.type` to `vllm` or `hf` to force using vLLM or HuggingFace, respectively. Otherwise, it will automatically switch between the two based on the model's support. Please also remember to update the parameters of the chosen backend as well, by referring to the LLM's model page.
+- Distribute different components to different GPUs if you have more than one.
+- Adjust VAD parameters for sensitivity and end-of-turn detection timeout.
+
+**If you want to access the server from a different machine, you need to change the `baseUrl` in `client/src/app.ts` to the actual ip address of the server machine.**
+
+
+
+### Start the server
+
+Open a terminal and run the server via:
+
+```bash
+NEMO_PATH=???  # Use your local NeMo path with the latest main branch from: https://github.com/NVIDIA-NeMo/NeMo
+export PYTHONPATH=$NEMO_PATH:$PYTHONPATH
+# export HF_TOKEN="hf_..."  # Use your own HuggingFace API token if needed, as some models may require.
+# export HF_HUB_CACHE="/path/to/your/huggingface/cache"  # change where HF cache is stored if you don't want to use the default cache
+# export SERVER_CONFIG_PATH="/path/to/your/server/config.yaml"  # change to the server config you want to use, otherwise it will use the default config in `server/server_configs/default.yaml`
+# export SERVER_PUBLIC_HOST="127.0.0.1"  # set this to the host/IP clients should use for the WebSocket server
+python ./server/server.py
+```
+
+### Launch the client
+In another terminal on the server machine, start the client via:
+
+```bash
+cd client
+npm install
+npm run dev
+```
+
+There should be a message in terminal showing the address and port of the client.
+
+### Connect to the client via browser
+
+Open the client via browser: `http://[YOUR MACHINE IP ADDRESS]:5173/` (or whatever address and port is shown in the terminal where the client was launched). 
+
+You can mute/unmute your microphone via the "Mute" button, and reset the LLM context history and speaker cache by clicking the "Reset" button. 
+
+**If using chrome browser, you need to add `http://[YOUR MACHINE IP ADDRESS]:5173/` to the allow list via `chrome://flags/#unsafely-treat-insecure-origin-as-secure`.** You may also need to restart the browser for the changes to take effect.
+
+If you want to use a different port for client connection, you can modify `client/vite.config.js` to change the `port` variable.
+
+## 📑 Supported Models and Features
+
+### 🤖 LLM
+
+Most LLMs from HuggingFace are supported. A few examples are:
+- [nvidia/NVIDIA-Nemotron-Nano-9B-v2](https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2) (default)
+    - Please use `server/server_configs/llm_configs/nemotron_nano_v2.yaml` as the server config.
+    - Tool calling is enabled for this model.
+- [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16)
+    - Please use `server/server_configs/llm_configs/nemotron_nano_v3.yaml` as the server config. It needs more than 60GB VRAM to host the model, thus the config by default is set to use tensor parallelism of 2. Expect additional 5GB for kv-cache and other components in the voice agent. To better monitor the vllm status, `start_vllm_on_init` is set to `false`, so that you can manually start the vllm server in another terminal via: 
+    ```bash
+        vllm serve nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 \
+            --trust-remote-code --max-num-seqs 1 --gpu-memory-utilization 0.8 --max-model-len 8192 \
+            --tensor-parallel-size 2 --enable-auto-tool-choice --tool-call-parser qwen3_coder --enable-prefix-caching \
+            --reasoning-parser-plugin server/parsers/nano_v3_reasoning_parser.py --reasoning-parser nano_v3
+    ```
+    - If you have a GPU with FP8 support, the VRAM requirement is reduced to about 30GB. You can switch to [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8) by modifying the llm config accordingly.
+    - Tool calling is enabled for this model.
+- [nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4)
+    - Can be used by modifying `server/server_configs/llm_configs/nemotron_nano_v3.yaml` accordingly.
+- [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
+    - Please use `server/server_configs/llm_configs/qwen2.5-7B.yaml` as the server config.
+- [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+    - Please use `server/server_configs/llm_configs/qwen3-8B.yaml` as the server config.
+    - Please use `server/server_configs/llm_configs/qwen3-8B_think.yaml` if you want to enable thinking mode.
+- [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)
+    - Please use `server/server_configs/llm_configs/llama3.1-8B-instruct.yaml` as the server config.
+    - Note that you need to get access to the model first, and specify `export HF_TOKEN="hf_..."` when launching the server.
+- [nvidia/Llama-3.1-Nemotron-Nano-8B-v1](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1) 
+- [nvidia/Nemotron-Mini-4B-Instruct](https://huggingface.co/nvidia/Nemotron-Mini-4B-Instruct)
+
+
+Please refer to the homepage of each model to configure the model parameters:
+- If `llm.type=hf`, please set `llm.generation_kwargs` and `llm.apply_chat_template_kwargs` in the server config as needed.
+- If `llm.type=vllm`, please set `llm.vllm_server_params` and `llm.vllm_generation_params`in the server config as needed.
+- If `llm.type=auto`, the server will first try to use vLLM, and if it fails, it will try to use HuggingFace. In this case, you need to make sure parameters for both backends are set properly.
+
+You can change the `llm.system_prompt` in `server/server_configs/default.yaml` to configure the behavior of the LLM, by either putting a local path to a text file or the whole prompt string. See `server/example_prompts/` for examples to start with.
+
+#### Thinking/reasoning Mode for LLMs
+
+A lot of LLMs support thinking/reasoning mode, which is useful for complex tasks, but it will create a long latency for the final answer. By default, we turn off the thinking/reasoning mode for all models for best latency.
+
+Different models may have different ways to support thinking/reasoning mode, please refer to the model's homepage for details on their thinking/reasoning mode support. Meanwhile, in many cases, they support enabling thinking/reasoning can be achieved by adding `/think` or `/no_think` to the end of the system prompt, and the thinking/reasoning content is wrapped by the tokens `["<think>", "</think>"]`. Some models may also support enabling thinking/reasoning by setting `llm.apply_chat_template_kwargs.enable_thinking=true/false` in the server config when `llm.type=hf`.
+
+If thinking/reasoning mode is enabled (e.g., in `server/server_configs/llm_configs/qwen3-8B_think.yaml`), the voice agent server will print out the thinking/reasoning content so that you can see the process of the LLM thinking and still have a smooth conversation experience. The thinking/reasoning content will not go through the TTS process, so you will only hear the final answer, and this is achieved by specifying the pair of thinking tokens `tts.think_tokens=["<think>", "</think>"]` in the server config.
+
+For vLLM server, if you specify `--reasoning_parser` in `vllm_server_params`, the thinking/reasoning content will be filtered out and does not show up in the output.
+
+### 🎤 ASR 
+
+We use [cache-aware streaming FastConformer](https://arxiv.org/abs/2312.17279) to transcribe the user's speech into text. While new models will be released soon, we use the existing English models for now:
+- [nvidia/parakeet_realtime_eou_120m-v1](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1) (default)
+  - This model supports EOU prediction and optimized for lowest latency, but does not support punctuation and capitalization.
+- [nvidia/nemotron-speech-streaming-en-0.6b](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b)
+  - This model has better ASR accuracy and supports punctuation and capitalization, but does not predict EOU.
+- [stt_en_fastconformer_hybrid_large_streaming_80ms](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_en_fastconformer_hybrid_large_streaming_80ms)
+- [nvidia/stt_en_fastconformer_hybrid_large_streaming_multi](https://huggingface.co/nvidia/stt_en_fastconformer_hybrid_large_streaming_multi)
+
+
+### 💬 Speaker Diarization
+
+Speaker diarization aims to distinguish different speakers in the input speech audio. We use [streaming Sortformer](http://arxiv.org/abs/2507.18446) to detect the speaker for each user turn. 
+
+As of now, we only support detecting 1 speaker per user turn, but different turns come from different speakers, with a maximum of 4 speakers in the whole conversation. 
+
+Currently supported models are:
+ - [nvidia/diar_streaming_sortformer_4spk-v2.1](https://huggingface.co/nvidia/diar_streaming_sortformer_4spk-v2.1) (default)
+ - [nvidia/diar_streaming_sortformer_4spk-v2](https://huggingface.co/nvidia/diar_streaming_sortformer_4spk-v2)
+
+Please note that in some circumstances, the diarization model might not work well in noisy environments, or it may confuse the speakers. In this case, you can disable the diarization by setting `diar.enabled` to `false` in `server/server_configs/default.yaml`.
+
+### 🔉 TTS
+
+Here are the supported TTS models:
+- [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) is a lightweight TTS model. This model is the default speech generation backend.
+    - Please use `server/server_configs/tts_configs/kokoro_82M.yaml` as the server config.
+- [FastPitch-HiFiGAN](https://huggingface.co/nvidia/tts_en_fastpitch) is an NVIDIA-NeMo TTS model. It only supports English output. 
+    - Please use `server/server_configs/tts_configs/nemo_fastpitch-hifigan.yaml` as the server config.
+- [magpie_tts_multilingual_357m](https://huggingface.co/nvidia/magpie_tts_multilingual_357m) is a multilingual TTS model.
+    - Please use `server/server_configs/tts_configs/magpie_tts_multilingual_357m.yaml` as the server config.
+We will support more TTS models in the future.
+
+
+### 🔄 Turn-taking
+
+As the new turn-taking prediction model is not yet released, we use the VAD-based turn-taking prediction for now. You can set the `vad.stop_secs` to the desired value in `server/server_configs/default.yaml` to control the amount of silence needed to indicate the end of a user's turn.
+
+Additionally, the voice agent supports ignoring back-channel phrases while the bot is talking, which means phrases such as "uh-huh", "yeah", "okay"  will not interrupt the bot while it's talking. To control the backchannel phrases to be used, you can set the `turn_taking.backchannel_phrases` in the server config to the desired list of phrases or a file path to a yaml file containing the list of phrases. By default, it will use the phrases in `server/backchannel_phrases.yaml`. Setting it to `null` will disable detecting backchannel phrases, and that the VAD will interrupt the bot immediately when the user starts speaking.
+
+
+### 🔧 Tool Calling
+
+We support tool calling for LLMs to use external tools (e.g., getting the current weather of a city) or adjust its own behavior (e.g., changing the speaking speed). Some example queries to try with the default server config:
+
+1. Getting the current weather of a city:
+   - "What's the weather in New York city?"
+   - "What's the weather in Paris?"
+   - "What's the weather in Paris, Texas, USA?"
+
+2. Changing the speaking speed of the voice agent:
+   - "Can you speak faster?"
+   - "Can you speak slower?"
+   - "Reset to the original speaking speed."
+   - "Speak twice as fast."
+   - "Speak half as slow."
+  
+3. Switching between British and American accents, and changing the gender of the voice:
+   - "Speak in British accent."
+   - "Switch to a male voice."
+   - "Switch to a female voice."
+   - "Reset to the original language and voice."
+
+Currently, tool calling is only supported for vLLM server and specific LLM models:
+- [nvidia/NVIDIA-Nemotron-Nano-9B-v2](https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2) (default)
+- [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16)
+  - Other quantized versions can also be used.
+- [nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4)
+
+More LLMs can be supported by referring to their documentation on how to enable tool calling in vLLM. Note that the system prompt may need to be tuned accordingly.
+
+More tools will be added later. However, if you cannot wait to hack and add your own tools, please read the following section.
+
+#### Adding new tools
+
+Additional tools can be added in two ways:
+- Adding a new [direct function](https://docs.pipecat.ai/guides/learn/function-calling#using-direct-functions-shorthand) such as the `get_city_weather` function in `nemo/agents/voice_agent/pipecat/utils/tool_calling/basic_tools.py`.
+- Adding new tools to adjust the behavior of each of the STT/TTS/Diar/LLM/TurnTaking components, by adding the `ToolCallingMixin` to the component and implementing the `setup_tool_calling` method as the `KokoroTTSService` class in `nemo/agents/voice_agent/pipecat/services/nemo/tts.py`.
+
+The tools are then registered to the LLM via the `register_direct_tools_to_llm` function in `nemo/agents/voice_agent/pipecat/utils/tool_calling/mixins.py`, as shown in the example in `examples/voice_agent/server/server.py`.
+
+More details on tool calling with Pipecat can be found in the [Pipecat documentation](https://docs.pipecat.ai/guides/learn/function-calling).
+
+#### Notes on tool calling issues
+
+We notice that sometimes the LLM cannot do anything that's not related to the provided tools, or it might not actually use the tools even though it says it's using them. To alleviate this issue, we insert additional instructions to the system prompt to regulate its behavior (e.g., in `server/server_configs/llm_configs/nemotron_nano_v2.yaml`).
+
+Sometimes, after answering a question related to the tools, the LLM might refuse to answer questions that are not related to the tools, or vice versa. This phenomenon can be called "commitment bias" or "tunnel vision". To alleviate this issue, we can insert additional instructions to the system prompt and explicitly asking the LLM to use or not use the tools in the user's query.
+
+
+## 📝 Notes & FAQ
+- Only one connection to the server is supported at a time, a new connection will disconnect the previous one, but the context will be preserved.
+- If directly loading from HuggingFace and got I/O errors, you can set `llm.model=<local_path>`, where the model is downloaded using a command like `huggingface-cli download Qwen/Qwen2.5-7B-Instruct --local-dir <local_path>`. Same for TTS models.
+- The current ASR and diarization models are not noise-robust, you might need to use a noise-cancelling microphone or a quiet environment. But we will release better models soon.
+- The diarization model works best with speakers that have much more different voices from each other, while it might not work well on some accents due to the limited training data.
+- If you see errors like `SyntaxError: Unexpected reserved word` when running `npm run dev`, please update the Node.js version.
+- If you see the error `Error connecting: Cannot read properties of undefined (reading 'enumerateDevices')`, it usually means the browser is not allowed to access the microphone. Please check the browser settings and add `http://[YOUR MACHINE IP ADDRESS]:5173/` to the allow list, e.g., via `chrome://flags/#unsafely-treat-insecure-origin-as-secure` for chrome browser.
+- If you see something like `node:internal/errors:496` when running `npm run dev`, remove the `client/node_modules` folder and run `npm install` again, then run `npm run dev` again.
+
+
+
+## ☁️ NVIDIA NIM Services
+
+NVIDIA also provides a variety of [NIM](https://developer.nvidia.com/nim?sortBy=developer_learning_library%2Fsort%2Ffeatured_in.nim%3Adesc%2Ctitle%3Aasc&hitsPerPage=12) services for better ASR, TTS and LLM performance with more efficient deployment on either cloud or local servers.
+
+You can also modify the `server/server.py` to use NVIDIA NIM services for better LLM, ASR and TTS performance, by referring to these Pipecat services:
+- [NVIDIA NIM LLM Service](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/nim/llm.py)
+- [NVIDIA Riva ASR Service](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/riva/stt.py)
+- [NVIDIA Riva TTS Service](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/riva/tts.py)
+- Please refer to this [NVIDIA ACE Controller example](https://github.com/NVIDIA/ace-controller/blob/main/examples/speech-to-speech/bot.py#L63) for more details on how to use NVIDIA NIM services in the voice agent.
+
+For details of available NVIDIA NIM services, please refer to:
+- [NVIDIA NIM LLM Service](https://docs.nvidia.com/nim/large-language-models/latest/introduction.html)
+- [NVIDIA Riva ASR NIM Service](https://docs.nvidia.com/nim/riva/asr/latest/overview.html)
+- [NVIDIA Riva TTS NIM Service](https://docs.nvidia.com/nim/riva/tts/latest/overview.html)
+
+
+## Acknowledgments
+
+- This example uses the [Pipecat](https://github.com/pipecat-ai/pipecat) orchestrator framework.
+
+
+
+## Contributing
+
+We welcome contributions to this project. Please feel free to submit a pull request or open an issue.
@@ -0,0 +1,85 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>NeMo Voice Agent</title>
+  <style>
+    .server-selection {
+      display: flex;
+      align-items: center;
+      gap: 8px;
+      margin-bottom: 10px;
+    }
+    
+    .server-selection label {
+      font-weight: bold;
+      color: #333;
+    }
+    
+    .server-selection select {
+      padding: 6px 12px;
+      border: 1px solid #ccc;
+      border-radius: 4px;
+      background-color: white;
+      font-size: 14px;
+      cursor: pointer;
+    }
+    
+    .server-selection select:focus {
+      outline: none;
+      border-color: #2196F3;
+      box-shadow: 0 0 0 2px rgba(33, 150, 243, 0.2);
+    }
+    
+    .server-selection select:disabled {
+      background-color: #f5f5f5;
+      cursor: not-allowed;
+      opacity: 0.6;
+    }
+  </style>
+</head>
+
+<body>
+<div class="container">
+  <div class="status-bar">
+    <div class="status">
+      Transport: <span id="connection-status">Disconnected</span>
+    </div>
+    <div class="server-selection">
+      <label for="server-select">Server:</label>
+      <select id="server-select">
+        <option value="websocket">WebSocket Server (Port 8765)</option>
+        <option value="fastapi">FastAPI Server (Port 8000)</option>
+      </select>
+    </div>
+    <div class="controls">
+      <button id="connect-btn">Connect</button>
+      <button id="disconnect-btn" disabled>Disconnect</button>
+      <button id="mute-btn" disabled>Mute</button>
+      <button id="reset-btn" disabled>Reset</button>
+    </div>
+  </div>
+
+  <div class="volume-indicator">
+    <div class="volume-label">Microphone Volume:</div>
+    <div class="volume-bar-container">
+      <div class="volume-bar" id="volume-bar"></div>
+    </div>
+    <div class="volume-text" id="volume-text">0%</div>
+  </div>
+
+  <audio id="bot-audio" autoplay></audio>
+
+  <div class="debug-panel">
+    <h3>Debug Info</h3>
+    <div id="debug-log"></div>
+  </div>
+</div>
+
+<script type="module" src="/src/app.ts"></script>
+<link rel="stylesheet" href="/src/style.css">
+</body>
+
+</html>
@@ -0,0 +1,30 @@
+{
+  "name": "client",
+  "version": "1.0.0",
+  "main": "index.js",
+  "scripts": {
+    "dev": "vite",
+    "build": "tsc && vite build",
+    "preview": "vite preview"
+  },
+  "keywords": [],
+  "author": "",
+  "license": "ISC",
+  "description": "",
+  "devDependencies": {
+    "@types/node": "^22.15.30",
+    "@types/protobufjs": "^6.0.0",
+    "@types/react": "^19.2.2",
+    "@types/react-dom": "^19.2.2",
+    "@vitejs/plugin-react-swc": "^3.10.1",
+    "typescript": "^5.8.3",
+    "vite": "^6.3.5"
+  },
+  "dependencies": {
+    "@pipecat-ai/client-js": "^0.4.0",
+    "@pipecat-ai/websocket-transport": "^0.4.1",
+    "protobufjs": "^7.4.0",
+    "react": "^19.2.0",
+    "react-dom": "^19.2.0"
+  }
+}
@@ -0,0 +1,572 @@
+/**
+ * Copyright (c) 2024–2025, Daily
+ *
+ * SPDX-License-Identifier: BSD 2-Clause License
+ */
+
+/**
+ * RTVI Client Implementation
+ *
+ * This client connects to an RTVI-compatible bot server using WebSocket.
+ *
+ * Requirements:
+ * - A running RTVI bot server (defaults to http://localhost:7860)
+ */
+
+import {
+  RTVIClient,
+  RTVIClientOptions,
+  RTVIEvent,
+} from '@pipecat-ai/client-js';
+import {
+  WebSocketTransport
+} from "@pipecat-ai/websocket-transport";
+
+class WebsocketClientApp {
+  private rtviClient: RTVIClient | null = null;
+  private connectBtn: HTMLButtonElement | null = null;
+  private disconnectBtn: HTMLButtonElement | null = null;
+  private muteBtn: HTMLButtonElement | null = null;
+  private resetBtn: HTMLButtonElement | null = null;
+  private serverSelect: HTMLSelectElement | null = null;
+  private statusSpan: HTMLElement | null = null;
+  private debugLog: HTMLElement | null = null;
+  private volumeBar: HTMLElement | null = null;
+  private volumeText: HTMLElement | null = null;
+  private botAudio: HTMLAudioElement;
+  private isConnecting: boolean = false;
+  private isDisconnecting: boolean = false;
+  private isMuted: boolean = false;
+  private audioContext: AudioContext | null = null;
+  private analyser: AnalyserNode | null = null;
+  private microphone: MediaStreamAudioSourceNode | null = null;
+  private volumeUpdateInterval: number | null = null;
+  private currentBotMessageElement: HTMLDivElement | null = null;
+  private currentBotMessage: string = '';
+
+  // Server configurations
+  private readonly serverConfigs = {
+    websocket: {
+      name: 'WebSocket Server',
+      baseUrl: `http://${window.location.hostname}:7860`,
+      port: 8765
+    },
+    fastapi: {
+      name: 'FastAPI Server', 
+      baseUrl: `http://${window.location.hostname}:8000`,
+      port: 8000
+    }
+  };
+
+  constructor() {
+    console.log("WebsocketClientApp");
+    this.botAudio = document.createElement('audio');
+    this.botAudio.autoplay = true;
+    //this.botAudio.playsInline = true;
+    document.body.appendChild(this.botAudio);
+
+    this.setupDOMElements();
+    this.setupEventListeners();
+  }
+
+  /**
+   * Set up references to DOM elements and create necessary media elements
+   */
+  private setupDOMElements(): void {
+    this.connectBtn = document.getElementById('connect-btn') as HTMLButtonElement;
+    this.disconnectBtn = document.getElementById('disconnect-btn') as HTMLButtonElement;
+    this.muteBtn = document.getElementById('mute-btn') as HTMLButtonElement;
+    this.resetBtn = document.getElementById('reset-btn') as HTMLButtonElement;
+    this.serverSelect = document.getElementById('server-select') as HTMLSelectElement;
+    this.statusSpan = document.getElementById('connection-status');
+    this.debugLog = document.getElementById('debug-log');
+    this.volumeBar = document.getElementById('volume-bar');
+    this.volumeText = document.getElementById('volume-text');
+  }
+
+  /**
+   * Set up event listeners for connect/disconnect buttons
+   */
+  private setupEventListeners(): void {
+    this.connectBtn?.addEventListener('click', () => this.connect());
+    this.disconnectBtn?.addEventListener('click', () => this.disconnect());
+    this.muteBtn?.addEventListener('click', () => this.toggleMute());
+    this.resetBtn?.addEventListener('click', () => this.reset());
+    this.serverSelect?.addEventListener('change', () => this.updateServerUrl());
+  }
+
+  /**
+   * Add a timestamped message to the debug log
+   */
+  private log(message: string): void {
+    if (!this.debugLog) return;
+    const entry = document.createElement('div');
+    entry.textContent = `${new Date().toISOString()} - ${message}`;
+    if (message.startsWith('User: ')) {
+      entry.style.color = '#2196F3';
+    } else if (message.startsWith('Bot: ')) {
+      entry.style.color = '#4CAF50';
+    }
+    this.debugLog.appendChild(entry);
+    this.debugLog.scrollTop = this.debugLog.scrollHeight;
+    console.log(message);
+  }
+
+  /**
+   * Create a bot message element and add it to the debug log
+   */
+  private createBotMessageElement(initialText: string): HTMLDivElement | null {
+    if (!this.debugLog) return null;
+    const entry = document.createElement('div');
+    entry.style.color = '#4CAF50';
+    entry.textContent = `${new Date().toISOString()} - ${initialText}`;
+    this.debugLog.appendChild(entry);
+    this.debugLog.scrollTop = this.debugLog.scrollHeight;
+    return entry;
+  }
+
+  /**
+   * Update the connection status display
+   */
+  private updateStatus(status: string): void {
+    if (this.statusSpan) {
+      this.statusSpan.textContent = status;
+    }
+    this.log(`Status: ${status}`);
+  }
+
+  /**
+   * Check for available media tracks and set them up if present
+   * This is called when the bot is ready or when the transport state changes to ready
+   */
+  setupMediaTracks() {
+    if (!this.rtviClient) return;
+    const tracks = this.rtviClient.tracks();
+    if (tracks.bot?.audio) {
+      this.setupAudioTrack(tracks.bot.audio);
+    }
+  }
+
+  /**
+   * Set up listeners for track events (start/stop)
+   * This handles new tracks being added during the session
+   */
+  setupTrackListeners() {
+    if (!this.rtviClient) {
+      this.log('Cannot setup track listeners: client is null');
+      return;
+    }
+
+    try {
+      // Listen for new tracks starting
+      this.rtviClient.on(RTVIEvent.TrackStarted, (track, participant) => {
+        // Only handle non-local (bot) tracks
+        if (!participant?.local && track.kind === 'audio') {
+          this.setupAudioTrack(track);
+        }
+      });
+
+      // Listen for tracks stopping
+      this.rtviClient.on(RTVIEvent.TrackStopped, (track, participant) => {
+        this.log(`Track stopped: ${track.kind} from ${participant?.name || 'unknown'}`);
+      });
+    } catch (error) {
+      this.log(`Error setting up track listeners: ${error}`);
+    }
+  }
+
+  /**
+   * Set up an audio track for playback
+   * Handles both initial setup and track updates
+   */
+  private setupAudioTrack(track: MediaStreamTrack): void {
+    this.log('Setting up audio track');
+    if (this.botAudio.srcObject && "getAudioTracks" in this.botAudio.srcObject) {
+      const oldTrack = this.botAudio.srcObject.getAudioTracks()[0];
+      if (oldTrack?.id === track.id) return;
+    }
+    this.botAudio.srcObject = new MediaStream([track]);
+  }
+
+  /**
+   * Initialize and connect to the bot
+   * This sets up the RTVI client, initializes devices, and establishes the connection
+   */
+  public async connect(): Promise<void> {
+    if (this.isConnecting) {
+      this.log('Connection already in progress, ignoring...');
+      return;
+    }
+
+    try {
+      this.isConnecting = true;
+      const startTime = Date.now();
+
+      //const transport = new DailyTransport();
+      const transport = new WebSocketTransport();
+      const RTVIConfig: RTVIClientOptions = {
+        transport,
+        params: {
+          // The baseURL and endpoint of your bot server that the client will connect to
+          baseUrl: this.getSelectedServerConfig().baseUrl,
+          endpoints: { connect: '/connect' },
+        },
+        enableMic: true,
+        enableCam: false,
+        callbacks: {
+          onConnected: () => {
+            this.updateStatus('Connected');
+            if (this.connectBtn) this.connectBtn.disabled = true;
+            if (this.disconnectBtn) this.disconnectBtn.disabled = false;
+            if (this.muteBtn) {
+              this.muteBtn.disabled = false;
+              this.muteBtn.textContent = 'Mute';
+            }
+            if (this.resetBtn) this.resetBtn.disabled = false;
+            if (this.serverSelect) this.serverSelect.disabled = true;
+            // Start volume monitoring when connected
+            if (!this.isMuted) {
+              this.startVolumeMonitoring();
+            }
+          },
+          onDisconnected: () => {
+            // Only handle disconnect if we're not in the middle of error cleanup
+            if (!this.isConnecting) {
+              this.updateStatus('Disconnected');
+              if (this.connectBtn) this.connectBtn.disabled = false;
+              if (this.disconnectBtn) this.disconnectBtn.disabled = true;
+              if (this.muteBtn) {
+                this.muteBtn.disabled = true;
+                this.muteBtn.textContent = 'Mute';
+              }
+              if (this.resetBtn) this.resetBtn.disabled = true;
+              if (this.serverSelect) this.serverSelect.disabled = false;
+              // Stop volume monitoring when disconnected
+              this.stopVolumeMonitoring();
+              this.log('Client disconnected');
+            }
+          },
+          onBotReady: (data) => {
+            this.log(`Bot ready: ${JSON.stringify(data)}`);
+            this.setupMediaTracks();
+          },
+          onUserTranscript: (data) => {
+            if (data.final) {
+              this.log(`User: ${data.text}`);
+            }
+          },
+          onBotTranscript: (data) => {
+            // If no current element exists, create one (fallback in case BOT_LLM_STARTED didn't fire)
+            if (!this.currentBotMessageElement) {
+              this.currentBotMessage = '';
+              this.currentBotMessageElement = this.createBotMessageElement('Bot: ');
+            }
+            
+            // Accumulate the text
+            this.currentBotMessage += data.text;
+            
+            // Update the current element
+            if (this.currentBotMessageElement) {
+              const timestamp = new Date().toISOString();
+              this.currentBotMessageElement.textContent = `${timestamp} - Bot: ${this.currentBotMessage}`;
+              this.debugLog?.scrollTo({ top: this.debugLog.scrollHeight, behavior: 'smooth' });
+            }
+          },
+          onBotLlmStarted: () => {
+            // Only create a new bot message element if the current one has content
+            if (this.currentBotMessage !== '') {
+              this.currentBotMessage = '';
+              this.currentBotMessageElement = this.createBotMessageElement('Bot: ');
+            } else if (!this.currentBotMessageElement) {
+              // Create element if it doesn't exist at all
+              this.currentBotMessage = '';
+              this.currentBotMessageElement = this.createBotMessageElement('Bot: ');
+            }
+          },
+          onMessageError: (error) => console.error('Message error:', error),
+          onError: (error) => console.error('Error:', error),
+        },
+      }
+      
+      // Create the client with error handling
+      try {
+        this.rtviClient = new RTVIClient(RTVIConfig);
+        this.setupTrackListeners();
+      } catch (clientError) {
+        this.log(`Error creating RTVI client: ${clientError}`);
+        throw clientError;
+      }
+
+      this.log('Initializing devices...');
+      await this.rtviClient.initDevices();
+      this.log('Devices initialized successfully');
+
+      this.log('Connecting to bot...');
+      await this.rtviClient.connect();
+
+      const timeTaken = Date.now() - startTime;
+      this.log(`Connection complete, timeTaken: ${timeTaken}`);
+    } catch (error) {
+      this.log(`Error connecting: ${(error as Error).message}`);
+      this.updateStatus('Error');
+      // Clean up if there's an error
+      await this.cleanupOnError();
+    } finally {
+      this.isConnecting = false;
+    }
+  }
+
+  /**
+   * Clean up resources when there's an error during connection
+   */
+  private async cleanupOnError(): Promise<void> {
+    // Set disconnecting flag to prevent onDisconnected callback interference
+    this.isDisconnecting = true;
+    
+    // Store reference to client before it might become null
+    const client = this.rtviClient;
+    
+    if (client) {
+      try {
+        // Check if the client is in a state where disconnect can be called
+        if (typeof client.disconnect === 'function') {
+          await client.disconnect();
+        }
+      } catch (disconnectError) {
+        this.log(`Error during cleanup disconnect: ${disconnectError}`);
+      } finally {
+        // Always reset the client to null to allow reconnection
+        this.rtviClient = null;
+      }
+    } else {
+      this.log('Client was already null during cleanup');
+    }
+    
+    // Reset button states
+    if (this.connectBtn) this.connectBtn.disabled = false;
+    if (this.disconnectBtn) this.disconnectBtn.disabled = true;
+    if (this.muteBtn) {
+      this.muteBtn.disabled = true;
+      this.muteBtn.textContent = 'Mute';
+    }
+    if (this.resetBtn) this.resetBtn.disabled = true;
+    if (this.serverSelect) this.serverSelect.disabled = false;
+    
+    // Stop volume monitoring
+    this.stopVolumeMonitoring();
+    
+    // Clean up bot message state
+    this.currentBotMessage = '';
+    this.currentBotMessageElement = null;
+    
+    // Reset mute state
+    this.isMuted = false;
+    
+    // Reset disconnecting flag
+    this.isDisconnecting = false;
+  }
+
+  /**
+   * Disconnect from the bot and clean up media resources
+   */
+  public async disconnect(): Promise<void> {
+    if (this.isDisconnecting) {
+      this.log('Disconnection already in progress, ignoring...');
+      return;
+    }
+
+    this.isDisconnecting = true;
+    
+    // Store reference to client before it might become null
+    const client = this.rtviClient;
+    
+    if (client) {
+      try {
+        // Check if the client is in a state where disconnect can be called
+        if (typeof client.disconnect === 'function') {
+          await client.disconnect();
+        }
+      } catch (error) {
+        this.log(`Error disconnecting: ${(error as Error).message}`);
+      } finally {
+        // Always clean up resources and reset the client
+        this.rtviClient = null;
+        if (this.botAudio.srcObject && "getAudioTracks" in this.botAudio.srcObject) {
+          this.botAudio.srcObject.getAudioTracks().forEach((track) => track.stop());
+          this.botAudio.srcObject = null;
+        }
+      }
+    } else {
+      this.log('Client was already null during disconnect');
+    }
+    
+    // Stop volume monitoring
+    this.stopVolumeMonitoring();
+    
+    // Clean up bot message state
+    this.currentBotMessage = '';
+    this.currentBotMessageElement = null;
+    
+    // Reset mute state
+    this.isMuted = false;
+    
+    this.isDisconnecting = false;
+  }
+
+  /**
+   * Toggle microphone mute/unmute
+   */
+  private toggleMute(): void {
+    if (!this.rtviClient) {
+      this.log('Cannot toggle mute: client is null');
+      return;
+    }
+
+    this.isMuted = !this.isMuted;
+    this.rtviClient.enableMic(!this.isMuted);
+    
+    // Update button text
+    if (this.muteBtn) {
+      this.muteBtn.textContent = this.isMuted ? 'Unmute' : 'Mute';
+    }
+    
+    // Update volume monitoring
+    if (this.isMuted) {
+      this.stopVolumeMonitoring();
+    } else {
+      this.startVolumeMonitoring();
+    }
+    
+    this.log(this.isMuted ? 'Microphone muted' : 'Microphone unmuted');
+  }
+
+  /**
+   * Start monitoring microphone volume
+   */
+  private async startVolumeMonitoring(): Promise<void> {
+    try {
+      if (!this.audioContext) {
+        this.audioContext = new AudioContext();
+      }
+
+      const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+      
+      this.analyser = this.audioContext.createAnalyser();
+      this.analyser.fftSize = 256;
+      this.analyser.smoothingTimeConstant = 0.8;
+      
+      this.microphone = this.audioContext.createMediaStreamSource(stream);
+      this.microphone.connect(this.analyser);
+      
+      // Start continuous volume updates
+      this.volumeUpdateInterval = window.setInterval(() => {
+        this.updateVolumeDisplay();
+      }, 100); // Update every 100ms
+      
+      this.log('Volume monitoring started');
+    } catch (error) {
+      this.log(`Error starting volume monitoring: ${error}`);
+    }
+  }
+
+  /**
+   * Stop monitoring microphone volume
+   */
+  private stopVolumeMonitoring(): void {
+    if (this.volumeUpdateInterval) {
+      clearInterval(this.volumeUpdateInterval);
+      this.volumeUpdateInterval = null;
+    }
+    
+    if (this.microphone) {
+      this.microphone.disconnect();
+      this.microphone = null;
+    }
+    
+    // Reset volume display
+    this.updateVolumeDisplay(0);
+    this.log('Volume monitoring stopped');
+  }
+
+  /**
+   * Update the volume display
+   */
+  private updateVolumeDisplay(volume?: number): void {
+    if (!this.volumeBar || !this.volumeText) return;
+
+    if (volume === undefined && this.analyser) {
+      const dataArray = new Uint8Array(this.analyser.frequencyBinCount);
+      this.analyser.getByteFrequencyData(dataArray);
+      
+      // Calculate average volume
+      const average = dataArray.reduce((sum, value) => sum + value, 0) / dataArray.length;
+      volume = (average / 255) * 100;
+    }
+
+    const displayVolume = volume || 0;
+    const clampedVolume = Math.min(100, Math.max(0, displayVolume));
+    
+    this.volumeBar.style.width = `${clampedVolume}%`;
+    this.volumeText.textContent = `${Math.round(clampedVolume)}%`;
+    
+    // Update color based on volume level
+    if (clampedVolume < 30) {
+      this.volumeBar.style.background = '#4caf50'; // Green
+    } else if (clampedVolume < 70) {
+      this.volumeBar.style.background = '#ff9800'; // Orange
+    } else {
+      this.volumeBar.style.background = '#f44336'; // Red
+    }
+  }
+
+  /**
+   * Reset the conversation context by calling the server action
+   */
+  private async reset(): Promise<void> {
+    if (!this.rtviClient) {
+      this.log('Cannot reset: not connected to server');
+      return;
+    }
+
+    try {
+      this.log('Resetting conversation context...');
+      
+      // Call the reset action on the server
+      const result = await this.rtviClient.action({ service: 'context', action: 'reset', arguments: [] });
+      
+      if (result) {
+        this.log('Conversation context reset successfully');
+      } else {
+        this.log('Failed to reset conversation context');
+      }
+    } catch (error) {
+      this.log(`Error resetting context: ${error}`);
+    }
+  }
+
+  private getSelectedServerConfig(): { name: string; baseUrl: string; port: number } {
+    const selectedValue = this.serverSelect?.value || 'websocket';
+    return this.serverConfigs[selectedValue as keyof typeof this.serverConfigs];
+  }
+
+  private updateServerUrl(): void {
+    const selectedConfig = this.getSelectedServerConfig();
+    this.log(`Server changed to: ${selectedConfig.name} (${selectedConfig.baseUrl})`);
+    
+    // If connected, show a message that they need to reconnect
+    if (this.rtviClient) {
+      this.log('Please disconnect and reconnect to use the new server');
+    }
+  }
+}
+
+declare global {
+  interface Window {
+    WebsocketClientApp: typeof WebsocketClientApp;
+  }
+}
+
+window.addEventListener('DOMContentLoaded', () => {
+  window.WebsocketClientApp = WebsocketClientApp;
+  new WebsocketClientApp();
+});
@@ -0,0 +1,180 @@
+body {
+    margin: 0;
+    padding: 20px;
+    font-family: Arial, sans-serif;
+    background-color: #f0f0f0;
+}
+
+.container {
+    max-width: 1200px;
+    margin: 0 auto;
+}
+
+.status-bar {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    padding: 10px;
+    background-color: #fff;
+    border-radius: 8px;
+    margin-bottom: 20px;
+}
+
+.controls button {
+    padding: 8px 16px;
+    margin-left: 10px;
+    border: none;
+    border-radius: 4px;
+    cursor: pointer;
+}
+
+#connect-btn {
+    background-color: #4caf50;
+    color: white;
+}
+
+#disconnect-btn {
+    background-color: #f44336;
+    color: white;
+}
+
+#mute-btn {
+    background-color: #ff9800;
+    color: white;
+}
+
+#mute-btn:disabled {
+    background-color: #ccc;
+    color: #666;
+}
+
+button:disabled {
+    opacity: 0.5;
+    cursor: not-allowed;
+}
+
+.volume-indicator {
+    display: flex;
+    align-items: center;
+    gap: 10px;
+    padding: 10px;
+    background-color: #fff;
+    border-radius: 8px;
+    margin-bottom: 20px;
+}
+
+.volume-label {
+    font-weight: bold;
+    min-width: 120px;
+}
+
+.volume-bar-container {
+    flex: 1;
+    height: 20px;
+    background-color: #e0e0e0;
+    border-radius: 10px;
+    overflow: hidden;
+    position: relative;
+}
+
+.volume-bar {
+    height: 100%;
+    background: linear-gradient(90deg, #4caf50, #ff9800, #f44336);
+    width: 0%;
+    transition: width 0.1s ease;
+    border-radius: 10px;
+}
+
+.volume-text {
+    min-width: 40px;
+    text-align: right;
+    font-weight: bold;
+    font-size: 14px;
+}
+
+.main-content {
+    background-color: #fff;
+    border-radius: 8px;
+    padding: 20px;
+    margin-bottom: 20px;
+}
+
+.bot-container {
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+}
+
+#bot-video-container {
+    width: 640px;
+    height: 360px;
+    background-color: #e0e0e0;
+    border-radius: 8px;
+    margin: 20px auto;
+    overflow: hidden;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+}
+
+#bot-video-container video {
+    width: 100%;
+    height: 100%;
+    object-fit: cover;
+}
+
+.debug-panel {
+    background-color: #fff;
+    border-radius: 8px;
+    padding: 20px;
+}
+
+.debug-panel h3 {
+    margin: 0 0 10px 0;
+    font-size: 16px;
+    font-weight: bold;
+}
+
+#debug-log {
+    height: 500px;
+    overflow-y: auto;
+    background-color: #f8f8f8;
+    padding: 10px;
+    border-radius: 4px;
+    font-family: monospace;
+    font-size: 12px;
+    line-height: 1.4;
+}
+
+.server-selection {
+    display: flex;
+    align-items: center;
+    gap: 8px;
+    margin-bottom: 10px;
+}
+
+.server-selection label {
+    font-weight: bold;
+    color: #333;
+}
+
+.server-selection select {
+    padding: 6px 12px;
+    border: 1px solid #ccc;
+    border-radius: 4px;
+    background-color: white;
+    font-size: 14px;
+    cursor: pointer;
+}
+
+.server-selection select:focus {
+    outline: none;
+    border-color: #2196F3;
+    box-shadow: 0 0 0 2px rgba(33, 150, 243, 0.2);
+}
+
+.server-selection select:disabled {
+    background-color: #f5f5f5;
+    cursor: not-allowed;
+    opacity: 0.6;
+}
@@ -0,0 +1,111 @@
+{
+  "compilerOptions": {
+    /* Visit https://aka.ms/tsconfig to read more about this file */
+
+    /* Projects */
+    // "incremental": true,                              /* Save .tsbuildinfo files to allow for incremental compilation of projects. */
+    // "composite": true,                                /* Enable constraints that allow a TypeScript project to be used with project references. */
+    // "tsBuildInfoFile": "./.tsbuildinfo",              /* Specify the path to .tsbuildinfo incremental compilation file. */
+    // "disableSourceOfProjectReferenceRedirect": true,  /* Disable preferring source files instead of declaration files when referencing composite projects. */
+    // "disableSolutionSearching": true,                 /* Opt a project out of multi-project reference checking when editing. */
+    // "disableReferencedProjectLoad": true,             /* Reduce the number of projects loaded automatically by TypeScript. */
+
+    /* Language and Environment */
+    "target": "ES2020",                                  /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */
+    "lib": ["ES2020", "DOM", "DOM.Iterable"],            /* Specify a set of bundled library declaration files that describe the target runtime environment. */
+    "jsx": "react-jsx",                                  /* Specify what JSX code is generated. */
+    // "experimentalDecorators": true,                   /* Enable experimental support for legacy experimental decorators. */
+    // "emitDecoratorMetadata": true,                    /* Emit design-type metadata for decorated declarations in source files. */
+    // "jsxFactory": "",                                 /* Specify the JSX factory function used when targeting React JSX emit, e.g. 'React.createElement' or 'h'. */
+    // "jsxFragmentFactory": "",                         /* Specify the JSX Fragment reference used for fragments when targeting React JSX emit e.g. 'React.Fragment' or 'Fragment'. */
+    // "jsxImportSource": "",                            /* Specify module specifier used to import the JSX factory functions when using 'jsx: react-jsx*'. */
+    // "reactNamespace": "",                             /* Specify the object invoked for 'createElement'. This only applies when targeting 'react' JSX emit. */
+    // "noLib": true,                                    /* Disable including any library files, including the default lib.d.ts. */
+    // "useDefineForClassFields": true,                  /* Emit ECMAScript-standard-compliant class fields. */
+    // "moduleDetection": "auto",                        /* Control what method is used to detect module-format JS files. */
+
+    /* Modules */
+    "module": "ESNext",                                  /* Specify what module code is generated. */
+    // "rootDir": "./",                                  /* Specify the root folder within your source files. */
+    "moduleResolution": "bundler",                       /* Specify how TypeScript looks up a file from a given module specifier. */
+    // "baseUrl": "./",                                  /* Specify the base directory to resolve non-relative module names. */
+    // "paths": {},                                      /* Specify a set of entries that re-map imports to additional lookup locations. */
+    // "rootDirs": [],                                   /* Allow multiple folders to be treated as one when resolving modules. */
+    // "typeRoots": [],                                  /* Specify multiple folders that act like './node_modules/@types'. */
+    // "types": [],                                      /* Specify type package names to be included without being referenced in a source file. */
+    // "allowUmdGlobalAccess": true,                     /* Allow accessing UMD globals from modules. */
+    // "moduleSuffixes": [],                             /* List of file name suffixes to search when resolving a module. */
+    // "allowImportingTsExtensions": true,               /* Allow imports to include TypeScript file extensions. Requires '--moduleResolution bundler' and either '--noEmit' or '--emitDeclarationOnly' to be set. */
+    // "rewriteRelativeImportExtensions": true,          /* Rewrite '.ts', '.tsx', '.mts', and '.cts' file extensions in relative import paths to their JavaScript equivalent in output files. */
+    // "resolvePackageJsonExports": true,                /* Use the package.json 'exports' field when resolving package imports. */
+    // "resolvePackageJsonImports": true,                /* Use the package.json 'imports' field when resolving imports. */
+    // "customConditions": [],                           /* Conditions to set in addition to the resolver-specific defaults when resolving imports. */
+    // "noUncheckedSideEffectImports": true,             /* Check side effect imports. */
+    "resolveJsonModule": true,                           /* Enable importing .json files. */
+    // "allowArbitraryExtensions": true,                 /* Enable importing files with any extension, provided a declaration file is present. */
+    // "noResolve": true,                                /* Disallow 'import's, 'require's or '<reference>'s from expanding the number of files TypeScript should add to a project. */
+
+    /* JavaScript Support */
+    // "allowJs": true,                                  /* Allow JavaScript files to be a part of your program. Use the 'checkJS' option to get errors from these files. */
+    // "checkJs": true,                                  /* Enable error reporting in type-checked JavaScript files. */
+    // "maxNodeModuleJsDepth": 1,                        /* Specify the maximum folder depth used for checking JavaScript files from 'node_modules'. Only applicable with 'allowJs'. */
+
+    /* Emit */
+    // "declaration": true,                              /* Generate .d.ts files from TypeScript and JavaScript files in your project. */
+    // "declarationMap": true,                           /* Create sourcemaps for d.ts files. */
+    // "emitDeclarationOnly": true,                      /* Only output d.ts files and not JavaScript files. */
+    // "sourceMap": true,                                /* Create source map files for emitted JavaScript files. */
+    // "inlineSourceMap": true,                          /* Include sourcemap files inside the emitted JavaScript. */
+    // "noEmit": true,                                   /* Disable emitting files from a compilation. */
+    // "outFile": "./",                                  /* Specify a file that bundles all outputs into one JavaScript file. If 'declaration' is true, also designates a file that bundles all .d.ts output. */
+    // "outDir": "./",                                   /* Specify an output folder for all emitted files. */
+    // "removeComments": true,                           /* Disable emitting comments. */
+    // "importHelpers": true,                            /* Allow importing helper functions from tslib once per project, instead of including them per-file. */
+    // "downlevelIteration": true,                       /* Emit more compliant, but verbose and less performant JavaScript for iteration. */
+    // "sourceRoot": "",                                 /* Specify the root path for debuggers to find the reference source code. */
+    // "mapRoot": "",                                    /* Specify the location where debugger should locate map files instead of generated locations. */
+    // "inlineSources": true,                            /* Include source code in the sourcemaps inside the emitted JavaScript. */
+    // "emitBOM": true,                                  /* Emit a UTF-8 Byte Order Mark (BOM) in the beginning of output files. */
+    // "newLine": "crlf",                                /* Set the newline character for emitting files. */
+    // "stripInternal": true,                            /* Disable emitting declarations that have '@internal' in their JSDoc comments. */
+    // "noEmitHelpers": true,                            /* Disable generating custom helper functions like '__extends' in compiled output. */
+    // "noEmitOnError": true,                            /* Disable emitting files if any type checking errors are reported. */
+    // "preserveConstEnums": true,                       /* Disable erasing 'const enum' declarations in generated code. */
+    // "declarationDir": "./",                           /* Specify the output directory for generated declaration files. */
+
+    /* Interop Constraints */
+    "isolatedModules": true,                             /* Ensure that each file can be safely transpiled without relying on other imports. */
+    // "verbatimModuleSyntax": true,                     /* Do not transform or elide any imports or exports not marked as type-only, ensuring they are written in the output file's format based on the 'module' setting. */
+    // "isolatedDeclarations": true,                     /* Require sufficient annotation on exports so other tools can trivially generate declaration files. */
+    "allowSyntheticDefaultImports": true,                /* Allow 'import x from y' when a module doesn't have a default export. */
+    "esModuleInterop": true,                             /* Emit additional JavaScript to ease support for importing CommonJS modules. This enables 'allowSyntheticDefaultImports' for type compatibility. */
+    // "preserveSymlinks": true,                         /* Disable resolving symlinks to their realpath. This correlates to the same flag in node. */
+    "forceConsistentCasingInFileNames": true,            /* Ensure that casing is correct in imports. */
+
+    /* Type Checking */
+    "strict": true,                                      /* Enable all strict type-checking options. */
+    // "noImplicitAny": true,                            /* Enable error reporting for expressions and declarations with an implied 'any' type. */
+    // "strictNullChecks": true,                         /* When type checking, take into account 'null' and 'undefined'. */
+    // "strictFunctionTypes": true,                      /* When assigning functions, check to ensure parameters and the return values are subtype-compatible. */
+    // "strictBindCallApply": true,                      /* Check that the arguments for 'bind', 'call', and 'apply' methods match the original function. */
+    // "strictPropertyInitialization": true,             /* Check for class properties that are declared but not set in the constructor. */
+    // "strictBuiltinIteratorReturn": true,              /* Built-in iterators are instantiated with a 'TReturn' type of 'undefined' instead of 'any'. */
+    // "noImplicitThis": true,                           /* Enable error reporting when 'this' is given the type 'any'. */
+    // "useUnknownInCatchVariables": true,               /* Default catch clause variables as 'unknown' instead of 'any'. */
+    // "alwaysStrict": true,                             /* Ensure 'use strict' is always emitted. */
+    // "noUnusedLocals": true,                           /* Enable error reporting when local variables aren't read. */
+    // "noUnusedParameters": true,                       /* Raise an error when a function parameter isn't read. */
+    // "exactOptionalPropertyTypes": true,               /* Interpret optional property types as written, rather than adding 'undefined'. */
+    // "noImplicitReturns": true,                        /* Enable error reporting for codepaths that do not explicitly return in a function. */
+    // "noFallthroughCasesInSwitch": true,               /* Enable error reporting for fallthrough cases in switch statements. */
+    // "noUncheckedIndexedAccess": true,                 /* Add 'undefined' to a type when accessed using an index. */
+    // "noImplicitOverride": true,                       /* Ensure overriding members in derived classes are marked with an override modifier. */
+    // "noPropertyAccessFromIndexSignature": true,       /* Enforces using indexed accessors for keys declared using an indexed type. */
+    // "allowUnusedLabels": true,                        /* Disable error reporting for unused labels. */
+    // "allowUnreachableCode": true,                     /* Disable error reporting for unreachable code. */
+
+    /* Completeness */
+    // "skipDefaultLibCheck": true,                      /* Skip type checking .d.ts files that are included with TypeScript. */
+    "skipLibCheck": true                                 /* Skip type checking all .d.ts files. */
+  }
+}
@@ -0,0 +1,17 @@
+import { defineConfig } from 'vite';
+import react from '@vitejs/plugin-react-swc';
+
+export default defineConfig({
+    plugins: [react()],
+    server: {
+        host: '0.0.0.0',  // Bind to all interfaces
+        port: 5173,       // Back to default Vite port
+        proxy: {
+            // Proxy /api requests to the backend server
+            '/connect': {
+                target: 'http://0.0.0.0:7860', // Replace with your backend URL if needed
+                changeOrigin: true,
+            },
+        },
+    },
+});
@@ -0,0 +1,504 @@
+name: nemo-voice
+channels:
+  - defaults
+dependencies:
+  - _libgcc_mutex=0.1
+  - _openmp_mutex=5.1
+  - bzip2=1.0.8
+  - ca-certificates=2025.9.9
+  - expat=2.7.3
+  - ld_impl_linux-64=2.40
+  - libffi=3.4.4
+  - libgcc-ng=11.2.0
+  - libgomp=11.2.0
+  - libnsl=2.0.0
+  - libstdcxx-ng=11.2.0
+  - libuuid=1.41.5
+  - libxcb=1.17.0
+  - libzlib=1.3.1
+  - ncurses=6.5
+  - openssl=3.0.18
+  - pip=26.0.1
+  - pthread-stubs=0.3
+  - python=3.12.12
+  - readline=8.3
+  - sqlite=3.50.2
+  - tk=8.6.15
+  - wheel=0.45.1
+  - xorg-libx11=1.8.12
+  - xorg-libxau=1.0.12
+  - xorg-libxdmcp=1.1.5
+  - xorg-xorgproto=2024.1
+  - xz=5.6.4
+  - zlib=1.3.1
+  - pip:
+    - absl-py==2.3.1
+    - accelerate==1.10.1
+    - accelerated-scan==0.2.0
+    - addict==2.4.0
+    - aiofiles==24.1.0
+    - aiohappyeyeballs==2.6.1
+    - aiohttp==3.13.3
+    - aiosignal==1.4.0
+    - alabaster==1.0.0
+    - alembic==1.16.5
+    - aniso8601==10.0.1
+    - annotated-doc==0.0.4
+    - annotated-types==0.7.0
+    - anthropic==0.71.0
+    - antlr4-python3-runtime==4.9.3
+    - anyio==4.9.0
+    - apache-tvm-ffi==0.1.8.post2
+    - argcomplete==3.6.3
+    - asciitree==0.3.3
+    - astor==0.8.1
+    - asttokens==3.0.0
+    - async-timeout==5.0.1
+    - attrdict==2.0.1
+    - attrs==25.3.0
+    - audioread==3.0.1
+    - av==15.1.0
+    - babel==2.17.0
+    - backports-datetime-fromisoformat==2.0.3
+    - bcrypt==5.0.0
+    - beautifulsoup4==4.13.1
+    - bitsandbytes==0.46.0
+    - black==24.10.0
+    - blake3==1.0.7
+    - blinker==1.9.0
+    - blis==1.3.3
+    - boto3==1.40.44
+    - botocore==1.40.44
+    - braceexpand==0.1.7
+    - bracex==2.6
+    - cachetools==6.2.0
+    - catalogue==2.0.10
+    - cbor2==5.7.0
+    - cdifflib==1.2.6
+    - certifi==2025.8.3
+    - cffi==2.0.0
+    - chardet==5.2.0
+    - charset-normalizer==3.4.3
+    - click==8.3.0
+    - clip==0.2.0
+    - cloudpathlib==0.23.0
+    - cloudpickle==3.0.0
+    - colorama==0.4.6
+    - coloredlogs==15.0.1
+    - colorlog==6.9.0
+    - compressed-tensors==0.12.2
+    - confection==0.1.5
+    - contextlib2==21.6.0
+    - contourpy==1.3.2
+    - coverage==7.10.7
+    - cryptography==42.0.8
+    - csvw==3.7.0
+    - cuda-bindings==13.1.1
+    - cuda-pathfinder==1.3.3
+    - cuda-python==13.1.1
+    - cupy-cuda12x==13.6.0
+    - curated-tokenizers==0.0.9
+    - curated-transformers==0.1.1
+    - cycler==0.12.1
+    - cymem==2.0.13
+    - cytoolz==1.0.1
+    - dataproperty==1.1.0
+    - datasets==4.1.1
+    - decorator==5.2.1
+    - decord==0.6.0
+    - defusedxml==0.7.1
+    - deprecated==1.2.18
+    - depyf==0.20.0
+    - diffusers==0.35.1
+    - dill==0.4.0
+    - diskcache==5.6.3
+    - distro==1.9.0
+    - dlinfo==2.0.0
+    - dnspython==2.8.0
+    - docker==7.1.0
+    - docopt==0.6.2
+    - docstring-parser==0.17.0
+    - docutils==0.21.2
+    - einops==0.8.1
+    - einops-exts==0.0.4
+    - email-validator==2.3.0
+    - espeakng-loader==0.2.4
+    - evaluate==0.4.6
+    - exceptiongroup==1.3.0
+    - executing==2.2.1
+    - fabric==3.2.2
+    - faiss-cpu==1.12.0
+    - fastapi==0.127.0
+    - fastapi-cli==0.0.13
+    - fastapi-cloud-cli==0.3.0
+    - fasteners==0.20
+    - fastrlock==0.8.3
+    - fiddle==0.3.0
+    - filelock==3.20.3
+    - flashinfer-python==0.5.3
+    - flask==3.1.2
+    - flask-restful==0.3.10
+    - flatbuffers==25.9.23
+    - fonttools==4.60.1
+    - frozendict==2.4.6
+    - frozenlist==1.7.0
+    - fsspec==2024.12.0
+    - ftfy==6.3.1
+    - future==1.0.0
+    - gdown==5.2.0
+    - gguf==0.17.1
+    - gitdb==4.0.12
+    - gitpython==3.1.45
+    - greenlet==3.2.4
+    - grpcio==1.75.1
+    - h11==0.16.0
+    - h2==4.3.0
+    - h5py==3.14.0
+    - hf-xet==1.1.10
+    - hpack==4.1.0
+    - httpcore==1.0.9
+    - httptools==0.6.4
+    - httpx==0.27.2
+    - httpx-sse==0.4.3
+    - huggingface-hub==0.35.3
+    - humanfriendly==10.0
+    - hydra-core==1.3.2
+    - hyperframe==6.1.0
+    - idna==3.10
+    - ijson==3.4.0
+    - imageio==2.37.0
+    - imagesize==1.4.1
+    - immutabledict==4.2.0
+    - importlib-metadata==8.7.0
+    - indic-numtowords==1.1.0
+    - inflect==7.5.0
+    - iniconfig==2.1.0
+    - inquirerpy==0.3.4
+    - interegular==0.3.3
+    - intervaltree==3.1.0
+    - invoke==2.2.0
+    - ipython==8.37.0
+    - isodate==0.7.2
+    - isort==5.13.2
+    - itsdangerous==2.2.0
+    - janome==0.5.0
+    - jedi==0.19.2
+    - jieba==0.42.1
+    - jinja2==3.1.6
+    - jiter==0.11.0
+    - jmespath==1.0.1
+    - joblib==1.5.2
+    - jsonlines==4.0.0
+    - jsonschema==4.25.1
+    - jsonschema-specifications==2025.9.1
+    - kaldi-python-io==1.2.2
+    - kaldialign==0.9.1
+    - kiwisolver==1.4.9
+    - kokoro==0.9.4
+    - kornia==0.8.1
+    - kornia-rs==0.1.9
+    - langdetect==1.0.9
+    - language-tags==1.2.0
+    - lark==1.2.2
+    - latexcodec==3.0.1
+    - lazy-loader==0.4
+    - ledoc-ui==0.1.0
+    - leptonai==0.26.6
+    - lhotse>=1.32.2
+    - libcst==1.8.5
+    - librosa==0.11.0
+    - lightning==2.4.0
+    - lightning-utilities==0.15.2
+    - lilcom==1.8.1
+    - llguidance==1.3.0
+    - llvmlite==0.44.0
+    - lm-format-enforcer>=0.11.3
+    - loguru==0.7.3
+    - lxml==6.0.2
+    - mako==1.3.10
+    - markdown==3.9
+    - markdown-it-py==4.0.0
+    - markdown2==2.5.4
+    - markupsafe==3.0.3
+    - marshmallow==4.0.1
+    - matplotlib==3.10.6
+    - matplotlib-inline==0.1.7
+    - mbstrdecoder==1.1.4
+    - mcp==1.26.0
+    - mdurl==0.1.2
+    - mediapy==1.1.6
+    - megatron-core==0.13.1
+    - megatron-energon==5.2.0
+    - misaki==0.9.4
+    - mistral-common==1.8.5
+    - ml-dtypes==0.5.3
+    - model-hosting-container-standards==0.1.13
+    - more-itertools==10.8.0
+    - mpmath==1.3.0
+    - msgpack==1.1.1
+    - msgspec==0.19.0
+    - multi-storage-client==0.31.0
+    - multidict==6.6.4
+    - multiprocess==0.70.16
+    - murmurhash==1.0.15
+    - mypy-extensions==1.1.0
+    - nemo-evaluator==0.1.79
+    - nemo-run==0.5.0
+    - nemo-text-processing==1.1.0
+    - nemo-toolkit==2.5.0rc0
+    - nerfacc==0.5.3
+    - networkx==3.4.2
+    - ninja==1.13.0
+    - nltk==3.9.2
+    - num2words==0.5.14
+    - numba==0.61.2
+    - numcodecs==0.13.1
+    - numexpr==2.13.1
+    - numpy==1.26.4
+    - nv-one-logger-core==2.3.0
+    - nv-one-logger-pytorch-lightning-integration==2.3.0
+    - nv-one-logger-training-telemetry==2.3.0
+    - nvidia-cublas-cu12==12.8.4.1
+    - nvidia-cuda-cupti-cu12==12.8.90
+    - nvidia-cuda-nvrtc-cu12==12.8.93
+    - nvidia-cuda-runtime-cu12==12.8.90
+    - nvidia-cudnn-cu12==9.10.2.21
+    - nvidia-cudnn-frontend==1.18.0
+    - nvidia-cufft-cu12==11.3.3.83
+    - nvidia-cufile-cu12==1.13.1.3
+    - nvidia-curand-cu12==10.3.9.90
+    - nvidia-cusolver-cu12==11.7.3.90
+    - nvidia-cusparse-cu12==12.5.8.93
+    - nvidia-cusparselt-cu12==0.7.1
+    - nvidia-cutlass-dsl==4.3.5
+    - nvidia-lm-eval==25.11.1
+    - nvidia-ml-py==13.590.48
+    - nvidia-modelopt==0.41.0
+    - nvidia-nccl-cu12==2.27.5
+    - nvidia-nvjitlink-cu12==12.8.93
+    - nvidia-nvshmem-cu12==3.3.20
+    - nvidia-nvtx-cu12==12.8.90
+    - nvidia-resiliency-ext==0.4.1
+    - nvtx==0.2.13
+    - omegaconf==2.3.0
+    - onnx==1.19.0
+    - onnxruntime==1.23.0
+    - open-clip-torch==2.24.0
+    - openai==2.14.0
+    - openai-harmony==0.0.4
+    - opencc==1.1.9
+    - opencv-python-headless==4.11.0.86
+    - opentelemetry-api==1.37.0
+    - optuna==4.5.0
+    - orjson==3.11.3
+    - outlines-core>=0.2.11
+    - overrides==7.7.0
+    - packaging==24.2
+    - pandas==2.3.3
+    - pangu==4.0.6.1
+    - parameterized==0.9.0
+    - paramiko==4.0.0
+    - parso==0.8.5
+    - partial-json-parser==0.2.1.1.post6
+    - pathspec==0.12.1
+    - pathvalidate==3.3.1
+    - peft==0.17.1
+    - pesq==0.0.4
+    - pexpect==4.9.0
+    - pfzy==0.3.4
+    - phonemizer-fork==3.3.2
+    - pillow==11.3.0
+    - pipecat-ai==0.0.98
+    - platformdirs==4.4.0
+    - pluggy==1.6.0
+    - pooch==1.8.2
+    - portalocker==3.2.0
+    - preshed==3.0.12
+    - prettytable==3.16.0
+    - progress==1.6.1
+    - prometheus-client==0.23.1
+    - prometheus-fastapi-instrumentator==7.0.0
+    - prompt-toolkit==3.0.52
+    - propcache==0.3.2
+    - protobuf==5.29.5
+    - psutil==7.1.0
+    - ptyprocess==0.7.0
+    - pulp==3.3.0
+    - pure-eval==0.2.3
+    - py-cpuinfo==9.0.0
+    - pyarrow==21.0.0
+    - pybase64==1.4.2
+    - pybind11==3.0.1
+    - pybtex==0.25.1
+    - pybtex-docutils==1.0.3
+    - pycountry==24.6.1
+    - pycparser==2.23
+    - pydantic==2.12.5
+    - pydantic-core==2.41.5
+    - pydantic-extra-types==2.10.5
+    - pydantic-settings==2.11.0
+    - pydub==0.25.1
+    - pygments==2.19.2
+    - pyjwt==2.11.0
+    - pyloudnorm==0.1.1
+    - pynacl==1.6.0
+    - pynini==2.1.6.post1
+    - pynvml==13.0.1
+    - pyparsing==3.2.5
+    - pypinyin==0.55.0
+    - pypinyin-dict==0.9.0
+    - pyre-extensions==0.0.32
+    - pysocks==1.7.1
+    - pystoi==0.4.1
+    - pytablewriter==1.2.1
+    - pytest==8.4.2
+    - pytest-httpserver==1.1.3
+    - pytest-mock==3.15.1
+    - pytest-runner==6.0.1
+    - python-dateutil==2.9.0.post0
+    - python-dotenv==1.1.1
+    - graphviz==0.21  # need to change from python-graphviz==0.21 to avoid package not found
+    - python-json-logger==3.3.0
+    - python-multipart==0.0.20
+    - python-weather==2.1.1
+    - pytorch-lightning==2.5.5
+    - pytz==2025.2
+    - pyyaml==6.0.3
+    - pyzmq==27.1.0
+    - qwen-vl-utils==0.0.14
+    - rapidfuzz==3.14.1
+    - ray==2.49.2
+    - rdflib==7.5.0
+    - referencing==0.36.2
+    - regex==2025.9.18
+    - requests==2.32.5
+    - resampy==0.4.3
+    - rfc3986==1.5.0
+    - rich==14.1.0
+    - rich-toolkit==0.15.1
+    - rignore==0.7.0
+    - rouge-score==0.1.2
+    - rpds-py==0.27.1
+    - ruamel-yaml==0.18.15
+    - ruamel-yaml-clib==0.2.14
+    - s3fs==0.4.2
+    - s3transfer==0.14.0
+    - sacrebleu==2.5.1
+    - sacremoses==0.1.1
+    - safetensors==0.6.2
+    - scikit-learn==1.7.2
+    - scipy==1.15.3
+    - seaborn==0.13.2
+    - segments==2.3.0
+    - sentence-transformers==5.1.1
+    - sentencepiece==0.2.1
+    - sentry-sdk==2.39.0
+    - setproctitle==1.3.7
+    - setuptools==79.0.0
+    - shellingham==1.5.4
+    - six==1.17.0
+    - smart-open==7.5.0
+    - smmap==5.0.2
+    - sniffio==1.3.1
+    - snowballstemmer==3.0.1
+    - sortedcontainers==2.4.0
+    - soundfile==0.13.1
+    - soupsieve==2.8
+    - sox==1.5.0
+    - soxr==0.5.0.post1
+    - spacy==3.8.11
+    - spacy-curated-transformers==0.3.1
+    - spacy-legacy==3.0.12
+    - spacy-loggers==1.0.5
+    - sphinx==8.1.3
+    - sphinxcontrib-applehelp==2.0.0
+    - sphinxcontrib-bibtex==2.6.5
+    - sphinxcontrib-devhelp==2.0.0
+    - sphinxcontrib-htmlhelp==2.1.0
+    - sphinxcontrib-jsmath==1.0.1
+    - sphinxcontrib-qthelp==2.0.0
+    - sphinxcontrib-serializinghtml==2.0.0
+    - sqlalchemy==2.0.43
+    - srsly==2.5.2
+    - sse-starlette==3.2.0
+    - stack-data==0.6.3
+    - starlette==0.50.0
+    - strenum==0.4.15
+    - structlog==25.5.0
+    - supervisor==4.3.0
+    - sympy==1.14.0
+    - tabledata==1.3.4
+    - tabulate==0.9.0
+    - taming-transformers==0.0.1
+    - tcolorpy==0.1.7
+    - tenacity==9.1.2
+    - tensorboard==2.20.0
+    - tensorboard-data-server==0.7.2
+    - tensorstore==0.1.71
+    - termcolor==3.3.0
+    - text-unidecode==1.3
+    - thinc==8.3.10
+    - threadpoolctl==3.6.0
+    - tiktoken==0.7.0
+    - timm==1.0.20
+    - tokenizers==0.22.1
+    - toml==0.10.2
+    - tomli==2.2.1
+    - tomlkit==0.14.0
+    - toolz==1.0.0
+    - torch==2.9.0
+    - torch-c-dlpack-ext==0.1.5
+    - torchaudio==2.9.0
+    - torchdiffeq==0.2.5
+    - torchmetrics==1.8.2
+    - torchprofile==0.0.4
+    - torchsde==0.2.6
+    - torchvision==0.24.0
+    - torchx==0.7.0
+    - tqdm==4.67.1
+    - tqdm-multiprocess==0.0.11
+    - traitlets==5.14.3
+    - trampoline==0.1.2
+    - transformers==4.56.1
+    - tree-sitter==0.25.2
+    - tree-sitter-python==0.25.0
+    - trimesh==4.8.3
+    - triton==3.5.0
+    - typeguard==4.4.4
+    - typepy==1.3.4
+    - typer==0.19.2
+    - typer-slim==0.21.1
+    - typing-extensions==4.15.0
+    - typing-inspect==0.9.0
+    - typing-inspection==0.4.2
+    - tzdata==2025.2
+    - ujson==5.11.0
+    - uritemplate==4.2.0
+    - urllib3==1.26.20
+    - uvicorn==0.37.0
+    - uvloop==0.21.0
+    - vllm==0.13.0
+    - wait-for2==0.4.1
+    - wandb==0.22.1
+    - wasabi==1.1.3
+    - watchfiles==1.1.0
+    - wcmatch==10.1
+    - wcwidth==0.2.14
+    - weasel==0.4.3
+    - webdataset==1.0.2
+    - websockets==15.0.1
+    - werkzeug==3.1.3
+    - wget==3.2
+    - whisper-normalizer==0.1.12
+    - word2number==1.1
+    - wrapt==1.17.3
+    - xattr==1.2.0
+    - xformers==0.0.33.post1
+    - xgrammar==0.1.27
+    - xmltodict==1.0.2
+    - xxhash==3.6.0
+    - yarl==1.20.1
+    - yq==3.4.3
+    - zarr==2.18.3
+    - zipp==3.23.0
+    - zstandard==0.25.0
@@ -0,0 +1,78 @@
+- "absolutely"
+- "ah"
+- "all right"
+- "alright"
+- "but yeah"
+- "cool"
+- "definitely"
+- "exactly"
+- "go ahead"
+- "good"
+- "great"
+- "great thanks"
+- "ha ha"
+- "hmm"
+- "humm"
+- "huh"
+- "i know"
+- "i know right"
+- "i see"
+- "indeed"
+- "interesting"
+- "mhmm"
+- "mhmm mhmm"
+- "mhmm right"
+- "mhmm yeah"
+- "mhmm yes"
+- "mm hmm"
+- "mmhmm"
+- "nice"
+- "of course"
+- "oh"
+- "oh dear"
+- "oh man"
+- "oh okay"
+- "oh wow"
+- "oh yes"
+- "ok"
+- "ok thanks"
+- "okay"
+- "okay okay"
+- "okay thanks"
+- "perfect"
+- "really"
+- "right"
+- "right exactly"
+- "right right"
+- "right yeah"
+- "so yeah"
+- "sounds good"
+- "sure"
+- "sure thing"
+- "thank you"
+- "thanks"
+- "that's awesome"
+- "thats right"
+- "thats true"
+- "true"
+- "uh huh"
+- "uh-huh"
+- "uh-huh yeah"
+- "uhhuh"
+- "uhhuh okay"
+- "um-humm"
+- "well"
+- "what"
+- "wow"
+- "yeah"
+- "yeah i know"
+- "yeah i see"
+- "yeah mhmm"
+- "yeah okay"
+- "yeah right"
+- "yeah uh-huh"
+- "yeah yeah"
+- "yep"
+- "yes"
+- "yes please"
+- "yes yes"
@@ -0,0 +1,52 @@
+Fast Bites Lunch Menu
+
+Burgers and Sandwiches:
+1. Classic Cheeseburger – $5.99
+   Juicy beef patty, cheddar cheese, pickles, ketchup & mustard on a toasted bun.
+   - Make it a double cheeseburger by adding another patty - $1.50
+2. Crispy Chicken Sandwich – $6.49
+   Fried chicken filet, lettuce, mayo, and pickles on a brioche bun.
+3. Veggie Wrap – $5.49
+   Grilled vegetables, hummus, lettuce, and tomato in a spinach wrap.
+
+Combo Deals (includes small fries and fountain soda)
+4. Cheeseburger Combo – $8.99
+5. Chicken Sandwich Combo – $9.49
+6. Veggie Wrap Combo – $8.49
+
+Sides:
+7. French Fries
+ - Small - $2.49
+ - Medium - $3.49
+ - Large - $4.49
+8. Chicken Nuggets
+ - 4 pieces - $3.29
+ - 8 pieces -  $5.99
+ - 12 pieces -  $8.99
+9. Side Salad -  $2.99
+
+Drinks:
+10. Fountain Soda (16 oz, choices: Coke, Diet Coke, Sprite, Fanta) – $1.99
+11. Iced Tea or Lemonade – $2.29
+12. Bottled Water – $1.49
+
+You are a helpful assistant named Lisa that helps customers order food from the lunch menu.
+Start by greeting the user warmly and introducing yourself within one sentence "Hi welcome to Fast Bites! I'm Lisa, what can I help you with?".
+Your answer should be concise and to the point.
+Do not include the whole lunch menu in your response, only include the items that are relevant to the user's question.
+If the user asks about a specific item, you should include the price of that item. 
+If the user asks about the menu, you should include the entire lunch menu. 
+If the user asks about the prices, you should include the prices of the items. 
+If the user asks about the location, you should include the location of the restaurant (123 Main St, Anytown, USA). 
+If the user asks about the hours, you should include the hours of the restaurant (11:00 AM - 9:00 PM). 
+When a user asks for the total price of the order, you should include the total price of the order. 
+When the conversation is done, you should say "Thank you for your order! Your total is <total_price>. Please come back soon!", where <total_price> is the total price of the orders of all speakers. 
+If a speaker finishes their order and you don't know their name, you should ask them for their name and associate it with their order. 
+When introducing an item from the menu, you should include the name of the item and the price.
+Stick strictly to the lunch menu and do not make up any items.
+You might also see speaker tags (<speaker_0>, <speaker_1>, etc.) in the user context. 
+You should respond to the user based on the speaker tag and the context of that speaker. 
+Do not include the speaker tags in your response, use them only to identify the speaker.
+If there are multiple speakers, you should handle the order of each speaker separately and not mix up the speakers. 
+Do not respond only with "Hi" or "Hi there", you should focus on the task of taking the order and not just greeting the user.
+/no_think
@@ -0,0 +1,4 @@
+You are a helpful AI agent named Lisa. 
+Start by greeting the user warmly and introducing yourself within one sentence. 
+Your answer should be concise and to the point.
+/no_think
@@ -0,0 +1,8 @@
+You are a helpful AI agent named Lisa. 
+Start by greeting the user warmly and introducing yourself within one sentence. 
+Your answer should be concise and to the point. 
+You might also see speaker tags (<speaker_0>, <speaker_1>, etc.) in the user context. 
+You should respond to the user based on the speaker tag and the context of that speaker. 
+Do not include the speaker tags in your response, use them only to identify the speaker.
+If a speaker provides their name, use their name when addressing their requests.
+/no_think
@@ -0,0 +1,38 @@
+# Model registry configuration
+# Centralized registry of available models and their corresponding configuration files
+#
+# "reasoning_supported=true" means that there's a another llm yaml file with the same name but addidional "_think.yaml" suffix,
+# which is tested to work as expected, otherwise you need to mamually modify the llm yaml to enable their reasoning mode.
+
+llm_models:
+  "nvidia/NVIDIA-Nemotron-Nano-9B-v2":
+    yaml_id: "nemotron_nano_v2.yaml"
+    reasoning_supported: false
+    
+  "meta-llama/Llama-3.1-8B-Instruct":
+    yaml_id: "llama3.1-8B-instruct.yaml"
+    reasoning_supported: false
+
+  "Qwen/Qwen2.5-7B-Instruct":
+    yaml_id: "qwen2.5-7B.yaml"
+    reasoning_supported: false
+    
+  "Qwen/Qwen3-8B":
+    yaml_id: "qwen3-8B.yaml"
+    reasoning_supported: true
+
+  "hf_llm_generic":
+    yaml_id: "hf_llm_generic.yaml"
+    reasoning_supported: false
+
+tts_models:
+  "fastpitch-hifigan":
+    yaml_id: "nemo_fastpitch-hifigan.yaml"
+
+  "hexgrad/Kokoro-82M":
+    yaml_id: "kokoro_82M.yaml"
+
+stt_models:
+  "stt_en_fastconformer_hybrid_large_streaming_80ms":
+    yaml_id: "nemo_cache_aware_streaming.yaml"
+    type: "nemo"
@@ -0,0 +1,33 @@
+# Copyright (c) 2026, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Code adapted from https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/resolve/main/nano_v3_reasoning_parser.py
+
+from vllm.reasoning.abs_reasoning_parsers import ReasoningParserManager
+from vllm.reasoning.deepseek_r1_reasoning_parser import DeepSeekR1ReasoningParser
+
+
+@ReasoningParserManager.register_module("nano_v3")
+class NanoV3ReasoningParser(DeepSeekR1ReasoningParser):
+    def extract_reasoning(self, model_output, request):
+        reasoning_content, final_content = super().extract_reasoning(model_output, request)
+        if (
+            hasattr(request, "chat_template_kwargs")
+            and request.chat_template_kwargs
+            and request.chat_template_kwargs.get("enable_thinking") is False
+            and final_content is None
+        ):
+            reasoning_content, final_content = final_content, reasoning_content
+
+        return reasoning_content, final_content
@@ -0,0 +1,637 @@
+# Copyright (c) 2025, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Code adapted from https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2/resolve/main/nemotron_toolcall_parser_streaming.py
+
+import json
+from collections.abc import Sequence
+from random import choices
+from string import ascii_letters, digits
+from typing import Optional, Union
+
+import partial_json_parser
+import regex as re
+from partial_json_parser.core.options import Allow
+from pydantic import Field
+from vllm.entrypoints.openai.protocol import (
+    ChatCompletionRequest,
+    DeltaFunctionCall,
+    DeltaMessage,
+    DeltaToolCall,
+    ExtractedToolCallInformation,
+    FunctionCall,
+    ToolCall,
+)
+from vllm.logger import init_logger
+from vllm.tokenizers.mistral import MistralTokenizer
+from vllm.tool_parsers.abstract_tool_parser import ToolParser, ToolParserManager
+from vllm.transformers_utils.tokenizer import AnyTokenizer
+
+logger = init_logger(__name__)
+
+ALPHANUMERIC = ascii_letters + digits
+
+
+class NemotronToolCall(ToolCall):
+    id: str = Field(default_factory=lambda: NemotronToolCall.generate_random_id())
+
+    @staticmethod
+    def generate_random_id():
+        return "".join(choices(ALPHANUMERIC, k=9))
+
+    @staticmethod
+    def is_valid_id(id: str) -> bool:
+        return id.isalnum() and len(id) == 9
+
+
+def _is_fn_name_regex_support(model_tokenizer: AnyTokenizer) -> bool:
+    return isinstance(model_tokenizer, MistralTokenizer) and model_tokenizer.version >= 11
+
+
+@ToolParserManager.register_module("nemotron_json")
+class NemotronToolParser(ToolParser):
+    """
+    Streaming tool call parser specifically designed for the Nemotron-Nano-V2 model.
+
+    This parser functions as an active reconstruction engine, managing the realtime
+    transition from text generation to structured tool execution. Its primary responsibilities
+    during token streaming include:
+
+    - Interception: Detects and consumes the `<TOOLCALL>` control tokens to switch parsing modes.
+    - Buffering: Manages a lookahead buffer to prevent ambiguous partial tags (like `<TOO`)
+                 from leaking to the user.
+    - Restoration: Utilizes `partial_json_parser` to reconstruct valid objects from incomplete
+                   JSON fragments.
+    - Differentiation: Computes the precise "delta" between the current and previous JSON states
+                       to ensure monotonic streaming.
+    - Sanitization: Strips premature auto-completed closing characters (e.g., `}`)
+                    to prevent malformed updates.
+
+    Configuration:
+        Activate this parser in the vLLM server by setting the following mandatory arguments:
+        - `--enable-auto-tool-choice`
+        - `--tool-call-parser nemotron_json`
+    """
+
+    def __init__(self, tokenizer: AnyTokenizer):
+        super().__init__(tokenizer)
+        # initialize properties used for state when parsing tool calls in
+        # streaming mode
+        self.prev_tool_call_arr: list[dict] = []
+        self.current_tool_id: int = -1
+        self.current_tool_name_sent: bool = False
+        self.streamed_args_for_tool: list[str] = []  # map what has been streamed for each tool so far to a list
+        self.tool_args_emitted: list[bool] = []
+        self.bot_token = "<TOOLCALL>"
+        self.bot_token_id = self.vocab.get(self.bot_token)
+        logger.info(f"Nemotron Tool Parser: bot_token: {self.bot_token}, bot_token_id: {self.bot_token_id}")
+        self.tool_call_regex = re.compile(r"\[{.*}\]", re.DOTALL)
+        if _is_fn_name_regex_support(self.model_tokenizer):
+            self.fn_name_regex = re.compile(r'([a-zA-Z0-9_-]+)(\{[\s\S]*?\})(?=\s*$|,|\s)', re.DOTALL)
+        else:
+            self.fn_name_regex = None
+
+        # Buffer for partial tag sequences to disambiguate between normal content and
+        # a forthcoming <TOOLCALL> or </TOOLCALL> tag in streaming.
+        self._pending_tag_buffer: str = ""
+
+    def _reset_state(self) -> None:
+        """
+        Reset the parser state for a new request.
+        This is used to prevent state corruption across multiple sequential requests.
+        """
+        self.prev_tool_call_arr: list[dict] = []
+        self.current_tool_id: int = -1
+        self.current_tool_name_sent: bool = False
+        self.streamed_args_for_tool: list[str] = []
+        self.tool_args_emitted: list[bool] = []
+        self._pending_tag_buffer: str = ""
+
+    @staticmethod
+    def _strip_trailing_auto_closers(chunk: str) -> str:
+        """
+        Remove parser auto-completed closing braces/brackets plus trailing whitespace.
+        These should be flushed only when a tool call completes to avoid duplicate
+        argument fragments.
+
+        Args:
+            chunk (str):
+                The chunk of text to strip.
+        Return:
+            (str): The chunk of text with trailing auto-completed closing braces/brackets
+                   plus trailing whitespace removed.
+        """
+        idx = len(chunk)
+        while idx > 0 and chunk[idx - 1] in " \t\r\n}]":
+            idx -= 1
+        # Remove trailing non-escaped double quotes (partial JSON auto-closes strings)
+        while idx > 0 and chunk[idx - 1] == '"':
+            # keep escaped quotes (\"), only strip bare ones
+            if idx - 2 >= 0 and chunk[idx - 2] == '\\':
+                break
+            idx -= 1
+        return chunk[:idx]
+
+    @staticmethod
+    def _common_prefix_len(left: str, right: str) -> int:
+        """
+        Calculate the length of the longest initial substring shared by two strings.
+
+        This utility is used to determine how much of the tool arguments have already
+        been streamed to the client, allowing the system to send only the new 'delta'.
+
+        Args:
+            left (str): The first string to compare (typically the full current arguments).
+            right (str): The second string to compare (typically the previously streamed arguments).
+
+        Returns:
+            int: The count of identical characters starting from index 0.
+                 Returns 0 if the strings share no common prefix.
+        """
+        max_len = min(len(left), len(right))
+        idx = 0
+        while idx < max_len and left[idx] == right[idx]:
+            idx += 1
+        return idx
+
+    def _compute_arguments_delta(self, cur_arguments_json: str, end_of_call: bool) -> str:
+        """
+        Determine the incremental suffix to stream for the current tool call.
+        Ensures we only emit monotonic chunks by trimming our tracked prefix to
+        the longest common prefix with the latest JSON snapshot.
+
+        Args:
+            cur_arguments_json (str):
+                The current arguments JSON in string format.
+            end_of_call (bool):
+                Whether the current tool call is the last one in the array.
+
+        Return:
+            (str): The incremental suffix to stream for the current tool call.
+        """
+        tool_idx = self.current_tool_id
+        if tool_idx < 0 or tool_idx >= len(self.streamed_args_for_tool):
+            if tool_idx < 0:
+                logger.debug(f"current_tool_id is negative ({tool_idx}), no tool designated yet")
+            else:
+                logger.warning(
+                    f"tool_idx ({tool_idx}) is out of bounds for streamed_args_for_tool "
+                    f"(length: {len(self.streamed_args_for_tool)})"
+                )
+            return ""
+
+        streamed_prefix = self.streamed_args_for_tool[tool_idx]
+        had_any = self.tool_args_emitted[tool_idx] if tool_idx < len(self.tool_args_emitted) else False
+
+        lcp_len = self._common_prefix_len(cur_arguments_json, streamed_prefix)
+        if lcp_len != len(streamed_prefix):
+            streamed_prefix = streamed_prefix[:lcp_len]
+            self.streamed_args_for_tool[tool_idx] = streamed_prefix
+
+        if (
+            not had_any
+            and not end_of_call
+            and lcp_len == 0
+            and cur_arguments_json.endswith('": ""}')
+            and '": ""' in cur_arguments_json
+        ):
+            closing_pos = cur_arguments_json.rfind('": ""}')
+            if closing_pos != -1:
+                arguments_delta = cur_arguments_json[: closing_pos + 4]
+            else:
+                arguments_delta = cur_arguments_json
+        else:
+            arguments_delta = cur_arguments_json[lcp_len:]
+
+        if not arguments_delta:
+            return ""
+
+        if not end_of_call:
+            arguments_delta = self._strip_trailing_auto_closers(arguments_delta)
+
+        if not had_any and not end_of_call and arguments_delta and arguments_delta.endswith('}'):
+            arguments_delta = arguments_delta[:-1]
+            if arguments_delta.endswith('"'):
+                arguments_delta = arguments_delta[:-1]
+
+        return arguments_delta
+
+    def _visible_delta_outside_tool(
+        self, delta_text: str, start_token: Optional[str], end_token: Optional[str]
+    ) -> str:
+        """
+        Filters incoming streaming text to hide incomplete or complete tool call tags.
+
+        This method acts as a buffer for the streaming response. It consumes and holds
+        characters that resemble the start of `start_token` or `end_token` (e.g., "<", "<T", "<TOO").
+
+        - If the buffer eventually matches the full token exactly (e.g., "<TOOLCALL>"),
+          the buffer is discarded (suppressed).
+        - If the buffer diverges from the expected tokens (e.g., user types "<Think>"),
+          the buffered text is released (flushed) alongside the current character.
+        - Regular text that does not start with "<" passes through immediately.
+
+        Args:
+            delta_text (str):
+                The new chunk of text generated by the model in this streaming step.
+            start_token (Optional[str]):
+                The opening tag to suppress (e.g., "<TOOLCALL>"). If None, no start tag is tracked.
+            end_token (Optional[str]):
+                The closing tag to suppress (e.g., "</TOOLCALL>"). If None, no end tag is tracked.
+
+        Returns:
+            str: The portion of `delta_text` (plus any previously buffered ambiguous characters)
+            that has been confirmed as *not* being part of a tool call tag.
+        """
+        if not delta_text:
+            return delta_text
+
+        visible: list[str] = []
+        for ch in delta_text:
+            if self._pending_tag_buffer or ch == '<':
+                self._pending_tag_buffer += ch
+
+                if start_token and start_token.startswith(self._pending_tag_buffer):
+                    if self._pending_tag_buffer == start_token:
+                        self._pending_tag_buffer = ""
+                    continue
+
+                if end_token and end_token.startswith(self._pending_tag_buffer):
+                    if self._pending_tag_buffer == end_token:
+                        self._pending_tag_buffer = ""
+                    continue
+
+                # Not a tool tag; flush buffered characters as normal content.
+                visible.append(self._pending_tag_buffer)
+                self._pending_tag_buffer = ""
+            else:
+                visible.append(ch)
+
+        return "".join(visible)
+
+    def adjust_request(self, request: ChatCompletionRequest) -> ChatCompletionRequest:
+        if not isinstance(self.model_tokenizer, MistralTokenizer) and request.tools and request.tool_choice != 'none':
+            # Do not skip special tokens when using chat template
+            # with Mistral parser as TOOL_CALL token is needed
+            # for tool detection.
+            # Note: we don't want skip_special_tokens=False
+            # with MistralTokenizer as it is incompatible
+            request.skip_special_tokens = False
+        return request
+
+    def extract_tool_calls(
+        self,
+        model_output: str,
+        request: ChatCompletionRequest,
+    ) -> ExtractedToolCallInformation:
+        """
+        Parses a complete (non-streaming) model response to extract tool execution instructions.
+
+        This method attempts to convert the raw text output from the model into structured
+        `NemotronToolCall` objects. It employs a robust two-stage parsing strategy:
+        - Direct JSON Parsing: First attempts to parse the content following the
+           `<TOOLCALL>` token as valid JSON.
+        - Regex Fallback: If direct parsing fails (e.g., due to extra text or noise),
+           it uses a regular expression to locate and extract the specific JSON array pattern.
+
+        Args:
+            model_output (str): The full text generated by the model.
+            request (ChatCompletionRequest): The original request object (used for context if needed).
+
+        Returns:
+            ExtractedToolCallInformation: An object containing the parsed list of tool calls
+            and any preceding text content. If parsing fails entirely, it returns the raw
+            content as a standard text message.
+        """
+        # Reset state for each new non-streaming request
+        self._reset_state()
+
+        # case -- if a tool call token is not present, return a text response
+        if self.bot_token not in model_output:
+            return ExtractedToolCallInformation(tools_called=False, tool_calls=[], content=model_output)
+
+        # first remove the BOT token
+        tool_content = model_output.replace(self.bot_token, "").strip()
+
+        try:
+            # we first try to directly load the json as parsing very nested
+            # jsons is difficult
+            try:
+                if self.fn_name_regex:
+                    matches = self.fn_name_regex.findall(tool_content)
+
+                    function_call_arr = []
+                    for match in matches:
+                        fn_name = match[0]
+                        args = match[1]
+
+                        # fn_name is encoded outside serialized json dump
+                        # only arguments are serialized
+                        function_call_arr.append({"name": fn_name, "arguments": json.loads(args)})
+                else:
+                    function_call_arr = json.loads(tool_content)
+            except json.JSONDecodeError:
+                # use a regex to find the part corresponding to the tool call.
+                # NOTE: This use case should not happen if the model is trained
+                # correctly. It's an easy possible fix so it's included, but
+                # can be brittle for very complex / highly nested tool calls
+                matches = self.tool_call_regex.findall(tool_content)
+                if not matches:
+                    raise ValueError(f"No tool call pattern found in: {tool_content[:100]} ...")
+                raw_tool_call = matches[0]
+                function_call_arr = json.loads(raw_tool_call)
+
+            # Tool Call
+            tool_calls: list[NemotronToolCall] = [
+                NemotronToolCall(
+                    type="function",
+                    function=FunctionCall(
+                        name=raw_function_call["name"],
+                        # function call args are JSON but as a string
+                        arguments=json.dumps(raw_function_call["arguments"], ensure_ascii=False),
+                    ),
+                )
+                for raw_function_call in function_call_arr
+            ]
+
+            # get any content before  the tool call
+            content = model_output.split(self.bot_token)[0]
+            return ExtractedToolCallInformation(
+                tools_called=True, tool_calls=tool_calls, content=content if len(content) > 0 else None
+            )
+
+        except Exception:
+            logger.exception("Error in extracting tool call from response.")
+            # return information to just treat the tool call as regular JSON
+            return ExtractedToolCallInformation(tools_called=False, tool_calls=[], content=tool_content)
+
+    def extract_tool_calls_streaming(
+        self,
+        previous_text: str,
+        current_text: str,
+        delta_text: str,
+        previous_token_ids: Sequence[int],
+        current_token_ids: Sequence[int],
+        delta_token_ids: Sequence[int],
+        request: ChatCompletionRequest,
+    ) -> Union[DeltaMessage, None]:
+        """
+        Parses the raw text stream to identify and extract tool calls in real-time.
+        This method monitors for the `<TOOLCALL>` trigger to switch parsing modes, buffers
+        ambiguous tag prefixes to prevent leakage, and utilizes partial JSON parsing to
+        compute and emit precise incremental updates (deltas) for tool arguments while
+        suppressing auto-generated artifacts.
+
+        Args:
+            previous_text (str): (Placeholder) The generated text prior to the current step.
+            current_text (str): The total generated text including the new token.
+            delta_text (str): The specific text chunk generated in this step.
+            previous_token_ids (Sequence[int]): (Placeholder) Token IDs for previous text.
+            current_token_ids (Sequence[int]): (Placeholder) Token IDs for current text.
+            delta_token_ids (Sequence[int]): (Placeholder) Token IDs for the delta.
+            request (ChatCompletionRequest): (Placeholder) The original client request object.
+
+        Returns:
+            Union[DeltaMessage, None]: A `DeltaMessage` containing visible content or
+            tool call updates, or `None` if the output is currently buffered or unchanged.
+        """
+        # Reset state at the start of a new streaming request
+        # Detect new request: if we have stale state but previous_text indicates this is a fresh start
+        if not previous_text and (
+            self.current_tool_id != -1 or self.prev_tool_call_arr or self.streamed_args_for_tool
+        ):
+            logger.debug("Detected new streaming request, resetting parser state")
+            self._reset_state()
+
+        # if candidates tool call tokens are in the tokens generated so far, that
+        # means we're parsing as tool calls now. Suppress streaming if we are
+        # currently generating any prefix of the start or end tag.
+        visible_delta_text = delta_text
+        try:
+            start_token = self.bot_token
+            end_token = f"</{self.bot_token[1:]}" if self.bot_token.startswith('<') else None
+
+            visible_delta_text = self._visible_delta_outside_tool(delta_text, start_token, end_token)
+        except Exception:
+            # Fallback to conservative checks in case of any issues
+            if (
+                current_text.endswith('<')
+                or current_text.endswith('<T')
+                or current_text.endswith('<TO')
+                or current_text.endswith('<TOOL')
+                or current_text.endswith('<TOOLCALL')
+            ):
+                return None
+
+        # if the tool call token is not in the tokens generated so far, append
+        # output to contents since it's not a tool
+        if self.bot_token not in current_text:
+            if visible_delta_text:
+                return DeltaMessage(content=visible_delta_text)
+            # still waiting on a potential tag, so emit nothing yet
+            return None
+
+        # bit mask flags for partial JSON parsing. If the name hasn't been
+        # sent yet, don't allow sending
+        # an incomplete string since OpenAI only ever (as far as I have
+        # seen) allows sending the entire tool/ function name at once.
+        flags = Allow.ALL if self.current_tool_name_sent else Allow.ALL & ~Allow.STR
+        end_of_call: bool = False
+        try:
+
+            # replace BOT token with empty string, and convert single quotes
+            # to double to allow parsing as JSON since mistral uses single
+            # quotes instead of double for tool calls
+            parsable_arr = current_text.split(self.bot_token)[-1]
+
+            # Check if we're at the end of the tool call
+            if '</TOOLCALL>' in parsable_arr:
+                end_of_call = True
+                parsable_arr = parsable_arr.split('</TOOLCALL>')[0]
+
+            # tool calls are generated in an array, so do partial JSON
+            # parsing on the entire array
+            try:
+                tool_call_arr: list[dict] = partial_json_parser.loads(parsable_arr, flags)
+            except (partial_json_parser.core.exceptions.MalformedJSON, json.JSONDecodeError, ValueError):
+                return None
+
+            current_tool_call: dict = (
+                tool_call_arr[self.current_tool_id]
+                if len(tool_call_arr) > 0 and self.current_tool_id >= 0 and self.current_tool_id < len(tool_call_arr)
+                else {}
+            )
+
+            # case -- if no tokens have been streamed for the tool, e.g.
+            #   only the array brackets, stream nothing
+            if len(tool_call_arr) == 0:
+                return None
+
+            # case: we are starting a new tool in the array
+            #   -> array has > 0 length AND length has moved past cursor
+            elif len(tool_call_arr) > 0 and len(tool_call_arr) > self.current_tool_id + 1:
+
+                # if we're moving on to a new call, first make sure we
+                # haven't missed anything in the previous one that was
+                # auto-generated due to JSON completions, but wasn't
+                # streamed to the client yet.
+                if self.current_tool_id >= 0 and self.current_tool_id < len(self.streamed_args_for_tool):
+                    diff: Union[str, None] = current_tool_call.get("arguments")
+
+                    if diff:
+                        diff = json.dumps(diff, ensure_ascii=False).replace(
+                            self.streamed_args_for_tool[self.current_tool_id], ""
+                        )
+                        delta = DeltaMessage(
+                            tool_calls=[
+                                DeltaToolCall(
+                                    index=self.current_tool_id,
+                                    function=DeltaFunctionCall(arguments=diff).model_dump(exclude_none=True),
+                                )
+                            ]
+                        )
+                        self.streamed_args_for_tool[self.current_tool_id] += diff
+                    else:
+                        delta = None
+                else:
+                    delta = None
+                # re-set stuff pertaining to progress in the current tool
+                self.current_tool_id = len(tool_call_arr) - 1
+                self.current_tool_name_sent = False
+                self.streamed_args_for_tool.append("")
+                self.tool_args_emitted.append(False)
+                return delta
+
+            # case: update an existing tool - this is handled below
+
+            # if the current tool name hasn't been sent, send if available
+            # - otherwise send nothing
+            if not self.current_tool_name_sent:
+                function_name = current_tool_call.get("name")
+                if function_name:
+
+                    delta = DeltaMessage(
+                        tool_calls=[
+                            DeltaToolCall(
+                                index=self.current_tool_id,
+                                type="function",
+                                id=NemotronToolCall.generate_random_id(),
+                                function=DeltaFunctionCall(name=function_name).model_dump(exclude_none=True),
+                            )
+                        ]
+                    )
+                    self.current_tool_name_sent = True
+                else:
+                    delta = None
+
+            # now we know we're on the same tool call and we're streaming
+            # arguments
+            else:
+                if self.current_tool_id < 0 or self.current_tool_id >= len(self.prev_tool_call_arr):
+                    prev_arguments = None
+                else:
+                    prev_arguments = self.prev_tool_call_arr[self.current_tool_id].get("arguments")
+                cur_arguments = current_tool_call.get("arguments")
+
+                if not cur_arguments and not prev_arguments:
+                    delta = None
+                elif not cur_arguments and prev_arguments:
+                    logger.error("INVARIANT - impossible to have arguments reset " "mid-arguments")
+                    delta = None
+                elif cur_arguments:
+                    cur_arguments_json = json.dumps(cur_arguments, ensure_ascii=False)
+                    arguments_delta = self._compute_arguments_delta(cur_arguments_json, end_of_call)
+                    if arguments_delta:
+                        delta = DeltaMessage(
+                            tool_calls=[
+                                DeltaToolCall(
+                                    index=self.current_tool_id,
+                                    function=DeltaFunctionCall(arguments=arguments_delta).model_dump(
+                                        exclude_none=True
+                                    ),
+                                )
+                            ]
+                        )
+                        if self.current_tool_id >= 0 and self.current_tool_id < len(self.streamed_args_for_tool):
+                            self.streamed_args_for_tool[self.current_tool_id] += arguments_delta
+                        else:
+                            logger.warning(
+                                f"current_tool_id ({self.current_tool_id}) is out of bounds for streamed_args_for_tool "
+                                f"(length: {len(self.streamed_args_for_tool)})"
+                            )
+                        if self.current_tool_id >= 0 and self.current_tool_id < len(self.tool_args_emitted):
+                            self.tool_args_emitted[self.current_tool_id] = True
+                        else:
+                            logger.warning(
+                                f"current_tool_id ({self.current_tool_id}) is out of bounds for tool_args_emitted "
+                                f"(length: {len(self.tool_args_emitted)})"
+                            )
+                    else:
+                        # Do not flush final JSON here; let the serving layer
+                        # compute a minimal remaining suffix on finish.
+                        delta = None
+                else:
+                    # End-of-call or equal state; do not force a final flush here.
+                    delta = None
+
+            # check to see if the name is defined and has been sent. if so,
+            # stream the name - otherwise keep waiting
+            # finish by setting old and returning None as base case
+            self.prev_tool_call_arr = tool_call_arr
+            # If we've reached the end of a tool call, flush any remaining
+            # suffix (including a final '}') that hasn't been streamed yet.
+            if end_of_call and self.current_tool_id >= 0:
+                try:
+                    cur_arguments = current_tool_call.get("arguments")
+                    if cur_arguments is not None:
+                        cur_args_json = json.dumps(cur_arguments, ensure_ascii=False)
+                        remaining_suffix = self._compute_arguments_delta(cur_args_json, end_of_call=True)
+
+                        # Only send remaining suffix if it's non-empty and contains meaningful content
+                        # (not just whitespace or single characters like closing braces)
+                        if remaining_suffix and remaining_suffix.strip():
+                            extra = DeltaToolCall(
+                                index=self.current_tool_id,
+                                function=DeltaFunctionCall(arguments=remaining_suffix).model_dump(exclude_none=True),
+                            )
+                            if delta is None:
+                                delta = DeltaMessage(tool_calls=[extra])
+                            else:
+                                if getattr(delta, "tool_calls", None):
+                                    delta.tool_calls.append(extra)
+                                else:
+                                    delta.tool_calls = [extra]
+                            if self.current_tool_id >= 0 and self.current_tool_id < len(self.streamed_args_for_tool):
+                                self.streamed_args_for_tool[self.current_tool_id] += remaining_suffix
+                            else:
+                                logger.warning(
+                                    f"current_tool_id ({self.current_tool_id}) is out of bounds for streamed_args_for_tool "
+                                    f"(length: {len(self.streamed_args_for_tool)})"
+                                )
+                            if self.current_tool_id >= 0 and self.current_tool_id < len(self.tool_args_emitted):
+                                self.tool_args_emitted[self.current_tool_id] = True
+                            else:
+                                logger.warning(
+                                    f"current_tool_id ({self.current_tool_id}) is out of bounds for tool_args_emitted "
+                                    f"(length: {len(self.tool_args_emitted)})"
+                                )
+                except Exception as e:
+                    # Failure to flush the remaining arguments suffix is non-fatal; log for debugging.
+                    logger.warning(f"Error in flushing remaining suffix for tool call: {e}")
+
+            return delta
+
+        except Exception as e:
+            logger.exception(f"Error trying to handle streaming tool call: {e}")
+            return None
@@ -0,0 +1,456 @@
+# Copyright (c) 2025, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+import asyncio
+import copy
+import os
+import signal
+import sys
+from contextlib import asynccontextmanager
+from datetime import datetime
+from typing import Any, Dict
+
+import uvicorn
+from dotenv import load_dotenv
+from fastapi import FastAPI, WebSocket
+from fastapi.middleware.cors import CORSMiddleware
+from loguru import logger
+from omegaconf import OmegaConf
+from pipecat.audio.vad.silero import SileroVADAnalyzer
+from pipecat.frames.frames import EndTaskFrame
+from pipecat.pipeline.pipeline import Pipeline
+from pipecat.pipeline.runner import PipelineRunner
+from pipecat.pipeline.task import PipelineParams, PipelineTask
+from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
+from pipecat.processors.frameworks.rtvi import RTVIAction, RTVIConfig, RTVIObserverParams, RTVIProcessor
+from pipecat.serializers.protobuf import ProtobufFrameSerializer
+from websocket_url import build_websocket_url
+
+from nemo.agents.voice_agent.pipecat.processors.frameworks.rtvi import RTVIObserver
+from nemo.agents.voice_agent.pipecat.services.nemo.audio_logger import AudioLogger, RTVIAudioLoggerObserver
+from nemo.agents.voice_agent.pipecat.services.nemo.diar import NemoDiarService
+from nemo.agents.voice_agent.pipecat.services.nemo.llm import get_llm_service_from_config
+from nemo.agents.voice_agent.pipecat.services.nemo.stt import NemoSTTService
+from nemo.agents.voice_agent.pipecat.services.nemo.tts import get_tts_service_from_config
+from nemo.agents.voice_agent.pipecat.services.nemo.turn_taking import NeMoTurnTakingService
+from nemo.agents.voice_agent.pipecat.transports.network.websocket_server import (
+    WebsocketServerParams,
+    WebsocketServerTransport,
+)
+from nemo.agents.voice_agent.utils.config_manager import ConfigManager
+from nemo.agents.voice_agent.utils.tool_calling.basic_tools import tool_get_city_weather
+from nemo.agents.voice_agent.utils.tool_calling.mixins import register_direct_tools_to_llm
+
+# Load environment variables
+load_dotenv(override=True)
+
+
+def setup_logging():
+    # Configure loguru to output to both console and file
+    logger.remove()  # Remove default handler
+    logger.add(
+        sys.stderr,
+        format="<green>{time:YYYY-MM-DD HH:mm:ss.SSSS}</green> | <level>{level: <8}</level> | <cyan>{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> - <level>{message}</level>",
+        level="DEBUG",
+    )
+
+    logger.add("bot_server.log", rotation="1 day", level="DEBUG")
+
+
+setup_logging()
+
+# Global flag for graceful shutdown
+shutdown_event = asyncio.Event()
+
+# Initialize configuration manager
+config_manager = ConfigManager(
+    server_base_path=os.path.dirname(__file__), server_config_path=os.environ.get("SERVER_CONFIG_PATH", None)
+)
+server_config = config_manager.get_server_config()
+
+logger.info(f"Server config: {OmegaConf.to_container(server_config, resolve=True)}")
+
+# Access configuration parameters from ConfigManager
+SAMPLE_RATE = config_manager.SAMPLE_RATE
+RAW_AUDIO_FRAME_LEN_IN_SECS = config_manager.RAW_AUDIO_FRAME_LEN_IN_SECS
+SYSTEM_PROMPT = config_manager.SYSTEM_PROMPT
+SYSTEM_ROLE = config_manager.SYSTEM_ROLE
+
+# Transport configuration
+TRANSPORT_AUDIO_OUT_10MS_CHUNKS = config_manager.TRANSPORT_AUDIO_OUT_10MS_CHUNKS
+RECORD_AUDIO_DATA = server_config.transport.get("record_audio_data", False)
+AUDIO_LOG_DIR = server_config.transport.get("audio_log_dir", "./audio_logs")
+SERVER_HOST = os.getenv("SERVER_HOST", "0.0.0.0")
+SERVER_PUBLIC_HOST = os.getenv("SERVER_PUBLIC_HOST", "127.0.0.1")
+WEBSOCKET_SCHEME = os.getenv("WEBSOCKET_SCHEME", "ws")
+WEBSOCKET_PORT = int(os.getenv("WEBSOCKET_PORT", 8765))
+FASTAPI_PORT = int(os.getenv("FASTAPI_PORT", 7860))
+
+# VAD configuration
+vad_params = config_manager.get_vad_params()
+
+# STT configuration
+STT_MODEL = config_manager.STT_MODEL
+STT_DEVICE = config_manager.STT_DEVICE
+stt_params = config_manager.get_stt_params()
+
+# Diarization configuration
+DIAR_MODEL = config_manager.DIAR_MODEL
+USE_DIAR = config_manager.USE_DIAR
+diar_params = config_manager.get_diar_params()
+
+# Turn taking configuration
+TURN_TAKING_BACKCHANNEL_PHRASES_PATH = config_manager.TURN_TAKING_BACKCHANNEL_PHRASES_PATH
+TURN_TAKING_MAX_BUFFER_SIZE = config_manager.TURN_TAKING_MAX_BUFFER_SIZE
+TURN_TAKING_BOT_STOP_DELAY = config_manager.TURN_TAKING_BOT_STOP_DELAY
+
+# TTS configuration
+TTS_TYPE = config_manager.server_config.tts.type
+
+
+def signal_handler(signum, frame):
+    """Handle shutdown signals gracefully"""
+    logger.info(f"Received signal {signum}, initiating graceful shutdown...")
+    shutdown_event.set()
+
+
+async def run_bot_websocket_server(host: str = "0.0.0.0", port: int = None):
+    """
+    NO-TIMEOUT CONFIGURATION:
+    - session_timeout=None: Disables WebSocket session timeout
+    - idle_timeout=None: Disables pipeline idle timeout
+    - asyncio.wait_for(timeout=None): No timeout on pipeline runner
+    - Server will run indefinitely until manually stopped (Ctrl+C)
+    """
+    if port is None:
+        port = WEBSOCKET_PORT
+    logger.info(f"Starting websocket server on {host}:{port}")
+    logger.info(f"Server configured to run indefinitely with no timeouts, use Ctrl+C to quit.")
+
+    # Set up signal handlers for graceful shutdown
+    signal.signal(signal.SIGINT, signal_handler)
+    signal.signal(signal.SIGTERM, signal_handler)
+
+    logger.info("Initializing WebSocket server transport...")
+    logger.info("Server configured to run indefinitely with no timeouts")
+    # Initialize AudioLogger if recording is enabled
+    audio_logger = None
+    if RECORD_AUDIO_DATA:
+        session_id = f"session_{datetime.now().strftime('%Y%m%d_%H%M%S')}"
+        audio_logger = AudioLogger(
+            log_dir=AUDIO_LOG_DIR,
+            session_id=session_id,
+            enabled=True,
+        )
+        logger.info(f"AudioLogger initialized for session: {session_id} at {AUDIO_LOG_DIR}")
+
+    vad_analyzer = SileroVADAnalyzer(
+        sample_rate=SAMPLE_RATE,
+        params=vad_params,
+    )
+    logger.info("VAD analyzer initialized")
+
+    ws_transport = WebsocketServerTransport(
+        params=WebsocketServerParams(
+            serializer=ProtobufFrameSerializer(),
+            audio_in_enabled=True,
+            audio_out_enabled=True,
+            add_wav_header=False,
+            vad_analyzer=vad_analyzer,
+            session_timeout=None,  # Disable session timeout
+            audio_in_sample_rate=SAMPLE_RATE,
+            can_create_user_frames=False,
+            audio_out_10ms_chunks=TRANSPORT_AUDIO_OUT_10MS_CHUNKS,
+        ),
+        host=host,
+        port=port,
+    )
+
+    logger.info("Initializing STT service...")
+
+    stt = NemoSTTService(
+        model=STT_MODEL,
+        device=STT_DEVICE,
+        params=stt_params,
+        sample_rate=SAMPLE_RATE,
+        audio_passthrough=True,
+        backend="legacy",
+        decoder_type="rnnt",
+        audio_logger=audio_logger,
+    )
+    logger.info("STT service initialized")
+
+    if USE_DIAR:
+        diar = NemoDiarService(
+            model=DIAR_MODEL,
+            device=STT_DEVICE,
+            params=diar_params,
+            sample_rate=SAMPLE_RATE,
+            backend="legacy",
+            enabled=USE_DIAR,
+        )
+        logger.info("Diarization service initialized")
+    else:
+        diar = None
+
+    turn_taking = NeMoTurnTakingService(
+        use_diar=USE_DIAR,
+        max_buffer_size=TURN_TAKING_MAX_BUFFER_SIZE,
+        bot_stop_delay=TURN_TAKING_BOT_STOP_DELAY,
+        backchannel_phrases=TURN_TAKING_BACKCHANNEL_PHRASES_PATH,
+        audio_logger=audio_logger,
+    )
+    logger.info("Turn taking service initialized")
+
+    if TTS_TYPE == "nemo":
+        tts = get_tts_service_from_config(config_manager.server_config.tts, audio_logger)
+    else:
+        raise ValueError(f"Invalid TTS type: {TTS_TYPE}")
+
+    logger.info("TTS service initialized")
+
+    # Setup logging again to avoid logger from being overwritten during setting up the pipeline components
+    setup_logging()
+
+    # Put LLM in the end of model initialization to reduce the chance of running out of HBM memory
+    logger.info("Initializing LLM service...")
+    llm = get_llm_service_from_config(server_config.llm)
+    logger.info("LLM service initialized")
+
+    messages = [
+        {
+            "role": SYSTEM_ROLE,
+            "content": SYSTEM_PROMPT,
+        }
+    ]
+    inject_dummy_user_message = server_config.llm.get("inject_dummy_user_message", False)
+    if inject_dummy_user_message:
+        messages.append(
+            {
+                "role": "user",
+                "content": "Hello, who are you?",
+            }
+        )
+    context = OpenAILLMContext(messages=messages)
+
+    if server_config.llm.get("enable_tool_calling", False):
+        logger.info("Tools calling for LLM is enabled by config, registering tools...")
+        register_direct_tools_to_llm(llm=llm, context=context, tool_mixins=[tts], tools=[tool_get_city_weather])
+    else:
+        logger.info("Tools calling for LLM is disabled by config, skipping tool registration.")
+
+    original_messages = copy.deepcopy(context.get_messages())
+    original_context = copy.deepcopy(context)
+    original_context.set_llm_adapter(llm.get_llm_adapter())
+
+    context_aggregator = llm.create_context_aggregator(context)
+    user_context_aggregator = context_aggregator.user()
+    assistant_context_aggregator = context_aggregator.assistant()
+
+    # RTVI events for Pipecat client UI
+    rtvi = RTVIProcessor(config=RTVIConfig(config=[]))
+
+    # Add reset action to RTVI processor
+    async def reset_context_handler(rtvi_processor: RTVIProcessor, service: str, arguments: dict[str, Any]) -> bool:
+        """Reset both user and assistant context aggregators"""
+        logger.info("Resetting conversation context...")
+        try:
+            user_context_aggregator.reset()
+            assistant_context_aggregator.reset()
+            user_context_aggregator.set_messages(copy.deepcopy(original_messages))
+            assistant_context_aggregator.set_messages(copy.deepcopy(original_messages))
+            tts.reset()
+            if diar is not None:
+                diar.reset()
+            turn_taking.reset()
+            logger.info("Conversation context reset successfully")
+            return True
+        except Exception as e:
+            logger.error(f"Error resetting context: {e}")
+            return False
+
+    reset_action = RTVIAction(
+        service="context",
+        action="reset",
+        result="bool",
+        arguments=[],
+        handler=reset_context_handler,
+    )
+    rtvi.register_action(reset_action)
+
+    logger.info("Setting up pipeline...")
+
+    pipeline = [
+        ws_transport.input(),
+        rtvi,
+        stt,
+    ]
+
+    if USE_DIAR:
+        pipeline.append(diar)
+
+    pipeline.extend(
+        [turn_taking, user_context_aggregator, llm, tts, ws_transport.output(), assistant_context_aggregator]
+    )
+
+    pipeline = Pipeline(pipeline)
+
+    rtvi_params = RTVIObserverParams(bot_llm_enabled=False)
+    task = PipelineTask(
+        pipeline,
+        params=PipelineParams(
+            allow_interruptions=True,
+            enable_metrics=False,
+            enable_usage_metrics=False,
+            send_initial_empty_metrics=True,
+            report_only_initial_ttfb=True,
+            idle_timeout=None,  # Disable idle timeout
+        ),
+        observers=[
+            RTVIObserver(rtvi, params=rtvi_params),
+            RTVIAudioLoggerObserver(audio_logger=audio_logger),
+        ],
+        idle_timeout_secs=None,
+        cancel_on_idle_timeout=False,
+    )
+
+    # Track task state
+    task_running = True
+
+    # Setup logging again to avoid logger from being overwritten during setting up the pipeline components
+    setup_logging()
+
+    @rtvi.event_handler("on_client_ready")
+    async def on_client_ready(rtvi: RTVIProcessor):
+        logger.info("Pipecat client ready.")
+        await rtvi.set_bot_ready()
+        # Kick off the conversation.
+        try:
+            await task.queue_frames([user_context_aggregator.get_context_frame()])
+        except Exception as e:
+            logger.error(f"Error queuing context frame: {e}")
+
+    @ws_transport.event_handler("on_client_connected")
+    async def on_client_connected(transport, client):
+        logger.info(f"Pipecat Client connected from {client.remote_address}")
+        # Reset RTVI state for new connection
+        rtvi._client_ready = False
+        rtvi._bot_ready = False
+
+    @ws_transport.event_handler("on_client_disconnected")
+    async def on_client_disconnected(transport, client):
+        logger.info(f"Pipecat Client disconnected from {client.remote_address}")
+        # Finalize audio logger session if enabled
+        if audio_logger:
+            audio_logger.finalize_session()
+            logger.info("Audio logger session finalized")
+        # Don't cancel the task immediately - let it handle the disconnection gracefully
+        # The task will continue running and can accept new connections
+        # Only send an EndTaskFrame to clean up the current session
+        if task_running:
+            try:
+                await task.queue_frames([EndTaskFrame()])
+            except Exception as e:
+                # Don't log warnings for normal connection closures
+                if "ConnectionClosedOK" not in str(e) and "1005" not in str(e):
+                    logger.warning(f"Error sending EndTaskFrame: {e}")
+                else:
+                    logger.info(f"Normal connection closure: {e}")
+
+    @ws_transport.event_handler("on_session_timeout")
+    async def on_session_timeout(transport, client):
+        logger.info(f"Session timeout for {client.remote_address}")
+        # Don't cancel the task - keep server running indefinitely
+        logger.info("Session timeout occurred but keeping server running")
+        # Note: With session_timeout=None, this handler should never be called
+
+    logger.info("Starting pipeline runner...")
+
+    try:
+        runner = PipelineRunner()
+        # Run the task until shutdown is requested
+        await asyncio.wait_for(runner.run(task), timeout=None)  # No timeout - run indefinitely
+    except asyncio.TimeoutError:
+        logger.info("Pipeline runner timeout (should not happen with no timeout)")
+    except Exception as e:
+        logger.error(f"Pipeline runner error: {e}")
+        task_running = False
+    finally:
+        # Finalize audio logger on shutdown
+        if audio_logger:
+            audio_logger.finalize_session()
+            logger.info("Audio logger session finalized on shutdown")
+        logger.info("Pipeline runner stopped")
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    """Handles FastAPI startup and shutdown."""
+    yield  # Run app
+
+
+# Initialize FastAPI app with lifespan manager
+app = FastAPI(lifespan=lifespan)
+
+# Configure CORS to allow requests from any origin
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+
+@app.websocket("/ws")
+async def websocket_endpoint(websocket: WebSocket):
+    await websocket.accept()
+    print("WebSocket connection accepted")
+    try:
+        # TODO: [heh] Implement FastAPI websocket endpoint
+        # await run_bot_fastapi_server(websocket)
+        raise NotImplementedError("FastAPI websocket endpoint is not implemented")
+    except Exception as e:
+        print(f"Exception in run_bot: {e}")
+
+
+@app.post("/connect")
+async def bot_connect() -> Dict[Any, Any]:
+    print("Received /connect request")
+    ws_url = build_websocket_url(SERVER_PUBLIC_HOST, WEBSOCKET_PORT, WEBSOCKET_SCHEME)
+    print(f"Returning WebSocket URL: {ws_url}")
+    return {"ws_url": ws_url}
+
+
+async def main():
+    """Main function to run both websocket server and FastAPI server concurrently."""
+    logger.info(f"Starting servers - WebSocket on port {WEBSOCKET_PORT}, FastAPI on port {FASTAPI_PORT}")
+    tasks = []
+    try:
+        # Start websocket server
+        tasks.append(run_bot_websocket_server(host=SERVER_HOST, port=WEBSOCKET_PORT))
+
+        # Start FastAPI server
+        config = uvicorn.Config(app, host=SERVER_HOST, port=FASTAPI_PORT)
+        server = uvicorn.Server(config)
+        tasks.append(server.serve())
+
+        await asyncio.gather(*tasks)
+    except asyncio.CancelledError:
+        logger.info("Tasks cancelled (probably due to shutdown).")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
@@ -0,0 +1,49 @@
+# This is an example config for setting up a NeMo Voice Agent server only with NVIDIA NeMo models.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+# STT, LLM and TTS models have standalone configs in the folder "server/server_configs/{stt,llm,tts}_configs".
+# Specify the type and an a model identifier to automatically configure the model.
+
+transport:
+  audio_out_10ms_chunks: 10  # use 4 as websocket default, but increasing to a larger number might have less glitches in TTS output
+
+vad:
+  type: silero
+  confidence: 0.6  # VAD threshold for detecting speech versus non-speech
+  start_secs: 0.1  # min amount of speech to trigger UserStartSpeaking
+  stop_secs: 1.2  # min amount of silence to trigger UserStopSpeaking
+  min_volume: 0.4  # Microphone volumn threshold for VAD
+
+stt:
+  type: nemo # choices in ['nemo'] currently only NeMo is supported
+  model: "stt_en_fastconformer_hybrid_large_streaming_80ms"
+  model_config: "./server_configs/stt_configs/nemo_cache_aware_streaming.yaml"
+  device: "cuda"
+
+diar:
+  type: nemo
+  enabled: true # set to false to disable
+  model: "nvidia/diar_streaming_sortformer_4spk-v2"
+  device: "cuda"
+  threshold: 0.4  # threshold value used to determine if a speaker exists or not, setting it to a lower value will increaset the sensitivity of the model
+  frame_len_in_secs: 0.08  # default for Sortformer, do not change unless using other architechtures
+
+turn_taking:
+  backchannel_phrases_path: "./server/backchannel_phrases.yaml"  # set it to the actual path of the file, or specify a list of backchannel phrases here
+  max_buffer_size: 2  # num of words more than this amount will interrupt the LLM immediately if not backchannel phrases
+  bot_stop_delay: 0.5  # a delay in seconds allowed between server and client audio output, so that the BotStopSpeaking signal is handled not too far away from the actual time that the user hears all audio output
+
+llm:
+  type: auto  # choices in ['auto', 'hf', 'vllm'], if `auto`, it will try to use vllm and fall back to hf if vllm not available
+  model: "nvidia/NVIDIA-Nemotron-Nano-9B-v2"  # model name for HF models, will be used via `AutoModelForCausalLM.from_pretrained()`
+  model_config: "./server_configs/llm_configs/nemotron_nano_v2.yaml"
+  device: "cuda"
+  enable_reasoning: false  # it's best to turn-off reasoning for lowest latency
+  # `system_prompt` is used as the sytem prompt to the LLM, please refer to differnt LLM webpage for spcial functions like enabling/disabling thinking
+  # system_prompt: /path/to/prompt.txt  # or use path to a txt file that contains a long prompt, for example in `../example_prompts/fast_bite.txt`
+  system_prompt: "You are a helpful AI agent named Lisa. Start by greeting the user warmly and introducing yourself within one sentence. Your answer should be concise and to the point. You might also see speaker tags (<speaker_0>, <speaker_1>, etc.) in the user context. You should respond to the user based on the speaker tag and the context of that speaker. Do not include the speaker tags in your response, use them only to identify the speaker."
+
+tts:
+  type: nemo # choices in ['nemo', 'kokoro']
+  model: fastpitch-hifigan
+  model_config: "./server_configs/tts_configs/nemo_fastpitch-hifigan.yaml"
+  device: "cuda"
@@ -0,0 +1,63 @@
+# This is an example config for setting up a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+# STT, LLM and TTS models have standalone configs in the folder "server/server_configs/{stt,llm,tts}_configs".
+# Specify the type and an a model identifier to automatically configure the model.
+
+transport:
+  audio_out_10ms_chunks: 10  # use 4 as websocket default, but increasing to a larger number might have less glitches in TTS output
+  record_audio_data: false
+  audio_log_dir: "./audio_logs"
+
+vad:
+  type: silero
+  confidence: 0.6  # VAD threshold for detecting speech versus non-speech
+  start_secs: 0.1  # min amount of speech to trigger UserStartSpeaking
+  stop_secs: 1.2  # min amount of silence to trigger UserStopSpeaking
+  min_volume: 0.4  # Microphone volumn threshold for VAD
+
+stt:
+  type: nemo # choices in ['nemo'] currently only NeMo is supported
+  model: "nvidia/parakeet_realtime_eou_120m-v1"
+  # model: "nvidia/nemotron-speech-streaming-en-0.6b"
+  model_config: "./server_configs/stt_configs/nemo_cache_aware_streaming.yaml"
+  device: "cuda"
+
+diar:
+  type: nemo
+  enabled: true # set to false to disable
+  model: "nvidia/diar_streaming_sortformer_4spk-v2.1"
+  device: "cuda"
+  threshold: 0.4  # threshold value used to determine if a speaker exists or not, setting it to a lower value will increaset the sensitivity of the model
+  frame_len_in_secs: 0.08  # default for Sortformer, do not change unless using other architechtures
+
+turn_taking:
+  backchannel_phrases_path: "./server/backchannel_phrases.yaml"  # set it to the actual path of the file, or specify a list of backchannel phrases here
+  max_buffer_size: 2  # num of words more than this amount will interrupt the LLM immediately if not backchannel phrases
+  bot_stop_delay: 0.5  # a delay in seconds allowed between server and client audio output, so that the BotStopSpeaking signal is handled not too far away from the actual time that the user hears all audio output
+
+llm:
+  type: auto  # choices in ['auto', 'hf', 'vllm'], if `auto`, it will try to use vllm and fall back to hf if vllm not available
+  model: "nvidia/NVIDIA-Nemotron-Nano-9B-v2"  # model name for HF models, will be used via `AutoModelForCausalLM.from_pretrained()`
+  model_config: "./server_configs/llm_configs/nemotron_nano_v2.yaml"
+  # model: "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
+  # model_config: "./server_configs/llm_configs/nemotron_nano_v3.yaml"
+  # if `model_config` is not specified, and the llm.model is not in model_registry.yaml, it will use `llm_configs/hf_llm_generic.yaml`
+  # model: "Qwen/Qwen2.5-7B-Instruct" 
+  # model_config: "./server_configs/llm_configs/qwen2.5-7B.yaml"
+  # model: "Qwen/Qwen3-8B"
+  # model_config: "./server_configs/llm_configs/qwen3-8B.yaml"
+  # model: meta-llama/Llama-3.1-8B-Instruct
+  device: "cuda"
+  enable_reasoning: false  # it's best to turn-off reasoning for lowest latency, setting it to True will use the same config ending with `_think.yaml` instead
+  # `system_prompt` is used as the sytem prompt to the LLM, please refer to differnt LLM webpage for spcial functions like enabling/disabling thinking
+  # system_prompt: /path/to/prompt.txt  # or use path to a txt file that contains a long prompt, for example in `../example_prompts/fast_bite.txt`
+  system_prompt: "You are a helpful AI agent named Lisa. Start by greeting the user with 'Hi, I'm Lisa, your helpful AI assistant. How can I help you today?'. Keep your responses concise and conversational since they will be spoken aloud. Avoid special characters. Use only simple, plain text sentences. Always punctuate your responses using standard sentence punctuation: commas, periods, question marks, exclamation points, etc. Always spell out numbers as words. You might also see speaker tags (<speaker_0>, <speaker_1>, etc.) in the user context. You should respond to the user based on the speaker tag and the context of that speaker. Do not include the speaker tags in your response, use them only to identify the speaker. Avoid using emoji in your response."
+
+tts:
+  type: nemo
+  model: "kokoro"
+  model_config: "./server_configs/tts_configs/kokoro_82M.yaml"
+  device: "cuda"
+  ignore_strings:  # strings/characters to ignore in TTS
+    - "*"
+    - "<unk>"
@@ -0,0 +1,57 @@
+# This is an example config for setting up a generic HuggingFace LLM for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+# type: auto  # choices in ['auto', 'hf', 'vllm']
+# device: "cuda"
+dtype: bfloat16  # torch.dtype for LLM
+system_role: "system"  # role for system prompt, set it to `user` for models that do not support system prompt
+system_prompt_suffix: "/no_think"  # a string that would be appended to the system prompt, used to enable/disable thinking
+enable_tool_calling: False  # Tool calling is not supported by default, since different LLMs have diffferent implementations
+
+##############################
+## Common generation config ##
+##############################
+temperature: 0.7  # LLM sampling params
+top_k: 20  # LLM sampling params
+top_p: 0.8  # LLM sampling params
+min_p: 0.0  # LLM sampling params
+max_new_tokens: 256  # max num of output tokens from LLM
+##############################
+##### HuggingFace config #####
+##############################
+# Please refer to the model page of each HF LLM model to set following params properly.
+# kwargs that will be passed into tokenizer.apply_chat_template() function
+apply_chat_template_kwargs:  
+  add_generation_prompt: true  # This is required in most cases, do not change unless you're sure of it
+  tokenize: false  # This is required, do not change
+# kwargs that will be passed into model.generate() function of HF models
+generation_kwargs:  
+  temperature: ${llm.temperature}  # LLM sampling params
+  top_k: ${llm.top_k}  # LLM sampling params
+  top_p: ${llm.top_p}  # LLM sampling params
+  min_p: ${llm.min_p} # LLM sampling params
+  max_new_tokens: ${llm.max_new_tokens}  # max num of output tokens from LLM
+  do_sample: true # enable sampling
+##############################
+######## vLLM config #########
+##############################
+api_key: "EMPTY"
+base_url: "http://localhost:8000/v1"  
+# Set `start_vllm_on_init` to automatically start vllm server if it's not manually started yet
+start_vllm_on_init: true
+# Specifying vllm_server_params with the parameters you want to pass to the vllm server command `vllm serve $model $vllm_server_params`
+# Refer to each LLM's model page for details on the recommended parameters
+# It's recommended to stay with `--max-num-seqs` 1 as the voice agent currently supports one connection at a time.
+# You can try increasing the model's max context len `--max-model-len` if GPU memory allows, or decrease it if GPU OOM occurs.
+vllm_server_params: "--trust-remote-code --max-num-seqs 1 --gpu-memory-utilization 0.85"  
+# `params` are the inference parameters that would be passed into OpenAI API, 
+# please put additional model-specific parameters in `extra`
+vllm_generation_params:
+  frequency_penalty: 0.0  # Penalty for frequent tokens (-2.0 to 2.0).
+  presence_penalty: 0.0  # Penalty for new tokens (-2.0 to 2.0).
+  seed: 42  # Random seed for deterministic outputs.
+  temperature: ${llm.temperature}  # Sampling temperature (0.0 to 2.0).
+  top_k: null  # Top-k sampling parameter (currently ignored by OpenAI).
+  top_p: ${llm.top_p}  # Top-p (nucleus) sampling parameter (0.0 to 1.0).
+  max_completion_tokens: ${llm.max_new_tokens}  # max number of tokens to generate
+  extra: null  # additional model specific params can be specified in dict format
@@ -0,0 +1,58 @@
+# This is an example config for setting up Qwen2.5-7B model for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+# type: auto  # choices in ['auto', 'hf', 'vllm']
+# model: meta-llama/Llama-3.1-8B-Instruct
+# device: "cuda"
+dtype: bfloat16  # torch.dtype for LLM
+system_role: "system"  # role for system prompt, set it to `user` for models that do not support system prompt
+system_prompt_suffix: null  # a string that would be appended to the system prompt, used to enable/disable thinking
+enable_tool_calling: False  # Tool calling is not supported yet in this specific configuration of the model
+
+##############################
+## Common generation config ##
+##############################
+temperature: 0.7  # LLM sampling params
+top_k: 20  # LLM sampling params
+top_p: 0.8  # LLM sampling params
+min_p: 0.0  # LLM sampling params
+max_new_tokens: 256  # max num of output tokens from LLM
+##############################
+##### HuggingFace config #####
+##############################
+# Please refer to the model page of each HF LLM model to set following params properly.
+# kwargs that will be passed into tokenizer.apply_chat_template() function
+apply_chat_template_kwargs:  
+  add_generation_prompt: true  # This is required in most cases, do not change unless you're sure of it
+  tokenize: false  # This is required, do not change
+# kwargs that will be passed into model.generate() function of HF models
+generation_kwargs:  
+  temperature: ${llm.temperature}  # LLM sampling params
+  top_k: ${llm.top_k}  # LLM sampling params
+  top_p: ${llm.top_p}  # LLM sampling params
+  min_p: ${llm.min_p} # LLM sampling params
+  max_new_tokens: ${llm.max_new_tokens}  # max num of output tokens from LLM
+  do_sample: true # enable sampling
+##############################
+######## vLLM config #########
+##############################
+api_key: "EMPTY"
+base_url: "http://localhost:8000/v1"  
+# Set `start_vllm_on_init` to automatically start vllm server if it's not manually started yet
+start_vllm_on_init: true
+# Specifying vllm_server_params with the parameters you want to pass to the vllm server command `vllm serve $model $vllm_server_params`
+# Refer to each LLM's model page for details on the recommended parameters
+# It's recommended to stay with `--max-num-seqs` 1 as the voice agent currently supports one connection at a time.
+# You can try increasing the model's max context len `--max-model-len` if GPU memory allows, or decrease it if GPU OOM occurs.
+vllm_server_params: "--trust-remote-code --max-num-seqs 1 --gpu-memory-utilization 0.85"  
+# `params` are the inference parameters that would be passed into OpenAI API, 
+# please put additional model-specific parameters in `extra`
+vllm_generation_params:
+  frequency_penalty: 0.0  # Penalty for frequent tokens (-2.0 to 2.0).
+  presence_penalty: 0.0  # Penalty for new tokens (-2.0 to 2.0).
+  seed: 42  # Random seed for deterministic outputs.
+  temperature: ${llm.temperature}  # Sampling temperature (0.0 to 2.0).
+  top_k: null  # Top-k sampling parameter (currently ignored by OpenAI).
+  top_p: ${llm.top_p}  # Top-p (nucleus) sampling parameter (0.0 to 1.0).
+  max_completion_tokens: ${llm.max_new_tokens}  # max number of tokens to generate
+  extra: null  # additional model specific params can be specified in dict format
@@ -0,0 +1,58 @@
+# This is an example config for setting up nemotron_nano_v2 model for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+# model: "nvidia/NVIDIA-Nemotron-Nano-9B-v2"  # model name for HF models, will be used via `AutoModelForCausalLM.from_pretrained()`
+type: vllm  # Overwrite to vllm to enable tool calling, the HF backend currently doesn't support tools
+dtype: bfloat16  # torch.dtype for LLM
+device: "cuda"
+system_role: "system"  # role for system prompt, set it to `user` for models that do not support system prompt
+system_prompt_suffix: "Before responding to the user, check if the user request requires using external tools, and use the tools if they match with the user's intention. Otherwise, use your internal knowledge to answer the user's question. Do not use tools for casual conversation or when the tools don't fit the use cases. You should still try to address the user's request when it's not related to the provided tools. If you are provided with a set of tools, use them only when needed, do not limit your capabilities to the scope of the tools. If the purpose of a tool matches well with a user's request, always try to call the tool first. Conversation history should not limit your behavior on whether you can use tools. You must answer questions not related to the tools. /no_think"  # a string that would be appended to the system prompt, `/think` and `/no_think` are used to enable/disable thinking
+enable_tool_calling: True  # set to True since the vllm config below supports tool calling
+
+##############################
+## Common generation config ##
+##############################
+temperature: 0.7  # LLM sampling params
+top_k: 20  # LLM sampling params
+top_p: 0.9  # LLM sampling params
+min_p: 0.0  # LLM sampling params
+max_new_tokens: 256  # max num of output tokens from LLM
+##############################
+##### HuggingFace config #####
+##############################
+# Please refer to the model page of each HF LLM model to set following params properly.
+# kwargs that will be passed into tokenizer.apply_chat_template() function
+apply_chat_template_kwargs:  
+  add_generation_prompt: true  # This is required in most cases, do not change unless you're sure of it
+  tokenize: false  # This is required, do not change
+# kwargs that will be passed into model.generate() function of HF models
+generation_kwargs:  
+  temperature: ${llm.temperature}  # LLM sampling params
+  top_k: ${llm.top_k}  # LLM sampling params
+  top_p: ${llm.top_p}  # LLM sampling params
+  min_p: ${llm.min_p} # LLM sampling params
+  max_new_tokens: ${llm.max_new_tokens}  # max num of output tokens from LLM
+  do_sample: true # enable sampling
+##############################
+######## vLLM config #########
+##############################
+api_key: "EMPTY"
+base_url: "http://localhost:8000/v1"  
+# Set `start_vllm_on_init` to automatically start vllm server if it's not manually started yet
+start_vllm_on_init: true
+# Specifying vllm_server_params with the parameters you want to pass to the vllm server command `vllm serve $model $vllm_server_params`
+# Refer to each LLM's model page for details on the recommended parameters.
+# It's recommended to stay with `--max-num-seqs` 1 as the voice agent currently supports one connection at a time.
+# You can try increasing the model's max context len `--max-model-len` if GPU memory allows, or decrease it if GPU OOM occurs.
+vllm_server_params: "--trust-remote-code --enable-prefix-caching --max-num-seqs 1 --gpu-memory-utilization 0.85 --max-model-len 8192 --enable-auto-tool-choice --tool-parser-plugin server/parsers/nemotron_toolcall_parser_streaming.py --tool-call-parser nemotron_json --mamba_ssm_cache_dtype float32"  
+# `params` are the inference parameters that would be passed into OpenAI API, 
+# please put additional model-specific parameters in `extra`
+vllm_generation_params:
+  frequency_penalty: 0.0  # Penalty for frequent tokens (-2.0 to 2.0).
+  presence_penalty: 0.0  # Penalty for new tokens (-2.0 to 2.0).
+  seed: 42  # Random seed for deterministic outputs.
+  temperature: ${llm.temperature}  # Sampling temperature (0.0 to 2.0).
+  top_k: null  # Top-k sampling parameter (currently ignored by OpenAI).
+  top_p: ${llm.top_p}  # Top-p (nucleus) sampling parameter (0.0 to 1.0).
+  max_completion_tokens: ${llm.max_new_tokens}  # max number of tokens to generate
+  extra: null  # additional model specific params can be specified in dict format
@@ -0,0 +1,62 @@
+# This is an example config for setting up nemotron_nano_v3 model for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+# model: "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"  # model name for HF models, will be used via `AutoModelForCausalLM.from_pretrained()`
+type: vllm  # Overwrite to vllm to enable tool calling, the HF backend currently doesn't support tools
+dtype: auto  # torch.dtype for LLM, `auto` is only available for vllm
+device: "cuda"
+system_role: "system"  # role for system prompt, set it to `user` for models that do not support system prompt
+system_prompt_suffix: "Before responding to the user, check if the user request requires using external tools, and use the tools if they match with the user's intention. Otherwise, use your internal knowledge to answer the user's question. Do not use tools for casual conversation or when the tools don't fit the use cases. You should still try to address the user's request when it's not related to the provided tools. If you are provided with a set of tools, use them only when needed, do not limit your capabilities to the scope of the tools. If the purpose of a tool matches well with a user's request, always try to call the tool first. Conversation history should not limit your behavior on whether you can use tools. You must answer questions not related to the tools. Avoid any emoji in your response."  
+enable_tool_calling: true  # set to True since the vllm config below supports tool calling
+inject_dummy_user_message: true
+
+##############################
+## Common generation config ##
+##############################
+temperature: 0.6  # LLM sampling params
+top_k: 20  # LLM sampling params, NO effect for vllm
+top_p: 0.95  # LLM sampling params
+min_p: 0.0  # LLM sampling params
+max_new_tokens: 256  # max num of output tokens from LLM
+##############################
+##### HuggingFace config #####
+##############################
+# Please refer to the model page of each HF LLM model to set following params properly.
+# kwargs that will be passed into tokenizer.apply_chat_template() function
+apply_chat_template_kwargs:  
+  add_generation_prompt: true  # This is required in most cases, do not change unless you're sure of it
+  tokenize: false  # This is required, do not change
+# kwargs that will be passed into model.generate() function of HF models
+generation_kwargs:  
+  temperature: ${llm.temperature}  # LLM sampling params
+  top_k: ${llm.top_k}  # LLM sampling params
+  top_p: ${llm.top_p}  # LLM sampling params
+  min_p: ${llm.min_p} # LLM sampling params
+  max_new_tokens: ${llm.max_new_tokens}  # max num of output tokens from LLM
+  do_sample: true # enable sampling
+##############################
+######## vLLM config #########
+##############################
+api_key: "EMPTY"
+base_url: "http://localhost:8000/v1"  
+# Set `start_vllm_on_init` to automatically start vllm server if it's not manually started yet
+start_vllm_on_init: false
+# Specifying vllm_server_params with the parameters you want to pass to the vllm server command `vllm serve $model $vllm_server_params`
+# Refer to each LLM's model page for details on the recommended parameters.
+# It's recommended to stay with `--max-num-seqs` 1 as the voice agent currently supports one connection at a time.
+# You can try increasing the model's max context len `--max-model-len` if GPU memory allows, or decrease it if GPU OOM occurs.
+vllm_server_params: "--trust-remote-code --tensor-parallel-size 2 --enable-prefix-caching --max-num-seqs 1 --gpu-memory-utilization 0.8 --max-model-len 8192 --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser-plugin server/parsers/nano_v3_reasoning_parser.py --reasoning-parser nano_v3"  
+# `params` are the inference parameters that would be passed into OpenAI API, 
+# please put additional model-specific parameters in `extra`
+vllm_generation_params:
+  frequency_penalty: 0.0  # Penalty for frequent tokens (-2.0 to 2.0).
+  presence_penalty: 0.0  # Penalty for new tokens (-2.0 to 2.0).
+  seed: 42  # Random seed for deterministic outputs.
+  temperature: ${llm.temperature}  # Sampling temperature (0.0 to 2.0).
+  top_k: null  # Top-k sampling parameter (currently ignored by OpenAI).
+  top_p: ${llm.top_p}  # Top-p (nucleus) sampling parameter (0.0 to 1.0).
+  max_completion_tokens: ${llm.max_new_tokens}  # max number of tokens to generate
+  extra:  # additional model specific params can be specified in dict format
+    extra_body:
+      chat_template_kwargs:
+        enable_thinking: False  # disable thinking
@@ -0,0 +1,58 @@
+# This is an example config for setting up Qwen2.5-7B model for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+# type: hf  # choices in ['auto', 'hf', 'vllm']
+# model: Qwen/Qwen2.5-7B-Instruct
+# device: "cuda"
+dtype: bfloat16  # torch.dtype for LLM
+system_role: "system"  # role for system prompt, set it to `user` for models that do not support system prompt
+system_prompt_suffix: null  # a string that would be appended to the system prompt, used to enable/disable thinking
+enable_tool_calling: False  # Tool calling is not supported yet in this specific configuration of the model
+
+##############################
+## Common generation config ##
+##############################
+temperature: 0.7  # LLM sampling params
+top_k: 20  # LLM sampling params
+top_p: 0.8  # LLM sampling params
+min_p: 0.0  # LLM sampling params
+max_new_tokens: 256  # max num of output tokens from LLM
+##############################
+##### HuggingFace config #####
+##############################
+# Please refer to the model page of each HF LLM model to set following params properly.
+# kwargs that will be passed into tokenizer.apply_chat_template() function
+apply_chat_template_kwargs:  
+  add_generation_prompt: true  # This is required in most cases, do not change unless you're sure of it
+  tokenize: false  # This is required, do not change
+# kwargs that will be passed into model.generate() function of HF models
+generation_kwargs:  
+  temperature: ${llm.temperature}  # LLM sampling params
+  top_k: ${llm.top_k}  # LLM sampling params
+  top_p: ${llm.top_p}  # LLM sampling params
+  min_p: ${llm.min_p} # LLM sampling params
+  max_new_tokens: ${llm.max_new_tokens}  # max num of output tokens from LLM
+  do_sample: true # enable sampling
+##############################
+######## vLLM config #########
+##############################
+api_key: "EMPTY"
+base_url: "http://localhost:8000/v1"  
+# Set `start_vllm_on_init` to automatically start vllm server if it's not manually started yet
+start_vllm_on_init: true
+# Specifying vllm_server_params with the parameters you want to pass to the vllm server command `vllm serve $model $vllm_server_params`
+# Refer to each LLM's model page for details on the recommended parameters
+# It's recommended to stay with `--max-num-seqs` 1 as the voice agent currently supports one connection at a time.
+# You can try increasing the model's max context len `--max-model-len` if GPU memory allows, or decrease it if GPU OOM occurs.
+vllm_server_params: "--trust-remote-code --max-num-seqs 1 --gpu-memory-utilization 0.85"  
+# `params` are the inference parameters that would be passed into OpenAI API, 
+# please put additional model-specific parameters in `extra`
+vllm_generation_params:
+  frequency_penalty: 0.0  # Penalty for frequent tokens (-2.0 to 2.0).
+  presence_penalty: 0.0  # Penalty for new tokens (-2.0 to 2.0).
+  seed: 42  # Random seed for deterministic outputs.
+  temperature: ${llm.temperature}  # Sampling temperature (0.0 to 2.0).
+  top_k: null  # Top-k sampling parameter (currently ignored by OpenAI).
+  top_p: ${llm.top_p}  # Top-p (nucleus) sampling parameter (0.0 to 1.0).
+  max_completion_tokens: ${llm.max_new_tokens}  # max number of tokens to generate
+  extra: null  # additional model specific params can be specified in dict format
@@ -0,0 +1,58 @@
+# This is an example config for setting up Qwen3-8B model for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+# type: auto  # choices in ['auto', 'hf', 'vllm']
+# model: "Qwen/Qwen3-8B"  # model name for HF models, will be used via `AutoModelForCausalLM.from_pretrained()`
+# device: "cuda"
+dtype: bfloat16  # torch.dtype for LLM
+system_role: "system"  # role for system prompt, set it to `user` for models that do not support system prompt
+system_prompt_suffix: "/no_think"  # a string that would be appended to the system prompt, used to enable/disable thinking
+enable_tool_calling: False  # Tool calling is not supported yet in this specific configuration of the model
+
+##############################
+## Common generation config ##
+##############################
+temperature: 0.7  # LLM sampling params
+top_k: 20  # LLM sampling params
+top_p: 0.8  # LLM sampling params
+min_p: 0.0  # LLM sampling params
+max_new_tokens: 256  # max num of output tokens from LLM
+##############################
+##### HuggingFace config #####
+##############################
+# Please refer to the model page of each HF LLM model to set following params properly.
+# kwargs that will be passed into tokenizer.apply_chat_template() function
+apply_chat_template_kwargs:  
+  add_generation_prompt: true  # This is required in most cases, do not change unless you're sure of it
+  tokenize: false  # This is required, do not change
+# kwargs that will be passed into model.generate() function of HF models
+generation_kwargs:  
+  temperature: ${llm.temperature}  # LLM sampling params
+  top_k: ${llm.top_k}  # LLM sampling params
+  top_p: ${llm.top_p}  # LLM sampling params
+  min_p: ${llm.min_p} # LLM sampling params
+  max_new_tokens: ${llm.max_new_tokens}  # max num of output tokens from LLM
+  do_sample: true # enable sampling
+##############################
+######## vLLM config #########
+##############################
+api_key: "EMPTY"
+base_url: "http://localhost:8000/v1"  
+# Set `start_vllm_on_init` to automatically start vllm server if it's not manually started yet
+start_vllm_on_init: true
+# Specifying vllm_server_params with the parameters you want to pass to the vllm server command `vllm serve $model $vllm_server_params`
+# Refer to each LLM's model page for details on the recommended parameters
+# It's recommended to stay with `--max-num-seqs` 1 as the voice agent currently supports one connection at a time.
+# You can try increasing the model's max context len `--max-model-len` if GPU memory allows, or decrease it if GPU OOM occurs.
+vllm_server_params: "--trust-remote-code --max-num-seqs 1 --gpu-memory-utilization 0.85"  
+# `params` are the inference parameters that would be passed into OpenAI API, 
+# please put additional model-specific parameters in `extra`
+vllm_generation_params:
+  frequency_penalty: 0.0  # Penalty for frequent tokens (-2.0 to 2.0).
+  presence_penalty: 0.0  # Penalty for new tokens (-2.0 to 2.0).
+  seed: 42  # Random seed for deterministic outputs.
+  temperature: ${llm.temperature}  # Sampling temperature (0.0 to 2.0).
+  top_k: null  # Top-k sampling parameter (currently ignored by OpenAI).
+  top_p: ${llm.top_p}  # Top-p (nucleus) sampling parameter (0.0 to 1.0).
+  max_completion_tokens: ${llm.max_new_tokens}  # max number of tokens to generate
+  extra: null  # additional model specific params can be specified in dict format
@@ -0,0 +1,58 @@
+# This is an example config for setting up Qwen3-8B model in thinking mode for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+# type: auto  # choices in ['auto', 'hf', 'vllm']
+# model: "Qwen/Qwen3-8B"  # model name for HF models, will be used via `AutoModelForCausalLM.from_pretrained()`
+# device: "cuda"
+dtype: bfloat16  # torch.dtype for LLM
+system_role: "system"  # role for system prompt, set it to `user` for models that do not support system prompt
+system_prompt_suffix: "/think"  # a string that would be appended to the system prompt, used to enable/disable thinking
+enable_tool_calling: False  # Tool calling is not supported yet in this specific configuration of the model
+
+##############################
+## Common generation config ##
+##############################
+temperature: 0.7  # LLM sampling params
+top_k: 20  # LLM sampling params
+top_p: 0.8  # LLM sampling params
+min_p: 0.0  # LLM sampling params
+max_new_tokens: 256  # max num of output tokens from LLM
+##############################
+##### HuggingFace config #####
+##############################
+# Please refer to the model page of each HF LLM model to set following params properly.
+# kwargs that will be passed into tokenizer.apply_chat_template() function
+apply_chat_template_kwargs:  
+  add_generation_prompt: true  # This is required in most cases, do not change unless you're sure of it
+  tokenize: false  # This is required, do not change
+# kwargs that will be passed into model.generate() function of HF models
+generation_kwargs:  
+  temperature: ${llm.temperature}  # LLM sampling params
+  top_k: ${llm.top_k}  # LLM sampling params
+  top_p: ${llm.top_p}  # LLM sampling params
+  min_p: ${llm.min_p} # LLM sampling params
+  max_new_tokens: ${llm.max_new_tokens}  # max num of output tokens from LLM
+  do_sample: true # enable sampling
+##############################
+######## vLLM config #########
+##############################
+api_key: "EMPTY"
+base_url: "http://localhost:8000/v1"  
+# Set `start_vllm_on_init` to automatically start vllm server if it's not manually started yet
+start_vllm_on_init: true
+# Specifying vllm_server_params with the parameters you want to pass to the vllm server command `vllm serve $model $vllm_server_params`
+# Refer to each LLM's model page for details on the recommended parameters
+# It's recommended to stay with `--max-num-seqs` 1 as the voice agent currently supports one connection at a time.
+# You can try increasing the model's max context len `--max-model-len` if GPU memory allows, or decrease it if GPU OOM occurs.
+vllm_server_params: "--trust-remote-code --max-num-seqs 1 --gpu-memory-utilization 0.85"  
+# `params` are the inference parameters that would be passed into OpenAI API, 
+# please put additional model-specific parameters in `extra`
+vllm_generation_params:
+  frequency_penalty: 0.0  # Penalty for frequent tokens (-2.0 to 2.0).
+  presence_penalty: 0.0  # Penalty for new tokens (-2.0 to 2.0).
+  seed: 42  # Random seed for deterministic outputs.
+  temperature: ${llm.temperature}  # Sampling temperature (0.0 to 2.0).
+  top_k: null  # Top-k sampling parameter (currently ignored by OpenAI).
+  top_p: ${llm.top_p}  # Top-p (nucleus) sampling parameter (0.0 to 1.0).
+  max_completion_tokens: ${llm.max_new_tokens}  # max number of tokens to generate
+  extra: null  # additional model specific params can be specified in dict format
@@ -0,0 +1,6 @@
+# This is an example config for setting up a NeMo cache-aware streaming ASR model for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+  att_context_size: [70,1] # left and right attention context sizes for streaming ASR
+  frame_len_in_secs: 0.08  # default for FastConformer, do not change unless using other architechtures
+  audio_chunk_size_in_secs: 0.08 
@@ -0,0 +1,15 @@
+# This is an example config for setting up a NeMo FastPitch-Hifigan TTS model for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+main_model_id: "hexgrad/Kokoro-82M"
+sub_model_id: "af_heart" # "af_heart" "af_bella" "am_fenrir" "am_michael"
+device: "cuda"
+speed: 1.25 # Speaking rate
+extra_separator:  # a list of additional punctuations to chunk LLM response into segments for faster TTS output, e.g., ",". Set to `null` to use default behavior 
+  - ','
+  - '\n'
+  - "."
+  - "?"
+  - "!"
+  - ";"
+think_tokens: ["<think>", "</think>"]  # specify them to avoid TTS for thinking process, set to `null` to allow thinking out loud
@@ -0,0 +1,17 @@
+# This is an example config for setting up a nvidia/magpie_tts_multilingual_357m TTS model for a NeMo Voice Agent server.
+# Please refer to https://huggingface.co/nvidia/magpie_tts_multilingual_357m for more details
+
+main_model_id: "nvidia/magpie_tts_multilingual_357m"
+sub_model_id: null
+language: "en"
+speaker: "Sofia"  # choices in ["Sofia", "Aria", "John", "Jason", "Leo"]
+apply_TN: false  # whether to apply text normalization
+device: "cuda"
+extra_separator:  # a list of additional punctuations to chunk LLM response into segments for faster TTS output, e.g., ",". Set to `null` to use default behavior 
+  - ','
+  - '\n'
+  - "."
+  - "?"
+  - "!"
+  - ";"
+think_tokens: ["<think>", "</think>"]  # specify them to avoid TTS for thinking process, set to `null` to allow thinking out loud
@@ -0,0 +1,14 @@
+# This is an example config for setting up a NeMo FastPitch-Hifigan TTS model for a NeMo Voice Agent server.
+# Please refer to https://github.com/NVIDIA-NeMo/NeMo/tree/main/examples/voice_agent/README.md for more details
+
+main_model_id: "nvidia/tts_en_fastpitch"
+sub_model_id: "nvidia/tts_hifigan"
+device: "cuda"
+extra_separator:  # a list of additional punctuations to chunk LLM response into segments for faster TTS output, e.g., ",". Set to `null` to use default behavior 
+  - ','
+  - '\n'
+  - "."
+  - "?"
+  - "!"
+  - ";"
+think_tokens: ["<think>", "</think>"]  # specify them to avoid TTS for thinking process, set to `null` to allow thinking out loud
@@ -0,0 +1,43 @@
+# Copyright (c) 2026, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from urllib.parse import urlsplit
+
+
+def _normalize_websocket_scheme(scheme: str) -> str:
+    scheme = scheme.strip().lower()
+    if scheme not in {"ws", "wss"}:
+        raise ValueError("WEBSOCKET_SCHEME must be either 'ws' or 'wss'")
+    return scheme
+
+
+def _normalize_websocket_host(host: str) -> str:
+    host = host.strip()
+    if not host:
+        raise ValueError("SERVER_PUBLIC_HOST must not be empty")
+    if "://" in host:
+        parsed_host = urlsplit(host).hostname
+        if not parsed_host:
+            raise ValueError("SERVER_PUBLIC_HOST must include a host name")
+        host = parsed_host
+    return host
+
+
+def build_websocket_url(host: str, port: int, scheme: str = "ws") -> str:
+    """Build the client-facing WebSocket URL from trusted server configuration."""
+    scheme = _normalize_websocket_scheme(scheme)
+    host = _normalize_websocket_host(host)
+    if ":" in host and not host.startswith("["):
+        host = f"[{host}]"
+    return f"{scheme}://{host}:{port}"
@@ -0,0 +1,246 @@
+# Copyright (c) 2025, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import sys
+from pathlib import Path
+
+# Add the local NeMo directory to Python path to use development version
+nemo_root = Path(__file__).resolve().parents[3]
+sys.path.insert(0, str(nemo_root))
+
+import pytest
+from omegaconf import DictConfig, OmegaConf
+from pipecat.audio.vad.silero import VADParams
+
+from nemo.agents.voice_agent.pipecat.services.nemo.diar import NeMoDiarInputParams
+from nemo.agents.voice_agent.pipecat.services.nemo.stt import NeMoSTTInputParams
+from nemo.agents.voice_agent.utils.config_manager import ConfigManager
+
+
+@pytest.fixture
+def voice_agent_server_base_path():
+    """Retrieve the NeMo root path from __file__ variable"""
+    nemo_root_path = Path(__file__).resolve().parents[3]
+
+    # Check if the expected directories exist in the NeMo root
+    expected_dirs = ["nemo", "tests", "examples", "requirements"]
+    existing_dirs = [d.name for d in nemo_root_path.iterdir() if d.is_dir()]
+
+    if not all(sub in existing_dirs for sub in expected_dirs):
+        raise ValueError(
+            f"{nemo_root_path} is not a NeMo root path. Expected dirs: {expected_dirs}, Found dirs: {existing_dirs}"
+        )
+
+    voice_agent_root_path = os.path.join(nemo_root_path, "examples", "voice_agent", "server")
+    return voice_agent_root_path
+
+
+class TestDefaultConfigs:
+    """Test suite for ConfigManager class."""
+
+    @pytest.mark.unit
+    def test_constructor_with_valid_path(self, voice_agent_server_base_path):
+        """Test ConfigManager initialization with valid configuration files."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        # Verify initialization
+        assert config_manager._server_base_path == voice_agent_server_base_path
+        assert config_manager.SAMPLE_RATE == 16000
+        assert config_manager.RAW_AUDIO_FRAME_LEN_IN_SECS == 0.016
+        assert isinstance(config_manager.vad_params, VADParams)
+        assert isinstance(config_manager.stt_params, NeMoSTTInputParams)
+        assert isinstance(config_manager.diar_params, NeMoDiarInputParams)
+
+    @pytest.mark.unit
+    def test_constructor_with_invalid_path(self):
+        """Test ConfigManager initialization with invalid path."""
+        with pytest.raises(FileNotFoundError):
+            ConfigManager("/nonexistent/path")
+
+    @pytest.mark.unit
+    def test_load_model_registry_success(self, voice_agent_server_base_path):
+        """Test successful model registry loading."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+
+        assert config_manager.model_registry is not None
+        assert "llm_models" in config_manager.model_registry
+        assert "tts_models" in config_manager.model_registry
+        assert "stt_models" in config_manager.model_registry
+
+    @pytest.mark.unit
+    def test_configure_stt_nemo_model(self, voice_agent_server_base_path):
+        """Test STT configuration for NeMo model."""
+        # Create necessary files
+        config_manager = ConfigManager(voice_agent_server_base_path)
+
+        # STT_MODEL can be either a fastconformer model or an EOU model (e.g., parakeet_realtime_eou)
+        assert (
+            "stt_en_fastconformer" in config_manager.STT_MODEL or "parakeet_realtime_eou" in config_manager.STT_MODEL
+        )
+        assert isinstance(config_manager.stt_params, NeMoSTTInputParams)
+
+    @pytest.mark.unit
+    def test_configure_stt_with_model_config(self, voice_agent_server_base_path):
+        """Test STT configuration with custom model config."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        assert hasattr(config_manager, "STT_MODEL")
+
+    @pytest.mark.unit
+    def test_configure_diarization(self, voice_agent_server_base_path):
+        """Test diarization configuration."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+
+        assert hasattr(config_manager, "DIAR_MODEL") and isinstance(config_manager.DIAR_MODEL, str)
+        assert hasattr(config_manager, "USE_DIAR") and isinstance(config_manager.USE_DIAR, bool)
+        assert isinstance(config_manager.diar_params, NeMoDiarInputParams)
+
+    @pytest.mark.unit
+    def test_configure_turn_taking(self, voice_agent_server_base_path):
+        """Test turn taking configuration."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        assert hasattr(config_manager, "TURN_TAKING_BACKCHANNEL_PHRASES_PATH") and isinstance(
+            config_manager.TURN_TAKING_BACKCHANNEL_PHRASES_PATH, str
+        )
+        assert hasattr(config_manager, "TURN_TAKING_MAX_BUFFER_SIZE") and isinstance(
+            config_manager.TURN_TAKING_MAX_BUFFER_SIZE, int
+        )
+        assert hasattr(config_manager, "TURN_TAKING_BOT_STOP_DELAY") and isinstance(
+            config_manager.TURN_TAKING_BOT_STOP_DELAY, float
+        )
+
+    @pytest.mark.unit
+    def test_configure_turn_taking_backchannel_phrases(self, voice_agent_server_base_path):
+        """Test turn taking configuration."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        # Load backchannel phrases yaml file
+        file_path = os.path.join(
+            voice_agent_server_base_path, os.path.basename(config_manager.TURN_TAKING_BACKCHANNEL_PHRASES_PATH)
+        )
+        assert os.path.exists(file_path)
+        with open(file_path, "r") as f:
+            backchannel_phrases = OmegaConf.load(f)
+        backchannel_phrases = list(backchannel_phrases)
+        assert isinstance(backchannel_phrases, list)
+        assert all(isinstance(item, str) for item in backchannel_phrases)
+
+    @pytest.mark.unit
+    def test_configure_llm_with_registry_model(self, voice_agent_server_base_path):
+        """Test LLM configuration with model from registry."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+
+        assert hasattr(config_manager, "SYSTEM_ROLE") and isinstance(config_manager.SYSTEM_ROLE, str)
+        assert hasattr(config_manager, "SYSTEM_PROMPT") and isinstance(config_manager.SYSTEM_PROMPT, str)
+
+    @pytest.mark.unit
+    def test_configure_llm_with_file_system_prompt(self, voice_agent_server_base_path):
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        assert hasattr(config_manager, "SYSTEM_PROMPT") and isinstance(config_manager.SYSTEM_PROMPT, str)
+
+    @pytest.mark.unit
+    def test_configure_llm_reasoning_model(self, voice_agent_server_base_path):
+        """Test LLM configuration for reasoning model."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        assert hasattr(config_manager, "SYSTEM_ROLE") and isinstance(config_manager.SYSTEM_ROLE, str)
+
+    @pytest.mark.unit
+    def test_configure_llm_fallback_to_generic(self, voice_agent_server_base_path):
+        """Test LLM configuration fallback to generic HF model."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        assert hasattr(config_manager, "SYSTEM_ROLE") and isinstance(config_manager.SYSTEM_ROLE, str)
+
+    @pytest.mark.unit
+    def test_configure_tts_nemo_model(self, voice_agent_server_base_path):
+        """Test TTS configuration for NeMo model."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+
+        assert hasattr(config_manager, "TTS_MAIN_MODEL_ID")
+        assert hasattr(config_manager, "TTS_SUB_MODEL_ID")
+        assert hasattr(config_manager, "TTS_DEVICE")
+
+    @pytest.mark.unit
+    def test_configure_tts_with_optional_params(self, voice_agent_server_base_path):
+        """Test TTS configuration with optional parameters."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+
+        assert hasattr(config_manager, "TTS_THINK_TOKENS") and isinstance(config_manager.TTS_THINK_TOKENS, list)
+        assert all(isinstance(item, str) for item in config_manager.TTS_THINK_TOKENS)
+        assert hasattr(config_manager, "TTS_EXTRA_SEPARATOR") and isinstance(config_manager.TTS_EXTRA_SEPARATOR, list)
+        assert all(isinstance(item, str) for item in config_manager.TTS_EXTRA_SEPARATOR)
+
+    @pytest.mark.unit
+    def test_get_server_config(self, voice_agent_server_base_path):
+        """Test get_server_config method."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        server_config = config_manager.get_server_config()
+
+        assert isinstance(server_config, DictConfig)
+        assert hasattr(server_config.transport, "audio_out_10ms_chunks")
+        assert isinstance(server_config.transport.audio_out_10ms_chunks, int)
+
+    @pytest.mark.unit
+    def test_get_model_registry(self, voice_agent_server_base_path):
+        """Test get_model_registry method."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        model_registry = config_manager.get_model_registry()
+        assert isinstance(model_registry, DictConfig)
+        assert "llm_models" in model_registry
+        assert "tts_models" in model_registry
+        assert "stt_models" in model_registry
+
+    @pytest.mark.unit
+    def test_get_vad_params(self, voice_agent_server_base_path):
+        """Test get_vad_params method."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        vad_params = config_manager.get_vad_params()
+
+        assert isinstance(vad_params, VADParams)
+        assert isinstance(vad_params.confidence, float) and 0.0 <= vad_params.confidence <= 1.0
+        assert isinstance(vad_params.start_secs, float) and vad_params.start_secs >= 0.0
+        assert isinstance(vad_params.stop_secs, float) and vad_params.stop_secs >= 0.0
+        assert isinstance(vad_params.min_volume, float) and 0.0 <= vad_params.min_volume <= 1.0
+
+    @pytest.mark.unit
+    def test_get_stt_params(self, voice_agent_server_base_path):
+        """Test get_stt_params method."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        stt_params = config_manager.get_stt_params()
+
+        assert isinstance(stt_params, NeMoSTTInputParams)
+        assert isinstance(stt_params.att_context_size, list)
+        assert all(isinstance(item, int) for item in stt_params.att_context_size)
+        assert isinstance(stt_params.frame_len_in_secs, float) and 0.0 <= stt_params.frame_len_in_secs <= 1.0
+        assert (
+            isinstance(stt_params.raw_audio_frame_len_in_secs, float)
+            and 0.0 <= stt_params.raw_audio_frame_len_in_secs <= 1.0
+        )
+
+    @pytest.mark.unit
+    def test_get_diar_params(self, voice_agent_server_base_path):
+        """Test get_diar_params method."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        diar_params = config_manager.get_diar_params()
+
+        assert isinstance(diar_params, NeMoDiarInputParams)
+        assert hasattr(diar_params, "frame_len_in_secs") and isinstance(diar_params.frame_len_in_secs, float)
+        assert hasattr(diar_params, "threshold") and isinstance(diar_params.threshold, float)
+
+    @pytest.mark.unit
+    def test_transport_configuration(self, voice_agent_server_base_path):
+        """Test transport configuration."""
+        config_manager = ConfigManager(voice_agent_server_base_path)
+        assert hasattr(config_manager, "TRANSPORT_AUDIO_OUT_10MS_CHUNKS")
+        if not isinstance(config_manager.TRANSPORT_AUDIO_OUT_10MS_CHUNKS, int):
+            raise ValueError(
+                f"TRANSPORT_AUDIO_OUT_10MS_CHUNKS is not an integer: {config_manager.TRANSPORT_AUDIO_OUT_10MS_CHUNKS}"
+            )
@@ -0,0 +1,45 @@
+# Copyright (c) 2026, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+from pathlib import Path
+
+import pytest
+
+voice_agent_server_path = Path(__file__).resolve().parents[1] / "server"
+sys.path.insert(0, str(voice_agent_server_path))
+
+from websocket_url import build_websocket_url
+
+
+@pytest.mark.unit
+def test_build_websocket_url_uses_configured_host():
+    assert build_websocket_url("voice-agent.example", 8765) == "ws://voice-agent.example:8765"
+
+
+@pytest.mark.unit
+def test_build_websocket_url_does_not_accept_request_host():
+    forged_request_host = "evil.example"
+
+    ws_url = build_websocket_url("voice-agent.example", 8765)
+
+    assert forged_request_host not in ws_url
+    assert ws_url == "ws://voice-agent.example:8765"
+
+
+@pytest.mark.unit
+@pytest.mark.parametrize("scheme", ["http", "https", ""])
+def test_build_websocket_url_rejects_non_websocket_schemes(scheme):
+    with pytest.raises(ValueError):
+        build_websocket_url("voice-agent.example", 8765, scheme=scheme)