microsoft--vibevoice/docs/setup_gradio_demo.md

VibeVoice Gradio Demo Setup Guide

End-to-end instructions to deploy the VibeVoice ASR server and launch the Gradio web demo.

Prerequisites

• CUDA-capable GPU(s)
• Docker with GPU support (nvidia-docker)
• VibeVoice repository cloned locally

git clone https://github.com/microsoft/VibeVoice.git
cd VibeVoice

---

Step 1 — Start the ASR Server

Launch a Docker container running the vLLM ASR server. The launcher script handles everything automatically (system deps, pip install, model download, tokenizer generation, server start).

Single GPU (default)

docker run -d --gpus '"device=0"' --name vibevoice-asr-demo \
  --ipc=host \
  -p 6001:6001 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --port 6001"

Multi-GPU with Data Parallel (load balancing)

Run 4 independent replicas, one per GPU. vLLM distributes requests automatically:

docker run -d --gpus '"device=0,1,2,3"' --name vibevoice-asr-demo \
  --ipc=host \
  -p 6001:6001 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --port 6001 --dp 4"

Tip

: Use --dp N for N-way data parallel (throughput scaling). Use --tp N for tensor parallel (large models). See docs/vibevoice-vllm-asr.md for details.

Check Logs

docker logs -f vibevoice-asr-demo

Wait until you see Application startup complete. — this means the server is ready.

---

Step 2 — Verify the Server

# Check the model is loaded
curl http://localhost:6001/v1/models

Expected output:

{
  "data": [{ "id": "vibevoice", ... }]
}

Quick Test with Audio File

docker exec -it vibevoice-asr-demo \
  python3 /app/vllm_plugin/tests/test_api.py /app/en-Alice_woman.wav \
  --url http://localhost:6001

---

Step 3 — Launch the Gradio Demo

Install tmux inside the container (to keep it running)

docker exec vibevoice-asr-demo apt-get install -y tmux

Start Gradio in tmux

docker exec vibevoice-asr-demo bash -c \
  "PYTHONUNBUFFERED=1 tmux new-session -d -s gradio \
  'PYTHONUNBUFFERED=1 python3 /app/vllm_plugin/scripts/gradio_asr_demo_api_video.py \
  --api_url http://localhost:6001 --share \
  2>&1 | tee /tmp/gradio.log'"

Get the Share Link

Wait ~20 seconds, then:

docker exec vibevoice-asr-demo cat /tmp/gradio.log

You should see:

✅ Connected to API: http://localhost:6001 | Model: vibevoice
🚀 Starting VibeVoice ASR Demo
* Running on local URL:  http://0.0.0.0:7860
* Running on public URL: https://xxxxxx.gradio.live

The gradio.live link is publicly accessible (valid for 1 week).

Gradio Options

Flag	Description	Default
--api_url URL	vLLM server URL	http://localhost:8000
--share	Create a public Gradio link	off
--port PORT	Local Gradio port	7860
--cloudflared	Use Cloudflare tunnel instead of Gradio share	off
--max_video_size MB	Max upload video size	50

---

Managing the Service

Stop Gradio (keep ASR server running)

docker exec vibevoice-asr-demo tmux kill-session -t gradio

Restart Gradio

Re-run the tmux command from Step 3.

Stop Everything

docker stop vibevoice-asr-demo
docker rm vibevoice-asr-demo

---

Example: Full Setup on GPU 0 with Port 6001

# 1. Start server
docker run -d --gpus '"device=0"' --name vibevoice-asr-demo \
  --ipc=host -p 6001:6001 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v $(pwd):/app -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_server.py --port 6001"

# 2. Wait for startup (~2 min), then verify
docker logs -f vibevoice-asr-demo  # wait for "Application startup complete."
curl http://localhost:6001/v1/models

# 3. Install tmux and launch Gradio
docker exec vibevoice-asr-demo apt-get install -y tmux
docker exec vibevoice-asr-demo bash -c \
  "PYTHONUNBUFFERED=1 tmux new-session -d -s gradio \
  'PYTHONUNBUFFERED=1 python3 /app/vllm_plugin/scripts/gradio_asr_demo_api_video.py \
  --api_url http://localhost:6001 --share \
  2>&1 | tee /tmp/gradio.log'"

# 4. Get the public link
sleep 20 && docker exec vibevoice-asr-demo cat /tmp/gradio.log

Troubleshooting

Issue	Fix
CUDA out of memory	Use a different GPU (device=X) or reduce --gpu-memory-utilization 0.7 in start_server.py
Gradio log is empty	Wait longer (~30s); Gradio buffers output. Use PYTHONUNBUFFERED=1 as shown above
Port already in use	Pick a different port or stop the existing container: docker stop <name> && docker rm <name>
Share link shows "No interface"	Gradio is still loading. Wait for Application startup complete in the log
tmux: command not found	Run docker exec <container> apt-get install -y tmux first