A low-latency bidirectional streaming pure-python application that enables voice conversations with Gemini using Gradio. This demo showcases real-time audio streaming capabilities, allowing natural conversations where you can speak with Gemini and receive audio responses back.

Features

Real-time audio streaming using WebRTC
Voice-to-voice conversations with Gemini
Multiple voice options for Gemini's responses
Simple web interface built with Gradio
Low-latency bidirectional communication

Prerequisites

Before running the application, you need to:

Enable Vertex AI in your Google Cloud Project
- Visit Enable Vertex AI
Set up authentication
- Configure default credentials following Google Cloud Authentication Guide

Installation

Clone this repository:

git clone git@github.com:GoogleCloudPlatform/generative-ai.git
cd generative-ai/gemini/multimodal-live-api/gradio-voice

Install the required dependencies:
```
pip install -r requirements.txt
```

The main dependencies are:

fastrtc
librosa
google-genai

Usage

Run the application:
```
python app.py
```
Open your web browser and navigate to the local Gradio interface (typically http://localhost:7860)
Configure the application:
- Enter your Google Cloud Project ID
- Select your preferred region (default: us-central1)
- Choose a voice for Gemini (options: Puck, Charon, Kore, Fenrir, Aoede)
- Click "Submit"
Start the conversation:
- Click the record button to start speaking
- Listen to Gemini's responses through your speakers

Configuration Options

Regions

The application supports any Google Cloud regions, with presets for:

us-central1 (default)
us-east5
us-south1
us-west4
us-east4
us-east1
us-west1

Additional locations can be found in the Vertex AI documentation.

Voice Options

Gemini can respond using different voice personalities:

Puck (default)
Charon
Kore
Fenrir
Aoede

Technical Details

The application uses:

Gradio for the web interface
WebRTC for real-time audio streaming
Google Gen AI SDK for Gemini integration
Vertex AI for model hosting and inference

The audio streaming implementation uses:

16kHz input sample rate
24kHz output sample rate
PCM audio format
Base64 encoding for audio transmission

Deployment Notes

For deploying behind a firewall, you may need to modify the WebRTC configuration. See the Gradio WebRTC deployment documentation for more details.