wehub/google--adk-python
1
0
Fork 0
Code Issues Pull Requests Actions 6 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
Continuous Integration / Pre-commit Linter (push) Has been cancelled
Details
Continuous Integration / Mypy Check (Python 3.10) (push) Has been cancelled
Details
Continuous Integration / Mypy Check (Python 3.11) (push) Has been cancelled
Details
Continuous Integration / Mypy Check (Python 3.12) (push) Has been cancelled
Details
Continuous Integration / Mypy Check (Python 3.13) (push) Has been cancelled
Details
Continuous Integration / Unit Tests (Python 3.10) (push) Has been cancelled
Details
Continuous Integration / Unit Tests (Python 3.11) (push) Has been cancelled
Details
Continuous Integration / Unit Tests (Python 3.12) (push) Has been cancelled
Details
Continuous Integration / Unit Tests (Python 3.13) (push) Has been cancelled
Details
Continuous Integration / Unit Tests (Python 3.14) (push) Has been cancelled
Details
Continuous Integration / A2A v0.3 Tests (Python 3.10) (push) Has been cancelled
Details
Continuous Integration / A2A v0.3 Tests (Python 3.11) (push) Has been cancelled
Details
Continuous Integration / A2A v0.3 Tests (Python 3.12) (push) Has been cancelled
Details
Copybara PR Handler / close-imported-pr (push) Has been cancelled
Details
Continuous Integration / A2A v0.3 Tests (Python 3.13) (push) Has been cancelled
Details
Continuous Integration / A2A v0.3 Tests (Python 3.14) (push) Has been cancelled
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:25:13 +08:00
commit ec2b666284
2231 changed files with 491535 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
contributing/samples/live/live_agent_api_server_example/check_prime_11.wav
BIN
View File
Binary file not shown.
contributing/samples/live/live_agent_api_server_example/check_prime_15.wav
BIN
View File
Binary file not shown.
contributing/samples/live/live_agent_api_server_example/live_agent_example.py
+825
View File
@@ -0,0 +1,825 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import asyncio
import base64
import json
import logging
import os
import re
import sys
import urllib.parse
import httpx
import pyaudio
import websockets
# --- Optional: For Audio Recording ---
# This is used to record audios for debugging purposes.
try:
import numpy as np # PyAudio will need NumPy for WAV conversion if not already int16
import sounddevice as sd # Sounddevice is for recording in this setup
AUDIO_RECORDING_ENABLED = True
except ImportError:
print(
"WARNING: Sounddevice or numpy not found. Audio RECORDING will be"
" disabled."
)
AUDIO_RECORDING_ENABLED = False
# --- PyAudio Playback Enabled Flag ---
# We assume PyAudio is for playback. If its import failed, this would be an issue.
# For simplicity, we'll try to initialize it and handle errors there.
AUDIO_PLAYBACK_ENABLED = True # Will be set to False if init fails
# --- Configure Logging ---
LOG_FILE_NAME = "websocket_client.log"
LOG_FILE_PATH = os.path.abspath(LOG_FILE_NAME)
logging.basicConfig(
level=logging.INFO,
format=(
"%(asctime)s - %(levelname)s - [%(filename)s:%(lineno)d] (%(funcName)s)"
" - %(message)s"
),
handlers=[
logging.FileHandler(LOG_FILE_PATH, mode="w"),
logging.StreamHandler(sys.stdout),
],
)
print(
f"INFO: Logging to console and to file. Log file location: {LOG_FILE_PATH}",
flush=True,
)
logging.info(f"Logging configured. Logs will also be saved to: {LOG_FILE_PATH}")
if not AUDIO_RECORDING_ENABLED:
logging.warning("Audio RECORDING is disabled due to missing libraries.")
# --- Configuration ---
SERVER_HOST = "127.0.0.1"
SERVER_PORT = 8000
# APP_NAME is the folder name of your agent.
APP_NAME = "hello_world"
# The following default ones also work
USER_ID = "your_user_id_123"
SESSION_ID = "your_session_id_abc"
MODALITIES = ["TEXT", "AUDIO"]
REC_AUDIO_SAMPLE_RATE = 16000 # Matches SEND_SAMPLE_RATE from old code
REC_AUDIO_CHANNELS = 1 # Matches CHANNELS from old code
REC_AUDIO_FORMAT_PYAUDIO = pyaudio.paInt16 # Matches FORMAT from old code
REC_AUDIO_CHUNK_SIZE = 1024 # Matches CHUNK_SIZE from old code
REC_AUDIO_MIME_TYPE = "audio/pcm" # This remains critical
# Recording parameters
REC_AUDIO_SAMPLE_RATE = 16000
REC_AUDIO_CHANNELS = 1
REC_AUDIO_FORMAT_DTYPE = "int16" # Sounddevice dtype
# REC_AUDIO_MIME_TYPE = "audio/wav" # We'll send WAV to server
REC_AUDIO_MIME_TYPE = "audio/pcm"
REC_AUDIO_SOUNDFILE_SUBTYPE = "PCM_16" # Soundfile subtype for WAV
# PyAudio Playback Stream Parameters
PYAUDIO_PLAY_RATE = 24000
PYAUDIO_PLAY_CHANNELS = 1
PYAUDIO_PLAY_FORMAT = pyaudio.paInt16
PYAUDIO_PLAY_FORMAT_NUMPY = np.int16 if AUDIO_RECORDING_ENABLED else None # type: ignore
PYAUDIO_FRAMES_PER_BUFFER = 1024
AUDIO_DURATION_SECONDS = 5 # For single "audio" command
# Global PyAudio instances
pya_interface_instance = None
pya_output_stream_instance = None
# --- Globals for Continuous Audio Streaming ---
is_streaming_audio = False
global_input_stream = None # Holds the sounddevice.InputStream object
audio_stream_task = None # Holds the asyncio.Task for audio streaming
debug_audio_save_count = 0
MAX_DEBUG_AUDIO_SAMPLES = 3 # Save first 3 chunks
CHUNK = 4200
FORMAT = pyaudio.paInt16
CHANNELS = 1
RECORD_SECONDS = 5
INPUT_RATE = 16000
OUTPUT_RATE = 24000
config = {
"response_modalities": ["AUDIO"],
"input_audio_transcription": {},
"output_audio_transcription": {},
}
# --- PyAudio Initialization and Cleanup ---
def init_pyaudio_playback():
global pya_interface_instance, pya_output_stream_instance, AUDIO_PLAYBACK_ENABLED
if (
not AUDIO_PLAYBACK_ENABLED
): # If already marked as disabled (e.g. previous attempt failed)
logging.warning("PyAudio playback init skipped as it's marked disabled.")
return False
try:
pya_interface_instance = pyaudio.PyAudio()
logging.info(
f"Initializing PyAudio output stream: Rate={PYAUDIO_PLAY_RATE},"
f" Channels={PYAUDIO_PLAY_CHANNELS}, Format=paInt16"
)
pya_output_stream_instance = pya_interface_instance.open(
format=PYAUDIO_PLAY_FORMAT,
channels=PYAUDIO_PLAY_CHANNELS,
rate=PYAUDIO_PLAY_RATE,
output=True,
frames_per_buffer=PYAUDIO_FRAMES_PER_BUFFER,
)
logging.info("PyAudio output stream initialized successfully.")
AUDIO_PLAYBACK_ENABLED = True
return True
except Exception as e:
logging.error(
f"Failed to initialize PyAudio: {e}. Playback will be disabled.",
exc_info=True,
)
print(
f"ERROR: Failed to initialize PyAudio for playback: {e}. Check"
" PortAudio installation if on Linux/macOS.",
flush=True,
)
if pya_interface_instance: # Terminate if open failed mid-way
try:
pya_interface_instance.terminate()
except:
pass
pya_interface_instance = None
pya_output_stream_instance = None
AUDIO_PLAYBACK_ENABLED = False # Mark as disabled
return False
# --- Payload Creation ---
def create_text_request_payload(text: str) -> str:
live_request_data = {"content": {"parts": [{"text": text}]}}
logging.debug(
f"Created LiveRequest text payload: {json.dumps(live_request_data)}"
)
return json.dumps(live_request_data)
def create_audio_request_payload(audio_bytes: bytes, mime_type: str) -> str:
base64_encoded_audio = base64.b64encode(audio_bytes)
base64_encoded_audio = base64_encoded_audio.decode("utf-8")
live_request_data = {
"blob": {
"mime_type": mime_type,
"data": base64_encoded_audio,
}
}
return json.dumps(live_request_data)
class AudioStreamingComponent:
async def stop_audio_streaming(self):
global is_streaming_audio
if is_streaming_audio:
logging.info("Requesting to stop audio streaming (flag set).")
is_streaming_audio = False
else:
logging.info("Audio streaming is not currently active.")
async def start_audio_streaming(
self,
websocket: websockets.WebSocketClientProtocol,
):
print("Starting continuous audio streaming...")
global is_streaming_audio, global_input_stream, debug_audio_save_count
# IMPORTANT: Reinstate this check
if not AUDIO_RECORDING_ENABLED:
logging.warning("Audio recording disabled. Cannot start stream.")
is_streaming_audio = (
False # Ensure flag is correctly set if we bail early
)
return
is_streaming_audio = True
debug_audio_save_count = 0 # Reset counter for each stream start
logging.info("Starting continuous audio streaming...")
global pya_interface_instance
try:
stream = pya_interface_instance.open(
format=FORMAT,
channels=CHANNELS,
rate=INPUT_RATE,
input=True,
frames_per_buffer=CHUNK,
)
while is_streaming_audio:
try:
audio_data_bytes = stream.read(CHUNK)
if audio_data_bytes:
payload_str = create_audio_request_payload(
audio_data_bytes,
REC_AUDIO_MIME_TYPE, # REC_AUDIO_MIME_TYPE is likely "audio/wav"
)
await websocket.send(payload_str)
# Make sure we sleep to yield control back to other threads(like audio playing)
await asyncio.sleep(10**-12)
else:
logging.warning("Empty audio data chunk from queue, not sending.")
except asyncio.TimeoutError:
continue
except websockets.exceptions.ConnectionClosed as e:
logging.warning(
f"WebSocket connection closed while sending audio stream: {e}"
)
is_streaming_audio = False
break
except Exception as e:
logging.error(
f"Error in audio streaming send loop: {e}", exc_info=True
)
is_streaming_audio = False
break
except Exception as e:
logging.error(
f"Failed to start or run audio InputStream: {e}", exc_info=True
)
is_streaming_audio = False # Ensure flag is reset
finally:
logging.info("Cleaning up audio stream...")
if global_input_stream:
try:
if global_input_stream.active:
global_input_stream.stop()
global_input_stream.close()
logging.info("Sounddevice InputStream stopped and closed.")
except Exception as e_sd_close:
logging.error(
f"Error stopping/closing Sounddevice InputStream: {e_sd_close}"
)
global_input_stream = None
is_streaming_audio = False # Critical to reset this
logging.info("Continuous audio streaming task finished.")
class AgentResponseAudioPlayer:
def cleanup_pyaudio_playback(self):
global pya_interface_instance, pya_output_stream_instance
logging.info("Attempting PyAudio cleanup...")
if pya_output_stream_instance:
try:
if pya_output_stream_instance.is_active(): # Check if stream is active
pya_output_stream_instance.stop_stream()
pya_output_stream_instance.close()
logging.info("PyAudio output stream stopped and closed.")
except Exception as e:
logging.error(f"Error closing PyAudio stream: {e}", exc_info=True)
finally:
pya_output_stream_instance = None
if pya_interface_instance:
try:
pya_interface_instance.terminate()
logging.info("PyAudio interface terminated.")
except Exception as e:
logging.error(
f"Error terminating PyAudio interface: {e}", exc_info=True
)
finally:
pya_interface_instance = None
logging.info("PyAudio cleanup process finished.")
# --- Audio Playback Handler (using PyAudio) ---
def _play_audio_pyaudio_handler(
self, audio_bytes: bytes, mime_type_full: str
):
if not AUDIO_PLAYBACK_ENABLED or not pya_output_stream_instance:
logging.warning(
"PyAudio stream not available or playback disabled. Cannot play"
" audio."
)
return
try:
logging.debug(
f"PyAudio handler: Mime='{mime_type_full}', Size={len(audio_bytes)}"
)
playable_data_bytes = None
mime_type_base = mime_type_full.split(";")[0].strip().lower()
if mime_type_base == "audio/pcm":
# Check rate from MIME type like "audio/pcm;rate=24000"
match = re.search(r"rate=(\d+)", mime_type_full, re.IGNORECASE)
current_audio_rate = PYAUDIO_PLAY_RATE # Fallback to stream's rate
if match:
try:
current_audio_rate = int(match.group(1))
except ValueError:
logging.warning(
f"Could not parse rate from '{mime_type_full}', using stream"
f" default {PYAUDIO_PLAY_RATE}Hz."
)
if current_audio_rate != PYAUDIO_PLAY_RATE:
logging.warning(
f"Received PCM audio at {current_audio_rate}Hz but PyAudio stream"
f" is {PYAUDIO_PLAY_RATE}Hz. Playback speed/pitch will be"
" affected. Resampling would be needed for correct playback."
)
# We will play it at PYAUDIO_PLAY_RATE, which will alter speed/pitch if rates differ.
# We assume the incoming PCM data is 1 channel, 16-bit, matching the stream.
# If server sent different channel count or bit depth, conversion would be needed.
playable_data_bytes = audio_bytes
logging.info(
"Preparing raw PCM for PyAudio stream (target rate"
f" {PYAUDIO_PLAY_RATE}Hz)."
)
else:
logging.warning(
f"Unsupported MIME type for PyAudio playback: {mime_type_full}"
)
return
if playable_data_bytes:
pya_output_stream_instance.write(playable_data_bytes)
logging.info(
"Audio chunk written to PyAudio stream (Size:"
f" {len(playable_data_bytes)} bytes)."
)
else:
logging.warning("No playable bytes prepared for PyAudio.")
except Exception as e:
logging.error(
f"Error in _blocking_play_audio_pyaudio_handler: {e}", exc_info=True
)
async def play_audio_data(self, audio_bytes: bytes, mime_type: str):
if not AUDIO_PLAYBACK_ENABLED:
logging.debug(
"PyAudio Playback is disabled, skipping play_audio_data call."
)
return
print(f"Scheduling PyAudio playback for {mime_type} audio.")
await asyncio.to_thread(
self._play_audio_pyaudio_handler, audio_bytes, mime_type
)
# --- Session Management ---
async def ensure_session_exists(
app_name: str,
user_id: str,
session_id: str,
server_host: str,
server_port: int,
) -> bool:
session_url = f"http://{server_host}:{server_port}/apps/{app_name}/users/{user_id}/sessions/{session_id}"
try:
async with httpx.AsyncClient() as client:
logging.info(f"Checking if session exists via GET: {session_url}")
response_get = await client.get(session_url, timeout=10)
if response_get.status_code == 200:
logging.info(f"Session '{session_id}' already exists.")
return True
elif response_get.status_code == 404:
logging.info(
f"Session '{session_id}' not found. Attempting to create via POST."
)
response_post = await client.post(session_url, json={}, timeout=10)
if response_post.status_code == 200:
logging.info(f"Session '{session_id}' created.")
return True
else:
logging.error(
f"Failed to create session '{session_id}'. POST Status:"
f" {response_post.status_code}"
)
return False
else:
logging.warning(
f"Could not verify session '{session_id}'. GET Status:"
f" {response_get.status_code}"
)
return False
except Exception as e:
logging.error(f"Error ensuring session '{session_id}': {e}", exc_info=True)
return False
async def websocket_client():
global audio_stream_task
logging.info("websocket_client function started.")
# --- ADD THIS SECTION FOR DEVICE DIAGNOSTICS ---
if AUDIO_RECORDING_ENABLED:
try:
print("-" * 30)
print("Available audio devices:")
devices = sd.query_devices()
print(devices)
print(f"Default input device: {sd.query_devices(kind='input')}")
print(f"Default output device: {sd.query_devices(kind='output')}")
print("-" * 30)
except Exception as e_dev:
logging.error(f"Could not query audio devices: {e_dev}")
# --- END DEVICE DIAGNOSTICS ---
if not init_pyaudio_playback():
logging.warning("PyAudio playback could not be initialized.")
agent_response_audio_player = AgentResponseAudioPlayer()
audio_streaming_component = AudioStreamingComponent()
if (
APP_NAME == "hello_world"
or USER_ID.startswith("your_user_id")
or SESSION_ID.startswith("your_session_id")
):
logging.warning("Using default/example APP_NAME, USER_ID, or SESSION_ID.")
session_ok = await ensure_session_exists(
APP_NAME, USER_ID, SESSION_ID, SERVER_HOST, SERVER_PORT
)
if not session_ok:
logging.error(
f"Critical: Could not ensure session '{SESSION_ID}'. Aborting."
)
return
params = {
"app_name": APP_NAME,
"user_id": USER_ID,
"session_id": SESSION_ID,
"modalities": MODALITIES,
}
uri = (
f"ws://{SERVER_HOST}:{SERVER_PORT}/run_live?{urllib.parse.urlencode(params, doseq=True)}"
)
logging.info(f"Attempting to connect to WebSocket: {uri}")
try:
async with websockets.connect(
uri, open_timeout=10, close_timeout=10
) as websocket:
logging.info(f"Successfully connected to WebSocket: {uri}.")
async def receive_messages(websocket: websockets.WebSocketClientProtocol):
# ... (Logic for parsing event_data and finding audio part is the same) ...
# ... (When audio part is found, call `await play_audio_data(audio_bytes_decoded, mime_type_full)`) ...
logging.info("Receiver task started: Listening for server messages...")
try:
async for message in websocket:
# logging.info(f"<<< Raw message from server: {message[:500]}...")
try:
event_data = json.loads(message)
logging.info(
"<<< Parsed event from server: (Keys:"
f" {list(event_data.keys())})"
)
if "content" in event_data and isinstance(
event_data["content"], dict
):
content_obj = event_data["content"]
if "parts" in content_obj and isinstance(
content_obj["parts"], list
):
for part in content_obj["parts"]:
if isinstance(part, dict) and "inlineData" in part:
inline_data = part["inlineData"]
if (
isinstance(inline_data, dict)
and "mimeType" in inline_data
and isinstance(inline_data["mimeType"], str)
and inline_data["mimeType"].startswith("audio/")
and "data" in inline_data
and isinstance(inline_data["data"], str)
):
audio_b64 = inline_data["data"]
mime_type_full = inline_data["mimeType"]
logging.info(
f"Audio part found: Mime='{mime_type_full}',"
f" Base64Len={len(audio_b64)}"
)
try:
standard_b64_string = audio_b64.replace(
"-", "+"
).replace("_", "/")
missing_padding = len(standard_b64_string) % 4
if missing_padding:
standard_b64_string += "=" * (4 - missing_padding)
audio_bytes_decoded = base64.b64decode(
standard_b64_string
)
if audio_bytes_decoded:
await agent_response_audio_player.play_audio_data(
audio_bytes_decoded, mime_type_full
)
else:
logging.warning(
"Decoded audio data is empty after sanitization"
" and padding."
)
except base64.binascii.Error as b64e:
# Log details if decoding still fails
logging.error(
"Base64 decode error after sanitization and"
" padding."
f" Error: {b64e}"
)
except Exception as e:
logging.error(
"Error processing audio for playback (original"
f" string prefix: '{audio_b64[:50]}...'): {e}",
exc_info=True,
)
except json.JSONDecodeError:
logging.warning(f"Received non-JSON: {message}")
except Exception as e:
logging.error(f"Error processing event: {e}", exc_info=True)
except websockets.exceptions.ConnectionClosed as e:
logging.warning(
f"Receiver: Connection closed (Code: {e.code}, Reason:"
f" '{e.reason if e.reason else 'N/A'}')"
)
except Exception as e:
logging.error("Receiver: Unhandled error", exc_info=True)
finally:
logging.info("Receiver task finished.")
async def send_messages_local(ws: websockets.WebSocketClientProtocol):
global audio_stream_task, is_streaming_audio
logging.info(
"Sender task started: Type 'start_stream', 'stop_stream', text,"
"sendfile, or 'quit'."
)
while True:
await asyncio.sleep(10**-12)
try:
user_input = await asyncio.to_thread(input, "Enter command: ")
if user_input.lower() == "quit":
logging.info("Sender: 'quit' received.")
if audio_stream_task and not audio_stream_task.done():
logging.info(
"Sender: Stopping active audio stream due to quit command."
)
await audio_streaming_component.stop_audio_streaming()
await audio_stream_task
audio_stream_task = None
break
elif user_input.lower() == "start_stream":
if audio_stream_task and not audio_stream_task.done():
logging.warning("Sender: Audio stream is already running.")
continue
audio_stream_task = asyncio.create_task(
audio_streaming_component.start_audio_streaming(ws)
)
logging.info("Sender: Audio streaming task initiated.")
elif user_input.lower() == "stop_stream":
if audio_stream_task and not audio_stream_task.done():
logging.info("Sender: Requesting to stop audio stream.")
await audio_streaming_component.stop_audio_streaming()
await audio_stream_task
audio_stream_task = None
logging.info("Sender: Audio streaming task stopped and joined.")
else:
logging.warning(
"Sender: Audio stream is not currently running or already"
" stopped."
)
# The 'audio' command for single recording was commented out in your version.
# If you need it, uncomment the block from my previous response.
elif user_input.lower().startswith("sendfile "):
if (
audio_stream_task
and isinstance(audio_stream_task, asyncio.Task)
and not audio_stream_task.done()
):
logging.warning(
"Please stop the current audio stream with 'stop_stream'"
" before sending a file."
)
continue
filepath = user_input[len("sendfile ") :].strip()
# fix filepath for testing
# filepath = "roll_and_check_audio.wav"
# Remove quotes if user added them around the filepath
filepath = filepath.strip("\"'")
if not os.path.exists(filepath):
logging.error(f"Audio file not found: {filepath}")
print(
f"Error: File not found at '{filepath}'. Please check the"
" path."
)
continue
if not filepath.lower().endswith(".wav"):
logging.warning(
f"File {filepath} does not end with .wav. Attempting to"
" send anyway."
)
print(
f"Warning: File '{filepath}' is not a .wav file. Ensure"
" it's a compatible WAV."
)
try:
logging.info(f"Reading audio file: {filepath}")
with open(filepath, "rb") as f:
audio_file_bytes = f.read()
# We assume the file is already in WAV format.
# REC_AUDIO_MIME_TYPE is "audio/wav"
payload_str = create_audio_request_payload(
audio_file_bytes, REC_AUDIO_MIME_TYPE
)
logging.info(
">>> Sending audio file"
f" {os.path.basename(filepath)} (Size:"
f" {len(audio_file_bytes)} bytes) with MIME type"
f" {REC_AUDIO_MIME_TYPE}"
)
await ws.send(payload_str)
logging.info("Audio file sent.")
print(f"Successfully sent {os.path.basename(filepath)}.")
except Exception as e_sendfile:
logging.error(
f"Error sending audio file {filepath}: {e_sendfile}",
exc_info=True,
)
print(f"Error sending file: {e_sendfile}")
else: # Text input
if not user_input.strip(): # Prevent sending empty messages
logging.info("Sender: Empty input, not sending.")
continue
payload_str = create_text_request_payload(user_input)
logging.info(f">>> Sending text: {user_input[:100]}")
await ws.send(payload_str)
except EOFError: # Handles Ctrl+D
logging.info("Sender: EOF detected (Ctrl+D).")
if audio_stream_task and not audio_stream_task.done():
await audio_streaming_component.stop_audio_streaming()
await audio_stream_task
audio_stream_task = None
break
except websockets.exceptions.ConnectionClosed as e:
logging.warning(
f"Sender: WebSocket connection closed. Code: {e.code}, Reason:"
f" {e.reason}"
)
if audio_stream_task and not audio_stream_task.done():
is_streaming_audio = False # Signal loop
try:
await asyncio.wait_for(audio_stream_task, timeout=2.0)
except asyncio.TimeoutError:
audio_stream_task.cancel()
except Exception as ex:
logging.error(f"Error during stream stop on conn close: {ex}")
audio_stream_task = None
break
except Exception as e_send_loop:
logging.error(
f"Sender: Unhandled error: {e_send_loop}", exc_info=True
)
if audio_stream_task and not audio_stream_task.done():
await audio_streaming_component.stop_audio_streaming()
await audio_stream_task
audio_stream_task = None
break
logging.info("Sender task finished.")
receive_task = asyncio.create_task(
receive_messages(websocket), name="ReceiverThread"
)
send_task = asyncio.create_task(
send_messages_local(websocket), name="SenderThread"
)
done, pending = await asyncio.wait(
[receive_task, send_task], return_when=asyncio.FIRST_COMPLETED
)
logging.info(
f"Main task completion: Done={len(done)}, Pending={len(pending)}"
)
current_active_audio_task = audio_stream_task
if current_active_audio_task and not current_active_audio_task.done():
logging.info(
"A main task finished. Ensuring audio stream is stopped if active."
)
await audio_streaming_component.stop_audio_streaming()
try:
await asyncio.wait_for(current_active_audio_task, timeout=5.0)
logging.info(
"Audio streaming task gracefully stopped after main task"
" completion."
)
except asyncio.TimeoutError:
logging.warning(
"Timeout waiting for audio stream to stop post main task."
" Cancelling."
)
current_active_audio_task.cancel()
except Exception as e_stream_stop:
logging.error(
f"Error during audio stream stop after main task: {e_stream_stop}"
)
if audio_stream_task is current_active_audio_task:
audio_stream_task = None
for task in pending:
if not task.done():
task.cancel()
logging.info(f"Cancelled pending main task: {task.get_name()}")
all_tasks_to_await = list(done) + list(pending)
for task in all_tasks_to_await:
try:
await task
except asyncio.CancelledError:
logging.info(f"Main task {task.get_name()} cancelled as expected.")
except Exception as e:
logging.error(
f"Error awaiting main task {task.get_name()}: {e}", exc_info=True
)
logging.info("All main tasks awaited.")
except Exception as e:
logging.error(f"Outer error in websocket_client: {e}", exc_info=True)
finally:
final_check_audio_task = audio_stream_task
if final_check_audio_task and not final_check_audio_task.done():
logging.warning("Performing final cleanup of active audio stream task.")
await audio_streaming_component.stop_audio_streaming()
try:
await asyncio.wait_for(final_check_audio_task, timeout=2.0)
except asyncio.TimeoutError:
final_check_audio_task.cancel()
except Exception:
pass
audio_stream_task = None
agent_response_audio_player.cleanup_pyaudio_playback()
logging.info("websocket_client function finished.")
if __name__ == "__main__":
logging.info("Script's main execution block started.")
if (
APP_NAME == "hello_world"
or USER_ID.startswith("your_user_id")
or SESSION_ID.startswith("your_session_id")
):
print(
"WARNING: Using default/example APP_NAME, USER_ID, or SESSION_ID."
" Please update these.",
flush=True,
)
try:
asyncio.run(websocket_client())
except KeyboardInterrupt:
logging.info("Client execution interrupted by user (KeyboardInterrupt).")
print("\nClient interrupted. Exiting.", flush=True)
except Exception as e:
logging.critical(
"A critical unhandled exception occurred in __main__.", exc_info=True
)
print(f"CRITICAL ERROR: {e}. Check logs. Exiting.", flush=True)
finally:
logging.info(
"Script's main execution block finished. Shutting down logging."
)
logging.shutdown()
print("Script execution finished.", flush=True)
contributing/samples/live/live_agent_api_server_example/readme.md
+26
View File
@@ -0,0 +1,26 @@
# What's this?
This is a sample that shows how to start the ADK api server, and how to connect
your agents in a live(bidi-streaming) way. It works text and audio input, and
the response is always audio.
## Prerequisite
- Make sure you go through https://google.github.io/adk-docs/streaming/
## Instruction for this sample
- The audio libraries we used here doesn't have noise cancellation. So the noise
may feed back to the model. You can use headset to avoid this or tune down
voice volume, or implement your own noise cancellation logic.
- Please ensure you grant the right mic/sound device permission to the terminal
that runs the script. Sometimes, terminal inside VSCode etc doesn't really work
well. So try native terminals if you have permission issue.
- start api server first for your agent folder. For example, my agents are
located in contributing/samples. So I will run
`adk api_server contributing/samples/`. Keep this running.
- then in a separate window, run `python3 live_agent_example.py`
## Misc
- Provide a few pre-recorded audio files for testing.
contributing/samples/live/live_bidi_debug_utils/pcm_audio_player.py
+39
View File
@@ -0,0 +1,39 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import numpy as np
import sounddevice as sd
# input audio example. replace with the input audio you want to test
FILE_PATH = 'adk_live_audio_storage_input_audio_1762910896736.pcm'
# output audio example. replace with the input audio you want to test
FILE_PATH = 'adk_live_audio_storage_output_audio_1762910893258.pcm;rate=24000'
# PCM rate is always 24,000 for input and output
SAMPLE_RATE = 24000
CHANNELS = 1
DTYPE = np.int16 # Common types: int16, float32
# Read and play
with open(FILE_PATH, 'rb') as f:
# Load raw data into numpy array
raw_data = f.read()
audio_array = np.frombuffer(raw_data, dtype=DTYPE)
# Reshape if stereo (interleaved)
if CHANNELS > 1:
audio_array = audio_array.reshape((-1, CHANNELS))
# Play
print('Playing...')
sd.play(audio_array, SAMPLE_RATE)
sd.wait()
contributing/samples/live/live_bidi_streaming_multi_agent/__init__.py
+15
View File
@@ -0,0 +1,15 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from . import agent
contributing/samples/live/live_bidi_streaming_multi_agent/agent.py
+166
View File
@@ -0,0 +1,166 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import random
from google.adk.agents.llm_agent import Agent
from google.adk.examples.example import Example
from google.adk.models.google_llm import Gemini
from google.adk.tools.example_tool import ExampleTool
from google.genai import types
# --- Roll Die Sub-Agent ---
def roll_die(sides: int) -> int:
"""Roll a die and return the rolled result."""
return random.randint(1, sides)
roll_agent = Agent(
name="roll_agent",
model=Gemini(
# Find supported models in Vertex here: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/live-api
model="gemini-live-2.5-flash-native-audio", # Vertex
# Find supported models in Gemini API here: https://ai.google.dev/gemini-api/docs/models
# model='gemini-2.5-flash-native-audio-preview-12-2025', # Gemini API
speech_config=types.SpeechConfig(
voice_config=types.VoiceConfig(
prebuilt_voice_config=types.PrebuiltVoiceConfig(
voice_name="Kore",
)
)
),
),
description="Handles rolling dice of different sizes.",
instruction="""
You are responsible for rolling dice based on the user's request.
When asked to roll a die, you must call the roll_die tool with the number of sides as an integer.
""",
tools=[roll_die],
generate_content_config=types.GenerateContentConfig(
safety_settings=[
types.SafetySetting( # avoid false alarm about rolling dice.
category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
threshold=types.HarmBlockThreshold.OFF,
),
]
),
)
# --- Prime Check Sub-Agent ---
def check_prime(nums: list[int]) -> str:
"""Check if a given list of numbers are prime."""
primes = set()
for number in nums:
number = int(number)
if number <= 1:
continue
is_prime = True
for i in range(2, int(number**0.5) + 1):
if number % i == 0:
is_prime = False
break
if is_prime:
primes.add(number)
return (
"No prime numbers found."
if not primes
else f"{', '.join(str(num) for num in primes)} are prime numbers."
)
prime_agent = Agent(
name="prime_agent",
model=Gemini(
# Find supported models in Vertex here: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/live-api
model="gemini-live-2.5-flash-native-audio", # Vertex
# Find supported models in Gemini API here: https://ai.google.dev/gemini-api/docs/models
# model='gemini-2.5-flash-native-audio-preview-12-2025', # Gemini API
speech_config=types.SpeechConfig(
voice_config=types.VoiceConfig(
prebuilt_voice_config=types.PrebuiltVoiceConfig(
voice_name="Puck",
)
)
),
),
description="Handles checking if numbers are prime.",
instruction="""
You are responsible for checking whether numbers are prime.
When asked to check primes, you must call the check_prime tool with a list of integers.
Never attempt to determine prime numbers manually.
Return the prime number results to the root agent.
""",
tools=[check_prime],
generate_content_config=types.GenerateContentConfig(
safety_settings=[
types.SafetySetting( # avoid false alarm about rolling dice.
category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
threshold=types.HarmBlockThreshold.OFF,
),
]
),
)
def get_current_weather(location: str):
"""
Returns the current weather.
"""
if location == "New York":
return "Sunny"
else:
return "Raining"
root_agent = Agent(
model=Gemini(
# Find supported models in Vertex here: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/live-api
model="gemini-live-2.5-flash-native-audio", # Vertex
# Find supported models in Gemini API here: https://ai.google.dev/gemini-api/docs/models
# model='gemini-2.5-flash-native-audio-preview-12-2025', # Gemini API
speech_config=types.SpeechConfig(
voice_config=types.VoiceConfig(
prebuilt_voice_config=types.PrebuiltVoiceConfig(
voice_name="Zephyr",
)
)
),
),
name="root_agent",
instruction="""
You are a helpful assistant that can check time, roll dice and check if numbers are prime.
You can check time on your own.
You delegate rolling dice tasks to the roll_agent and prime checking tasks to the prime_agent.
Follow these steps:
1. If the user asks to roll a die, delegate to the roll_agent.
2. If the user asks to check primes, delegate to the prime_agent.
3. If the user asks to roll a die and then check if the result is prime, call roll_agent first, then pass the result to prime_agent.
Always clarify the results before proceeding.
""",
global_instruction=(
"You are DicePrimeBot, ready to roll dice and check prime numbers."
),
sub_agents=[roll_agent, prime_agent],
tools=[get_current_weather],
generate_content_config=types.GenerateContentConfig(
safety_settings=[
types.SafetySetting( # avoid false alarm about rolling dice.
category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
threshold=types.HarmBlockThreshold.OFF,
),
]
),
)
contributing/samples/live/live_bidi_streaming_multi_agent/readme.md
+43
View File
@@ -0,0 +1,43 @@
# Simplistic Live (Bidi-Streaming) Multi-Agent
This project provides a basic example of a live, [bidirectional streaming](https://google.github.io/adk-docs/streaming/) multi-agent
designed for testing and experimentation.
## Getting Started
Follow these steps to get the agent up and running:
1. **Start the ADK Web Server**
Open your terminal, navigate to the root directory that contains the
`live_bidi_streaming_agent` folder, and execute the following command:
```bash
adk web
```
1. **Access the ADK Web UI**
Once the server is running, open your web browser and navigate to the URL
provided in the terminal (it will typically be `http://localhost:8000`).
1. **Select the Agent**
In the top-left corner of the ADK Web UI, use the dropdown menu to select
this agent.
1. **Start Streaming**
Click on either the **Audio** or **Video** icon located near the chat input
box to begin the streaming session.
1. **Interact with the Agent**
You can now begin talking to the agent, and it will respond in real-time.
## Usage Notes
- You only need to click the **Audio** or **Video** button once to initiate the
stream. The current version does not support stopping and restarting the stream
by clicking the button again during a session.
## Sample Queries
- Hello, what's the weather in Seattle and New York?
- Could you roll a 6-sided dice for me?
- Could you check if the number you rolled is a prime number or not?
contributing/samples/live/live_bidi_streaming_parallel_tools_agent/README.md
+40
View File
@@ -0,0 +1,40 @@
# Simple Live (Bidi-Streaming) Agent with Parallel Tools
This project provides a basic example of a live, [bidirectional streaming](https://google.github.io/adk-docs/streaming/) agent that demonstrates parallel tool execution.
## Getting Started
Follow these steps to get the agent up and running:
1. **Start the ADK Web Server**
Open your terminal, navigate to the root directory that contains the
`live_bidi_streaming_parallel_tools_agent` folder, and execute the following
command:
```bash
adk web
```
1. **Access the ADK Web UI**
Once the server is running, open your web browser and navigate to the URL
provided in the terminal (it will typically be `http://localhost:8000`).
1. **Select the Agent**
In the top-left corner of the ADK Web UI, use the dropdown menu to select
this agent (`live_bidi_streaming_parallel_tools_agent`).
1. **Start Streaming**
Click on the **Audio** icon located near the chat input
box to begin the streaming session.
1. **Interact with the Agent**
You can now begin talking to the agent, and it will respond in real-time.
Try asking it to perform multiple actions at once, for example: "Turn on the
lights and the TV at the same time." The agent will be able to invoke both
`turn_on_lights` and `turn_on_tv` tools in parallel.
## Usage Notes
- You only need to click the **Audio** button once to initiate the
stream. The current version does not support stopping and restarting the stream
by clicking the button again during a session.
contributing/samples/live/live_bidi_streaming_parallel_tools_agent/__init__.py
+15
View File
@@ -0,0 +1,15 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from . import agent
contributing/samples/live/live_bidi_streaming_parallel_tools_agent/agent.py
+39
View File
@@ -0,0 +1,39 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from google.adk.agents.llm_agent import Agent
def turn_on_lights():
"""Turn on the lights."""
print("turn_on_lights")
return {"status": "OK"}
def turn_on_tv():
"""Turn on the tv."""
print("turn_on_tv")
return {"status": "OK"}
root_agent = Agent(
# Find supported models in Vertex here: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/live-api
model="gemini-live-2.5-flash-native-audio", # Vertex
# Find supported models in Gemini API here: https://ai.google.dev/gemini-api/docs/models
# model='gemini-2.5-flash-native-audio-preview-12-2025', # Gemini API
name="Home_helper",
instruction="Be polite and answer all user's questions.",
tools=[turn_on_lights, turn_on_tv],
)
contributing/samples/live/live_bidi_streaming_single_agent/__init__.py
Executable
+15
View File
@@ -0,0 +1,15 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from . import agent
contributing/samples/live/live_bidi_streaming_single_agent/agent.py
Executable
+106
View File
@@ -0,0 +1,106 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import random
from google.adk.agents.llm_agent import Agent
from google.adk.tools.tool_context import ToolContext
from google.genai import types
def roll_die(sides: int, tool_context: ToolContext) -> int:
"""Roll a die and return the rolled result.
Args:
sides: The integer number of sides the die has.
Returns:
An integer of the result of rolling the die.
"""
result = random.randint(1, sides)
if not 'rolls' in tool_context.state:
tool_context.state['rolls'] = []
tool_context.state['rolls'] = tool_context.state['rolls'] + [result]
return result
async def check_prime(nums: list[int]) -> str:
"""Check if a given list of numbers are prime.
Args:
nums: The list of numbers to check.
Returns:
A str indicating which number is prime.
"""
primes = set()
for number in nums:
number = int(number)
if number <= 1:
continue
is_prime = True
for i in range(2, int(number**0.5) + 1):
if number % i == 0:
is_prime = False
break
if is_prime:
primes.add(number)
return (
'No prime numbers found.'
if not primes
else f"{', '.join(str(num) for num in primes)} are prime numbers."
)
root_agent = Agent(
# Find supported models in Vertex here: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/live-api
model='gemini-live-2.5-flash-native-audio', # Vertex
# Find supported models in Gemini API here: https://ai.google.dev/gemini-api/docs/models
# model='gemini-2.5-flash-native-audio-preview-12-2025', # Gemini API
name='roll_dice_agent',
description=(
'hello world agent that can roll a dice of 6 sides and check prime'
' numbers.'
),
instruction="""
You roll dice and answer questions about the outcome of the dice rolls.
You can roll dice of different sizes. When the user doesn't specify the number of sides, you should assume 6 sides.
You can use multiple tools in parallel by calling functions in parallel(in one request and in one round).
It is ok to discuss previous dice roles, and comment on the dice rolls.
When you are asked to roll a die, you must call the roll_die tool with the number of sides. Be sure to pass in an integer. Do not pass in a string.
You should never roll a die on your own.
When checking prime numbers, call the check_prime tool with a list of integers. Be sure to pass in a list of integers. You should never pass in a string.
You should not check prime numbers before calling the tool.
When you are asked to roll a die and check prime numbers, you should always make the following two function calls:
1. You should first call the roll_die tool to get a roll. Wait for the function response before calling the check_prime tool.
2. After you get the function response from roll_die tool, you should call the check_prime tool with the roll_die result.
2.1 If user asks you to check primes based on previous rolls, make sure you include the previous rolls in the list.
3. When you respond, you must include the roll_die result from step 1.
You should always perform the previous 3 steps when asking for a roll and checking prime numbers.
You should not rely on the previous history on prime results.
""",
tools=[
roll_die,
check_prime,
],
generate_content_config=types.GenerateContentConfig(
safety_settings=[
types.SafetySetting( # avoid false alarm about rolling dice.
category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
threshold=types.HarmBlockThreshold.OFF,
),
]
),
)
contributing/samples/live/live_bidi_streaming_single_agent/readme.md
+37
View File
@@ -0,0 +1,37 @@
# Simplistic Live (Bidi-Streaming) Agent
This project provides a basic example of a live, [bidirectional streaming](https://google.github.io/adk-docs/streaming/) agent
designed for testing and experimentation.
## Getting Started
Follow these steps to get the agent up and running:
1. **Start the ADK Web Server**
Open your terminal, navigate to the root directory that contains the
`live_bidi_streaming_single_agent` folder, and execute the following command:
```bash
adk web
```
1. **Access the ADK Web UI**
Once the server is running, open your web browser and navigate to the URL
provided in the terminal (it will typically be `http://localhost:8000`).
1. **Select the Agent**
In the top-left corner of the ADK Web UI, use the dropdown menu to select
this agent.
1. **Start Streaming**
Click on either the **Audio** or **Video** icon located near the chat input
box to begin the streaming session.
1. **Interact with the Agent**
You can now begin talking to the agent, and it will respond in real-time.
## Usage Notes
- You only need to click the **Audio** or **Video** button once to initiate the
stream. The current version does not support stopping and restarting the stream
by clicking the button again during a session.
contributing/samples/live/live_bidi_streaming_tools_agent/__init__.py
+15
View File
@@ -0,0 +1,15 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from . import agent
contributing/samples/live/live_bidi_streaming_tools_agent/agent.py
+145
View File
@@ -0,0 +1,145 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import asyncio
from typing import AsyncGenerator
from google.adk.agents import LiveRequestQueue
from google.adk.agents.llm_agent import Agent
from google.adk.tools.function_tool import FunctionTool
from google.genai import types as genai_types
async def monitor_stock_price(stock_symbol: str) -> AsyncGenerator[str, None]:
"""This function will monitor the price for the given stock_symbol in a continuous, streaming and asynchronously way."""
print(f"Start monitor stock price for {stock_symbol}!")
# Let's mock stock price change.
await asyncio.sleep(4)
price_alert1 = f"the price for {stock_symbol} is 300"
yield price_alert1
print(price_alert1)
await asyncio.sleep(4)
price_alert1 = f"the price for {stock_symbol} is 400"
yield price_alert1
print(price_alert1)
await asyncio.sleep(20)
price_alert1 = f"the price for {stock_symbol} is 900"
yield price_alert1
print(price_alert1)
await asyncio.sleep(20)
price_alert1 = f"the price for {stock_symbol} is 500"
yield price_alert1
print(price_alert1)
# for video streaming, `input_stream: LiveRequestQueue` is required and reserved key parameter for ADK to pass the video streams in.
async def monitor_video_stream(
input_stream: LiveRequestQueue,
) -> AsyncGenerator[str, None]:
"""Monitor how many people are in the video streams."""
from google.genai import Client
print("start monitor_video_stream!")
from google.genai import Client
client = Client(enterprise=False)
prompt_text = (
"Count the number of people in this image. Just respond with a numeric"
" number."
)
last_count = None
while True:
last_valid_req = None
print("Start monitoring loop")
# use this loop to pull the latest images and discard the old ones
while input_stream._queue.qsize() != 0:
live_req = await input_stream.get()
if live_req.blob is not None and live_req.blob.mime_type == "image/jpeg":
last_valid_req = live_req
# If we found a valid image, process it
if last_valid_req is not None:
print("Processing the most recent frame from the queue")
# Create an image part using the blob's data and mime type
image_part = genai_types.Part.from_bytes(
data=last_valid_req.blob.data, mime_type=last_valid_req.blob.mime_type
)
contents = genai_types.Content(
role="user",
parts=[image_part, genai_types.Part.from_text(text=prompt_text)],
)
# Call the model to generate content based on the provided image and prompt
response = client.models.generate_content(
contents=contents,
config=genai_types.GenerateContentConfig(
system_instruction=(
"You are a helpful video analysis assistant. You can count"
" the number of people in this image or video. Just respond"
" with a numeric number."
)
),
)
if not last_count:
last_count = response.candidates[0].content.parts[0].text
elif last_count != response.candidates[0].content.parts[0].text:
last_count = response.candidates[0].content.parts[0].text
yield response
print("response:", response)
# Wait before checking for new images
await asyncio.sleep(0.5)
# Use this exact function to help ADK stop your streaming tools when requested.
# for example, if we want to stop `monitor_stock_price`, then the agent will
# invoke this function with stop_streaming(function_name=monitor_stock_price).
def stop_streaming(function_name: str):
"""Stop the streaming
Args:
function_name: The name of the streaming function to stop.
"""
pass
root_agent = Agent(
# Find supported models in Vertex here: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/live-api
model="gemini-live-2.5-flash-native-audio", # Vertex
# Find supported models in Gemini API here: https://ai.google.dev/gemini-api/docs/models
# model='gemini-2.5-flash-native-audio-preview-12-2025', # Gemini API
name="video_streaming_agent",
instruction="""
You are a monitoring agent. You can do video monitoring and stock price monitoring
using the provided tools/functions.
When users want to monitor a video stream,
You can use monitor_video_stream function to do that. When monitor_video_stream
returns the alert, you should tell the users.
When users want to monitor a stock price, you can use monitor_stock_price.
Don't ask too many questions. Don't be too talkative.
""",
tools=[
monitor_video_stream,
monitor_stock_price,
FunctionTool(stop_streaming),
],
)
contributing/samples/live/live_bidi_streaming_tools_agent/readme.md
+19
View File
@@ -0,0 +1,19 @@
This is only supported in streaming(live) agents/api.
Streaming tools allows tools(functions) to stream intermediate results back to agents and agents can respond to those intermediate results.
For example, we can use streaming tools to monitor the changes of the stock price and have the agent react to it. Another example is we can have the agent monitor the video stream, and when there is changes in video stream, the agent can report the changes.
To define a streaming tool, you must adhere to the following:
1. **Asynchronous Function:** The tool must be an `async` Python function.
1. **AsyncGenerator Return Type:** The function must be typed to return an `AsyncGenerator`. The first type parameter to `AsyncGenerator` is the type of the data you `yield` (e.g., `str` for text messages, or a custom object for structured data). The second type parameter is typically `None` if the generator doesn't receive values via `send()`.
We support two types of streaming tools:
- Simple type. This is a one type of streaming tools that only take non video/audio streams(the streams that you feed to adk web or adk runner) as input.
- Video streaming tools. This only works in video streaming and the video stream(the streams that you feed to adk web or adk runner) will be passed into this function.
Here are some sample queries to test:
- Help me monitor the stock price for $XYZ stock.
- Help me monitor how many people are there in the video stream.
contributing/samples/live/live_non_blocking_tool_agent/README.md
+27
View File
@@ -0,0 +1,27 @@
# Live Non-Blocking Tool Agent Sample
## Overview
This sample provides a minimal agent to demonstrate non-blocking tool execution in ADK Live mode (`adk web` / `run_live`).
When a tool declaration is configured with `response_scheduling` set to `WHEN_IDLE`, `SILENT`, or `INTERRUPT`, it indicates to the model that response handling can occur asynchronously.
## Sample Inputs
- `Please start a slow background task for data processing, and then let's keep talking.`
*Triggers `slow_background_task` which sleeps for 10 seconds. While it runs, continue speaking to the agent.*
## Reproduction Instructions
1. Run the sample via `adk web`:
```bash
uv run adk web contributing/samples/live/live_non_blocking_tool_agent
```
1. Open the ADK web interface and start a Live Session with the agent.
1. Trigger the tool by saying: *"Please start a slow background task and keep talking with me."*
1. Continue speaking to the agent while the background task runs in console (`[Tool] Starting slow background task...`).
### Expected Behavior
The model should continue conversing and generating audio/transcription responses immediately while the tool executes in the background. The tool result is delivered later per the `response_scheduling` mode.
contributing/samples/live/live_non_blocking_tool_agent/__init__.py
+15
View File
@@ -0,0 +1,15 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from . import agent
contributing/samples/live/live_non_blocking_tool_agent/agent.py
+72
View File
@@ -0,0 +1,72 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import asyncio
from typing import Any
from typing import Dict
from google.adk.agents.llm_agent import Agent
from google.adk.tools.function_tool import FunctionTool
from google.genai import types
async def slow_background_task(task_description: str) -> Dict[str, Any]:
"""Performs a long-running background computation or data retrieval.
Args:
task_description: Description of the task to run in the background.
Returns:
A dictionary containing the completion status and task result.
"""
print(f"[Tool] Starting slow background task: {task_description}")
# Simulate a 10-second non-blocking background operation
await asyncio.sleep(10)
print(f"[Tool] Completed slow background task: {task_description}")
return {
"status": "completed",
"task": task_description,
"result": "Background task finished successfully after 10 seconds.",
}
# Create a FunctionTool wrapping the long-running async function
non_blocking_tool = FunctionTool(slow_background_task)
# Configure response_scheduling to indicate non-blocking behavior for Live mode.
# Options: WHEN_IDLE, SILENT, or INTERRUPT.
non_blocking_tool.response_scheduling = (
types.FunctionResponseScheduling.WHEN_IDLE
)
root_agent = Agent(
model="gemini-live-2.5-flash-native-audio",
name="non_blocking_tool_agent",
description=(
"Agent demonstrating non-blocking tool execution in ADK Live mode."
),
instruction="""
You are a helpful assistant for testing live mode non-blocking tool execution.
You have access to a tool `slow_background_task` which is configured with
NON_BLOCKING response scheduling (WHEN_IDLE).
When the user asks you to run a long-running or background task, call the `slow_background_task` tool.
Inform the user that the task has started and continue conversing with them normally while the task runs in the background.
""",
tools=[
non_blocking_tool,
],
)
contributing/samples/live/live_tool_callbacks_agent/__init__.py
+15
View File
@@ -0,0 +1,15 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from . import agent
contributing/samples/live/live_tool_callbacks_agent/agent.py
+271
View File
@@ -0,0 +1,271 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from datetime import datetime
import random
import time
from typing import Any
from typing import Dict
from typing import Optional
from google.adk.agents.llm_agent import Agent
from google.adk.tools.tool_context import ToolContext
from google.genai import types
def get_weather(location: str, tool_context: ToolContext) -> Dict[str, Any]:
"""Get weather information for a location.
Args:
location: The city or location to get weather for.
Returns:
A dictionary containing weather information.
"""
# Simulate weather data
temperatures = [-10, -5, 0, 5, 10, 15, 20, 25, 30, 35]
conditions = ["sunny", "cloudy", "rainy", "snowy", "windy"]
return {
"location": location,
"temperature": random.choice(temperatures),
"condition": random.choice(conditions),
"humidity": random.randint(30, 90),
"timestamp": datetime.now().isoformat(),
}
async def calculate_async(operation: str, x: float, y: float) -> Dict[str, Any]:
"""Perform async mathematical calculations.
Args:
operation: The operation to perform (add, subtract, multiply, divide).
x: First number.
y: Second number.
Returns:
A dictionary containing the calculation result.
"""
# Simulate some async work
await asyncio.sleep(0.1)
operations = {
"add": x + y,
"subtract": x - y,
"multiply": x * y,
"divide": x / y if y != 0 else float("inf"),
}
result = operations.get(operation.lower(), "Unknown operation")
return {
"operation": operation,
"x": x,
"y": y,
"result": result,
"timestamp": datetime.now().isoformat(),
}
def log_activity(message: str, tool_context: ToolContext) -> Dict[str, str]:
"""Log an activity message with timestamp.
Args:
message: The message to log.
Returns:
A dictionary confirming the log entry.
"""
if "activity_log" not in tool_context.state:
tool_context.state["activity_log"] = []
log_entry = {"timestamp": datetime.now().isoformat(), "message": message}
tool_context.state["activity_log"].append(log_entry)
return {
"status": "logged",
"entry": log_entry,
"total_entries": len(tool_context.state["activity_log"]),
}
# Before tool callbacks
def before_tool_audit_callback(
tool, args: Dict[str, Any], tool_context: ToolContext
) -> Optional[Dict[str, Any]]:
"""Audit callback that logs all tool calls before execution."""
print(f"🔍 AUDIT: About to call tool '{tool.name}' with args: {args}")
# Add audit info to tool context state
if "audit_log" not in tool_context.state:
tool_context.state["audit_log"] = []
tool_context.state["audit_log"].append({
"type": "before_call",
"tool_name": tool.name,
"args": args,
"timestamp": datetime.now().isoformat(),
})
# Return None to allow normal tool execution
return None
def before_tool_security_callback(
tool, args: Dict[str, Any], tool_context: ToolContext
) -> Optional[Dict[str, Any]]:
"""Security callback that can block certain tool calls."""
# Example: Block weather requests for restricted locations
if tool.name == "get_weather" and args.get("location", "").lower() in [
"classified",
"secret",
]:
print(
"🚫 SECURITY: Blocked weather request for restricted location:"
f" {args.get('location')}"
)
return {
"error": "Access denied",
"reason": "Location access is restricted",
"requested_location": args.get("location"),
}
# Allow other calls to proceed
return None
async def before_tool_async_callback(
tool, args: Dict[str, Any], tool_context: ToolContext
) -> Optional[Dict[str, Any]]:
"""Async before callback that can add preprocessing."""
print(f"⚡ ASYNC BEFORE: Processing tool '{tool.name}' asynchronously")
# Simulate some async preprocessing
await asyncio.sleep(0.05)
# For calculation tool, we could add validation
if (
tool.name == "calculate_async"
and args.get("operation") == "divide"
and args.get("y") == 0
):
print("🚫 VALIDATION: Prevented division by zero")
return {
"error": "Division by zero",
"operation": args.get("operation"),
"x": args.get("x"),
"y": args.get("y"),
}
return None
# After tool callbacks
def after_tool_enhancement_callback(
tool,
args: Dict[str, Any],
tool_context: ToolContext,
tool_response: Dict[str, Any],
) -> Optional[Dict[str, Any]]:
"""Enhance tool responses with additional metadata."""
print(f"✨ ENHANCE: Adding metadata to response from '{tool.name}'")
# Add enhancement metadata
enhanced_response = tool_response.copy()
enhanced_response.update({
"enhanced": True,
"enhancement_timestamp": datetime.now().isoformat(),
"tool_name": tool.name,
"execution_context": "live_streaming",
})
return enhanced_response
async def after_tool_async_callback(
tool,
args: Dict[str, Any],
tool_context: ToolContext,
tool_response: Dict[str, Any],
) -> Optional[Dict[str, Any]]:
"""Async after callback for post-processing."""
print(
f"🔄 ASYNC AFTER: Post-processing response from '{tool.name}'"
" asynchronously"
)
# Simulate async post-processing
await asyncio.sleep(0.05)
# Add async processing metadata
processed_response = tool_response.copy()
processed_response.update({
"async_processed": True,
"processing_time": "0.05s",
"processor": "async_after_callback",
})
return processed_response
import asyncio
# Create the agent with tool callbacks
root_agent = Agent(
# Find supported models in Vertex here: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/live-api
model="gemini-live-2.5-flash-native-audio", # Vertex
# Find supported models in Gemini API here: https://ai.google.dev/gemini-api/docs/models
# model='gemini-2.5-flash-native-audio-preview-12-2025', # Gemini API
name="tool_callbacks_agent",
description=(
"Live streaming agent that demonstrates tool callbacks functionality. "
"It can get weather, perform calculations, and log activities while "
"showing how before and after tool callbacks work in live mode."
),
instruction="""
You are a helpful assistant that can:
1. Get weather information for any location using the get_weather tool
2. Perform mathematical calculations using the calculate_async tool
3. Log activities using the log_activity tool
Important behavioral notes:
- You have several callbacks that will be triggered before and after tool calls
- Before callbacks can audit, validate, or even block tool calls
- After callbacks can enhance or modify tool responses
- Some locations like "classified" or "secret" are restricted for weather requests
- Division by zero will be prevented by validation callbacks
- All your tool responses will be enhanced with additional metadata
When users ask you to test callbacks, explain what's happening with the callback system.
Be conversational and explain the callback behavior you observe.
""",
tools=[
get_weather,
calculate_async,
log_activity,
],
# Multiple before tool callbacks (will be processed in order until one returns a response)
before_tool_callback=[
before_tool_audit_callback,
before_tool_security_callback,
before_tool_async_callback,
],
# Multiple after tool callbacks (will be processed in order until one returns a response)
after_tool_callback=[
after_tool_enhancement_callback,
after_tool_async_callback,
],
generate_content_config=types.GenerateContentConfig(
safety_settings=[
types.SafetySetting(
category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
threshold=types.HarmBlockThreshold.OFF,
),
]
),
)
contributing/samples/live/live_tool_callbacks_agent/readme.md
+110
View File
@@ -0,0 +1,110 @@
# Live Tool Callbacks Agent
This sample demonstrates how tool callbacks work in live (bidirectional streaming) mode. It showcases both `before_tool_callback` and `after_tool_callback` functionality with multiple callback chains, async callbacks, and various callback behaviors.
## Features Demonstrated
### Before Tool Callbacks
1. **Audit Callback**: Logs all tool calls before execution
1. **Security Callback**: Can block tool calls based on security rules (e.g., restricted locations)
1. **Async Validation Callback**: Performs async validation and can prevent invalid operations
### After Tool Callbacks
1. **Enhancement Callback**: Adds metadata to tool responses
1. **Async Post-processing Callback**: Performs async post-processing of responses
### Tools Available
- `get_weather`: Get weather information for any location
- `calculate_async`: Perform mathematical calculations asynchronously
- `log_activity`: Log activities with timestamps
## Testing Scenarios
### 1. Basic Callback Flow
```
"What's the weather in New York?"
```
Watch the console output to see:
- Audit logging before the tool call
- Security check (will pass for New York)
- Response enhancement after the tool call
### 2. Security Blocking
```
"What's the weather in classified?"
```
The security callback will block this request and return an error response.
### 3. Validation Prevention
```
"Calculate 10 divided by 0"
```
The async validation callback will prevent division by zero.
### 4. Multiple Tool Calls
```
"Get weather for London and calculate 5 + 3"
```
See how callbacks work with multiple parallel tool calls.
### 5. Callback Chain Testing
```
"Log this activity: Testing callback chains"
```
Observe how multiple callbacks in the chain are processed.
## Getting Started
1. **Start the ADK Web Server**
```bash
adk web
```
1. **Access the ADK Web UI**
Navigate to `http://localhost:8000`
1. **Select the Agent**
Choose "tool_callbacks_agent" from the dropdown in the top-left corner
1. **Start Streaming**
Click the **Audio** or **Video** icon to begin streaming
1. **Test Callbacks**
Try the testing scenarios above and watch both the chat responses and the console output to see callbacks in action
## What to Observe
- **Console Output**: Watch for callback logs with emojis:
- 🔍 AUDIT: Audit callback logging
- 🚫 SECURITY: Security callback blocking
- ⚡ ASYNC BEFORE: Async preprocessing
- ✨ ENHANCE: Response enhancement
- 🔄 ASYNC AFTER: Async post-processing
- **Enhanced Responses**: Tool responses will include additional metadata added by after callbacks
- **Error Handling**: Security blocks and validation errors will be returned as proper error responses
## Technical Notes
- This sample demonstrates that tool callbacks now work identically in both regular and live streaming modes
- Multiple callbacks are supported and processed in order
- Both sync and async callbacks are supported
- Callbacks can modify, enhance, or block tool execution
- The callback system provides full control over the tool execution pipeline