Files
googlecloudplatform--genera…/gemini/logging/intro_request_response_logging.ipynb
2026-07-13 13:30:30 +08:00

566 lines
22 KiB
Plaintext
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "yFeds7eiwI6x"
},
"outputs": [],
"source": [
"# Copyright 2025 Google LLC\n",
"#\n",
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4ircZQgHwRdy"
},
"source": [
"# Intro to Request and Response Logging with Gemini\n",
"\n",
"<table align=\"left\">\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/logging/intro_request_response_logging.ipynb\">\n",
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Flogging%2Fintro_request_response_logging.ipynb\">\n",
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/logging/intro_request_response_logging.ipynb\">\n",
" <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/logging/intro_request_response_logging.ipynb\">\n",
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
" </a>\n",
" </td>\n",
"</table>\n",
"\n",
"<div style=\"clear: both;\"></div>\n",
"\n",
"<p>\n",
"<b>Share to:</b>\n",
"\n",
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/logging/intro_request_response_logging.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/logging/intro_request_response_logging.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/logging/intro_request_response_logging.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/logging/intro_request_response_logging.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/logging/intro_request_response_logging.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
"</a>\n",
"</p>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MJDxNAGItD51"
},
"source": [
"| Authors |\n",
"| --- |\n",
"| [Eric Dong](https://github.com/gericdong) |"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "896Uhs2Ww6_E"
},
"source": [
"## Overview\n",
"\n",
"Vertex AI allows you to log samples of requests and responses for Gemini models. This capability is useful for analyzing model usage, debugging issues, and understanding how your model is performing. The logs are saved directly to a BigQuery table that you specify, making them easy to query and analyze.\n",
"\n",
"### Objectives\n",
"\n",
"In this tutorial, you'll learn how to:\n",
"\n",
"- Enable logging to an automatically created BigQuery destination.\n",
"- Send a request to the model to generate a log entry.\n",
"- Programmatically identify the auto-created dataset and table.\n",
"- Verify the logged content in BigQuery.\n",
"- Disable request-response logging."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gPiTOAHURvTM"
},
"source": [
"## Getting Started"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CHRZUpfWSEpp"
},
"source": [
"### Install the Vertex AI SDK and other required libraries\n",
"\n",
"> ⚠️ This logging feature is a preview feature and is only available in the `preview` module of the Vertex AI SDK."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "sG3_LKsWSD3A"
},
"outputs": [],
"source": [
"%pip install --upgrade --quiet google-cloud-aiplatform google-cloud-bigquery"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HlMVjiAWSMNX"
},
"source": [
"### Authenticate your notebook environment\n",
"\n",
"If you are running this notebook on Google Colab, run the cell below to authenticate your environment."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "12fnq4V0SNV3"
},
"outputs": [],
"source": [
"import sys\n",
"\n",
"if \"google.colab\" in sys.modules:\n",
" from google.colab import auth\n",
"\n",
" auth.authenticate_user()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "be18ac9c5ec8"
},
"source": [
"### Set your project information\n",
"\n",
"Please update the following variables with your Google Cloud project details and the desired BigQuery destination for your logs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "6wXh1aH7BlPl"
},
"outputs": [],
"source": [
"import os\n",
"\n",
"# fmt: off\n",
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
"# fmt: on\n",
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
"\n",
"LOCATION = \"us-central1\" # @param {type: \"string\"}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rK3jJDR5lfiT"
},
"source": [
"### Import libraries\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "T6DyDoNclVEn"
},
"outputs": [],
"source": [
"import time\n",
"import warnings\n",
"\n",
"import vertexai\n",
"from IPython.display import display\n",
"from google.api_core import exceptions\n",
"from google.cloud import bigquery\n",
"\n",
"# Use the preview module for this feature\n",
"from vertexai.preview.generative_models import GenerativeModel\n",
"\n",
"# Ignore all warnings\n",
"warnings.filterwarnings(\"ignore\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EdvJRUWRNGHE"
},
"source": [
"### Initialize the Vertex AI SDK for Python\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qgdSpVmDbdQ9"
},
"outputs": [],
"source": [
"vertexai.init(project=PROJECT_ID, location=LOCATION)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "n4yRkFg6BBu4"
},
"source": [
"### Supported Models\n",
"\n",
"Learn more about [supported models for logging](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging#supported_api_methods_for_logging). This tutorial uses the Gemini 2.5 Flash model."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-coEslfWPrxo"
},
"outputs": [],
"source": [
"# fmt: off\n",
"MODEL_ID = \"gemini-2.5-flash\" # @param {type: \"string\"}\n",
"# fmt: on\n",
"\n",
"model = GenerativeModel(MODEL_ID)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KqjCltg20IgR"
},
"source": [
"## Enable request-response logging\n",
"\n",
"To enable logging, we'll use the `set_request_response_logging_config` method on a `GenerativeModel` instance. This method can be used to both create a new logging configuration or update an existing one.\n",
"\n",
"You need to provide a few key parameters:\n",
"\n",
"- `enabled`: Set to `True` to turn on logging.\n",
"- `sampling_rate`: A value from `0.0` to `1.0` indicating the fraction of requests to log. `1.0` means log 100% of requests.\n",
"- `bigquery_destination`: The path to your BigQuery table in the format `bq://PROJECT_ID.DATASET_NAME.TABLE_NAME`.\n",
"\n",
"**Notes**:\n",
"\n",
"- To enable logging with automatic resource creation, we will set the `bigquery_destination` to only specify the project ID. When this method is used:\n",
" - A **new BigQuery dataset** is created with a name following the pattern `ENDPOINT_DISPLAY_NAME_ENDPOINT_ID`.\n",
" - A **new table** is created within that dataset with the name `request_response_logging`.\n",
"\n",
"- If a configuration already exists, the API will return an `AlreadyExists` error. The code below wraps the call in a `try...except` block to handle this case gracefully."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-uz0nnRSlGw-"
},
"outputs": [],
"source": [
"output_uri = None\n",
"\n",
"try:\n",
" logging_config_result = model.set_request_response_logging_config(\n",
" enabled=True,\n",
" sampling_rate=1.0,\n",
" bigquery_destination=f\"bq://{PROJECT_ID}\",\n",
" )\n",
" print(\"Successfully enabled request-response logging.\")\n",
"\n",
" output_uri = logging_config_result.logging_config.bigquery_destination.output_uri\n",
" print(f\"Retrieved Output URI: {output_uri}\")\n",
"\n",
"except exceptions.AlreadyExists:\n",
" print(\"Request-response logging is already enabled for this model.\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4lp5TQM0guow"
},
"source": [
"## Generate Content to Create Logs\n",
"\n",
"After enabling the configuration, we need to send a request to the model. This first request will trigger the creation of the dataset and table in BigQuery, and the request/response will be logged.\n",
"\n",
"> ️ Note: It can take a few minutes for the resources to be created and for the first log to appear."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "LfyMq5F9guoy"
},
"outputs": [],
"source": [
"response = model.generate_content(\"Why is the sky blue?\")\n",
"print(response.text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4SHROeLdhK-z"
},
"source": [
"### Wait for the BQ dataset and table to be created and populated"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "QwxhcT8gpsum"
},
"outputs": [],
"source": [
"# It can take a few minutes for the dataset to be created and for logs to show up\n",
"print(\"Waiting 2 minutes for resources and logs to propagate...\")\n",
"time.sleep(120)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "verify_logs_markdown"
},
"source": [
"## Identify Log Destination\n",
"\n",
"Since the dataset name is generated automatically, we need to discover it before we can query the logs. Since the SDK's `set_request_response_logging_config` method is idempotent (it creates or updates), we can use it to retrieve the current configuration by making a trivial change and then immediately changing it back.\n",
"\n",
"The returned object from the successful update call will contain the full configuration, including the output_uri you want."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4VyY2CoOjETW"
},
"outputs": [],
"source": [
"if output_uri is None:\n",
" # Define the original and a temporary sampling rate for the toggle\n",
" original_sampling_rate = 1.0\n",
" temp_sampling_rate = 0.99\n",
"\n",
" try:\n",
" # Update the config with a temporary value to get the return object.\n",
" print(\n",
" f\"Temporarily updating sampling_rate to {temp_sampling_rate} to fetch config...\"\n",
" )\n",
" config = model.set_request_response_logging_config(\n",
" enabled=True,\n",
" sampling_rate=temp_sampling_rate,\n",
" bigquery_destination=f\"bq://{PROJECT_ID}\",\n",
" )\n",
"\n",
" # Extract the output_uri from the returned object.\n",
" output_uri = config.logging_config.bigquery_destination.output_uri\n",
" print(f\"\\nSuccessfully retrieved Output URI: {output_uri}\")\n",
"\n",
" finally:\n",
" # Always change the setting back to the original value, even if errors occur.\n",
" if output_uri: # Only run this if the config was successfully fetched\n",
" print(f\"\\nRestoring original sampling_rate to {original_sampling_rate}...\")\n",
" model.set_request_response_logging_config(\n",
" enabled=True,\n",
" sampling_rate=original_sampling_rate,\n",
" bigquery_destination=f\"bq://{PROJECT_ID}\",\n",
" )\n",
" print(\"Configuration restored.\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gtcf1n9MqKgh"
},
"source": [
"## Query Logs\n",
"\n",
"Now that you have the `output_uri`, you can use the BigQuery client library to parse it, query the correct table, and display the logs.\n",
"\n",
"### Understanding the BigQuery Table Schema\n",
"\n",
"When you enable logging, Vertex AI writes the logs to a BigQuery table with a predefined schema. Here are the fields you will find in that table:\n",
"\n",
"- **`api_method`** (STRING): The API method used, such as `generateContent` or `streamGenerateContent`.\n",
"- **`deployed_model_id`** (STRING): The ID for a tuned model that has been deployed to an endpoint.\n",
"- **`endpoint`** (STRING): The resource name of the endpoint if a tuned model is deployed.\n",
"- **`full_request`** (JSON): The complete `GenerateContentRequest`.\n",
"- **`full_response`** (JSON): The complete `GenerateContentResponse`.\n",
"- **`logging_time`** (TIMESTAMP): The time the log was recorded, which is roughly when the response was returned.\n",
"- **`metadata`** (JSON): Contains metadata about the call, including request latency.\n",
"- **`model`** (STRING): The resource name of the model used.\n",
"- **`model_version`** (STRING): The version of the model, which is often `\"default\"` for Gemini models.\n",
"- **`otel_log`** (JSON): Logs formatted in the OpenTelemetry schema, which is only available if `otel_logging` is enabled in the configuration.\n",
"- **`request_id`** (NUMERIC): The auto-generated integer ID for the API request.\n",
"- **`request_payload`** (STRING): Included for logging with partner models and for backward compatibility.\n",
"- **`response_payload`** (STRING): Included for logging with partner models and for backward compatibility.\n",
"\n",
"### ⚠️ Important Limitation\n",
"\n",
"It's important to note that request-response pairs that are larger than the **10MB** row limit for the BigQuery write API will not be recorded in the table.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "bf90BzNyoOy6"
},
"outputs": [],
"source": [
"# Parse the URI to get project, dataset, and table IDs\n",
"uri_parts = output_uri.replace(\"bq://\", \"\").split(\".\")\n",
"if len(uri_parts) != 3:\n",
" raise ValueError(\n",
" \"The output_uri format is incorrect. Expected 'bq://project.dataset.table'\"\n",
" )\n",
"\n",
"project_id, dataset_id, table_id = uri_parts\n",
"\n",
"# Construct the full table ID for the query, ensuring it's properly quoted\n",
"full_table_id = f\"`{project_id}.{dataset_id}.{table_id}`\"\n",
"print(f\"Querying table: {full_table_id}\")\n",
"\n",
"# Query the BigQuery table\n",
"bq_client = bigquery.Client(project=project_id)\n",
"\n",
"# Construct a SQL query to get the 10 most recent logs\n",
"query = f\"\"\"\n",
" SELECT\n",
" logging_time,\n",
" full_request,\n",
" full_response,\n",
" model\n",
" FROM {full_table_id}\n",
" ORDER BY logging_time DESC\n",
" LIMIT 10\n",
"\"\"\"\n",
"\n",
"# Execute the query and load the results into a pandas DataFrame\n",
"df = bq_client.query(query).to_dataframe()\n",
"\n",
"if df.empty:\n",
" print(\n",
" \"\\\\nNo logs found in the table. It may take a few minutes for new logs to appear.\"\n",
" )\n",
"else:\n",
" print(f\"\\\\nSuccessfully fetched {len(df)} recent log(s):\")\n",
" display(df)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "disable_logging_markdown"
},
"source": [
"## Disable Request-Response Logging\n",
"\n",
"You can disable logging at any time by calling the same `set_request_response_logging_config` method and setting `enabled=False`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "disable_logging_code"
},
"outputs": [],
"source": [
"model.set_request_response_logging_config(\n",
" enabled=False,\n",
" sampling_rate=1.0,\n",
" bigquery_destination=f\"bq://{PROJECT_ID}\",\n",
")\n",
"\n",
"print(\"Request-response logging has been disabled.\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "whats_next"
},
"source": [
"## What's next\n",
"\n",
"You can now navigate to your BigQuery project to view the table and analyze the logged data. To learn more, check out the official documentation on [logging requests and responses](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging)."
]
}
],
"metadata": {
"colab": {
"name": "intro_request_response_logging.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}