wehub-resource-sync
429 lines
15 KiB
Plaintext
429 lines
15 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "2RexaI2BoZZy"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Copyright 2026 Google LLC\n",
|
|
"#\n",
|
|
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
|
"# you may not use this file except in compliance with the License.\n",
|
|
"# You may obtain a copy of the License at\n",
|
|
"#\n",
|
|
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
|
"#\n",
|
|
"# Unless required by applicable law or agreed to in writing, software\n",
|
|
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
|
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
|
"# See the License for the specific language governing permissions and\n",
|
|
"# limitations under the License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "rJ43O7oc6xGP"
|
|
},
|
|
"source": [
|
|
"# Configuring Retries in the Gen AI SDK\n",
|
|
"\n",
|
|
"<table align=\"left\">\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/sdk/retries/configure_retries.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fsdk%2Fretries%2Fconfigure_retries.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/sdk/retries/configure_retries.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/sdk/retries/configure_retries.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
"</table>\n",
|
|
"\n",
|
|
"<div style=\"clear: both;\"></div>\n",
|
|
"\n",
|
|
"<p>\n",
|
|
"<b>Share to:</b>\n",
|
|
"\n",
|
|
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/sdk/retries/configure_retries.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/sdk/retries/configure_retries.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/sdk/retries/configure_retries.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/sdk/retries/configure_retries.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/sdk/retries/configure_retries.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
|
"</a>\n",
|
|
"</p>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "uYusEvgifp9U"
|
|
},
|
|
"source": [
|
|
"| Author |\n",
|
|
"| --- |\n",
|
|
"| [Eric Dong](https://github.com/gericdong) |"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "8sg1R6UQ7Q3t"
|
|
},
|
|
"source": [
|
|
"## Overview\n",
|
|
"\n",
|
|
"When interacting with Gemini through Gemini API, transient network errors or service disruptions can occur. Implementing a retry strategy is crucial for building robust and reliable applications. This guide explains how to configure HTTP retry options for the [Google Gen AI SDK](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/sdks/overview).\n",
|
|
"\n",
|
|
"\n",
|
|
"### **Why Configure Retries?**\n",
|
|
"\n",
|
|
"By default, the Gen AI SDK has some level of retry for certain types of errors. However, for fine-grained control over the retry behavior, you can configure your own retry strategy. This is particularly important for:\n",
|
|
"\n",
|
|
"- **Handling transient errors**: Automatically retry requests that fail due to temporary issues, such as network timeouts or temporary server unavailability (e.g., 5xx errors).\n",
|
|
"- **Managing rate limits**: If you anticipate hitting rate limits (e.g., 429 errors), a retry strategy with exponential backoff can help manage the request rate and avoid overwhelming the service.\n",
|
|
"- **Improving application resilience**: Make your application more resilient to intermittent issues, leading to a better user experience.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "gPiTOAHURvTM"
|
|
},
|
|
"source": [
|
|
"## Getting Started"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "-tn3uw268iw4"
|
|
},
|
|
"source": [
|
|
"### Install Google Gen AI SDK for Python\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "3CaXL22k8iw4"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install --upgrade --quiet google-genai"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "xW5WwfAOfp9V"
|
|
},
|
|
"source": [
|
|
"### Import libraries\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "o0JUCkvVfp9V"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import sys\n",
|
|
"\n",
|
|
"from IPython.display import Markdown, display\n",
|
|
"from google import genai\n",
|
|
"from google.genai import types"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "SY-GRP3m8iw5"
|
|
},
|
|
"source": [
|
|
"### Authenticate your notebook environment\n",
|
|
"\n",
|
|
"If you are running this notebook on Google Colab, run the cell below to authenticate your environment."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "06RAe75C8iw5"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"if \"google.colab\" in sys.modules:\n",
|
|
" from google.colab import auth\n",
|
|
"\n",
|
|
" auth.authenticate_user()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "wkInCpT9fp9V"
|
|
},
|
|
"source": [
|
|
"### Authenticate your Google Cloud Project for Vertex AI\n",
|
|
"\n",
|
|
"You can use a Google Cloud Project or an API Key for authentication. This tutorial uses a Google Cloud Project.\n",
|
|
"\n",
|
|
"- [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "z-jVOPQVfp9V"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# fmt: off\n",
|
|
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
|
"# fmt: on\n",
|
|
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
|
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
|
"\n",
|
|
"LOCATION = \"global\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "LBGTO9MUFWP0"
|
|
},
|
|
"source": [
|
|
"## **Option 1**: Configuring Retries at the Client Level\n",
|
|
"\n",
|
|
"The Gen AI SDK provides the `genai.types.HttpRetryOptions` class to configure a custom retry policy. You can then apply this policy to the `genai.Client` using `genai.types.HttpOptions`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "RR5RO9KxFvRR"
|
|
},
|
|
"source": [
|
|
"### **Step 1**: Define `HttpRetryOptions`\n",
|
|
"\n",
|
|
"First, create an instance of `HttpRetryOptions` and specify the desired retry behavior. Here are some of the key parameters:\n",
|
|
"\n",
|
|
"- `initial_delay`: The initial delay in seconds before the first retry.\n",
|
|
"- `attempts`: The maximum number of retry attempts.\n",
|
|
"- `exp_base`: The base for the exponential backoff calculation.\n",
|
|
"- `max_delay`: The maximum delay in seconds between retries.\n",
|
|
"- `jitter`: A factor to add a random delay to the backoff, which can help avoid a \"thundering herd\" problem.\n",
|
|
"- `http_status_codes`: A list of HTTP status codes that should trigger a retry."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "pzHKE306dSJ2"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Configure the retry policy.\n",
|
|
"retry_options = types.HttpRetryOptions(\n",
|
|
" initial_delay=1.0, # Initial delay of 1 second.\n",
|
|
" attempts=5, # Retry up to 5 times.\n",
|
|
" exp_base=2, # Double the delay for each subsequent retry.\n",
|
|
" http_status_codes=[429, 500, 502, 503, 504], # Retry on these server errors.\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "sP0bwwcoGJKE"
|
|
},
|
|
"source": [
|
|
"### **Step 2**: Apply to the Client with `HttpOptions`\n",
|
|
"\n",
|
|
"Next, create an instance of `HttpOptions` and pass your `retry_options` to it.\n",
|
|
"\n",
|
|
"You can also configure other HTTP options, such as `timeout` - Timeout for the request in milliseconds."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "D90UFQOpGaEh"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Configure HTTP options with the retry settings.\n",
|
|
"http_options = types.HttpOptions(\n",
|
|
" retry_options=retry_options,\n",
|
|
" timeout=120 * 1000, # Set a 120-second timeout for requests.\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "u8Vh9ePQGp_A"
|
|
},
|
|
"source": [
|
|
"### **Step 3**: Initialize the `genai.Client`\n",
|
|
"\n",
|
|
"Finally, initialize the `genai.Client` with the `http_options`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "lSvDL3LoGzDx"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Initialize the client with the configured HTTP options.\n",
|
|
"client = genai.Client(\n",
|
|
" vertexai=True,\n",
|
|
" project=PROJECT_ID,\n",
|
|
" location=LOCATION,\n",
|
|
" http_options=http_options,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "k5aRhD50HBSM"
|
|
},
|
|
"source": [
|
|
"Now, any API calls made with this client will use the configured retry policy."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "5kFIrV5NfD0d"
|
|
},
|
|
"source": [
|
|
"### Complete Code Example\n",
|
|
"\n",
|
|
"Here is the complete code sample."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "pdoPLP6lfMFH"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"client = genai.Client(\n",
|
|
" vertexai=True,\n",
|
|
" project=PROJECT_ID,\n",
|
|
" location=LOCATION,\n",
|
|
" http_options=types.HttpOptions(\n",
|
|
" retry_options=types.HttpRetryOptions(\n",
|
|
" initial_delay=1.0,\n",
|
|
" attempts=10,\n",
|
|
" http_status_codes=[429, 500, 502, 503, 504],\n",
|
|
" ),\n",
|
|
" timeout=120 * 1000,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"# You can now use this client to make API calls with the custom retry policy.\n",
|
|
"# For example:\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=\"gemini-3.5-flash\", contents=\"Tell me a story\"\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "b5386769b966"
|
|
},
|
|
"source": [
|
|
"## **Option 2**: Configuring Retries at the Request Level\n",
|
|
"\n",
|
|
"You can also set `HttpRetryOptions` per-request using the `config` parameter."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "620c8177fc44"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=\"gemini-3.5-flash\",\n",
|
|
" contents=\"Tell me a joke about a rabbit.\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" http_options=types.HttpOptions(\n",
|
|
" retry_options=types.HttpRetryOptions(\n",
|
|
" initial_delay=1.0,\n",
|
|
" attempts=10,\n",
|
|
" http_status_codes=[429, 500, 502, 503, 504],\n",
|
|
" ),\n",
|
|
" timeout=120 * 1000,\n",
|
|
" )\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"name": "configure_retries.ipynb",
|
|
"toc_visible": true
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"name": "python3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
}
|