chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,19 @@
|
||||
# Getting Started with Gemini
|
||||
|
||||
This directory contains notebooks that will help you get started with the Gemini family of models on Google Cloud.
|
||||
|
||||
## Notebooks
|
||||
|
||||
- [Intro to Gemini 2.0 Flash](intro_gemini_2_0_flash.ipynb): Learn how to use the Gemini 2.0 Flash model with the Gemini API in Vertex AI and the Google Gen AI SDK for Python.
|
||||
- [Intro to Gemini 2.0 Flash-Lite](intro_gemini_2_0_flash_lite.ipynb): Learn how to use the Gemini 2.0 Flash-Lite model, our fastest and most cost-efficient Flash model.
|
||||
- [Gemini 2.0 Flash Image Generation in Vertex AI](intro_gemini_2_0_image_gen.ipynb): Learn how to use Gemini 2.0 Flash for image generation and editing in Vertex AI.
|
||||
- [Gemini 2.0 Flash Image Generation in Vertex AI with REST API](intro_gemini_2_0_image_gen_rest_api.ipynb): Learn how to use the Gemini 2.0 Flash for image generation and editing in Vertex AI using the REST API.
|
||||
- [Intro to Gemini 2.5 Flash](intro_gemini_2_5_flash.ipynb): Learn how to use the Gemini 2.5 Flash model, a hybrid reasoning model with extended thinking capabilities.
|
||||
- [Intro to Gemini 2.5 Flash-Lite](intro_gemini_2_5_flash_lite.ipynb): Learn how to use Gemini 2.5 Flash-Lite, Google's most cost-effective Gemini 2.5 model for high-volume workloads.
|
||||
- [Gemini 2.5 Flash Image (Nano Banana 🍌) Generation in Vertex AI](intro_gemini_2_5_image_gen.ipynb): Learn how to use Gemini 2.5 Flash Image for image generation and conversational image editing.
|
||||
- [Gemini 3 Pro Image (Nano Banana Pro 🍌) Generation in Vertex AI](intro_gemini_3_image_gen.ipynb): Learn how to use Gemini 3 Pro Image for image generation and multi-turn image editing.
|
||||
- [Intro to Gemini 2.5 Pro](intro_gemini_2_5_pro.ipynb): Learn how to use Gemini 2.5 Pro, Google's most advanced reasoning model, to solve complex problems.
|
||||
- [Getting Started with Chat with Gemini](intro_gemini_chat.ipynb): Learn how to send chat prompts to the Gemini model using the Google Gen AI SDK for Python and LangChain.
|
||||
- [Getting Started with the Gemini API in Vertex AI with cURL / REST API](intro_gemini_curl.ipynb): Learn how to use the Vertex AI REST API with cURL commands to interact with the Gemini 2.0 Flash model.
|
||||
- [Getting started with Gemini using Vertex AI in Express Mode](intro_gemini_express.ipynb): Learn how to use Vertex AI in Express Mode to quickly try out core generative AI features.
|
||||
- [Getting started with Google Generative AI using the Gen AI SDK](intro_genai_sdk.ipynb): Learn about the key features of the Google Gen AI SDK for Python to get started with Google generative AI services and models.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,633 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "sqi5B7V_Rjim"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2025 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "VyPmicX9RlZX"
|
||||
},
|
||||
"source": [
|
||||
"# Intro to Gemini 2.5 Flash-Lite\n",
|
||||
"\n",
|
||||
"> **NOTE:** On June 15, 2026, we will remove access to Gemini 2.5 Flash-Lite on Gemini Enterprise Agent Platform for new and inactive projects only. We will also turn off model tuning on this date. If you have a new project, please use Gemini 3.1 Flash-Lite.\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_flash_lite.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_2_5_flash_lite.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_2_5_flash_lite.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_flash_lite.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>\n",
|
||||
"\n",
|
||||
"<div style=\"clear: both;\"></div>\n",
|
||||
"\n",
|
||||
"<p>\n",
|
||||
"<b>Share to:</b>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_flash_lite.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_flash_lite.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_flash_lite.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_flash_lite.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_flash_lite.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
||||
"</a>\n",
|
||||
"</p>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "8MqT58L6Rm_q"
|
||||
},
|
||||
"source": [
|
||||
"| Authors |\n",
|
||||
"| --- |\n",
|
||||
"| [Eric Dong](https://github.com/gericdong) |\n",
|
||||
"| [Holt Skinner](https://github.com/holtskinner) |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "nVxnv1D5RoZw"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"Gemini 2.5 Flash-Lite is Google's most cost-effective Gemini 2.5 model yet, optimized for performance in high-volume workloads. Delivering higher performance than the previous 2.0 Flash and Flash-Lite models and significantly improved latency, it's ideal for tasks like classification, translation, intelligent routing, and other cost-sensitive, high-scale operations.\n",
|
||||
"\n",
|
||||
"### Objectives\n",
|
||||
"\n",
|
||||
"In this tutorial, you will learn how to use the Gemini API and the Google Gen AI SDK for Python with the Gemini 2.5 Flash-Lite model.\n",
|
||||
"\n",
|
||||
"You will complete the following tasks:\n",
|
||||
"\n",
|
||||
"- Generate text\n",
|
||||
"- Control thinking budget\n",
|
||||
"- View summarized thoughts\n",
|
||||
"- Set model parameters and system instruction\n",
|
||||
"- Use safety filters\n",
|
||||
"- Use controlled generation\n",
|
||||
"- Use search as a tool\n",
|
||||
"- Use code execution\n",
|
||||
"\n",
|
||||
"Gemini 2.5 Flash-Lite also supports other standard Gemini features such as multimodal input, automatic and manual function calling, counting tokens, chat completions. These features are not covered in this tutorial.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "gPiTOAHURvTM"
|
||||
},
|
||||
"source": [
|
||||
"## Getting Started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "CHRZUpfWSEpp"
|
||||
},
|
||||
"source": [
|
||||
"### Install the Google Gen AI SDK for Python\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "sG3_LKsWSD3A"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install --upgrade --quiet google-genai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "HlMVjiAWSMNX"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment\n",
|
||||
"\n",
|
||||
"If you are running this notebook on Google Colab, run the cell below to authenticate your environment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "12fnq4V0SNV3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "be18ac9c5ec8"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate to Vertex AI on Google Cloud\n",
|
||||
"\n",
|
||||
"You'll need to set up authentication by choosing **one** of the following methods:\n",
|
||||
"\n",
|
||||
"1. **Use a Google Cloud Project:** (Recommended for most users)\n",
|
||||
" - See instructions [Set up a project and development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)\n",
|
||||
"2. **Use a Vertex AI API Key (Express Mode):** For quick experimentation.\n",
|
||||
" - [Get an API Key](https://cloud.google.com/vertex-ai/generative-ai/docs/start/express-mode/overview)\n",
|
||||
" - See tutorial [Getting started with Gemini using Vertex AI in Express Mode](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_express.ipynb).\n",
|
||||
"\n",
|
||||
"This tutorial uses a Google Cloud Project for authentication."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6wXh1aH7BlPl"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# fmt: off\n",
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
||||
"# fmt: on\n",
|
||||
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
||||
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
||||
"\n",
|
||||
"LOCATION = \"global\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "EdvJRUWRNGHE"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "qgdSpVmDbdQ9"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from IPython.display import HTML, Markdown, display\n",
|
||||
"from google import genai\n",
|
||||
"from google.genai.types import (\n",
|
||||
" GenerateContentConfig,\n",
|
||||
" GoogleSearch,\n",
|
||||
" HarmBlockThreshold,\n",
|
||||
" HarmCategory,\n",
|
||||
" SafetySetting,\n",
|
||||
" ThinkingConfig,\n",
|
||||
" Tool,\n",
|
||||
" ToolCodeExecution,\n",
|
||||
")\n",
|
||||
"from pydantic import BaseModel"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uhAZMeVrCcKS"
|
||||
},
|
||||
"source": [
|
||||
"### Connect to the generative AI service on Vertex AI"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "pv_22mKMCjQY"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "n4yRkFg6BBu4"
|
||||
},
|
||||
"source": [
|
||||
"## Use Gemini 2.5 Flash-Lite\n",
|
||||
"\n",
|
||||
"Learn more about the [Gemini 2.5 Flash-Lite model](https://cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/2-5-flash-lite). The model ID used in this notebook is `gemini-2.5-flash-lite`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "-coEslfWPrxo"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_ID = \"gemini-2.5-flash-lite\" # @param {type: \"string\"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "37CH91ddY9kG"
|
||||
},
|
||||
"source": [
|
||||
"### Send Your First Prompt\n",
|
||||
"\n",
|
||||
"- Use the `generate_content` method to generate responses to your prompts. You can pass text and other multimodal input to `generate_content`.\n",
|
||||
"- Use the `generate_content_stream` method to stream the response as it is being generated, and the model will return chunks of the response as soon as they are generated.\n",
|
||||
"- Use the `.text` property to get the text content of the response. By default, Gemini outputs formatted text using [Markdown](https://daringfireball.net/projects/markdown/) syntax."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "xRJuHj0KZ8xz"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=\"Why is the sky blue?\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"display(Markdown(response.text))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "hIJVEr0RQY8S"
|
||||
},
|
||||
"source": [
|
||||
"### Set Model Parameters and System Instruction\n",
|
||||
"\n",
|
||||
"- You can set [Gemini API parameters](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#parameters) in each model request to control how the model generates a response.\n",
|
||||
"- A [system instruction](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instruction-introduction) gives the model additional context to understand the task, provide more customized responses, and adhere to guidelines over the user interaction.\n",
|
||||
"- You can also configure [safety filters](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-filters) to adjust how the model handles different content categories.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "d9NXP5N2Pmfo"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_instruction = \"\"\"\n",
|
||||
"Persona: You are 'The Curator,' a sophisticated AI historian. Your tone is eloquent and engaging.\n",
|
||||
"Task: Create a thematic travel itinerary that creatively combines the user's interests.\n",
|
||||
"Rules:\n",
|
||||
"- Use Markdown for all formatting.\n",
|
||||
"- Each day must have a ### Theme:.\n",
|
||||
"- Every day must include these exact bolded sections: Morning:, Afternoon:, Evening:, The Curator's Note:, and Required Attire:.\n",
|
||||
"- Directly connect the itinerary to all of the user's stated interests.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"prompt = \"\"\"\n",
|
||||
"Destination: Florence, Italy\n",
|
||||
"Duration: 2 Days\n",
|
||||
"Interests: Medici family banking and the use of light/optics in Renaissance art.\n",
|
||||
"Goal: Create an itinerary showing how money and light converged to create Florentine masterpieces.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"safety_settings = [\n",
|
||||
" SafetySetting(\n",
|
||||
" category=HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,\n",
|
||||
" threshold=HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
|
||||
" ),\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" system_instruction=system_instruction,\n",
|
||||
" temperature=0.7,\n",
|
||||
" top_p=0.95,\n",
|
||||
" candidate_count=1,\n",
|
||||
" max_output_tokens=2048,\n",
|
||||
" safety_settings=safety_settings,\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"display(Markdown(response.text))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "EBtGmbvaD9Mk"
|
||||
},
|
||||
"source": [
|
||||
"### Control Thinking Budget\n",
|
||||
"\n",
|
||||
"Gemini 2.5 Flash-Lite is a hybrid reasoning model that can apply an extended amount of thinking to tasks and use tools to maximize response accuracy. Unlike Gemini 2.5 Flash, the thinking mode is OFF by default in Gemini 2.5 Flash-Lite. You can set the optional `thinking_budget` parameter to control how much the model thinks for a given prompt.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"- `thinking_budget=0` : Thinking OFF (default)\n",
|
||||
"- `thinking_budget=-1`: Dynamic thinking\n",
|
||||
"- `thinking_budget=[512-24576]`: Allocate thinking budget\n",
|
||||
"\n",
|
||||
"Then use the `generate_content` or `generate_content_stream` method to send a request to generate content with the `ThinkingConfig`.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6LT7dm2FDTo3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"THINKING_BUDGET = 1024 # @param {type: \"integer\"}\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=\"How many R's are in the word strawberry?\",\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" thinking_config=ThinkingConfig(\n",
|
||||
" include_thoughts=True,\n",
|
||||
" thinking_budget=THINKING_BUDGET,\n",
|
||||
" )\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"display(Markdown(response.text))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BzXrgiUTzsPw"
|
||||
},
|
||||
"source": [
|
||||
"Optionally, you can print the `usage_metadata` and token counts from the model response."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "xOiHH_vCymZ_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(f\"prompt_token_count: {response.usage_metadata.prompt_token_count}\")\n",
|
||||
"print(f\"candidates_token_count: {response.usage_metadata.candidates_token_count}\")\n",
|
||||
"print(f\"thoughts_token_count: {response.usage_metadata.thoughts_token_count}\")\n",
|
||||
"print(f\"total_token_count: {response.usage_metadata.total_token_count}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "e59e82a73a3e"
|
||||
},
|
||||
"source": [
|
||||
"### View Summarized Thoughts\n",
|
||||
"\n",
|
||||
"You can set the `include_thoughts` flag to `True` to have the model return a summary of its \"thoughts\" in addition to the final answer. When you make a request with this setting, the response will contain multiple parts. You can check the `part.thought` field to identify which part contains the model's thoughts."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "2cd0767506a6"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=\"How many R's are in the word strawberry?\",\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" thinking_config=ThinkingConfig(include_thoughts=True, thinking_budget=-1)\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" display(Markdown(f\"**Thoughts**: {part.text}\"))\n",
|
||||
" else:\n",
|
||||
" display(Markdown(f\"**Answer**: {part.text}\"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "y0GaUgecqXLy"
|
||||
},
|
||||
"source": [
|
||||
"## Use Controlled Generation\n",
|
||||
"\n",
|
||||
"Controlled generation allows you to define a response schema to specify the structure of the model's output, including field names and expected data types. The model's output will strictly follow the provided schema."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "eqHIrPOPpflz"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class Recipe(BaseModel):\n",
|
||||
" name: str\n",
|
||||
" description: str\n",
|
||||
" ingredients: list[str]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class RecipeList(BaseModel):\n",
|
||||
" recipes: list[Recipe]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=\"List a few popular cookie recipes and their ingredients.\",\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" response_mime_type=\"application/json\",\n",
|
||||
" response_schema=RecipeList,\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"recipes: list[Recipe] = response.parsed\n",
|
||||
"for recipe in recipes.recipes:\n",
|
||||
" print(f\"Recipe: {recipe.name}\")\n",
|
||||
" print(f\"Ingredients: {recipe.ingredients}\")\n",
|
||||
" print(\"\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "_BsP0vXOY7hg"
|
||||
},
|
||||
"source": [
|
||||
"## Grounding with Google Search (Search as a Tool)\n",
|
||||
"\n",
|
||||
"Google Search is available as a built-in tool in Gemini 2.5 Flash-Lite, which is not a feature of the 2.0 Flash-Lite model. This allows the model to decide when to use Google Search to improve the accuracy and recency of its responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "yeR09J3AZT4U"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"google_search_tool = Tool(google_search=GoogleSearch())\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=\"What is the current temperature in Austin, TX?\",\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" tools=[google_search_tool],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"display(Markdown(response.text))\n",
|
||||
"\n",
|
||||
"HTML(response.candidates[0].grounding_metadata.search_entry_point.rendered_content)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "MhDs2X3o0neK"
|
||||
},
|
||||
"source": [
|
||||
"## Code Execution\n",
|
||||
"\n",
|
||||
"The Gemini API's code execution feature enables the model to generate and run Python code. The model can learn iteratively from the code's results to arrive at a final output."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "1W-3c7sy0nyz"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"code_execution_tool = Tool(code_execution=ToolCodeExecution())\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=\"Calculate 20th fibonacci number. Then find the nearest palindrome to it.\",\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" tools=[code_execution_tool],\n",
|
||||
" temperature=0,\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"display(\n",
|
||||
" Markdown(\n",
|
||||
" f\"\"\"\n",
|
||||
"## Code\n",
|
||||
"```py\n",
|
||||
"{response.executable_code}\n",
|
||||
"```\n",
|
||||
"### Output\n",
|
||||
"```\n",
|
||||
"{response.code_execution_result}\n",
|
||||
"```\n",
|
||||
"\"\"\"\n",
|
||||
" )\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "eQwiONFdVHw5"
|
||||
},
|
||||
"source": [
|
||||
"## What's next\n",
|
||||
"\n",
|
||||
"- Explore other notebooks in the [Google Cloud Generative AI GitHub repository](https://github.com/GoogleCloudPlatform/generative-ai).\n",
|
||||
"- Explore AI models in [Model Garden](https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models)."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "intro_gemini_2_5_flash_lite.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
@@ -0,0 +1,771 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "BCl9pcbOOeA0"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2025 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ZPC2X_a9ErW7"
|
||||
},
|
||||
"source": [
|
||||
"# Gemini 2.5 Flash Image (Nano Banana 🍌) Generation\n",
|
||||
"\n",
|
||||
"> **NOTE:** On October 2, 2026, Gemini 2.5 Flash Image will be retired on Gemini Enterprise Agent Platform. After this date, you will be unable to use this model. Please use Gemini 3.1 Flash Image (Nano Banana 2) or Gemini 3 Pro Image (Nano Banana Pro) instead.\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_2_5_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_2_5_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>\n",
|
||||
"\n",
|
||||
"<div style=\"clear: both;\"></div>\n",
|
||||
"\n",
|
||||
"<p>\n",
|
||||
"<b>Share to:</b>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_5_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
||||
"</a>\n",
|
||||
"</p>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f0cc0f48513b"
|
||||
},
|
||||
"source": [
|
||||
"| Author |\n",
|
||||
"| --- |\n",
|
||||
"| [Katie Nguyen](https://github.com/katiemn) |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "axauUzNXEl_R"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"Gemini 2.5 Flash Image is a powerful, generalist multimodal model that offers state-of-the-art image generation and conversational image editing capabilities. This enables you to converse with Gemini and create or edit images with interwoven text.\n",
|
||||
"\n",
|
||||
"In this tutorial, you'll learn how to use Gemini 2.5 Flash Image on Agent Platform using the Google Gen AI SDK to try out the following scenarios:\n",
|
||||
" - Image generation:\n",
|
||||
" - Text-to-image generation\n",
|
||||
" - Interleaved image and text sequences\n",
|
||||
" - Image editing:\n",
|
||||
" - Image-to-image with subject customization and style transfer\n",
|
||||
" - Multi-turn image editing with localization\n",
|
||||
" - Editing with multiple reference images"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "D50ekWXjEl_S"
|
||||
},
|
||||
"source": [
|
||||
"## Get started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "jLJQdbgSbb4M"
|
||||
},
|
||||
"source": [
|
||||
"### Install Google Gen AI SDK for Python"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "SQ0qcEWuXNXs"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install --upgrade --quiet google-genai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dmWOrTJ3gx13"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NyKGtVQjgx13"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "gfs2gxVrAN02"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "O6ZGaZlxP9L0"
|
||||
},
|
||||
"source": [
|
||||
"### Set Google Cloud project information\n",
|
||||
"\n",
|
||||
"To get started using Agent Platform, you must have an existing Google Cloud project and [enable the Agent Platform API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"Learn more about [setting up a project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects) and a [development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "u8IivOG5SqY6"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"from io import BytesIO\n",
|
||||
"\n",
|
||||
"import matplotlib.image as img\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import requests\n",
|
||||
"from IPython.display import Image, Markdown, display\n",
|
||||
"from PIL import Image as PIL_Image\n",
|
||||
"from google import genai\n",
|
||||
"from google.genai.types import FinishReason, GenerateContentConfig, ImageConfig, Part\n",
|
||||
"\n",
|
||||
"# fmt: off\n",
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
||||
"# fmt: on\n",
|
||||
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
||||
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
||||
"\n",
|
||||
"LOCATION = \"global\"\n",
|
||||
"\n",
|
||||
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "854fbf388e2b"
|
||||
},
|
||||
"source": [
|
||||
"### Load the image model\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "7eeb063ac6d4"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_ID = \"gemini-2.5-flash-image\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "xgOucVQlVR4t"
|
||||
},
|
||||
"source": [
|
||||
"## Image generation\n",
|
||||
"\n",
|
||||
"First, you'll send text prompts to Gemini 2.5 Flash Image describing the images you'd like to generate.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "MmA_RAbFwED4"
|
||||
},
|
||||
"source": [
|
||||
"### Text to image\n",
|
||||
"\n",
|
||||
"In the cell below, you'll call the `generate_content` method and pass in the following arguments:\n",
|
||||
"\n",
|
||||
" - `model`: The ID of the model you want to use.\n",
|
||||
" - `contents`: This is your prompt, in this case, a text only user message describing the image to be generated.\n",
|
||||
" - `config`: A config for specifying content settings.\n",
|
||||
" - `response_modalities`: To generate an image, you must include `IMAGE` in the `response_modalities` list.\n",
|
||||
" - `ImageConfig`: Set the `aspect_ratio`. Valid ratios are: \"1:1\", \"3:2\", \"2:3\", \"3:4\", \"4:3\", \"4:5\", \"5:4\", \"9:16\", \"16:9\", \"21:9\"\n",
|
||||
" - `candidate_count`: The number of candidates to generate.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"All generated images include a [SynthID watermark](https://deepmind.google/technologies/synthid/)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "QK2Mi3zmkHSA"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=\"a cartoon infographic on flying sneakers\",\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" image_config=ImageConfig(\n",
|
||||
" aspect_ratio=\"9:16\",\n",
|
||||
" ),\n",
|
||||
" candidate_count=1,\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Check for errors if an image is not generated\n",
|
||||
"if response.candidates[0].finish_reason != FinishReason.STOP:\n",
|
||||
" reason = response.candidates[0].finish_reason\n",
|
||||
" raise ValueError(f\"Prompt Content Error: {reason}\")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "5l4YLy8Vq_-v"
|
||||
},
|
||||
"source": [
|
||||
"### Text to image and text\n",
|
||||
"\n",
|
||||
"In addition to generating images, Gemini can also create interleaved sequences of images and text.\n",
|
||||
"\n",
|
||||
"For example, you could ask the model to generate a recipe for banana bread with images showing different stages of the cooking process. Or, you could ask the model to generate images of different wildflowers with accompanying titles and descriptions.\n",
|
||||
"\n",
|
||||
"Let's try out the interleaved text and image functionality by prompting Gemini 2.5 Flash Image to create a tutorial for assembling a peanut butter and jelly sandwich.\n",
|
||||
"\n",
|
||||
"You'll notice that in the prompt we ask the model to generate both text and images for each step. This encourages the model to create interleaved text and images. Also, make sure to set `response_modalities=[\"TEXT\", \"IMAGE\"]` so you'll get both modalities in the response."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "FwCeB0Hxlrz2"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=\"Create a tutorial explaining how to make a peanut butter and jelly sandwich in three easy steps. For each step, provide a title with the number of the step, an explanation, and also generate an image to illustrate the content. Label each image with the step number but no other words.\",\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=ImageConfig(\n",
|
||||
" aspect_ratio=\"4:3\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "R3g5n23lDtsN"
|
||||
},
|
||||
"source": [
|
||||
"## Image editing\n",
|
||||
"\n",
|
||||
"Gemini 2.5 Flash Image can generate image-to-image outputs from multiple reference images. This is useful for tasks like ensuring character consistency, generating logos, transferring styles, and inserting or removing objects.\n",
|
||||
"\n",
|
||||
"### Subject customization\n",
|
||||
"\n",
|
||||
"Let's try out a subject customization example by asking Gemini 2.5 Flash Image to create an image of this dog in a pencil sketch style.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ZFI-Q53Bg0oK"
|
||||
},
|
||||
"source": [
|
||||
"#### Download the dog image\n",
|
||||
"\n",
|
||||
"The following example uses an image from Cloud Storage. If you prefer to use a different image, you can either change the URL in the `wget` command or, if you have a local file, update the `subject_image` variable in the subsequent step."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "3kEFbvgHhlvI"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/dog-1.jpg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "Bd9CASBlNc1p"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"subject_image = \"dog-1.jpg\" # @param {type: 'string'}\n",
|
||||
"\n",
|
||||
"# Display the image\n",
|
||||
"fig, axis = plt.subplots(1, 1, figsize=(6, 12))\n",
|
||||
"axis.imshow(img.imread(subject_image))\n",
|
||||
"axis.axis(\"off\")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "z284xua1iVXm"
|
||||
},
|
||||
"source": [
|
||||
"#### Send the request\n",
|
||||
"\n",
|
||||
"Since you're including data from a local image in the request, you'll need to include `Part.from_bytes` to the request `contents`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "UAiUyW1VMzPn"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"with open(subject_image, \"rb\") as f:\n",
|
||||
" image = f.read()\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" Part.from_bytes(\n",
|
||||
" data=image,\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" \"Create a pencil sketch image of this dog wearing a cowboy hat in a western-themed setting.\",\n",
|
||||
" ],\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" image_config=ImageConfig(\n",
|
||||
" aspect_ratio=\"1:1\",\n",
|
||||
" ),\n",
|
||||
" candidate_count=1,\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "2lQcs5x0jJLx"
|
||||
},
|
||||
"source": [
|
||||
"### Style transfer\n",
|
||||
"\n",
|
||||
"In this next example, you'll use the style from a living room to reimagine a kitchen in the same style."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "9IFuYABFjz24"
|
||||
},
|
||||
"source": [
|
||||
"#### Download the living room image\n",
|
||||
"\n",
|
||||
"Again, the following example uses an image from Cloud Storage. If you prefer to use a different image, you can either change the URL in the `wget` command or, if you have a local file, update the `style_image` variable in the subsequent step."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "FSl1zOVJwmMU"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/living-room.png"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "BC0oPWSDwvn4"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"style_image = \"living-room.png\" # @param {type: 'string'}\n",
|
||||
"\n",
|
||||
"# Display the image\n",
|
||||
"fig, axis = plt.subplots(1, 1, figsize=(6, 12))\n",
|
||||
"axis.imshow(img.imread(style_image))\n",
|
||||
"axis.axis(\"off\")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "zsjSSGCLkAxW"
|
||||
},
|
||||
"source": [
|
||||
"#### Send the request"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "vuCxm1YOxCKn"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"with open(style_image, \"rb\") as f:\n",
|
||||
" image = f.read()\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" Part.from_bytes(\n",
|
||||
" data=image,\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" \"Using the concepts, colors, and themes from this living room generate a kitchen and dining room with the same aesthetic.\",\n",
|
||||
" ],\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" image_config=ImageConfig(\n",
|
||||
" aspect_ratio=\"21:9\",\n",
|
||||
" ),\n",
|
||||
" candidate_count=1,\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "2vnEO2OfPkjn"
|
||||
},
|
||||
"source": [
|
||||
"### Multi-turn image editing\n",
|
||||
"\n",
|
||||
"In this next section, you supply a starting image and iteratively alter certain aspects of the image by chatting with Gemini 2.5 Flash Image.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Rather than using local images in the next example, you'll use images stored in Google Cloud Storage. Run the next step to view the starting image of a perfume bottle. If you'd like to use a different image from Cloud Storage, replace the `perfume_url` and `perfume_uri` below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "GGK3jQ86reqy"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"perfume_url = (\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/perfume.jpg\"\n",
|
||||
")\n",
|
||||
"perfume_uri = \"gs://cloud-samples-data/generative-ai/image/perfume.jpg\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Display the image\n",
|
||||
"fig, axes = plt.subplots(1, 2, figsize=(12, 6))\n",
|
||||
"perfume_image = PIL_Image.open(BytesIO(requests.get(perfume_url).content))\n",
|
||||
"axes[0].imshow(perfume_image)\n",
|
||||
"for i, ax in enumerate(axes):\n",
|
||||
" ax.axis(\"off\")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "XF3brYa9lSON"
|
||||
},
|
||||
"source": [
|
||||
"#### Start a chat\n",
|
||||
"\n",
|
||||
"In this next step, you'll start a chat so you can continuously edit your images by talking with Gemini. Since you're now using reference images stored in Cloud Storage, you'll use `Part.from_uri` in the `message` content."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "B-2KXCNgQEYu"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"chat = client.chats.create(model=MODEL_ID)\n",
|
||||
"\n",
|
||||
"response = chat.send_message(\n",
|
||||
" message=[\n",
|
||||
" Part.from_uri(\n",
|
||||
" file_uri=perfume_uri,\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" \"change the perfume color to a light purple\",\n",
|
||||
" ],\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" image_config=ImageConfig(\n",
|
||||
" aspect_ratio=\"3:2\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"data = perfume_uri\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))\n",
|
||||
" data = part.inline_data.data"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Loi4drVpdCjn"
|
||||
},
|
||||
"source": [
|
||||
"Now, you'll include the previous image data in a new message in the existing chat, along with a new text prompt, to update the previously generated image. This time, you'll ask to write a word on the perfume bottle, and since Gemini is able to handle different languages, let's ask for the word in French."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "99B7me3HGC1I"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = chat.send_message(\n",
|
||||
" message=[\n",
|
||||
" Part.from_bytes(\n",
|
||||
" data=data,\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" \"inscribe the word flowers in French on the perfume bottle in a delicate white cursive font\",\n",
|
||||
" ],\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" image_config=ImageConfig(\n",
|
||||
" aspect_ratio=\"3:2\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1l19iG4CZMEr"
|
||||
},
|
||||
"source": [
|
||||
"### Multiple reference images\n",
|
||||
"\n",
|
||||
"When editing images with Gemini 2.5 Flash Image, you can also supply multiple input images to create new ones. In this next example, you'll prompt Gemini with an image of a woman and a suitcase. You'll then ask Gemini to combine the objects from these images in order to create a new one. You'll also ask Gemini to supply text to accompany the image.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Run the following cell to visualize the starting images stored in Cloud Storage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "HuLVkC3edP6s"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"person_url = (\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/woman.jpg\"\n",
|
||||
")\n",
|
||||
"suitcase_url = (\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/suitcase.png\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"fig, axes = plt.subplots(1, 2, figsize=(12, 6))\n",
|
||||
"person_image = PIL_Image.open(BytesIO(requests.get(person_url).content))\n",
|
||||
"axes[0].imshow(person_image)\n",
|
||||
"suitcase_image = PIL_Image.open(BytesIO(requests.get(suitcase_url).content))\n",
|
||||
"axes[1].imshow(suitcase_image)\n",
|
||||
"for i, ax in enumerate(axes):\n",
|
||||
" ax.axis(\"off\")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "EYobavQRnZ6Q"
|
||||
},
|
||||
"source": [
|
||||
"Now, you'll send the request. The only difference from previous image editing calls is that you'll supply multiple `Part.from_uri` instances depending on the number of reference images you have."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "WD3XoWjuZQFr"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/suitcase.png\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/woman.jpg\",\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" \"Generate an image of the woman pulling the suitcase in an airport. Separately, write a short caption for this image that would be suitable for a social media post.\",\n",
|
||||
" ],\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=ImageConfig(\n",
|
||||
" aspect_ratio=\"9:16\",\n",
|
||||
" ),\n",
|
||||
" candidate_count=1,\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "intro_gemini_2_5_image_gen.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,995 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ikIep-HBcvvC"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2026 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Qw6ttkOtrQ_D"
|
||||
},
|
||||
"source": [
|
||||
"# Gemini 3.1 Flash Image (Nano Banana 2 🍌) Generation\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_3_1_flash_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_3_1_flash_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>\n",
|
||||
"\n",
|
||||
"<div style=\"clear: both;\"></div>\n",
|
||||
"\n",
|
||||
"<p>\n",
|
||||
"<b>Share to:</b>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
||||
"</a>\n",
|
||||
"</p>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uDN8B4CBdMNs"
|
||||
},
|
||||
"source": [
|
||||
"| Author |\n",
|
||||
"| --- |\n",
|
||||
"| [Katie Nguyen](https://github.com/katiemn) |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "b0e4d036833c"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"This notebook will show you how to use the Gemini 3.1 Flash Image (Nano Banana 2) image model. This model is a powerful, generalist multimodal model that offers state-of-the-art image generation and conversational image editing capabilities. It's also able to show its work, allowing you to set the thinking level and see the 'thought process' behind the generated output.\n",
|
||||
"\n",
|
||||
"In this tutorial, you'll learn how to use gemini-3.1-flash-image on Agent Platform using the Google Gen AI SDK to try out the following scenarios:\n",
|
||||
"\n",
|
||||
"- Image generation:\n",
|
||||
" - Text-to-image generation\n",
|
||||
" - Model thoughts\n",
|
||||
" - Prominent people filter\n",
|
||||
" - Grounding with web and image search\n",
|
||||
" - Image sizing\n",
|
||||
" - Video-to-image generation\n",
|
||||
"- Image editing:\n",
|
||||
" - Localization\n",
|
||||
" - Multi-turn image editing (chat)\n",
|
||||
" - Editing with reference images"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Mfk6YY3G5kqp"
|
||||
},
|
||||
"source": [
|
||||
"## Get started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uJqUEH_mg6kb"
|
||||
},
|
||||
"source": [
|
||||
"### Install Google Gen AI SDK for Python\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "-VBT2jIXLD7h"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install --upgrade --quiet google-genai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "eLIrxLFihSoE"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"id": "hP-_lnBZhUjZ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "oukaeL9Thgy4"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"id": "227VoQtmhjRa"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import warnings\n",
|
||||
"\n",
|
||||
"from IPython.display import Image, Markdown, display, Video\n",
|
||||
"from google import genai\n",
|
||||
"from google.genai import types\n",
|
||||
"\n",
|
||||
"warnings.filterwarnings(\"ignore\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "VdO2n52RhwBG"
|
||||
},
|
||||
"source": [
|
||||
"### Set Google Cloud project information\n",
|
||||
"\n",
|
||||
"To get started using Agent Platform, you must have an existing Google Cloud project and [enable the Agent Platform API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"Learn more about [setting up a project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects) and a [development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"id": "lpI4Mo0phyq8"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# fmt: off\n",
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
||||
"# fmt: on\n",
|
||||
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
||||
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
||||
"\n",
|
||||
"LOCATION = \"global\"\n",
|
||||
"\n",
|
||||
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "QOov6dpG99rY"
|
||||
},
|
||||
"source": [
|
||||
"### Load the model"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"id": "27Fikag0xSaB"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_ID = \"gemini-3.1-flash-image\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "xuHBu3aRiYYv"
|
||||
},
|
||||
"source": [
|
||||
"## Image generation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "M2i8O36nTHI1"
|
||||
},
|
||||
"source": [
|
||||
"### Text-to-image\n",
|
||||
"\n",
|
||||
"In the cell below, you'll call the `generate_content` method and modify the following arguments:\n",
|
||||
"\n",
|
||||
" - `prompt`: A text-only user message describing the image to be generated.\n",
|
||||
" - `config`: A config for specifying content settings.\n",
|
||||
" - `response_modalities`: To generate an image, you must include `IMAGE` in the `response_modalities` list. To get both text and images, specify `IMAGE` and `TEXT`.\n",
|
||||
" - `ImageConfig`: Set the `aspect_ratio`. Valid ratios are: 1:1, 3:2, 2:3, 3:4, 4:3, 1:4, 4:1, 4:5, 5:4, 1:8, 8:1, 9:16, 16:9, 21:9\n",
|
||||
" - `ThinkingConfig`: Set the `thinking_level` to `HIGH` or `MINIMAL`, and if you'd like to view the model thoughts, set `include_thoughts` to `True`.\n",
|
||||
" - `HIGH`: Allows the model to use more tokens for thinking and is suitable for complex prompts requiring deep reasoning.\n",
|
||||
" - `MINIMAL`: Constrains the model to use as few tokens as possible for thinking and is best used for low-complexity tasks.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"All generated images include a [SynthID watermark](https://deepmind.google/technologies/synthid/)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NZsZMcA-iPSj"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"\"\"\n",
|
||||
"A high-contrast, grainy black and white street photography shot. A woman in dark sunglasses is captured in mid-stride with elegant motion blur. Overlaid on the image are large, white, pillowy bubble lines that curve around her to trace her silhouette. A word is added to the top of the image in the same white, bubble font: STYLE.\n",
|
||||
"\"\"\"\n",
|
||||
"aspect_ratio = \"3:2\"\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\", \"TEXT\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=aspect_ratio,\n",
|
||||
" output_mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" thinking_config=types.ThinkingConfig(\n",
|
||||
" include_thoughts=True, thinking_level=types.ThinkingLevel.HIGH\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Check for errors if an image is not generated\n",
|
||||
"if response.candidates[0].finish_reason != types.FinishReason.STOP:\n",
|
||||
" reason = response.candidates[0].finish_reason\n",
|
||||
" raise ValueError(f\"Prompt Content Error: {reason}\")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue # Skip displaying thoughts\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "mbpegA7YhkxI"
|
||||
},
|
||||
"source": [
|
||||
"### See the thoughts\n",
|
||||
"\n",
|
||||
"Since this is a thinking model, you can check the thoughts that led to the image being produced."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KzKLlFCYhzov"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for part in response.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" elif part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "xS1G2QWV7A3e"
|
||||
},
|
||||
"source": [
|
||||
"### Prominent people\n",
|
||||
"\n",
|
||||
"You can block the model from generating photorealistic images of prominent people by setting `prominent_people` to `types.ProminentPeople.BLOCK_PROMINENT_PEOPLE`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "uFJQcgbJ7Dry"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"Generate an image of Beyonce\"\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\", \"TEXT\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"1:1\",\n",
|
||||
" prominent_people=types.ProminentPeople.BLOCK_PROMINENT_PEOPLE,\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Check for errors if an image is not generated\n",
|
||||
"if response.candidates[0].finish_reason != types.FinishReason.STOP:\n",
|
||||
" print(response.candidates[0].finish_reason, response.candidates[0].finish_message)\n",
|
||||
"else:\n",
|
||||
" for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Ys_8nuIENJP9"
|
||||
},
|
||||
"source": [
|
||||
"### Grounding with web search results\n",
|
||||
"\n",
|
||||
"With this model, you can also generate responses that are grounded in the results of a Google Search.\n",
|
||||
"\n",
|
||||
"To display the grounding data, use the helper function in the following cell."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {
|
||||
"id": "_H9eISeEJNZQ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def print_search_grounding_data(response: types.GenerateContentResponse) -> None:\n",
|
||||
" \"\"\"Prints response with grounding citations in Markdown format.\"\"\"\n",
|
||||
" grounding_metadata = response.candidates[0].grounding_metadata\n",
|
||||
" lines = []\n",
|
||||
"\n",
|
||||
" if response.text:\n",
|
||||
" ENCODING = \"utf-8\"\n",
|
||||
" text_bytes = response.text.encode(ENCODING)\n",
|
||||
" last_byte_index = 0\n",
|
||||
"\n",
|
||||
" if grounding_metadata.grounding_supports:\n",
|
||||
" for support in grounding_metadata.grounding_supports:\n",
|
||||
" lines.append(\n",
|
||||
" text_bytes[last_byte_index : support.segment.end_index].decode(\n",
|
||||
" ENCODING\n",
|
||||
" )\n",
|
||||
" )\n",
|
||||
" footnotes = \"\".join(\n",
|
||||
" [f\"[{i + 1}]\" for i in support.grounding_chunk_indices]\n",
|
||||
" )\n",
|
||||
" lines.append(f\" {footnotes}\")\n",
|
||||
" last_byte_index = support.segment.end_index\n",
|
||||
"\n",
|
||||
" if last_byte_index < len(text_bytes):\n",
|
||||
" lines.append(text_bytes[last_byte_index:].decode(ENCODING))\n",
|
||||
"\n",
|
||||
" lines.append(\"\\n\\n----\\n## Grounding Sources\\n\")\n",
|
||||
"\n",
|
||||
" if grounding_metadata.grounding_chunks:\n",
|
||||
" lines.append(\"### Grounding Chunks\\n\")\n",
|
||||
" for i, chunk in enumerate(grounding_metadata.grounding_chunks, start=1):\n",
|
||||
" context = chunk.web or chunk.retrieved_context or chunk.maps\n",
|
||||
" image = chunk.image\n",
|
||||
"\n",
|
||||
" if not context and not image:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" if image:\n",
|
||||
" title = image.title or \"Image Source\"\n",
|
||||
" uri = image.source_uri or \"#\"\n",
|
||||
" image_uri = image.image_uri\n",
|
||||
" domain = image.domain\n",
|
||||
"\n",
|
||||
" lines.append(f\"{i}. [{title}]({uri})\\n\")\n",
|
||||
" if domain:\n",
|
||||
" lines.append(f\" - Domain: `{domain}`\\n\")\n",
|
||||
" if image_uri:\n",
|
||||
" lines.append(f\" <img src='{image_uri}' width='250'>\\n\\n\")\n",
|
||||
"\n",
|
||||
" elif context:\n",
|
||||
" uri = context.uri\n",
|
||||
" title = context.title or \"Source\"\n",
|
||||
" if uri:\n",
|
||||
" uri = uri.replace(\" \", \"%20\")\n",
|
||||
" if uri.startswith(\"gs://\"):\n",
|
||||
" uri = uri.replace(\"gs://\", \"https://storage.googleapis.com/\", 1)\n",
|
||||
"\n",
|
||||
" lines.append(f\"{i}. [{title}]({uri})\\n\")\n",
|
||||
" if hasattr(context, \"text\") and context.text:\n",
|
||||
" lines.append(f\"{context.text}\\n\\n\")\n",
|
||||
"\n",
|
||||
" web_queries = grounding_metadata.web_search_queries or []\n",
|
||||
" image_queries = getattr(grounding_metadata, \"image_search_queries\", []) or []\n",
|
||||
"\n",
|
||||
" all_queries = list(web_queries) + list(image_queries)\n",
|
||||
"\n",
|
||||
" if all_queries:\n",
|
||||
" lines.append(f\"\\n**Web Search Queries:** {all_queries}\\n\")\n",
|
||||
"\n",
|
||||
" if grounding_metadata.search_entry_point:\n",
|
||||
" lines.append(\n",
|
||||
" f\"\\n**Search Entry Point:**\\n{grounding_metadata.search_entry_point.rendered_content}\\n\"\n",
|
||||
" )\n",
|
||||
" elif grounding_metadata.retrieval_queries:\n",
|
||||
" lines.append(\n",
|
||||
" f\"\\n**Retrieval Queries:** {grounding_metadata.retrieval_queries}\\n\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" display(Markdown(\"\".join(lines)))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "QIRVJT5SJWVZ"
|
||||
},
|
||||
"source": [
|
||||
"Next, you'll create a Google Search tool and include it in the `tools` parameter of the following request to perform a web search."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "yWHgXTHldU_N"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"\"\"\n",
|
||||
"A cinematic, wide-angle illustration of the NFL team's stadium that won the 2026 Super Bowl. Digital art style with high-detail architecture.\n",
|
||||
"\"\"\"\n",
|
||||
"google_search = types.Tool(\n",
|
||||
" google_search=types.GoogleSearch(\n",
|
||||
" search_types=types.SearchTypes(web_search=types.WebSearch())\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"21:9\",\n",
|
||||
" ),\n",
|
||||
" thinking_config=types.ThinkingConfig(\n",
|
||||
" include_thoughts=True, thinking_level=types.ThinkingLevel.HIGH\n",
|
||||
" ),\n",
|
||||
" tools=[google_search],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))\n",
|
||||
"\n",
|
||||
"print_search_grounding_data(response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "y7akNXJC5pTA"
|
||||
},
|
||||
"source": [
|
||||
"Below, you'll do the same thing, except with an image search."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "4C0d3W4L5q6W"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"Generate a watercolor image of the Lansdowne Bridge\"\n",
|
||||
"google_search = types.Tool(\n",
|
||||
" google_search=types.GoogleSearch(\n",
|
||||
" search_types=types.SearchTypes(\n",
|
||||
" image_search=types.ImageSearch(),\n",
|
||||
" )\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"16:9\",\n",
|
||||
" ),\n",
|
||||
" thinking_config=types.ThinkingConfig(\n",
|
||||
" include_thoughts=True, thinking_level=types.ThinkingLevel.HIGH\n",
|
||||
" ),\n",
|
||||
" tools=[google_search],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"print_search_grounding_data(response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "hgO4Xqv9NRph"
|
||||
},
|
||||
"source": [
|
||||
"### Image sizes\n",
|
||||
"\n",
|
||||
"You can have Nano Banana 2 generate images with an `image_size` of `512`, `1K`, `2K`, or `4K`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "m6cGJoDDinY3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"\"\"\n",
|
||||
"Macro photography of a vibrant teal and lime green chameleon perched on a mossy branch.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"1:1\",\n",
|
||||
" image_size=\"4K\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "HFR2LKZHNmtZ"
|
||||
},
|
||||
"source": [
|
||||
"### Video-to-image"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "EH0Q_PalPIcv"
|
||||
},
|
||||
"source": [
|
||||
"Nano Banana 2 is also able to generate images from a video. The model is able to observe the frames and generate new images based on the visual content.\n",
|
||||
"\n",
|
||||
"Start by downloading and displaying a video below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "dXJICih8PdXZ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/video/animals.mp4\n",
|
||||
"\n",
|
||||
"video_file = \"animals.mp4\"\n",
|
||||
"display(Video(video_file, embed=True, width=600))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "O-rAlzxvOXHv"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"with open(video_file, \"rb\") as f:\n",
|
||||
" video = f.read()\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_bytes(\n",
|
||||
" data=video,\n",
|
||||
" mime_type=\"video/mp4\"\n",
|
||||
" ),\n",
|
||||
" \"Generate an illustration of every animal featured in this video\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "yAoLWBu4RUGP"
|
||||
},
|
||||
"source": [
|
||||
"You can also generate images from YouTube videos and videos in Cloud Storage. Simply provide the `file_uri` and `mime_type` in the request."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "oBVD4GdbRsIh"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"youtube_link = \"https://www.youtube.com/watch?v=rjoMZyxncUI\"\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=youtube_link,\n",
|
||||
" mime_type=\"video/mp4\"\n",
|
||||
" ),\n",
|
||||
" \"Generate an infographic of the topics covered in this video\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "5nlhVQCuT6DS"
|
||||
},
|
||||
"source": [
|
||||
"## Image editing\n",
|
||||
"\n",
|
||||
"You can also edit images with this model; simply pass the original image as part of the prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "qG0_O2yU9d99"
|
||||
},
|
||||
"source": [
|
||||
"### Localization\n",
|
||||
"\n",
|
||||
"You can also translate the text in images through image editing. Start by downloading the image and displaying it below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "nVOW78Qt9keo"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/city-street.png\n",
|
||||
"\n",
|
||||
"starting_image = \"city-street.png\"\n",
|
||||
"display(Image(filename=starting_image, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "V48CSOv995Z9"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"with open(starting_image, \"rb\") as f:\n",
|
||||
" image = f.read()\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_bytes(\n",
|
||||
" data=image,\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" \"Change the text in this infographic from English to Japanese.\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0sIquv-1lAzn"
|
||||
},
|
||||
"source": [
|
||||
"### Multi-turn image editing (chat)\n",
|
||||
"\n",
|
||||
"In this next section, you'll generate a starting image and iteratively alter certain aspects of the image by chatting with the model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NRMWpFgKY3Xx"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"chat = client.chats.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" config=types.GenerateContentConfig(response_modalities=[\"TEXT\", \"IMAGE\"]),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"message = \"A minimalist overhead shot of two dense florist buckets on a smooth grey pavement. One contains a bouquet of pink baby's breath, and the other contains a cluster of pale blush poppies.\"\n",
|
||||
"response = chat.send_message(message)\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "PR6T_rSBJwM1"
|
||||
},
|
||||
"source": [
|
||||
"Now, you'll send a new message in the existing chat to update the previously generated image."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "bMp_cFHplh-Z"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"message = \"Change the poppies to white dahlias and add a third bucket of pink hydrangeas in the top right corner.\"\n",
|
||||
"response = chat.send_message(message)\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "hQ_uiJOY5Sy9"
|
||||
},
|
||||
"source": [
|
||||
"### Multiple reference images\n",
|
||||
"\n",
|
||||
"With Nano Banana 2, you can include up to 14 reference images in a request to generate a new image that preserves the content of the original images.\n",
|
||||
"\n",
|
||||
"Run the following cell to visualize the starting images stored in Cloud Storage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "cUsResOwmBuS"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from io import BytesIO\n",
|
||||
"\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import requests\n",
|
||||
"from PIL import Image as PIL_Image\n",
|
||||
"\n",
|
||||
"image_urls = [\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/woman.jpeg\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/black-boots.jpg\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/black-bag.jpg\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/shirt.jpg\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/jacket.jpg\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/white-pants.jpg\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"fig, axes = plt.subplots(2, 3, figsize=(12, 8))\n",
|
||||
"for i, ax in enumerate(axes.flatten()):\n",
|
||||
" ax.imshow(PIL_Image.open(BytesIO(requests.get(image_urls[i]).content)))\n",
|
||||
" ax.axis(\"off\")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ErSjXfZ2qg7F"
|
||||
},
|
||||
"source": [
|
||||
"The process for sending the request is similar to previous image editing calls. The main difference is that you will provide multiple `Part.from_uri` instances, one for each reference image."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "_93g7aAeoyNP"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/woman.jpeg\",\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/black-boots.jpg\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/black-bag.jpg\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/shirt.jpg\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/jacket.jpg\",\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/white-pants.jpg\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" \"Generate an image of a woman wearing white pants, a navy blue shirt, a cropped trench coat, and black boots. She's carrying a black bag. The outline of the woman is cropped against a piece of cardstock with a giant paperclip attached at the top.\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"4:3\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "intro_gemini_3_1_flash_image_gen.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,614 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ikIep-HBcvvC"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2026 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Qw6ttkOtrQ_D"
|
||||
},
|
||||
"source": [
|
||||
"# Gemini 3.1 Flash Lite Image (Nano Banana 2 Lite 🍌) Generation\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_lite_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_3_1_flash_lite_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_3_1_flash_lite_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_lite_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>\n",
|
||||
"\n",
|
||||
"<div style=\"clear: both;\"></div>\n",
|
||||
"\n",
|
||||
"<p>\n",
|
||||
"<b>Share to:</b>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_lite_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_lite_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_lite_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_lite_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_1_flash_lite_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
||||
"</a>\n",
|
||||
"</p>\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uDN8B4CBdMNs"
|
||||
},
|
||||
"source": [
|
||||
"| Author |\n",
|
||||
"| --- |\n",
|
||||
"| [Katie Nguyen](https://github.com/katiemn) |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "b0e4d036833c"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook will show you how to use the Gemini 3.1 Flash Lite Image model. This is an image generation and editing model, designed for low-latency, high-volume use cases.\n",
|
||||
"\n",
|
||||
"In this tutorial, you'll learn how to use `gemini-3.1-flash-lite-image` on Agent Platform with the Google Gen AI SDK to try out the following scenarios:\n",
|
||||
"\n",
|
||||
"- Image generation:\n",
|
||||
" - Text-to-image generation\n",
|
||||
" - Video-to-image generation\n",
|
||||
"- Image editing:\n",
|
||||
" - From a local image\n",
|
||||
" - Multi-turn image editing (chat)\n",
|
||||
" - Editing with reference images"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Mfk6YY3G5kqp"
|
||||
},
|
||||
"source": [
|
||||
"## Get started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uJqUEH_mg6kb"
|
||||
},
|
||||
"source": [
|
||||
"### Install Google Gen AI SDK for Python\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "-VBT2jIXLD7h"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install --upgrade --quiet google-genai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "eLIrxLFihSoE"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "hP-_lnBZhUjZ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "oukaeL9Thgy4"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "227VoQtmhjRa"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import warnings\n",
|
||||
"\n",
|
||||
"from IPython.display import Image, display, Video\n",
|
||||
"from google import genai\n",
|
||||
"from google.genai import types\n",
|
||||
"\n",
|
||||
"warnings.filterwarnings(\"ignore\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "VdO2n52RhwBG"
|
||||
},
|
||||
"source": [
|
||||
"### Set Google Cloud project information\n",
|
||||
"\n",
|
||||
"To get started using Agent Platform, you must have an existing Google Cloud project and [enable the Agent Platform API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"Learn more about [setting up a project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects) and a [development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "lpI4Mo0phyq8"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# fmt: off\n",
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
||||
"# fmt: on\n",
|
||||
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
||||
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
||||
"\n",
|
||||
"LOCATION = \"global\"\n",
|
||||
"\n",
|
||||
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "QOov6dpG99rY"
|
||||
},
|
||||
"source": [
|
||||
"### Load the model"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "27Fikag0xSaB"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_ID = \"gemini-3.1-flash-lite-image\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "xuHBu3aRiYYv"
|
||||
},
|
||||
"source": [
|
||||
"## Image generation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "M2i8O36nTHI1"
|
||||
},
|
||||
"source": [
|
||||
"### Text-to-image\n",
|
||||
"\n",
|
||||
"In the cell below, you'll call the `generate_content` method and modify the following arguments:\n",
|
||||
"\n",
|
||||
" - `prompt`: A text-only user message describing the image to be generated.\n",
|
||||
" - `config`: A config for specifying content settings.\n",
|
||||
" - `response_modalities`: To generate an image, you must include `IMAGE` in the `response_modalities` list. To get both text and images, specify `IMAGE` and `TEXT`.\n",
|
||||
" - `ImageConfig`: Set the `aspect_ratio`. Valid ratios are: 1:1, 3:2, 2:3, 3:4, 4:3, 1:4, 4:1, 4:5, 5:4, 1:8, 8:1, 9:16, 16:9, 21:9\n",
|
||||
" - `ThinkingConfig`: Set the `thinking_level` to `HIGH` or `MINIMAL`, and if you'd like to view the model's thoughts, set `include_thoughts` to `True`.\n",
|
||||
" - `HIGH`: Allows the model to use more tokens for thinking and is suitable for complex prompts requiring deep reasoning.\n",
|
||||
" - `MINIMAL`: Constrains the model to use as few tokens as possible for thinking and is best used for low-complexity tasks.\n",
|
||||
"\n",
|
||||
"All images from `gemini-3.1-flash-lite-image` are generated in **1K** and include both [C2PA metadata](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/content-credentials) and a [SynthID watermark](https://deepmind.google/technologies/synthid/)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NZsZMcA-iPSj"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"\"\"\n",
|
||||
"A cute French Bulldog wearing a tiny chef hat, joyfully eating a California roll. Bright, vivid colors, thick black outlines, pop art sticker style. High-contrast, 2D flat vector illustration, isolated on a solid white background.\n",
|
||||
"\"\"\"\n",
|
||||
"aspect_ratio = \"3:2\"\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=aspect_ratio,\n",
|
||||
" output_mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" thinking_config=types.ThinkingConfig(\n",
|
||||
" include_thoughts=True, thinking_level=types.ThinkingLevel.MINIMAL\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Check for errors if an image is not generated\n",
|
||||
"if response.candidates[0].finish_reason != types.FinishReason.STOP:\n",
|
||||
" reason = response.candidates[0].finish_reason\n",
|
||||
" raise ValueError(f\"Prompt Content Error: {reason}\")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "bXP62vOehgBM"
|
||||
},
|
||||
"source": [
|
||||
"### Video-to-image"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Cqyz3tGkhjOW"
|
||||
},
|
||||
"source": [
|
||||
"Nano Banana 2 Lite is also able to generate images from a video. The model observes the frames and generates new images based on the visual content.\n",
|
||||
"\n",
|
||||
"Start by downloading and displaying a video below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "e6iAdvhWhnUQ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/video/animals.mp4\n",
|
||||
"\n",
|
||||
"video_file = \"animals.mp4\"\n",
|
||||
"display(Video(video_file, embed=True, width=600))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "BkKGy-kdhpkg"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"with open(video_file, \"rb\") as f:\n",
|
||||
" video = f.read()\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_bytes(\n",
|
||||
" data=video,\n",
|
||||
" mime_type=\"video/mp4\"\n",
|
||||
" ),\n",
|
||||
" \"Generate a postcard of the setting in this video.\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "5nlhVQCuT6DS"
|
||||
},
|
||||
"source": [
|
||||
"## Image editing\n",
|
||||
"\n",
|
||||
"You can also edit images with this model. Simply pass the original image as part of the prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "qG0_O2yU9d99"
|
||||
},
|
||||
"source": [
|
||||
"### Single-turn image editing\n",
|
||||
"\n",
|
||||
"Start by downloading a starting image and displaying it below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "nVOW78Qt9keo"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/flowers.png\n",
|
||||
"\n",
|
||||
"starting_image = \"flowers.png\"\n",
|
||||
"display(Image(filename=starting_image, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "V48CSOv995Z9"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"with open(starting_image, \"rb\") as f:\n",
|
||||
" image = f.read()\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_bytes(\n",
|
||||
" data=image,\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" \"Generate a doodle of the red flower in this image.\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0sIquv-1lAzn"
|
||||
},
|
||||
"source": [
|
||||
"### Multi-turn image editing (chat)\n",
|
||||
"\n",
|
||||
"In this next section, you'll generate a starting image and iteratively alter certain aspects of the image by chatting with the model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NRMWpFgKY3Xx"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"chat = client.chats.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" config=types.GenerateContentConfig(response_modalities=[\"IMAGE\"]),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"message = \"Professional sports logo for a fictitious Seattle basketball team called the Seattle Sasquatch. A stylized, fierce Bigfoot mascot bursting out from behind a basketball, holding a pine tree. Modern athletic branding, sharp angles, bold clean outlines. Color palette: moss green, cedar brown, and copper. Vector art, athletic typography, isolated on a solid black background.\"\n",
|
||||
"response = chat.send_message(message)\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "PR6T_rSBJwM1"
|
||||
},
|
||||
"source": [
|
||||
"Now, you'll send a new message in the existing chat to update the previously generated image."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "bMp_cFHplh-Z"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"message = \"Change the tree to an umbrella.\"\n",
|
||||
"response = chat.send_message(message)\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "hQ_uiJOY5Sy9"
|
||||
},
|
||||
"source": [
|
||||
"### Multiple reference images\n",
|
||||
"\n",
|
||||
"With Nano Banana 2 Lite, you can include up to 14 reference images in a request to generate a new image that preserves the content of the original images. However, consistency remains optimal when the number of input reference images is limited to 3.\n",
|
||||
"\n",
|
||||
"Run the following cell to visualize the starting images stored in Cloud Storage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "cUsResOwmBuS"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from io import BytesIO\n",
|
||||
"\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import requests\n",
|
||||
"from PIL import Image as PIL_Image\n",
|
||||
"\n",
|
||||
"image_urls = [\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/dog-ad-1.png\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/woman.jpeg\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"fig, axes = plt.subplots(1, 2, figsize=(12, 8))\n",
|
||||
"for i, ax in enumerate(axes.flatten()):\n",
|
||||
" ax.imshow(PIL_Image.open(BytesIO(requests.get(image_urls[i]).content)))\n",
|
||||
" ax.axis(\"off\")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ErSjXfZ2qg7F"
|
||||
},
|
||||
"source": [
|
||||
"The process for sending the request is similar to previous image editing calls. The main difference is that you will provide multiple `Part.from_uri` instances, one for each reference image."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "_93g7aAeoyNP"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/woman.jpeg\",\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/dog-ad-1.png\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" \"A 3D stylized character render of a cheerful woman and her cute dog sitting next to her. Vibrant colors, expressive eyes, clean solid-color background, centered avatar composition.\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"4:3\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,784 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ikIep-HBcvvC"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2025 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Qw6ttkOtrQ_D"
|
||||
},
|
||||
"source": [
|
||||
"# Gemini 3 Pro Image (Nano Banana Pro 🍌) Generation\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_3_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_3_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_image_gen.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>\n",
|
||||
"\n",
|
||||
"<div style=\"clear: both;\"></div>\n",
|
||||
"\n",
|
||||
"<p>\n",
|
||||
"<b>Share to:</b>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_image_gen.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
||||
"</a>\n",
|
||||
"</p>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uDN8B4CBdMNs"
|
||||
},
|
||||
"source": [
|
||||
"| Author |\n",
|
||||
"| --- |\n",
|
||||
"| [Katie Nguyen](https://github.com/katiemn) |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "b0e4d036833c"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"This notebook will show you how to use the Nano Banana Pro image model. This model is a powerful, generalist multimodal model that offers state-of-the-art image generation and conversational image editing capabilities. Nano Banana Pro is also able to show its work, allowing you to see the 'thought process' behind the generated output.\n",
|
||||
"\n",
|
||||
"In this tutorial, you'll learn how to use the model on Agent Platform using the Google Gen AI SDK to try out the following scenarios:\n",
|
||||
"\n",
|
||||
"- Image generation:\n",
|
||||
" - Text-to-image generation\n",
|
||||
" - Model thoughts\n",
|
||||
" - Grounding with search\n",
|
||||
" - Image sizes\n",
|
||||
"- Image editing:\n",
|
||||
" - Localization\n",
|
||||
" - Multi-turn image editing (chat)\n",
|
||||
" - Editing with multiple reference images\n",
|
||||
"\n",
|
||||
"**NOTE:** Expect higher latency when using this model compared to Gemini 2.5 Flash Image (Nano Banana) as a result of the more advanced capabilities."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Mfk6YY3G5kqp"
|
||||
},
|
||||
"source": [
|
||||
"## Get started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uJqUEH_mg6kb"
|
||||
},
|
||||
"source": [
|
||||
"### Install Google Gen AI SDK for Python\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 68,
|
||||
"metadata": {
|
||||
"id": "-VBT2jIXLD7h"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install --upgrade --quiet google-genai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "eLIrxLFihSoE"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 69,
|
||||
"metadata": {
|
||||
"id": "hP-_lnBZhUjZ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "oukaeL9Thgy4"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 70,
|
||||
"metadata": {
|
||||
"id": "227VoQtmhjRa"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import warnings\n",
|
||||
"\n",
|
||||
"from IPython.display import Image, Markdown, display\n",
|
||||
"from google import genai\n",
|
||||
"from google.genai import types\n",
|
||||
"\n",
|
||||
"warnings.filterwarnings(\"ignore\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "VdO2n52RhwBG"
|
||||
},
|
||||
"source": [
|
||||
"### Set Google Cloud project information\n",
|
||||
"\n",
|
||||
"To get started using Agent Platform, you must have an existing Google Cloud project and [enable the Agent Platform API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"Learn more about [setting up a project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects) and a [development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 72,
|
||||
"metadata": {
|
||||
"id": "lpI4Mo0phyq8"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# fmt: off\n",
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
||||
"# fmt: on\n",
|
||||
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
||||
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
||||
"\n",
|
||||
"LOCATION = \"global\"\n",
|
||||
"\n",
|
||||
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "QOov6dpG99rY"
|
||||
},
|
||||
"source": [
|
||||
"### Load the model"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 73,
|
||||
"metadata": {
|
||||
"id": "27Fikag0xSaB"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_ID = \"gemini-3-pro-image\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "xuHBu3aRiYYv"
|
||||
},
|
||||
"source": [
|
||||
"## Image generation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "M2i8O36nTHI1"
|
||||
},
|
||||
"source": [
|
||||
"### Text-to-image\n",
|
||||
"\n",
|
||||
"In the cell below, you'll call the `generate_content` method and modify the following arguments:\n",
|
||||
"\n",
|
||||
" - `prompt`: A text only user message describing the image to be generated.\n",
|
||||
" - `config`: A config for specifying content settings.\n",
|
||||
" - `response_modalities`: To generate an image, you must include `IMAGE` in the `response_modalities` list. To get both text and images, specify `IMAGE` and `TEXT`.\n",
|
||||
" - `ImageConfig`: Set the `aspect_ratio`. Valid ratios are: 1:1, 3:2, 2:3, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9\n",
|
||||
"\n",
|
||||
"All generated images include a [SynthID watermark](https://deepmind.google/technologies/synthid/)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NZsZMcA-iPSj"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"\"\"\n",
|
||||
"Generate an infographic of a seasonal produce guide. Include the months and a fun category name for each season as well as detailed illustrations of the produce.\n",
|
||||
"\"\"\"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"IMAGE\", \"TEXT\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"16:9\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Check for errors if an image is not generated\n",
|
||||
"if response.candidates[0].finish_reason != types.FinishReason.STOP:\n",
|
||||
" reason = response.candidates[0].finish_reason\n",
|
||||
" raise ValueError(f\"Prompt Content Error: {reason}\")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" continue # Skip displaying thoughts\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=1000))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "mbpegA7YhkxI"
|
||||
},
|
||||
"source": [
|
||||
"### See the thoughts\n",
|
||||
"\n",
|
||||
"This is a thinking model, you can check the thoughts that led to the image being produced."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KzKLlFCYhzov"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for part in response.parts:\n",
|
||||
" if part.thought:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" elif part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Ys_8nuIENJP9"
|
||||
},
|
||||
"source": [
|
||||
"### Grounding with search results\n",
|
||||
"\n",
|
||||
"With this model, you can also generate responses that are grounded in the results of a Google Search. Note that the model is only grounded on text results and not images that can be found on Google Search.\n",
|
||||
"\n",
|
||||
"To display the grounding data, use the helper function in the following cell."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 76,
|
||||
"metadata": {
|
||||
"id": "6yQDpb-zTl3H"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def print_grounding_data(response: types.GenerateContentResponse) -> None:\n",
|
||||
" \"\"\"Prints Gemini response with grounding citations in Markdown format.\"\"\"\n",
|
||||
" grounding_metadata = response.candidates[0].grounding_metadata\n",
|
||||
" lines = []\n",
|
||||
"\n",
|
||||
" if response.text:\n",
|
||||
" # Citation indexes are in bytes\n",
|
||||
" ENCODING = \"utf-8\"\n",
|
||||
" text_bytes = response.text.encode(ENCODING)\n",
|
||||
" last_byte_index = 0\n",
|
||||
"\n",
|
||||
" if grounding_metadata.grounding_supports:\n",
|
||||
" for support in grounding_metadata.grounding_supports:\n",
|
||||
" lines.append(\n",
|
||||
" text_bytes[last_byte_index : support.segment.end_index].decode(\n",
|
||||
" ENCODING\n",
|
||||
" )\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # Generate and append citation footnotes (e.g., \"[1][2]\")\n",
|
||||
" footnotes = \"\".join(\n",
|
||||
" [f\"[{i + 1}]\" for i in support.grounding_chunk_indices]\n",
|
||||
" )\n",
|
||||
" lines.append(f\" {footnotes}\")\n",
|
||||
"\n",
|
||||
" # Update index for the next segment\n",
|
||||
" last_byte_index = support.segment.end_index\n",
|
||||
"\n",
|
||||
" # Append any remaining text after the last citation\n",
|
||||
" if last_byte_index < len(text_bytes):\n",
|
||||
" lines.append(text_bytes[last_byte_index:].decode(ENCODING))\n",
|
||||
"\n",
|
||||
" lines.append(\"\\n\\n----\\n## Grounding Sources\\n\")\n",
|
||||
"\n",
|
||||
" if grounding_metadata.grounding_chunks:\n",
|
||||
" # Build Grounding Sources Section\n",
|
||||
" lines.append(\"### Grounding Chunks\\n\")\n",
|
||||
" for i, chunk in enumerate(grounding_metadata.grounding_chunks, start=1):\n",
|
||||
" context = chunk.web or chunk.retrieved_context or chunk.maps\n",
|
||||
" if not context:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" uri = context.uri\n",
|
||||
" title = context.title or \"Source\"\n",
|
||||
"\n",
|
||||
" # Convert GCS URIs to public HTTPS URLs\n",
|
||||
" if uri:\n",
|
||||
" uri = uri.replace(\" \", \"%20\")\n",
|
||||
" if uri.startswith(\"gs://\"):\n",
|
||||
" uri = uri.replace(\"gs://\", \"https://storage.googleapis.com/\", 1)\n",
|
||||
"\n",
|
||||
" lines.append(f\"{i}. [{title}]({uri})\\n\")\n",
|
||||
" if hasattr(context, \"place_id\") and context.place_id:\n",
|
||||
" lines.append(f\" - Place ID: `{context.place_id}`\\n\\n\")\n",
|
||||
" if hasattr(context, \"text\") and context.text:\n",
|
||||
" lines.append(f\"{context.text}\\n\\n\")\n",
|
||||
"\n",
|
||||
" # Add Search/Retrieval Queries\n",
|
||||
" if grounding_metadata.web_search_queries:\n",
|
||||
" lines.append(\n",
|
||||
" f\"\\n**Web Search Queries:** {grounding_metadata.web_search_queries}\\n\"\n",
|
||||
" )\n",
|
||||
" if grounding_metadata.search_entry_point:\n",
|
||||
" lines.append(\n",
|
||||
" f\"\\n**Search Entry Point:**\\n{grounding_metadata.search_entry_point.rendered_content}\\n\"\n",
|
||||
" )\n",
|
||||
" elif grounding_metadata.retrieval_queries:\n",
|
||||
" lines.append(\n",
|
||||
" f\"\\n**Retrieval Queries:** {grounding_metadata.retrieval_queries}\\n\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" display(Markdown(\"\".join(lines)))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Jj_HlbJpv_zx"
|
||||
},
|
||||
"source": [
|
||||
"Next, you'll create a Google Search tool and include it in the `tools` parameter of the following request."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "yWHgXTHldU_N"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"\"\"\n",
|
||||
"Search for and visualize the current weather forecast for the next 5 days in San Francisco in a clean, modern weather chart. Add a visual of what I could wear each day.\n",
|
||||
"\"\"\"\n",
|
||||
"google_search = types.Tool(google_search=types.GoogleSearch())\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"21:9\",\n",
|
||||
" ),\n",
|
||||
" tools=[google_search],\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.parts:\n",
|
||||
" if part.text and part.thought:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" elif part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))\n",
|
||||
"\n",
|
||||
"print_grounding_data(response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "5Zk2GXsoaS1u"
|
||||
},
|
||||
"source": [
|
||||
"### Image sizes\n",
|
||||
"\n",
|
||||
"Nano Banana Pro supports the following image sizes: `1K`, `2K`, or `4K`.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "rHQvvEkDaehC"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"prompt = \"\"\"\n",
|
||||
"Generate a close up headshot of a person.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=prompt,\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"1:1\",\n",
|
||||
" image_size=\"2K\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "5nlhVQCuT6DS"
|
||||
},
|
||||
"source": [
|
||||
"## Image editing\n",
|
||||
"\n",
|
||||
"You can also edit images with this model, simply pass the original image as part of the prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Y9FuL02EdvuN"
|
||||
},
|
||||
"source": [
|
||||
"### Localization\n",
|
||||
"\n",
|
||||
"You can also translate the text in images through image editing. Start by downloading the image and displaying it below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "Jp0X8wEjhjLd"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/flying-sneakers.png\n",
|
||||
"\n",
|
||||
"starting_image = \"flying-sneakers.png\"\n",
|
||||
"display(Image(filename=starting_image, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "HjIBLjr-il1y"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"with open(starting_image, \"rb\") as f:\n",
|
||||
" image = f.read()\n",
|
||||
"\n",
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_bytes(\n",
|
||||
" data=image,\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" \"Change the text in this infographic from English to Spanish.\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" image_size=\"1K\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0sIquv-1lAzn"
|
||||
},
|
||||
"source": [
|
||||
"### Multi-turn image editing (chat)\n",
|
||||
"\n",
|
||||
"In this next section, you'll generate a starting image and iteratively alter certain aspects of the image by chatting with the model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "05m25YRrB9Wg"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"chat = client.chats.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" config=types.GenerateContentConfig(response_modalities=[\"TEXT\", \"IMAGE\"]),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"message = \"Create an image of a clear perfume bottle sitting on a vanity.\"\n",
|
||||
"response = chat.send_message(message)\n",
|
||||
"\n",
|
||||
"# Save the image data to pass in the next chat message\n",
|
||||
"data = b\"\"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" data = part.inline_data.data\n",
|
||||
" display(Image(data=data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "mggna_5UPTsu"
|
||||
},
|
||||
"source": [
|
||||
"Now, you'll include the previous image data in a new message in the existing chat, along with a new text prompt, to update the previously generated image."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "bMp_cFHplh-Z"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = chat.send_message(\n",
|
||||
" message=[\n",
|
||||
" types.Part.from_bytes(\n",
|
||||
" data=data,\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" \"Make the perfume bottle purple and add a vase of hydrangeas next to the bottle.\",\n",
|
||||
" ],\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "hQ_uiJOY5Sy9"
|
||||
},
|
||||
"source": [
|
||||
"### Multiple reference images\n",
|
||||
"\n",
|
||||
"With Nano Banana Pro, you can include multiple reference images in a request to generate a new image that preserves the content of the original images.\n",
|
||||
"\n",
|
||||
"Run the following cell to visualize the starting images stored in Cloud Storage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "cUsResOwmBuS"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from io import BytesIO\n",
|
||||
"\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import requests\n",
|
||||
"from PIL import Image as PIL_Image\n",
|
||||
"\n",
|
||||
"image_urls = [\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/woman.jpg\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/suitcase.png\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/armchair.png\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/man-in-field.png\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/shoes.jpg\",\n",
|
||||
" \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/living-room.png\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"fig, axes = plt.subplots(2, 3, figsize=(12, 8))\n",
|
||||
"for i, ax in enumerate(axes.flatten()):\n",
|
||||
" ax.imshow(PIL_Image.open(BytesIO(requests.get(image_urls[i]).content)))\n",
|
||||
" ax.axis(\"off\")\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ErSjXfZ2qg7F"
|
||||
},
|
||||
"source": [
|
||||
"The process for sending the request is similar to previous image editing calls. The main difference is that you will provide multiple `Part.from_uri` instances, one for each reference image."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "_93g7aAeoyNP"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = client.models.generate_content(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" contents=[\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/woman.jpg\",\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/suitcase.png\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/armchair.png\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/man-in-field.png\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/shoes.jpg\",\n",
|
||||
" mime_type=\"image/jpeg\",\n",
|
||||
" ),\n",
|
||||
" types.Part.from_uri(\n",
|
||||
" file_uri=\"gs://cloud-samples-data/generative-ai/image/living-room.png\",\n",
|
||||
" mime_type=\"image/png\",\n",
|
||||
" ),\n",
|
||||
" \"Generate an image of a woman sitting in a living room with a man, both wearing sneakers. The woman is sitting in a white armchair with a blue suitcase next to her.\",\n",
|
||||
" ],\n",
|
||||
" config=types.GenerateContentConfig(\n",
|
||||
" response_modalities=[\"TEXT\", \"IMAGE\"],\n",
|
||||
" image_config=types.ImageConfig(\n",
|
||||
" aspect_ratio=\"16:9\",\n",
|
||||
" ),\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"for part in response.candidates[0].content.parts:\n",
|
||||
" if part.text:\n",
|
||||
" display(Markdown(part.text))\n",
|
||||
" if part.inline_data:\n",
|
||||
" display(Image(data=part.inline_data.data, width=500))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "intro_gemini_3_image_gen.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
@@ -0,0 +1,655 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ijGzTHJJUCPY"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2024 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "VEqbX8OhE8y9"
|
||||
},
|
||||
"source": [
|
||||
"# Getting Started with Chat with Gemini\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_chat.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_chat.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>\n",
|
||||
"\n",
|
||||
"<div style=\"clear: both;\"></div>\n",
|
||||
"\n",
|
||||
"<p>\n",
|
||||
"<b>Share to:</b>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
||||
"</a>\n",
|
||||
"</p>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f95c904716cd"
|
||||
},
|
||||
"source": [
|
||||
"| Authors |\n",
|
||||
"| --- |\n",
|
||||
"| [Eric Dong](https://github.com/gericdong), [Holt Skinner](https://github.com/holtskinner) |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "CkHPv2myT2cx"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"**YouTube Video: Introduction to Gemini on Vertex AI**\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.youtube.com/watch?v=YfiLUpNejpE&list=PLIivdWyY5sqJio2yeg1dlfILOUO2FoFRx\" target=\"_blank\">\n",
|
||||
" <img src=\"https://img.youtube.com/vi/YfiLUpNejpE/maxresdefault.jpg\" alt=\"Introduction to Gemini on Vertex AI\" width=\"500\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"This notebook demonstrates how to send chat prompts to the Gemini model. Gemini supports prompts with multimodal input, including natural language tasks, multi-turn text, images, video, audio, and code generation. It can output text and code.\n",
|
||||
"\n",
|
||||
"Learn more about [Sending chat prompt requests (Gemini)](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/send-chat-prompts-gemini)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "DrkcqHrrwMAo"
|
||||
},
|
||||
"source": [
|
||||
"### Objectives\n",
|
||||
"\n",
|
||||
"In this tutorial, you learn how to send chat prompts to the Gemini model.\n",
|
||||
"\n",
|
||||
"You will complete the following tasks:\n",
|
||||
"\n",
|
||||
"- Sending chat prompts using Google Gen AI SDK for Python\n",
|
||||
"- Sending chat prompts using LangChain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "C9nEPojogw-g"
|
||||
},
|
||||
"source": [
|
||||
"### Costs\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"- Vertex AI\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "r11Gu7qNgx1p"
|
||||
},
|
||||
"source": [
|
||||
"## Getting Started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "No17Cw5hgx12"
|
||||
},
|
||||
"source": [
|
||||
"### Install libraries\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "tFy3H3aPgx12"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install --upgrade --quiet google-genai langchain-google-vertexai langchain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dmWOrTJ3gx13"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"If you are running this notebook on Google Colab, run the cell below to authenticate your environment.\n",
|
||||
"\n",
|
||||
"This step is not required if you are using [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NyKGtVQjgx13"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"# Additional authentication is required for Google Colab\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
" # Authenticate user to Google Cloud\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "DF4l8DTdWgPY"
|
||||
},
|
||||
"source": [
|
||||
"### Define Google Cloud project information and initialize Vertex AI\n",
|
||||
"\n",
|
||||
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "Nqwi-5ufWp_B"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Use the environment variable if the user doesn't provide Project ID.\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"from google import genai\n",
|
||||
"\n",
|
||||
"# fmt: off\n",
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
||||
"# fmt: on\n",
|
||||
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
||||
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
||||
"\n",
|
||||
"LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"global\")\n",
|
||||
"\n",
|
||||
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "jXHfaVS66_01"
|
||||
},
|
||||
"source": [
|
||||
"### Import libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "lslYAvw37JGQ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from IPython.display import Markdown, display\n",
|
||||
"from google.genai.types import GenerateContentConfig, ModelContent, UserContent\n",
|
||||
"from langchain.chains import ConversationChain\n",
|
||||
"from langchain.memory import ConversationBufferMemory\n",
|
||||
"from langchain.prompts import (\n",
|
||||
" ChatPromptTemplate,\n",
|
||||
" HumanMessagePromptTemplate,\n",
|
||||
" MessagesPlaceholder,\n",
|
||||
" SystemMessagePromptTemplate,\n",
|
||||
")\n",
|
||||
"from langchain_core.messages import HumanMessage, SystemMessage\n",
|
||||
"from langchain_google_vertexai import ChatVertexAI, HarmBlockThreshold, HarmCategory"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "4437b7608c8e"
|
||||
},
|
||||
"source": [
|
||||
"## Sending chat prompts using Gen AI SDK for Python\n",
|
||||
"\n",
|
||||
"### Load the Gemini model"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "2998506fe6d1"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_ID = \"gemini-3.5-flash\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "wl2AZceWjXoy"
|
||||
},
|
||||
"source": [
|
||||
"### Start a chat session\n",
|
||||
"\n",
|
||||
"You start a stateful chat session and then send chat prompts with configuration parameters including generation configurations and safety settings."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "_vLprtHAjNOO"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"chat = client.chats.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" system_instruction=\"You are an astronomer, knowledgeable about the solar system..\"\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response = chat.send_message(\n",
|
||||
" \"\"\"How many moons does Mars have? Tell me some fun facts about them.\"\"\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"display(Markdown(response.text))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "V2axT2nKuVzP"
|
||||
},
|
||||
"source": [
|
||||
"You can check out the metadata of the response including the `safety_ratings` and `usage_metadata`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ih0v9B1vspiF"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ZuHvZevgwdj5"
|
||||
},
|
||||
"source": [
|
||||
"You can retrieve the history of the chat session."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "FYuqKyyktFq7"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(chat.get_history())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "kiS2CywXxU2y"
|
||||
},
|
||||
"source": [
|
||||
"### Code chat\n",
|
||||
"\n",
|
||||
"Gemini also supports code generation from a text prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "1lWiPGQ-cDqC"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"code_chat = client.chats.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" config=GenerateContentConfig(\n",
|
||||
" system_instruction=\"You are an expert software engineer, proficient in Python.\"\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"response = code_chat.send_message(\n",
|
||||
" \"Write a function that checks if a year is a leap year\"\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"display(Markdown(response.text))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "kI781aqpy-lH"
|
||||
},
|
||||
"source": [
|
||||
"You can generate unit tests to test the function in this multi-turn chat."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "TGHToON4xyOV"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = code_chat.send_message(\"Write a unit test of the generated function\")\n",
|
||||
"\n",
|
||||
"display(Markdown(response.text))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "jeZNScL7Ci2A"
|
||||
},
|
||||
"source": [
|
||||
"### Add chat history\n",
|
||||
"\n",
|
||||
"You can add chat history to a chat by adding messages from role `user` and `model` alternately. System messages can be set in the first part for the first message."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "JzqUThJv77G9"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"chat2 = client.chats.create(\n",
|
||||
" model=MODEL_ID,\n",
|
||||
" history=[\n",
|
||||
" UserContent(\n",
|
||||
" \"\"\"My name is Ned. You are my personal assistant. My favorite movies are Lord of the Rings and Hobbit.\n",
|
||||
" Who do you work for?\n",
|
||||
" \"\"\"\n",
|
||||
" ),\n",
|
||||
" ModelContent(\"I work for Ned.\"),\n",
|
||||
" UserContent(\"What do I like?\"),\n",
|
||||
" ModelContent(\"Ned likes watching movies.\"),\n",
|
||||
" ],\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response = chat2.send_message(\"Are my favorite movies based on a book series?\")\n",
|
||||
"display(Markdown(response.text))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "G64BrDoxC-K3"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = chat2.send_message(\"When were these books published?\")\n",
|
||||
"display(Markdown(response.text))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "c4e8deb52116"
|
||||
},
|
||||
"source": [
|
||||
"### Multimodal "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "WFlnwb0u54iN"
|
||||
},
|
||||
"source": [
|
||||
"## Sending chat prompts using LangChain\n",
|
||||
"\n",
|
||||
"The Gemini API in Vertex AI is integrated with the LangChain Python SDK, making it convenient to build applications on top of Gemini models."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ga2YnFYPXZgh"
|
||||
},
|
||||
"source": [
|
||||
"### Start a chat session\n",
|
||||
"\n",
|
||||
"You can start a chat by sending chat prompts to the Gemini 3 model directly. Gemini 3 doesn't support `SystemMessage` at the moment, but `SystemMessage` can be added to the first human message by setting the `convert_system_message_to_human` to `True`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "s41n0UEwGVZV"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_message = \"You are a helpful assistant who translates English to French.\"\n",
|
||||
"human_message = \"Translate this sentence from English to French. I love programming.\"\n",
|
||||
"\n",
|
||||
"messages = [SystemMessage(content=system_message), HumanMessage(content=human_message)]\n",
|
||||
"\n",
|
||||
"chat = ChatVertexAI(\n",
|
||||
" project=PROJECT_ID,\n",
|
||||
" model_name=MODEL_ID,\n",
|
||||
" convert_system_message_to_human=True,\n",
|
||||
" safety_settings={\n",
|
||||
" HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE\n",
|
||||
" },\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"result = chat.generate([messages])\n",
|
||||
"print(result.generations[0][0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "4VprmGzu4FPO"
|
||||
},
|
||||
"source": [
|
||||
"You can check out the metadata of the generated content."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "hBLFXgzsIJhw"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(result.generations[0][0].generation_info)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "RgYHzIUP4PMo"
|
||||
},
|
||||
"source": [
|
||||
"### Use a chat chain with chat prompt template"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "9vZr5vyrV-eI"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_message = \"You are a helpful assistant who translates English to French.\"\n",
|
||||
"human_message = \"Translate this sentence from English to French. I love programming.\"\n",
|
||||
"\n",
|
||||
"messages = [SystemMessage(content=system_message), HumanMessage(content=human_message)]\n",
|
||||
"prompt = ChatPromptTemplate.from_messages(messages)\n",
|
||||
"\n",
|
||||
"chat = ChatVertexAI(\n",
|
||||
" project=PROJECT_ID,\n",
|
||||
" model_name=MODEL_ID,\n",
|
||||
" convert_system_message_to_human=True,\n",
|
||||
" safety_settings={\n",
|
||||
" HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE\n",
|
||||
" },\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"chain = prompt | chat\n",
|
||||
"chain.invoke({})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "lB50i-JG9RQp"
|
||||
},
|
||||
"source": [
|
||||
"### Use a conversation chain\n",
|
||||
"\n",
|
||||
"You also can wrap up a chat in `ConversationChain`, which has built-in memory for remembering past user inputs and model outputs."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "UFxQ6AN_5vDQ"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"model = ChatVertexAI(\n",
|
||||
" project=PROJECT_ID,\n",
|
||||
" model_name=MODEL_ID,\n",
|
||||
" convert_system_message_to_human=True,\n",
|
||||
" safety_settings={\n",
|
||||
" HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE\n",
|
||||
" },\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"prompt = ChatPromptTemplate(\n",
|
||||
" messages=[\n",
|
||||
" SystemMessagePromptTemplate.from_template(\n",
|
||||
" \"You are a helpful assistant who is good at language translation.\"\n",
|
||||
" ),\n",
|
||||
" MessagesPlaceholder(variable_name=\"history\"),\n",
|
||||
" HumanMessagePromptTemplate.from_template(\"{input}\"),\n",
|
||||
" ]\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"memory = ConversationBufferMemory(memory_key=\"history\", return_messages=True)\n",
|
||||
"conversation = ConversationChain(llm=model, prompt=prompt, verbose=True, memory=memory)\n",
|
||||
"\n",
|
||||
"conversation.invoke(\n",
|
||||
" input=\"Translate this sentence from English to French. I love programming.\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "AJMP2JrMDmS-"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"conversation.invoke(\"Translate it to Spanish\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "intro_gemini_chat.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
@@ -0,0 +1,910 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "ijGzTHJJUCPY"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Copyright 2024 Google LLC\n",
|
||||
"#\n",
|
||||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||||
"# you may not use this file except in compliance with the License.\n",
|
||||
"# You may obtain a copy of the License at\n",
|
||||
"#\n",
|
||||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||||
"#\n",
|
||||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||||
"# See the License for the specific language governing permissions and\n",
|
||||
"# limitations under the License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ZPC2X_a9ErW7"
|
||||
},
|
||||
"source": [
|
||||
"# Getting Started with the Gemini API in Vertex AI with cURL / REST API\n",
|
||||
"\n",
|
||||
"<table align=\"left\">\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_curl.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_curl.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_curl.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
" <td style=\"text-align: center\">\n",
|
||||
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_curl.ipynb\">\n",
|
||||
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||||
" </a>\n",
|
||||
" </td>\n",
|
||||
"</table>\n",
|
||||
"\n",
|
||||
"<div style=\"clear: both;\"></div>\n",
|
||||
"\n",
|
||||
"<p>\n",
|
||||
"<b>Share to:</b>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_curl.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_curl.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_curl.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_curl.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_curl.ipynb\" target=\"_blank\">\n",
|
||||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
||||
"</a>\n",
|
||||
"</p>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "f0cc0f48513b"
|
||||
},
|
||||
"source": [
|
||||
"| Authors |\n",
|
||||
"| --- |\n",
|
||||
"| [Eric Dong](https://github.com/gericdong) |\n",
|
||||
"| [Polong Lin](https://github.com/polong-lin) |"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "axauUzNXEl_R"
|
||||
},
|
||||
"source": [
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"**YouTube Video: Introduction to Gemini on Vertex AI**\n",
|
||||
"\n",
|
||||
"<a href=\"https://www.youtube.com/watch?v=YfiLUpNejpE&list=PLIivdWyY5sqJio2yeg1dlfILOUO2FoFRx\" target=\"_blank\">\n",
|
||||
" <img src=\"https://img.youtube.com/vi/YfiLUpNejpE/maxresdefault.jpg\" alt=\"Introduction to Gemini on Vertex AI\" width=\"500\">\n",
|
||||
"</a>\n",
|
||||
"\n",
|
||||
"In this tutorial, you learn how to use the Vertex AI REST API with cURL commands to interact with the Gemini 3 Flash model.\n",
|
||||
"\n",
|
||||
"You will complete the following tasks:\n",
|
||||
"\n",
|
||||
"- Text generation\n",
|
||||
"- Streaming text generation\n",
|
||||
"- Chat\n",
|
||||
"- Function Calling\n",
|
||||
"- Multimodal Input\n",
|
||||
"- Controlled generation\n",
|
||||
"- Search as a tool\n",
|
||||
"- Code execution"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "wJf9sLIIEl_S"
|
||||
},
|
||||
"source": [
|
||||
"### Costs\n",
|
||||
"This tutorial uses billable components of Google Cloud:\n",
|
||||
"\n",
|
||||
"- Vertex AI\n",
|
||||
"\n",
|
||||
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "D50ekWXjEl_S"
|
||||
},
|
||||
"source": [
|
||||
"## Getting Started"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "5f7c203ffaa1"
|
||||
},
|
||||
"source": [
|
||||
"### Install required libraries"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "4e66b2f6d36f"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture\n",
|
||||
"\n",
|
||||
"!sudo apt install -q jq"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dmWOrTJ3gx13"
|
||||
},
|
||||
"source": [
|
||||
"### Authenticate your notebook environment (Colab only)\n",
|
||||
"\n",
|
||||
"If you are running this notebook on Google Colab, run the following cell to authenticate your environment.\n",
|
||||
"\n",
|
||||
"This step is not required if you are using [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "NyKGtVQjgx13"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"# Additional authentication is required for Google Colab\n",
|
||||
"if \"google.colab\" in sys.modules:\n",
|
||||
" # Authenticate user to Google Cloud\n",
|
||||
" from google.colab import auth\n",
|
||||
"\n",
|
||||
" auth.authenticate_user()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "O6ZGaZlxP9L0"
|
||||
},
|
||||
"source": [
|
||||
"### Set Google Cloud project\n",
|
||||
"\n",
|
||||
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
||||
"\n",
|
||||
"Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "u8IivOG5SqY6"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"# fmt: off\n",
|
||||
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
||||
"# fmt: on\n",
|
||||
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
||||
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
||||
"\n",
|
||||
"LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"global\")\n",
|
||||
"\n",
|
||||
"os.environ[\"GOOGLE_CLOUD_PROJECT\"] = PROJECT_ID\n",
|
||||
"os.environ[\"GOOGLE_CLOUD_REGION\"] = LOCATION"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "854fbf388e2b"
|
||||
},
|
||||
"source": [
|
||||
"## Use the Gemini 3 Flash model"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "7eeb063ac6d4"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"MODEL_ID = \"gemini-3.5-flash\"\n",
|
||||
"\n",
|
||||
"api_host = \"aiplatform.googleapis.com\"\n",
|
||||
"if LOCATION != \"global\":\n",
|
||||
" api_host = f\"{LOCATION}-aiplatform.googleapis.com\"\n",
|
||||
"\n",
|
||||
"os.environ[\"API_ENDPOINT\"] = (\n",
|
||||
" f\"{api_host}/v1/projects/{PROJECT_ID}/locations/{LOCATION}/publishers/google/models/{MODEL_ID}\"\n",
|
||||
")\n",
|
||||
"API_ENDPOINT = os.environ[\"API_ENDPOINT\"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0ZZUVBSzc0cR"
|
||||
},
|
||||
"source": [
|
||||
"## Text generation\n",
|
||||
"\n",
|
||||
"The `generateContent` method can handle a wide variety of use cases, including multi-turn chat and multimodal input, depending on what the underlying model supports. In this example, you send a text prompt and request the model response in text."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "1979afec8834"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": {\n",
|
||||
" \"role\": \"USER\",\n",
|
||||
" \"parts\": { \"text\": \"Why is the sky blue?\" },\n",
|
||||
" },\n",
|
||||
" \"generation_config\": {\n",
|
||||
" \"response_modalities\": \"TEXT\",\n",
|
||||
" },\n",
|
||||
" }' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].text\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "27701e417da6"
|
||||
},
|
||||
"source": [
|
||||
"### Streaming\n",
|
||||
"\n",
|
||||
"The Gemini API provides a streaming response mechanism. With this approach, you don't need to wait for the complete response; you can start processing fragments as soon as they're accessible."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "rzkCij_iS0we"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:streamGenerateContent \\\n",
|
||||
" \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": {\n",
|
||||
" \"role\": \"USER\",\n",
|
||||
" \"parts\": { \"text\": \"Why is the sky blue?\" }\n",
|
||||
" }\n",
|
||||
" }' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".[] | .candidates[] | .content.parts[].text\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "3e56BV7PH9t8"
|
||||
},
|
||||
"source": [
|
||||
"### Model parameters\n",
|
||||
"\n",
|
||||
"Every prompt you send to the model includes parameter values that control how the model generates a response. The model can generate different results for different parameter values. You can experiment with different model parameters to see how the results change. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "Px8hSHhiH9t8"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": {\n",
|
||||
" \"role\": \"USER\",\n",
|
||||
" \"parts\": [\n",
|
||||
" {\"text\": \"Tell me a story.\"}\n",
|
||||
" ]\n",
|
||||
" },\n",
|
||||
" \"generation_config\": {\n",
|
||||
" \"temperature\": 0.2,\n",
|
||||
" \"top_p\": 0.1,\n",
|
||||
" \"top_k\": 16,\n",
|
||||
" \"max_output_tokens\": 2048,\n",
|
||||
" \"candidate_count\": 1,\n",
|
||||
" \"stop_sequences\": []\n",
|
||||
" },\n",
|
||||
" \"safety_settings\": {\n",
|
||||
" \"category\": \"HARM_CATEGORY_SEXUALLY_EXPLICIT\",\n",
|
||||
" \"threshold\": \"BLOCK_LOW_AND_ABOVE\"\n",
|
||||
" }\n",
|
||||
" }' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].text\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "I4-XhmPn_Pb-"
|
||||
},
|
||||
"source": [
|
||||
"### Chat\n",
|
||||
"\n",
|
||||
"The Gemini API supports natural multi-turn conversations and is ideal for text tasks that require back-and-forth interactions.\n",
|
||||
"\n",
|
||||
"Specify the `role` field only if the content represents a turn in a conversation. You can set `role` to one of the following values: `user`, `model`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "YqSQSK-K-KVU"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"parts\": [\n",
|
||||
" { \"text\": \"Hello\" }\n",
|
||||
" ]\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"model\",\n",
|
||||
" \"parts\": [\n",
|
||||
" { \"text\": \"Hello! I am glad you could both make it.\" }\n",
|
||||
" ]\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"parts\": [\n",
|
||||
" { \"text\": \"So what is the first order of business?\" }\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
" }' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].text\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "5f0f5fe3b331"
|
||||
},
|
||||
"source": [
|
||||
"### Function calling\n",
|
||||
"\n",
|
||||
"Function calling lets you create a description of a function in their code, then pass that description to a language model in a request. This sample is an example of passing in a description of a function that returns information about where a movie is playing. Several function declarations are included in the request, such as `find_movies` and `find_theaters`.\n",
|
||||
"\n",
|
||||
"Learn more about [function calling](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/function-calling)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "680b11b0ba4c"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"parts\": {\n",
|
||||
" \"text\": \"Which theaters in Mountain View show Barbie movie?\"\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" \"tools\": [\n",
|
||||
" {\n",
|
||||
" \"function_declarations\": [\n",
|
||||
" {\n",
|
||||
" \"name\": \"find_movies\",\n",
|
||||
" \"description\": \"find movie titles currently playing in theaters based on any description, genre, title words, etc.\",\n",
|
||||
" \"parameters\": {\n",
|
||||
" \"type\": \"object\",\n",
|
||||
" \"properties\": {\n",
|
||||
" \"location\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"The city and state, e.g. San Francisco, CA or a zip code e.g. 95616\"\n",
|
||||
" },\n",
|
||||
" \"description\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"Any kind of description including category or genre, title words, attributes, etc.\"\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" \"required\": [\n",
|
||||
" \"description\"\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"name\": \"find_theaters\",\n",
|
||||
" \"description\": \"find theaters based on location and optionally movie title which are is currently playing in theaters\",\n",
|
||||
" \"parameters\": {\n",
|
||||
" \"type\": \"object\",\n",
|
||||
" \"properties\": {\n",
|
||||
" \"location\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"The city and state, e.g. San Francisco, CA or a zip code e.g. 95616\"\n",
|
||||
" },\n",
|
||||
" \"movie\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"Any movie title\"\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" \"required\": [\n",
|
||||
" \"location\"\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"name\": \"get_showtimes\",\n",
|
||||
" \"description\": \"Find the start times for movies playing in a specific theater\",\n",
|
||||
" \"parameters\": {\n",
|
||||
" \"type\": \"object\",\n",
|
||||
" \"properties\": {\n",
|
||||
" \"location\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"The city and state, e.g. San Francisco, CA or a zip code e.g. 95616\"\n",
|
||||
" },\n",
|
||||
" \"movie\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"Any movie title\"\n",
|
||||
" },\n",
|
||||
" \"theater\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"Name of theater\"\n",
|
||||
" },\n",
|
||||
" \"date\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"Date for requested showtime\"\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" \"required\": [\n",
|
||||
" \"location\",\n",
|
||||
" \"movie\",\n",
|
||||
" \"theater\",\n",
|
||||
" \"date\"\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
"}' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].functionCall\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "R3g5n23lDtsN"
|
||||
},
|
||||
"source": [
|
||||
"## Multimodal input\n",
|
||||
"\n",
|
||||
"Gemini is a multimodal model that supports adding image and video in text or chat prompts for a text response.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "uTfL2DDch4Lp"
|
||||
},
|
||||
"source": [
|
||||
"### Download an image from Google Cloud Storage"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "KmtWSNLtJ7oD"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! gsutil cp \"gs://cloud-samples-data/generative-ai/image/320px-Felis_catus-cat_on_snow.jpg\" ./image.jpg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "BlyyaPgmhpyv"
|
||||
},
|
||||
"source": [
|
||||
"### Generate text from a local image\n",
|
||||
"\n",
|
||||
"Specify the [base64](https://en.wikipedia.org/wiki/Base64) encoding of the image or video to include inline in the prompt and the `mime_type` field. The supported [MIME types](https://en.wikipedia.org/wiki/Media_type) for images include `image/png` and `image/jpeg`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "-uqZ-RWdtdit"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"# Encode image data in base64\n",
|
||||
"image_file=\"image.jpg\"\n",
|
||||
"if [[ -f \"$image_file\" ]]; then\n",
|
||||
" if command -v base64 &> /dev/null; then\n",
|
||||
" # base64 is available\n",
|
||||
" if [[ \"$(uname -s)\" == \"Darwin\" ]]; then\n",
|
||||
" # macOS -b 0 to avoid line wrapping\n",
|
||||
" data=$(base64 -b 0 -i \"$image_file\")\n",
|
||||
" else\n",
|
||||
" # Linux -w 0 to avoid line wrapping\n",
|
||||
" data=$(base64 -w 0 \"$image_file\")\n",
|
||||
" fi\n",
|
||||
" else\n",
|
||||
" echo \"Error: base64 command not found.\"\n",
|
||||
" exit 1\n",
|
||||
" fi\n",
|
||||
"else\n",
|
||||
" echo \"Error: Image file '$image_file' not found.\"\n",
|
||||
" exit 1\n",
|
||||
"fi\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d \"{\n",
|
||||
" 'contents': {\n",
|
||||
" 'role': 'USER',\n",
|
||||
" 'parts': [\n",
|
||||
" {\n",
|
||||
" 'text': 'Is it a cat?'\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" 'inline_data': {\n",
|
||||
" 'data': '${data}',\n",
|
||||
" 'mime_type':'image/jpeg'\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" }\" 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].text\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "fKr-BklmhjgP"
|
||||
},
|
||||
"source": [
|
||||
"### Generate text from an image on Google Cloud Storage\n",
|
||||
"\n",
|
||||
"Specify the Cloud Storage URI of the image to include in the prompt. The bucket that stores the file must be in the same Google Cloud project that's sending the request. You must also specify the `mime_type` field. The supported image MIME types include `image/png` and `image/jpeg`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "43pQE3_z3OjG"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": {\n",
|
||||
" \"role\": \"USER\",\n",
|
||||
" \"parts\": [\n",
|
||||
" {\n",
|
||||
" \"text\": \"Describe this image\"\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"file_data\": {\n",
|
||||
" \"mime_type\": \"image/png\",\n",
|
||||
" \"file_uri\": \"gs://cloud-samples-data/generative-ai/image/320px-Felis_catus-cat_on_snow.jpg\"\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
" },\n",
|
||||
" \"generation_config\": {\n",
|
||||
" \"temperature\": 0.2,\n",
|
||||
" \"top_p\": 0.1,\n",
|
||||
" \"top_k\": 16,\n",
|
||||
" \"max_output_tokens\": 2048,\n",
|
||||
" \"candidate_count\": 1,\n",
|
||||
" \"stop_sequences\": []\n",
|
||||
" },\n",
|
||||
" \"safety_settings\": {\n",
|
||||
" \"category\": \"HARM_CATEGORY_SEXUALLY_EXPLICIT\",\n",
|
||||
" \"threshold\": \"BLOCK_LOW_AND_ABOVE\"\n",
|
||||
" }\n",
|
||||
" }' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].text\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "TVF4vHuBOD8N"
|
||||
},
|
||||
"source": [
|
||||
"### Generate text from a video file\n",
|
||||
"\n",
|
||||
"Specify the Cloud Storage URI of the video to include in the prompt. The bucket that stores the file must be in the same Google Cloud project that's sending the request. You must also specify the `mime_type` field. The supported MIME types for video include `video/mp4`.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "F8kS5p0l_uHE"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d \\\n",
|
||||
"'{\n",
|
||||
" \"contents\": {\n",
|
||||
" \"role\": \"USER\",\n",
|
||||
" \"parts\": [\n",
|
||||
" {\n",
|
||||
" \"text\": \"Answer the following questions using the video only. What is the profession of the main person? What are the main features of the phone highlighted? Which city was this recorded in?\"\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"file_data\": {\n",
|
||||
" \"mime_type\": \"video/mp4\",\n",
|
||||
" \"file_uri\": \"gs://github-repo/img/gemini/multimodality_usecases_overview/pixel8.mp4\"\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" }' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].text\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "133ddb1bc7ff"
|
||||
},
|
||||
"source": [
|
||||
"### Controlled Generation\n",
|
||||
"\n",
|
||||
"Controlled generation allows you to define a response schema to specify the structure of a model's output, the field names, and the expected data type for each field."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "40db1e8d9061"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"parts\": {\n",
|
||||
" \"text\": \"List a few popular cookie recipes.\"\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" \"generationConfig\": {\n",
|
||||
" \"response_mime_type\": \"application/json\",\n",
|
||||
" \"response_schema\": {\"type\": \"object\", \"properties\": {\"recipe_name\": {\"type\": \"string\"}}}\n",
|
||||
" },\n",
|
||||
"}' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].text\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ebcbb1533401"
|
||||
},
|
||||
"source": [
|
||||
"## Search as a tool\n",
|
||||
"\n",
|
||||
"Using Grounding with Google Search, you can improve the accuracy and recency of responses from the model. Starting with Gemini 3, Google Search is available as a tool. This means that the model can decide when to use Google Search."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "a3d8a66bfb3c"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"parts\": [\n",
|
||||
" {\n",
|
||||
" \"text\": \"What is the weather today in San Jose CA?\"\n",
|
||||
" },\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" ],\n",
|
||||
" \"tools\": {\n",
|
||||
" \"google_search\": {}\n",
|
||||
" },\n",
|
||||
" \"generationConfig\": {\n",
|
||||
" \"response_modalities\": \"TEXT\"\n",
|
||||
" }\n",
|
||||
"}' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[].text\" response.json\n",
|
||||
"jq -r \".candidates[].groundingMetadata.groundingChunks\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "37223c8e3133"
|
||||
},
|
||||
"source": [
|
||||
"### Code Execution\n",
|
||||
"\n",
|
||||
"The Gemini API code execution feature enables the model to generate and run Python code and learn iteratively from the results until it arrives at a final output."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "7cebe2cd31c1"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%bash\n",
|
||||
"\n",
|
||||
"curl -X POST \\\n",
|
||||
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
|
||||
" -H \"Content-Type: application/json\" \\\n",
|
||||
" https://${API_ENDPOINT}:generateContent \\\n",
|
||||
" -d '{\n",
|
||||
" \"contents\": {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"parts\": {\n",
|
||||
" \"text\": \"Calculate 20th fibonacci number. Then find the nearest palindrome to it.\"\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" \"tools\": [\n",
|
||||
" {\"code_execution\": {},}\n",
|
||||
" ]\n",
|
||||
"}' 2>/dev/null >response.json\n",
|
||||
"\n",
|
||||
"jq -r \".candidates[].content.parts[]\" response.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6301d8abe89d"
|
||||
},
|
||||
"source": [
|
||||
"## What's next\n",
|
||||
"\n",
|
||||
"- Explore other notebooks in the [Google Cloud Generative AI GitHub repository](https://github.com/GoogleCloudPlatform/generative-ai).\n",
|
||||
"- Explore AI models in [Model Garden](https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models)."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"name": "intro_gemini_curl.ipynb",
|
||||
"toc_visible": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user