{ "cells": [ { "cell_type": "code", "execution_count": null, "id": "26882ecf", "metadata": { "cellView": "form", "id": "2RexaI2BoZZy" }, "outputs": [], "source": [ "# Copyright 2026 Google LLC\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "id": "d0204763", "metadata": { "id": "rJ43O7oc6xGP" }, "source": [ "# Intro to Gemini 3.5 Flash\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \"Google
Open in Colab\n", " \n", " 		\n", " \n", " \"Google
Open in Colab Enterprise\n", " \n", " 		\n", " \n", " \"Workbench
Open in Workbench\n", " \n", " 		\n", " \n", " \"GitHub
View on GitHub\n", " \n", "
\n", "\n", "
\n", "\n", "

\n", "Share to:\n", "\n", "\n", " \"LinkedIn\n", "\n", "\n", "\n", " \"Bluesky\n", "\n", "\n", "\n", " \"X\n", "\n", "\n", "\n", " \"Reddit\n", "\n", "\n", "\n", " \"Facebook\n", "\n", "

" ] }, { "cell_type": "markdown", "id": "6c2fad61", "metadata": { "id": "uYusEvgifp9U" }, "source": [ "| Authors |\n", "| --- |\n", "| [Eric Dong](https://github.com/gericdong) |\n", "| [Holt Skinner](https://github.com/holtskinner) |" ] }, { "cell_type": "markdown", "id": "c7366c8a", "metadata": { "id": "8sg1R6UQ7Q3t" }, "source": [ "## Overview\n", "\n", "This notebook serves as a comprehensive developer guide and quickstart for **[Gemini 3.5 Flash](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/gemini/3-5-flash)** using the Google Gen AI SDK.\n", "\n", "Gemini 3.5 Flash is engineered specifically to power high-speed, cost-efficient, and complex developer and agentic workflows. It brings powerful multi-step reasoning capabilities to a highly optimized, low-latency framework, making it highly effective for running massive reasoning loops at scale.\n", "\n", "### 🌟 Key Model & API Features Covered\n", "\n", "1. **Core Generation APIs & Standard Quickstart:** Familiarize yourself with standard content generation (`generate_content`), streaming, and async APIs.\n", "2. **Stateful Interactions API:** Learn how to use the recommended stateful API optimized for chronological reasoning, complex sub-agent coordination, and multi-turn workflows.\n", "3. **Flexible Thinking Control (`thinking_level`):** Control reasoning depth (`MINIMAL`, `LOW`, `MEDIUM`, or `HIGH`) to balance latency, cost, and intelligence.\n", "4. **Granular Multimodal Processing (`media_resolution`):** Optimize token consumption globally or per individual media part (images, video, documents).\n", "5. **Robust Tool Use & Thought Preservation:** Automate or manually handle encrypted **Thought Signatures** across multi-turn tool invocations, stream tool arguments, and return multimodal function responses.\n", "\n", "---" ] }, { "cell_type": "markdown", "id": "ffc9be79", "metadata": { "id": "gPiTOAHURvTM" }, "source": [ "## 🚀 Getting Started & Setup" ] }, { "cell_type": "markdown", "id": "59aa4204", "metadata": { "id": "-tn3uw268iw4" }, "source": [ "### Install Google Gen AI SDK for Python\n", "\n", "Gemini 3.5 Flash features require the Google Gen AI SDK for Python version `2.0.0` or later." ] }, { "cell_type": "code", "execution_count": null, "id": "2deb0b1c", "metadata": { "id": "3CaXL22k8iw4" }, "outputs": [], "source": [ "%pip install --upgrade --quiet \"google-genai[pyopenssl]>=2.4.0\"" ] }, { "cell_type": "markdown", "id": "ca6812e7", "metadata": { "id": "xW5WwfAOfp9V" }, "source": [ "### Import Libraries\n", "\n", "Import the required system and visual display components. We will also import the Pydantic library for validating structured outputs." ] }, { "cell_type": "code", "execution_count": null, "id": "fc4d4ff9", "metadata": { "id": "o0JUCkvVfp9V" }, "outputs": [], "source": [ "import os\n", "import sys\n", "\n", "from IPython.display import HTML, Image, Markdown, display\n", "from google.genai import types\n", "from pydantic import BaseModel" ] }, { "cell_type": "markdown", "id": "b2896b77", "metadata": { "id": "SY-GRP3m8iw5" }, "source": [ "### Authenticate your Notebook Environment\n", "\n", "If you are running this notebook in **Google Colab**, execute the cell below to authenticate." ] }, { "cell_type": "code", "execution_count": null, "id": "b3871f39", "metadata": { "id": "06RAe75C8iw5" }, "outputs": [], "source": [ "if \"google.colab\" in sys.modules:\n", " from google.colab import auth\n", "\n", " auth.authenticate_user()" ] }, { "cell_type": "markdown", "id": "8501ea4a", "metadata": { "id": "wkInCpT9fp9V" }, "source": [ "### Set Google Cloud Project Information\n", "\n", "To get started using Agent Platform, you must have an existing Google Cloud project and [enable the Agent Platform API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n", "\n", "Learn more about [setting up a project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects) and a [development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)." ] }, { "cell_type": "code", "execution_count": null, "id": "408d8e8b", "metadata": { "id": "z-jVOPQVfp9V" }, "outputs": [], "source": [ "from google import genai\n", "\n", "# fmt: off\n", "PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n", "# fmt: on\n", "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n", " PROJECT_ID = str(os.getenv(\"GOOGLE_CLOUD_PROJECT\"))\n", "\n", "LOCATION = \"global\"\n", "\n", "client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)" ] }, { "cell_type": "markdown", "id": "ba1fdf68", "metadata": { "id": "eXHJi5B6P5vd" }, "source": [ "### Set Model ID\n", "\n", "Define `gemini-3.5-flash` as the target model for all tutorial examples." ] }, { "cell_type": "code", "execution_count": null, "id": "7724bc98", "metadata": { "id": "eGbUoVu-931G" }, "outputs": [], "source": [ "# fmt: off\n", "MODEL_ID = \"gemini-3.5-flash\" # @param [\"gemini-3.5-flash\"] {type: \"string\"}\n", "# fmt: on" ] }, { "cell_type": "markdown", "id": "238cdec1", "metadata": { "id": "sHanNp-jfp9W" }, "source": [ "## 💻 Core Generation APIs" ] }, { "cell_type": "markdown", "id": "cc400393", "metadata": { "id": "sHanNp-jfp9W_sub" }, "source": [ "### Basic Content Generation (`generate_content`)\n", "\n", "Use the straightforward standard method to generate content synchronously from text inputs." ] }, { "cell_type": "code", "execution_count": null, "id": "5e369246", "metadata": { "id": "SzYgqNCNfp9W" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"How does AI work?\",\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "9132ecd2", "metadata": { "id": "lxZ--hIvfp9Z" }, "source": [ "### Streaming Content Generation (`generate_content_stream`)\n", "\n", "For user-facing or chat-based interfaces, use streaming to receive content chunks immediately as they are being computed." ] }, { "cell_type": "code", "execution_count": null, "id": "a192bfa1", "metadata": { "id": "xDDqwvWlfp9Z" }, "outputs": [], "source": [ "prompt = \"\"\"\n", "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball.\n", "How much does the ball cost?\n", "\"\"\"\n", "\n", "for chunk in client.models.generate_content_stream(\n", " model=MODEL_ID,\n", " contents=prompt,\n", "):\n", " print(chunk.text, end=\"\")" ] }, { "cell_type": "markdown", "metadata": { "id": "2d147e1e0225" }, "source": [ "### Multi-turn Chats\n", "\n", "Create stateful chats where the history is stored locally in the Chat instance." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "6c2864a9dafa" }, "outputs": [], "source": [ "chat = client.chats.create(\n", " model=MODEL_ID,\n", " config=types.GenerateContentConfig(\n", " thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.MINIMAL)\n", " ),\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "4a1dfdfae0a6" }, "outputs": [], "source": [ "response = chat.send_message(\n", " \"Write a Python function that checks if a year is a leap year.\"\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "metadata": { "id": "f61aedff23de" }, "source": [ "This follow-up prompt shows how the model responds based on the previous prompt:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "d436775ff147" }, "outputs": [], "source": [ "response = chat.send_message(\"Write a unit test of the generated function.\")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "db2f71da", "metadata": { "id": "SONkqnLRfp9a" }, "source": [ "### Asynchronous Generation (`client.aio`)\n", "\n", "Improve execution throughput in web apps and multi-agent systems by running non-blocking async generation requests." ] }, { "cell_type": "code", "execution_count": null, "id": "dbbbc098", "metadata": { "id": "PhPZyeN0fp9a" }, "outputs": [], "source": [ "response = await client.aio.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"Compose a song about the adventures of a time-traveling squirrel.\",\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "6a54ccb7", "metadata": { "id": "Mwpbk8wOfp9W" }, "source": [ "## 🧠 Reasoning & Thinking Control\n", "\n", "Gemini 3.5 Flash features a customizable reasoning depth. The **default thinking level is `medium`** (which replaced the `high` setting used in older Gemini 3 models to deliver rapid, highly intelligent, yet cost-efficient reasoning).\n", "\n", "Developers can adjust the `thinking_level` parameter inside `ThinkingConfig` to fine-tune this balance.\n", "\n", "| Thinking Level | Description | Recommended Use Case |\n", "| :--- | :--- | :--- |\n", "| `MINIMAL` | Lowest reasoning footprint; matches standard \"no thinking\" settings. | Fact retrieval, classification, high-throughput chat. |\n", "| `LOW` | Lightweight logical checks with high execution speed and low token cost. | Moderate concepts comparison, editing. |\n", "| `MEDIUM` (Default) | Highly balanced reasoning effort. | Standard problem solving, standard logical deduction. |\n", "| `HIGH` (Dynamic) | Maximizes internal chain-of-thought loops before first-token output. | Advanced math, logic puzzles, complex code generation. |\n", "\n", "> ⚠️ **NOTE:** You cannot combine `thinking_level` with the legacy `thinking_budget` in the same configuration block. Doing so returns a `400 Bad Request` API error." ] }, { "cell_type": "markdown", "id": "8dd35e0a", "metadata": { "id": "Mwpbk8wOfp9W_sub" }, "source": [ "### Example: High-Level Reasoning for Logic Puzzles\n", "\n", "Configure `thinking_level` to `HIGH` to solve a classic math puzzle." ] }, { "cell_type": "code", "execution_count": null, "id": "355a73d8", "metadata": { "id": "0UMeFK5Tfp9W" }, "outputs": [], "source": [ "prompt = \"\"\"A farmer has chickens and rabbits. There are 35 heads and 94 legs. How many chickens and how many rabbits does he have? Show your step-by-step logic.\"\"\"\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=prompt,\n", " config=types.GenerateContentConfig(\n", " thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.HIGH)\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "ad6e24e5", "metadata": { "id": "kbQAJXx6fp9Z" }, "source": [ "### Extracting Thought Summaries (`include_thoughts`)\n", "\n", "When using reasoning capabilities, you can inspect the model's inner thoughts separately from the final answer by setting `include_thoughts=True`." ] }, { "cell_type": "code", "execution_count": null, "id": "289383c2", "metadata": { "id": "LyfDWcvufp9Z" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"How many R's are in the word strawberry?\",\n", " config=types.GenerateContentConfig(\n", " thinking_config=types.ThinkingConfig(\n", " include_thoughts=True,\n", " thinking_level=types.ThinkingLevel.LOW,\n", " )\n", " ),\n", ")\n", "\n", "for part in response.candidates[0].content.parts:\n", " if part.thought:\n", " display(\n", " Markdown(\n", " f\"\"\"## Thoughts:\n", " {part.text}\n", " \"\"\"\n", " )\n", " )\n", " else:\n", " display(\n", " Markdown(\n", " f\"\"\"## Answer:\n", " {part.text}\n", " \"\"\"\n", " )\n", " )" ] }, { "cell_type": "markdown", "id": "eedd698f", "metadata": { "id": "6OFja2Qvfp9X" }, "source": [ "## 🎆 Multimodal Controls & Media Resolution\n", "\n", "Gemini 3.5 provides precise control over vision and multimodal tokens via the `media_resolution` config. Adjusting resolution helps optimize accuracy for fine text, details, and complex visual assets versus API latency and overall token usage.\n", "\n", "* **Global Configuration:** Set a general resolution limit for all media in the request.\n", "* **Per-Part Resolution (Recommended):** Set separate media resolutions per item (e.g., highly detailed schema image in `ULTRA_HIGH` and video in `LOW`).\n", "\n", "Available Levels: `LOW`, `MEDIUM`, `HIGH`, and `ULTRA_HIGH` (per-part only)." ] }, { "cell_type": "markdown", "id": "94d63da9", "metadata": { "id": "71DgyBXAKZtM" }, "source": [ "### Example A: Per-Part Media Resolution (Ultra-High Image & Low-Resolution Video)\n", "\n", "Assign different fidelity levels to separate components of the prompt list." ] }, { "cell_type": "code", "execution_count": null, "id": "1260fb8f", "metadata": { "id": "Lc7i2xBEfp9X" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=[\n", " types.Part(\n", " file_data=types.FileData(\n", " file_uri=\"gs://cloud-samples-data/generative-ai/image/a-man-and-a-dog.png\",\n", " mime_type=\"image/jpeg\",\n", " ),\n", " media_resolution=types.PartMediaResolution(\n", " level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_ULTRA_HIGH\n", " ),\n", " ),\n", " types.Part(\n", " file_data=types.FileData(\n", " file_uri=\"gs://cloud-samples-data/generative-ai/video/behind_the_scenes_pixel.mp4\",\n", " mime_type=\"video/mp4\",\n", " ),\n", " media_resolution=types.PartMediaResolution(\n", " level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_LOW\n", " ),\n", " ),\n", " \"When does the image appear in the video? What is the context?\",\n", " ],\n", " config=types.GenerateContentConfig(\n", " thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW)\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "edaffba1", "metadata": { "id": "MCBrvJiiKcyX" }, "source": [ "### Example B: Global Media Resolution Configuration\n", "\n", "Apply `LOW` resolution constraints globally to minimize tokens and speed up response times." ] }, { "cell_type": "code", "execution_count": null, "id": "de53653a", "metadata": { "id": "MGBBPXJCKLYX" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=[\n", " types.Part(\n", " file_data=types.FileData(\n", " file_uri=\"gs://cloud-samples-data/generative-ai/image/a-man-and-a-dog.png\",\n", " mime_type=\"image/jpeg\",\n", " ),\n", " ),\n", " \"What is in the image?\",\n", " ],\n", " config=types.GenerateContentConfig(\n", " media_resolution=types.MediaResolution.MEDIA_RESOLUTION_LOW\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "dbb03173", "metadata": { "id": "kYSiUnZzfp9a" }, "source": [ "### Multimodality Integration Formats\n", "\n", "Retrieve files from local storage (`from_bytes`) or reference them directly from Google Cloud Storage or the open web (`from_uri`)." ] }, { "cell_type": "markdown", "id": "8b398c57", "metadata": { "id": "wdSJ_Sunfp9a" }, "source": [ "#### 💡 Image (Local File Processing)" ] }, { "cell_type": "code", "execution_count": null, "id": "d7afe9f4", "metadata": { "id": "fTMhSuxWfp9a" }, "outputs": [], "source": [ "# Download and open an image locally.\n", "! wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/meal.png\n", "\n", "with open(\"meal.png\", \"rb\") as f:\n", " image = f.read()\n", " display(Image(image, width=500))\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=[\n", " types.Part.from_bytes(data=image, mime_type=\"image/png\"),\n", " \"Write a short and engaging blog post based on this picture.\",\n", " ],\n", " config=types.GenerateContentConfig(\n", " thinking_config=types.ThinkingConfig(\n", " thinking_level=types.ThinkingLevel.MINIMAL\n", " ),\n", " media_resolution=types.MediaResolution.MEDIA_RESOLUTION_LOW,\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "04d9cf47", "metadata": { "id": "P__jJRLVfp9a" }, "source": [ "#### 💡 PDF Document Analysis" ] }, { "cell_type": "code", "execution_count": null, "id": "37db0766", "metadata": { "id": "NCT96Py3fp9a" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=[\n", " types.Part.from_uri(\n", " file_uri=\"gs://cloud-samples-data/generative-ai/pdf/1706.03762v7.pdf\",\n", " mime_type=\"application/pdf\",\n", " ),\n", " \"Summarize the document.\",\n", " ],\n", " config=types.GenerateContentConfig(\n", " thinking_config=types.ThinkingConfig(\n", " thinking_level=types.ThinkingLevel.MINIMAL,\n", " ),\n", " media_resolution=types.MediaResolution.MEDIA_RESOLUTION_LOW,\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "25a474cb", "metadata": { "id": "yDYVS10kfp9a" }, "source": [ "#### 💡 Audio Analysis (Direct HTTP Support)" ] }, { "cell_type": "code", "execution_count": null, "id": "d41993a4", "metadata": { "id": "KDosCL1Ofp9a" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=[\n", " types.Part.from_uri(\n", " file_uri=\"https://traffic.libsyn.com/secure/e780d51f-f115-44a6-8252-aed9216bb521/KPOD242.mp3\",\n", " mime_type=\"audio/mpeg\",\n", " ),\n", " \"Write a summary of this podcast episode.\",\n", " ],\n", " config=types.GenerateContentConfig(\n", " audio_timestamp=True,\n", " thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW),\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "014e9f70", "metadata": { "id": "jeoh77yDfp9b" }, "source": [ "#### 💡 Video Analysis (YouTube Support)" ] }, { "cell_type": "code", "execution_count": null, "id": "b7cce307", "metadata": { "id": "mZ4wow7Rfp9b" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=[\n", " types.Part.from_uri(\n", " file_uri=\"https://www.youtube.com/watch?v=3KtWfp0UopM\",\n", " mime_type=\"video/mp4\",\n", " ),\n", " \"At what point in the video is Harry Potter shown?\",\n", " ],\n", " config=types.GenerateContentConfig(\n", " thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW)\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "ed23acff", "metadata": { "id": "AgnjHBphfp9b" }, "source": [ "#### 💡 Web Page Analysis (HTML Ingestion)" ] }, { "cell_type": "code", "execution_count": null, "id": "9f497031", "metadata": { "id": "H7fj17B9fp9b" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=[\n", " types.Part.from_uri(\n", " file_uri=\"https://cloud.google.com/vertex-ai/generative-ai/docs\",\n", " mime_type=\"text/html\",\n", " ),\n", " \"Write a summary of this documentation.\",\n", " ],\n", " config=types.GenerateContentConfig(\n", " thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW)\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "92b99d32", "metadata": { "id": "0c00UyBkfp9X" }, "source": [ "## 🤖 Agentic Workflows & Advanced Tooling\n", "\n", "When a reasoning model calls an external tool (Function Calling), it pauses its internal thinking. The **Thought Signature** is an encrypted token returned by the model acting as a \"save state\".\n", "\n", "To maintain complete context and prevent logical fragmentation, **you must return this signature** in subsequent conversation rounds back to the model.\n", "\n", "* **Automatic Function Calling (Recommended):** The SDK manages thought signatures behind the scenes.\n", "* **Manual Function Calling:** The developer must explicitly append the previous candidate's content block (containing the signature) when assembling history." ] }, { "cell_type": "markdown", "id": "8bbbb032", "metadata": { "id": "nEjLXZ-GTqb9" }, "source": [ "### Example A: Automatic Function Calling (Thought Signatures Handled Automatically)\n", "\n", "The SDK resolves intermediate execution paths and automatically manages signatures in the background." ] }, { "cell_type": "code", "execution_count": null, "id": "f0e118aa", "metadata": { "id": "riOjhr8Rfp9Y" }, "outputs": [], "source": [ "def get_weather(city: str):\n", " \"\"\"Gets the weather in a city.\"\"\"\n", " if \"london\" in city.lower():\n", " return \"Rainy\"\n", " if \"new york\" in city.lower():\n", " return \"Sunny\"\n", " return \"Cloudy\"\n", "\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"What's the weather in London and New York?\",\n", " config=types.GenerateContentConfig(\n", " tools=[get_weather],\n", " ),\n", ")\n", "\n", "# The SDK handles the function calls and thought signatures, and returns the final text\n", "display(Markdown(\"## Final Response\"))\n", "display(Markdown(response.text))\n", "\n", "# Print function calling history\n", "hist_turn = response.automatic_function_calling_history[1]\n", "print(\"\\nFunction Call 1:\", hist_turn.parts[1].function_call.name)" ] }, { "cell_type": "markdown", "id": "cab7dabb", "metadata": { "id": "XWJqKKVNTEHD" }, "source": [ "### Example B: Manual Function Calling (Explicit Thought Preservation)\n", "\n", "Here, we manually run the tool and pass the signature block back in the history array to maintain the model's reasoning chain." ] }, { "cell_type": "code", "execution_count": null, "id": "e10d4389", "metadata": { "id": "EVXwCkQET4qQ" }, "outputs": [], "source": [ "# 1. Define your tool\n", "get_weather_declaration = types.FunctionDeclaration(\n", " name=\"get_weather\",\n", " description=\"Gets the current weather temperature for a given location.\",\n", " parameters={\n", " \"type\": \"object\",\n", " \"properties\": {\"location\": {\"type\": \"string\"}},\n", " \"required\": [\"location\"],\n", " },\n", ")\n", "get_weather_tool = types.Tool(function_declarations=[get_weather_declaration])\n", "\n", "# 2. Send a message that triggers the tool\n", "prompt = \"What's the weather like in London?\"\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=prompt,\n", " config=types.GenerateContentConfig(\n", " tools=[get_weather_tool],\n", " thinking_config=types.ThinkingConfig(include_thoughts=True),\n", " ),\n", ")\n", "\n", "# 3. Handle the function call\n", "function_call = response.function_calls[0]\n", "location = function_call.args[\"location\"]\n", "print(f\"Model wants to call: {function_call.name}\")\n", "\n", "# Execute your tool (e.g., call an API)\n", "# (This is a mock response for the example)\n", "print(f\"Calling external tool for: {location}\")\n", "function_response_data = {\n", " \"location\": location,\n", " \"temperature\": \"30C\",\n", "}\n", "\n", "# 4. Send the tool's result back\n", "# Append this turn's messages to history for a final response.\n", "# The `content` object automatically attaches the required thought_signature behind the scenes.\n", "history = [\n", " types.Content(role=\"user\", parts=[types.Part(text=prompt)]),\n", " response.candidates[0].content, # Signature preserved here\n", " types.Content(\n", " role=\"tool\",\n", " parts=[\n", " types.Part.from_function_response(\n", " name=function_call.name,\n", " response=function_response_data,\n", " )\n", " ],\n", " ),\n", "]\n", "\n", "response_2 = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=history,\n", " config=types.GenerateContentConfig(\n", " tools=[get_weather_tool],\n", " thinking_config=types.ThinkingConfig(include_thoughts=True),\n", " ),\n", ")\n", "\n", "# 5. Get the final, natural-language answer\n", "print(f\"\\nFinal model response: {response_2.text}\")" ] }, { "cell_type": "markdown", "id": "a6ef3ccf", "metadata": { "id": "eNKS8D9Ofp9Y" }, "source": [ "### Streaming Function Call Arguments\n", "\n", "Turn on partial tool argument streaming using `stream_function_call_arguments=True` to improve streaming performance when calling tools." ] }, { "cell_type": "code", "execution_count": null, "id": "094f551d", "metadata": { "id": "VY_Ir0OdID5v" }, "outputs": [], "source": [ "get_weather_declaration = types.FunctionDeclaration(\n", " name=\"get_weather\",\n", " description=\"Gets the current weather temperature for a given location.\",\n", " parameters={\n", " \"type\": \"object\",\n", " \"properties\": {\"location\": {\"type\": \"string\"}},\n", " \"required\": [\"location\"],\n", " },\n", ")\n", "get_weather_tool = types.Tool(function_declarations=[get_weather_declaration])\n", "\n", "\n", "for chunk in client.models.generate_content_stream(\n", " model=MODEL_ID,\n", " contents=\"What's the weather in London and New York?\",\n", " config=types.GenerateContentConfig(\n", " tools=[get_weather_tool],\n", " tool_config=types.ToolConfig(\n", " function_calling_config=types.FunctionCallingConfig(\n", " mode=types.FunctionCallingConfigMode.AUTO,\n", " stream_function_call_arguments=True,\n", " )\n", " ),\n", " ),\n", "):\n", " function_call = chunk.function_calls[0]\n", " if function_call and function_call.name:\n", " print(f\"{function_call.name}\")\n", " print(f\"will_continue={function_call.will_continue}\")" ] }, { "cell_type": "markdown", "id": "d96ef8b2", "metadata": { "id": "gyoTiy1Cfp9Y" }, "source": [ "### Multimodal Function Responses\n", "\n", "Return structured multimodal file references directly inside function call tool responses." ] }, { "cell_type": "code", "execution_count": null, "id": "6189f128", "metadata": { "id": "nX7EDrhaPkVh" }, "outputs": [], "source": [ "# 1. Define the function tool\n", "get_image_declaration = types.FunctionDeclaration(\n", " name=\"get_image\",\n", " description=\"Retrieves the image file reference for a specific order item.\",\n", " parameters={\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"item_name\": {\n", " \"type\": \"string\",\n", " \"description\": \"The name or description of the item ordered (e.g., 'green shirt').\",\n", " }\n", " },\n", " \"required\": [\"item_name\"],\n", " },\n", ")\n", "tool_config = types.Tool(function_declarations=[get_image_declaration])\n", "\n", "# 2. Send a message that triggers the tool\n", "prompt = \"Show me the green shirt I ordered last month.\"\n", "response_1 = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=[prompt],\n", " config=types.GenerateContentConfig(\n", " tools=[tool_config],\n", " ),\n", ")\n", "\n", "# 3. Handle the function call\n", "function_call = response_1.function_calls[0]\n", "requested_item = function_call.args[\"item_name\"]\n", "print(f\"Model wants to call: {function_call.name}\")\n", "\n", "# Execute your tool (e.g., call an API)\n", "# (This is a mock response for the example)\n", "print(f\"Calling external tool for: {requested_item}\")\n", "\n", "function_response_data = {\n", " \"image_ref\": {\"$ref\": \"dress.jpg\"},\n", "}\n", "\n", "function_response_multimodal_data = types.FunctionResponsePart(\n", " file_data=types.FunctionResponseFileData(\n", " mime_type=\"image/png\",\n", " display_name=\"dress.jpg\",\n", " file_uri=\"gs://cloud-samples-data/generative-ai/image/dress.jpg\",\n", " )\n", ")\n", "\n", "# 4. Send the tool's result back\n", "# Append this turn's messages to history for a final response.\n", "history = [\n", " types.Content(role=\"user\", parts=[types.Part(text=prompt)]),\n", " response_1.candidates[0].content,\n", " types.Content(\n", " role=\"tool\",\n", " parts=[\n", " types.Part.from_function_response(\n", " name=function_call.name,\n", " response=function_response_data,\n", " parts=[function_response_multimodal_data],\n", " )\n", " ],\n", " ),\n", "]\n", "\n", "response_2 = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=history,\n", " config=types.GenerateContentConfig(\n", " tools=[tool_config], thinking_config=types.ThinkingConfig(include_thoughts=True)\n", " ),\n", ")\n", "\n", "print(f\"\\nFinal model response: {response_2.text}\")" ] }, { "cell_type": "markdown", "id": "102b3b25", "metadata": { "id": "MELl7Nmpfp9Z" }, "source": [ "## 🚉 Platform Features & Model Configurations" ] }, { "cell_type": "markdown", "id": "d39a6856", "metadata": { "id": "MELl7Nmpfp9Z_sub" }, "source": [ "### ✅ Set System Instructions\n", "\n", "Establish a durable role, persona, or behavioral rule set that persists across all turns in a generation session." ] }, { "cell_type": "code", "execution_count": null, "id": "3b8f502c", "metadata": { "id": "GBCwUvdTfp9Z" }, "outputs": [], "source": [ "system_instruction = \"\"\"\n", " You are a helpful language translator.\n", " Your mission is to translate text in English to Spanish.\n", "\"\"\"\n", "\n", "prompt = \"\"\"\n", " User input: I like bagels.\n", " Answer:\n", "\"\"\"\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=prompt,\n", " config=types.GenerateContentConfig(\n", " system_instruction=system_instruction,\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "d7bca235", "metadata": { "id": "ddMgdnZ3fp9Z" }, "source": [ "### ✅ Configure Model Parameters\n", "\n", "Define hyper-parameters like token constraints.\n", "\n", "> ⚠️ **Pro-tip:** For Gemini 3+, we strongly recommend keeping `temperature`, `top_p`, `top_k` at their default values. The advanced reasoning layers are calibrated and optimized specifically around the default." ] }, { "cell_type": "code", "execution_count": null, "id": "10e15aca", "metadata": { "id": "MMiH7-ilfp9Z" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"Tell me how the internet works, but pretend I'm a puppy who only understands squeaky toys.\",\n", " config=types.GenerateContentConfig(\n", " max_output_tokens=500,\n", " thinking_config=types.ThinkingConfig(\n", " thinking_level=types.ThinkingLevel.MINIMAL,\n", " ),\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "id": "95bba82f", "metadata": { "id": "NykdwhDHfp9b" }, "source": [ "### ✅ Structured JSON Output\n", "\n", "Enforce a schema constraint to guarantee that the model outputs structured JSON matching a predefined format." ] }, { "cell_type": "markdown", "id": "2c1dc164", "metadata": { "id": "O9NhOqV-fzfN" }, "source": [ "#### Option A: Pydantic Model Schema Support" ] }, { "cell_type": "code", "execution_count": null, "id": "277040cf", "metadata": { "id": "IWx_KWGyfp9b" }, "outputs": [], "source": [ "class CountryInfo(BaseModel):\n", " name: str\n", " population: int\n", " capital: str\n", " continent: str\n", " gdp: int\n", " official_language: str\n", " total_area_sq_mi: int\n", "\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"Give me information for the United States.\",\n", " config=types.GenerateContentConfig(\n", " response_mime_type=\"application/json\",\n", " response_schema=CountryInfo,\n", " ),\n", ")\n", "# Response as JSON\n", "print(response.text)\n", "# Response as Pydantic object\n", "print(response.parsed)" ] }, { "cell_type": "markdown", "id": "a22ca9a0", "metadata": { "id": "IrKldRiPgPr2" }, "source": [ "#### Option B: OpenAPI Schema Dictionary Support" ] }, { "cell_type": "code", "execution_count": null, "id": "4ee89b43", "metadata": { "id": "bRgilziNgCyo" }, "outputs": [], "source": [ "response_schema = {\n", " \"required\": [\n", " \"name\",\n", " \"population\",\n", " \"capital\",\n", " \"continent\",\n", " \"gdp\",\n", " \"official_language\",\n", " \"total_area_sq_mi\",\n", " ],\n", " \"properties\": {\n", " \"name\": {\"type\": \"STRING\"},\n", " \"population\": {\"type\": \"INTEGER\"},\n", " \"capital\": {\"type\": \"STRING\"},\n", " \"continent\": {\"type\": \"STRING\"},\n", " \"gdp\": {\"type\": \"INTEGER\"},\n", " \"official_language\": {\"type\": \"STRING\"},\n", " \"total_area_sq_mi\": {\"type\": \"INTEGER\"},\n", " },\n", " \"type\": \"OBJECT\",\n", "}\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"Give me information for the United States.\",\n", " config=types.GenerateContentConfig(\n", " response_mime_type=\"application/json\",\n", " response_schema=response_schema,\n", " ),\n", ")\n", "# As JSON\n", "print(response.text)\n", "# As Dict\n", "print(response.parsed)" ] }, { "cell_type": "markdown", "id": "07ee4041", "metadata": { "id": "tkwGORmQfp9b" }, "source": [ "### ✅ Google Search Grounding (Search as a Tool)\n", "\n", "Ground responses in real-time internet search results to eliminate hallucinations regarding current events." ] }, { "cell_type": "code", "execution_count": null, "id": "e36a0b3f", "metadata": { "id": "SnTT3W6Ifp9b" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"Where will the next FIFA World Cup be held?\",\n", " config=types.GenerateContentConfig(\n", " tools=[types.Tool(google_search=types.GoogleSearch())],\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))\n", "print(response.candidates[0].grounding_metadata.grounding_chunks)\n", "display(\n", " HTML(response.candidates[0].grounding_metadata.search_entry_point.rendered_content)\n", ")" ] }, { "cell_type": "markdown", "id": "73fd2bed", "metadata": { "id": "CjVbPV9wfp9b" }, "source": [ "### ✅ Python Code Execution Environment\n", "\n", "Give the model access to an isolated, secure Python sandbox to calculate exact values, verify algorithms, and eliminate mathematical calculation errors." ] }, { "cell_type": "code", "execution_count": null, "id": "8cece312", "metadata": { "id": "0rDZz6tdfp9b" }, "outputs": [], "source": [ "# Define code execution tool\n", "code_execution_tool = types.Tool(code_execution=types.ToolCodeExecution())\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=\"Calculate 20th fibonacci number. Then find the nearest palindrome to it.\",\n", " config=types.GenerateContentConfig(\n", " tools=[code_execution_tool],\n", " ),\n", ")\n", "\n", "display(\n", " Markdown(\n", " f\"\"\"\n", "## Code\n", "\n", "```py\n", "{response.executable_code}\n", "```\n", "\n", "### Output\n", "\n", "```\n", "{response.code_execution_result}\n", "```\n", "\"\"\"\n", " )\n", ")" ] }, { "cell_type": "markdown", "id": "08df603b", "metadata": { "id": "WSfbJod3fp9b" }, "source": [ "### ✅ URL Context (Website Grounding)\n", "\n", "Provide specific, long-form open web pages directly to the model as context." ] }, { "cell_type": "code", "execution_count": null, "id": "49a7c9a9", "metadata": { "id": "AbA6x42lfp9c" }, "outputs": [], "source": [ "# Define the Url context tool\n", "url_context_tool = types.Tool(url_context=types.UrlContext)\n", "\n", "url = \"https://blog.google/technology/developers/introducing-gemini-cli-open-source-ai-agent/\"\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=f\"Summarize this document: {url}\",\n", " config=types.GenerateContentConfig(\n", " tools=[url_context_tool],\n", " thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW),\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))\n", "print(response.candidates[0].grounding_metadata)" ] }, { "cell_type": "markdown", "id": "a63590f1", "metadata": { "id": "yjoQryztjBx2" }, "source": [ "### ✅ Token Management\n", "\n", "Count and compute prompt token volumes before dispatching requests." ] }, { "cell_type": "code", "execution_count": null, "id": "403a49d2", "metadata": { "id": "yOxaKemujDfQ" }, "outputs": [], "source": [ "# Count tokens\n", "response = client.models.count_tokens(\n", " model=MODEL_ID,\n", " contents=\"why is the sky blue?\",\n", ")\n", "\n", "print(response)" ] }, { "cell_type": "code", "execution_count": null, "id": "ef25b531", "metadata": { "id": "SrHw5Ip1lerP" }, "outputs": [], "source": [ "# Compute tokens\n", "response = client.models.compute_tokens(\n", " model=MODEL_ID,\n", " contents=\"why is the sky blue?\",\n", ")\n", "\n", "print(response)" ] }, { "cell_type": "markdown", "id": "99538c9e", "metadata": { "id": "dn_pfhEyfp9a" }, "source": [ "### ✅ Safety Filters\n", "\n", "Dynamically adjust threshold limits across multiple hazard and content categories." ] }, { "cell_type": "code", "execution_count": null, "id": "758d8a81", "metadata": { "id": "yud7PFzafp9a" }, "outputs": [], "source": [ "system_instruction = \"Be as mean and hateful as possible.\"\n", "\n", "prompt = \"\"\"\n", "Write a list of 5 disrespectful things that I might say to the universe after stubbing my toe in the dark.\n", "\"\"\"\n", "\n", "safety_settings = [\n", " types.SafetySetting(\n", " category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,\n", " threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n", " ),\n", " types.SafetySetting(\n", " category=types.HarmCategory.HARM_CATEGORY_HARASSMENT,\n", " threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n", " ),\n", " types.SafetySetting(\n", " category=types.HarmCategory.HARM_CATEGORY_HATE_SPEECH,\n", " threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n", " ),\n", " types.SafetySetting(\n", " category=types.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,\n", " threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n", " ),\n", " types.SafetySetting(\n", " category=types.HarmCategory.HARM_CATEGORY_JAILBREAK,\n", " threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n", " ),\n", "]\n", "\n", "response = client.models.generate_content(\n", " model=MODEL_ID,\n", " contents=prompt,\n", " config=types.GenerateContentConfig(\n", " system_instruction=system_instruction,\n", " safety_settings=safety_settings,\n", " ),\n", ")\n", "\n", "# Response will be `None` if it is blocked.\n", "print(response.text)\n", "# Finish Reason will be `SAFETY` if it is blocked.\n", "print(response.candidates[0].finish_reason)\n", "# Safety Ratings show the levels for each filter.\n", "for safety_rating in response.candidates[0].safety_ratings:\n", " if safety_rating.blocked:\n", " print(safety_rating)" ] } ], "metadata": { "colab": { "name": "intro_gemini_3_5_flash.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }