wehub-resource-sync
1617 lines
55 KiB
Plaintext
1617 lines
55 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "26882ecf",
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "2RexaI2BoZZy"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Copyright 2026 Google LLC\n",
|
|
"#\n",
|
|
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
|
"# you may not use this file except in compliance with the License.\n",
|
|
"# You may obtain a copy of the License at\n",
|
|
"#\n",
|
|
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
|
"#\n",
|
|
"# Unless required by applicable law or agreed to in writing, software\n",
|
|
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
|
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
|
"# See the License for the specific language governing permissions and\n",
|
|
"# limitations under the License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d0204763",
|
|
"metadata": {
|
|
"id": "rJ43O7oc6xGP"
|
|
},
|
|
"source": [
|
|
"# Intro to Gemini 3.5 Flash\n",
|
|
"\n",
|
|
"<table align=\"left\">\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_5_flash.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_3_5_flash.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_3_5_flash.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_5_flash.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
"</table>\n",
|
|
"\n",
|
|
"<div style=\"clear: both;\"></div>\n",
|
|
"\n",
|
|
"<p>\n",
|
|
"<b>Share to:</b>\n",
|
|
"\n",
|
|
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_5_flash.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_5_flash.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_5_flash.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_5_flash.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_3_5_flash.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
|
"</a>\n",
|
|
"</p>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6c2fad61",
|
|
"metadata": {
|
|
"id": "uYusEvgifp9U"
|
|
},
|
|
"source": [
|
|
"| Authors |\n",
|
|
"| --- |\n",
|
|
"| [Eric Dong](https://github.com/gericdong) |\n",
|
|
"| [Holt Skinner](https://github.com/holtskinner) |"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c7366c8a",
|
|
"metadata": {
|
|
"id": "8sg1R6UQ7Q3t"
|
|
},
|
|
"source": [
|
|
"## Overview\n",
|
|
"\n",
|
|
"This notebook serves as a comprehensive developer guide and quickstart for **[Gemini 3.5 Flash](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/gemini/3-5-flash)** using the Google Gen AI SDK.\n",
|
|
"\n",
|
|
"Gemini 3.5 Flash is engineered specifically to power high-speed, cost-efficient, and complex developer and agentic workflows. It brings powerful multi-step reasoning capabilities to a highly optimized, low-latency framework, making it highly effective for running massive reasoning loops at scale.\n",
|
|
"\n",
|
|
"### 🌟 Key Model & API Features Covered\n",
|
|
"\n",
|
|
"1. **Core Generation APIs & Standard Quickstart:** Familiarize yourself with standard content generation (`generate_content`), streaming, and async APIs.\n",
|
|
"2. **Stateful Interactions API:** Learn how to use the recommended stateful API optimized for chronological reasoning, complex sub-agent coordination, and multi-turn workflows.\n",
|
|
"3. **Flexible Thinking Control (`thinking_level`):** Control reasoning depth (`MINIMAL`, `LOW`, `MEDIUM`, or `HIGH`) to balance latency, cost, and intelligence.\n",
|
|
"4. **Granular Multimodal Processing (`media_resolution`):** Optimize token consumption globally or per individual media part (images, video, documents).\n",
|
|
"5. **Robust Tool Use & Thought Preservation:** Automate or manually handle encrypted **Thought Signatures** across multi-turn tool invocations, stream tool arguments, and return multimodal function responses.\n",
|
|
"\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ffc9be79",
|
|
"metadata": {
|
|
"id": "gPiTOAHURvTM"
|
|
},
|
|
"source": [
|
|
"## 🚀 Getting Started & Setup"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "59aa4204",
|
|
"metadata": {
|
|
"id": "-tn3uw268iw4"
|
|
},
|
|
"source": [
|
|
"### Install Google Gen AI SDK for Python\n",
|
|
"\n",
|
|
"Gemini 3.5 Flash features require the Google Gen AI SDK for Python version `2.0.0` or later."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2deb0b1c",
|
|
"metadata": {
|
|
"id": "3CaXL22k8iw4"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install --upgrade --quiet \"google-genai[pyopenssl]>=2.4.0\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ca6812e7",
|
|
"metadata": {
|
|
"id": "xW5WwfAOfp9V"
|
|
},
|
|
"source": [
|
|
"### Import Libraries\n",
|
|
"\n",
|
|
"Import the required system and visual display components. We will also import the Pydantic library for validating structured outputs."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "fc4d4ff9",
|
|
"metadata": {
|
|
"id": "o0JUCkvVfp9V"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import sys\n",
|
|
"\n",
|
|
"from IPython.display import HTML, Image, Markdown, display\n",
|
|
"from google.genai import types\n",
|
|
"from pydantic import BaseModel"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b2896b77",
|
|
"metadata": {
|
|
"id": "SY-GRP3m8iw5"
|
|
},
|
|
"source": [
|
|
"### Authenticate your Notebook Environment\n",
|
|
"\n",
|
|
"If you are running this notebook in **Google Colab**, execute the cell below to authenticate."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b3871f39",
|
|
"metadata": {
|
|
"id": "06RAe75C8iw5"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"if \"google.colab\" in sys.modules:\n",
|
|
" from google.colab import auth\n",
|
|
"\n",
|
|
" auth.authenticate_user()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8501ea4a",
|
|
"metadata": {
|
|
"id": "wkInCpT9fp9V"
|
|
},
|
|
"source": [
|
|
"### Set Google Cloud Project Information\n",
|
|
"\n",
|
|
"To get started using Agent Platform, you must have an existing Google Cloud project and [enable the Agent Platform API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
|
"\n",
|
|
"Learn more about [setting up a project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects) and a [development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "408d8e8b",
|
|
"metadata": {
|
|
"id": "z-jVOPQVfp9V"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from google import genai\n",
|
|
"\n",
|
|
"# fmt: off\n",
|
|
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
|
"# fmt: on\n",
|
|
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
|
" PROJECT_ID = str(os.getenv(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
|
"\n",
|
|
"LOCATION = \"global\"\n",
|
|
"\n",
|
|
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ba1fdf68",
|
|
"metadata": {
|
|
"id": "eXHJi5B6P5vd"
|
|
},
|
|
"source": [
|
|
"### Set Model ID\n",
|
|
"\n",
|
|
"Define `gemini-3.5-flash` as the target model for all tutorial examples."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "7724bc98",
|
|
"metadata": {
|
|
"id": "eGbUoVu-931G"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# fmt: off\n",
|
|
"MODEL_ID = \"gemini-3.5-flash\" # @param [\"gemini-3.5-flash\"] {type: \"string\"}\n",
|
|
"# fmt: on"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "238cdec1",
|
|
"metadata": {
|
|
"id": "sHanNp-jfp9W"
|
|
},
|
|
"source": [
|
|
"## 💻 Core Generation APIs"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cc400393",
|
|
"metadata": {
|
|
"id": "sHanNp-jfp9W_sub"
|
|
},
|
|
"source": [
|
|
"### Basic Content Generation (`generate_content`)\n",
|
|
"\n",
|
|
"Use the straightforward standard method to generate content synchronously from text inputs."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "5e369246",
|
|
"metadata": {
|
|
"id": "SzYgqNCNfp9W"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"How does AI work?\",\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9132ecd2",
|
|
"metadata": {
|
|
"id": "lxZ--hIvfp9Z"
|
|
},
|
|
"source": [
|
|
"### Streaming Content Generation (`generate_content_stream`)\n",
|
|
"\n",
|
|
"For user-facing or chat-based interfaces, use streaming to receive content chunks immediately as they are being computed."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a192bfa1",
|
|
"metadata": {
|
|
"id": "xDDqwvWlfp9Z"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\"\n",
|
|
"A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball.\n",
|
|
"How much does the ball cost?\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"for chunk in client.models.generate_content_stream(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=prompt,\n",
|
|
"):\n",
|
|
" print(chunk.text, end=\"\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "2d147e1e0225"
|
|
},
|
|
"source": [
|
|
"### Multi-turn Chats\n",
|
|
"\n",
|
|
"Create stateful chats where the history is stored locally in the Chat instance."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "6c2864a9dafa"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"chat = client.chats.create(\n",
|
|
" model=MODEL_ID,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.MINIMAL)\n",
|
|
" ),\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "4a1dfdfae0a6"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = chat.send_message(\n",
|
|
" \"Write a Python function that checks if a year is a leap year.\"\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "f61aedff23de"
|
|
},
|
|
"source": [
|
|
"This follow-up prompt shows how the model responds based on the previous prompt:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "d436775ff147"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = chat.send_message(\"Write a unit test of the generated function.\")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "db2f71da",
|
|
"metadata": {
|
|
"id": "SONkqnLRfp9a"
|
|
},
|
|
"source": [
|
|
"### Asynchronous Generation (`client.aio`)\n",
|
|
"\n",
|
|
"Improve execution throughput in web apps and multi-agent systems by running non-blocking async generation requests."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "dbbbc098",
|
|
"metadata": {
|
|
"id": "PhPZyeN0fp9a"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = await client.aio.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"Compose a song about the adventures of a time-traveling squirrel.\",\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6a54ccb7",
|
|
"metadata": {
|
|
"id": "Mwpbk8wOfp9W"
|
|
},
|
|
"source": [
|
|
"## 🧠 Reasoning & Thinking Control\n",
|
|
"\n",
|
|
"Gemini 3.5 Flash features a customizable reasoning depth. The **default thinking level is `medium`** (which replaced the `high` setting used in older Gemini 3 models to deliver rapid, highly intelligent, yet cost-efficient reasoning).\n",
|
|
"\n",
|
|
"Developers can adjust the `thinking_level` parameter inside `ThinkingConfig` to fine-tune this balance.\n",
|
|
"\n",
|
|
"| Thinking Level | Description | Recommended Use Case |\n",
|
|
"| :--- | :--- | :--- |\n",
|
|
"| `MINIMAL` | Lowest reasoning footprint; matches standard \"no thinking\" settings. | Fact retrieval, classification, high-throughput chat. |\n",
|
|
"| `LOW` | Lightweight logical checks with high execution speed and low token cost. | Moderate concepts comparison, editing. |\n",
|
|
"| `MEDIUM` (Default) | Highly balanced reasoning effort. | Standard problem solving, standard logical deduction. |\n",
|
|
"| `HIGH` (Dynamic) | Maximizes internal chain-of-thought loops before first-token output. | Advanced math, logic puzzles, complex code generation. |\n",
|
|
"\n",
|
|
"> ⚠️ **NOTE:** You cannot combine `thinking_level` with the legacy `thinking_budget` in the same configuration block. Doing so returns a `400 Bad Request` API error."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8dd35e0a",
|
|
"metadata": {
|
|
"id": "Mwpbk8wOfp9W_sub"
|
|
},
|
|
"source": [
|
|
"### Example: High-Level Reasoning for Logic Puzzles\n",
|
|
"\n",
|
|
"Configure `thinking_level` to `HIGH` to solve a classic math puzzle."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "355a73d8",
|
|
"metadata": {
|
|
"id": "0UMeFK5Tfp9W"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\"A farmer has chickens and rabbits. There are 35 heads and 94 legs. How many chickens and how many rabbits does he have? Show your step-by-step logic.\"\"\"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=prompt,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.HIGH)\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ad6e24e5",
|
|
"metadata": {
|
|
"id": "kbQAJXx6fp9Z"
|
|
},
|
|
"source": [
|
|
"### Extracting Thought Summaries (`include_thoughts`)\n",
|
|
"\n",
|
|
"When using reasoning capabilities, you can inspect the model's inner thoughts separately from the final answer by setting `include_thoughts=True`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "289383c2",
|
|
"metadata": {
|
|
"id": "LyfDWcvufp9Z"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"How many R's are in the word strawberry?\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" thinking_config=types.ThinkingConfig(\n",
|
|
" include_thoughts=True,\n",
|
|
" thinking_level=types.ThinkingLevel.LOW,\n",
|
|
" )\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"for part in response.candidates[0].content.parts:\n",
|
|
" if part.thought:\n",
|
|
" display(\n",
|
|
" Markdown(\n",
|
|
" f\"\"\"## Thoughts:\n",
|
|
" {part.text}\n",
|
|
" \"\"\"\n",
|
|
" )\n",
|
|
" )\n",
|
|
" else:\n",
|
|
" display(\n",
|
|
" Markdown(\n",
|
|
" f\"\"\"## Answer:\n",
|
|
" {part.text}\n",
|
|
" \"\"\"\n",
|
|
" )\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "eedd698f",
|
|
"metadata": {
|
|
"id": "6OFja2Qvfp9X"
|
|
},
|
|
"source": [
|
|
"## 🎆 Multimodal Controls & Media Resolution\n",
|
|
"\n",
|
|
"Gemini 3.5 provides precise control over vision and multimodal tokens via the `media_resolution` config. Adjusting resolution helps optimize accuracy for fine text, details, and complex visual assets versus API latency and overall token usage.\n",
|
|
"\n",
|
|
"* **Global Configuration:** Set a general resolution limit for all media in the request.\n",
|
|
"* **Per-Part Resolution (Recommended):** Set separate media resolutions per item (e.g., highly detailed schema image in `ULTRA_HIGH` and video in `LOW`).\n",
|
|
"\n",
|
|
"Available Levels: `LOW`, `MEDIUM`, `HIGH`, and `ULTRA_HIGH` (per-part only)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "94d63da9",
|
|
"metadata": {
|
|
"id": "71DgyBXAKZtM"
|
|
},
|
|
"source": [
|
|
"### Example A: Per-Part Media Resolution (Ultra-High Image & Low-Resolution Video)\n",
|
|
"\n",
|
|
"Assign different fidelity levels to separate components of the prompt list."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "1260fb8f",
|
|
"metadata": {
|
|
"id": "Lc7i2xBEfp9X"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=[\n",
|
|
" types.Part(\n",
|
|
" file_data=types.FileData(\n",
|
|
" file_uri=\"gs://cloud-samples-data/generative-ai/image/a-man-and-a-dog.png\",\n",
|
|
" mime_type=\"image/jpeg\",\n",
|
|
" ),\n",
|
|
" media_resolution=types.PartMediaResolution(\n",
|
|
" level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_ULTRA_HIGH\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
" types.Part(\n",
|
|
" file_data=types.FileData(\n",
|
|
" file_uri=\"gs://cloud-samples-data/generative-ai/video/behind_the_scenes_pixel.mp4\",\n",
|
|
" mime_type=\"video/mp4\",\n",
|
|
" ),\n",
|
|
" media_resolution=types.PartMediaResolution(\n",
|
|
" level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_LOW\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
" \"When does the image appear in the video? What is the context?\",\n",
|
|
" ],\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW)\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "edaffba1",
|
|
"metadata": {
|
|
"id": "MCBrvJiiKcyX"
|
|
},
|
|
"source": [
|
|
"### Example B: Global Media Resolution Configuration\n",
|
|
"\n",
|
|
"Apply `LOW` resolution constraints globally to minimize tokens and speed up response times."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "de53653a",
|
|
"metadata": {
|
|
"id": "MGBBPXJCKLYX"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=[\n",
|
|
" types.Part(\n",
|
|
" file_data=types.FileData(\n",
|
|
" file_uri=\"gs://cloud-samples-data/generative-ai/image/a-man-and-a-dog.png\",\n",
|
|
" mime_type=\"image/jpeg\",\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
" \"What is in the image?\",\n",
|
|
" ],\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" media_resolution=types.MediaResolution.MEDIA_RESOLUTION_LOW\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "dbb03173",
|
|
"metadata": {
|
|
"id": "kYSiUnZzfp9a"
|
|
},
|
|
"source": [
|
|
"### Multimodality Integration Formats\n",
|
|
"\n",
|
|
"Retrieve files from local storage (`from_bytes`) or reference them directly from Google Cloud Storage or the open web (`from_uri`)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8b398c57",
|
|
"metadata": {
|
|
"id": "wdSJ_Sunfp9a"
|
|
},
|
|
"source": [
|
|
"#### 💡 Image (Local File Processing)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "d7afe9f4",
|
|
"metadata": {
|
|
"id": "fTMhSuxWfp9a"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Download and open an image locally.\n",
|
|
"! wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/meal.png\n",
|
|
"\n",
|
|
"with open(\"meal.png\", \"rb\") as f:\n",
|
|
" image = f.read()\n",
|
|
" display(Image(image, width=500))\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=[\n",
|
|
" types.Part.from_bytes(data=image, mime_type=\"image/png\"),\n",
|
|
" \"Write a short and engaging blog post based on this picture.\",\n",
|
|
" ],\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" thinking_config=types.ThinkingConfig(\n",
|
|
" thinking_level=types.ThinkingLevel.MINIMAL\n",
|
|
" ),\n",
|
|
" media_resolution=types.MediaResolution.MEDIA_RESOLUTION_LOW,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "04d9cf47",
|
|
"metadata": {
|
|
"id": "P__jJRLVfp9a"
|
|
},
|
|
"source": [
|
|
"#### 💡 PDF Document Analysis"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "37db0766",
|
|
"metadata": {
|
|
"id": "NCT96Py3fp9a"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=[\n",
|
|
" types.Part.from_uri(\n",
|
|
" file_uri=\"gs://cloud-samples-data/generative-ai/pdf/1706.03762v7.pdf\",\n",
|
|
" mime_type=\"application/pdf\",\n",
|
|
" ),\n",
|
|
" \"Summarize the document.\",\n",
|
|
" ],\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" thinking_config=types.ThinkingConfig(\n",
|
|
" thinking_level=types.ThinkingLevel.MINIMAL,\n",
|
|
" ),\n",
|
|
" media_resolution=types.MediaResolution.MEDIA_RESOLUTION_LOW,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "25a474cb",
|
|
"metadata": {
|
|
"id": "yDYVS10kfp9a"
|
|
},
|
|
"source": [
|
|
"#### 💡 Audio Analysis (Direct HTTP Support)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "d41993a4",
|
|
"metadata": {
|
|
"id": "KDosCL1Ofp9a"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=[\n",
|
|
" types.Part.from_uri(\n",
|
|
" file_uri=\"https://traffic.libsyn.com/secure/e780d51f-f115-44a6-8252-aed9216bb521/KPOD242.mp3\",\n",
|
|
" mime_type=\"audio/mpeg\",\n",
|
|
" ),\n",
|
|
" \"Write a summary of this podcast episode.\",\n",
|
|
" ],\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" audio_timestamp=True,\n",
|
|
" thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "014e9f70",
|
|
"metadata": {
|
|
"id": "jeoh77yDfp9b"
|
|
},
|
|
"source": [
|
|
"#### 💡 Video Analysis (YouTube Support)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b7cce307",
|
|
"metadata": {
|
|
"id": "mZ4wow7Rfp9b"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=[\n",
|
|
" types.Part.from_uri(\n",
|
|
" file_uri=\"https://www.youtube.com/watch?v=3KtWfp0UopM\",\n",
|
|
" mime_type=\"video/mp4\",\n",
|
|
" ),\n",
|
|
" \"At what point in the video is Harry Potter shown?\",\n",
|
|
" ],\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW)\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ed23acff",
|
|
"metadata": {
|
|
"id": "AgnjHBphfp9b"
|
|
},
|
|
"source": [
|
|
"#### 💡 Web Page Analysis (HTML Ingestion)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9f497031",
|
|
"metadata": {
|
|
"id": "H7fj17B9fp9b"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=[\n",
|
|
" types.Part.from_uri(\n",
|
|
" file_uri=\"https://cloud.google.com/vertex-ai/generative-ai/docs\",\n",
|
|
" mime_type=\"text/html\",\n",
|
|
" ),\n",
|
|
" \"Write a summary of this documentation.\",\n",
|
|
" ],\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW)\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "92b99d32",
|
|
"metadata": {
|
|
"id": "0c00UyBkfp9X"
|
|
},
|
|
"source": [
|
|
"## 🤖 Agentic Workflows & Advanced Tooling\n",
|
|
"\n",
|
|
"When a reasoning model calls an external tool (Function Calling), it pauses its internal thinking. The **Thought Signature** is an encrypted token returned by the model acting as a \"save state\".\n",
|
|
"\n",
|
|
"To maintain complete context and prevent logical fragmentation, **you must return this signature** in subsequent conversation rounds back to the model.\n",
|
|
"\n",
|
|
"* **Automatic Function Calling (Recommended):** The SDK manages thought signatures behind the scenes.\n",
|
|
"* **Manual Function Calling:** The developer must explicitly append the previous candidate's content block (containing the signature) when assembling history."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8bbbb032",
|
|
"metadata": {
|
|
"id": "nEjLXZ-GTqb9"
|
|
},
|
|
"source": [
|
|
"### Example A: Automatic Function Calling (Thought Signatures Handled Automatically)\n",
|
|
"\n",
|
|
"The SDK resolves intermediate execution paths and automatically manages signatures in the background."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "f0e118aa",
|
|
"metadata": {
|
|
"id": "riOjhr8Rfp9Y"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def get_weather(city: str):\n",
|
|
" \"\"\"Gets the weather in a city.\"\"\"\n",
|
|
" if \"london\" in city.lower():\n",
|
|
" return \"Rainy\"\n",
|
|
" if \"new york\" in city.lower():\n",
|
|
" return \"Sunny\"\n",
|
|
" return \"Cloudy\"\n",
|
|
"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"What's the weather in London and New York?\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[get_weather],\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"# The SDK handles the function calls and thought signatures, and returns the final text\n",
|
|
"display(Markdown(\"## Final Response\"))\n",
|
|
"display(Markdown(response.text))\n",
|
|
"\n",
|
|
"# Print function calling history\n",
|
|
"hist_turn = response.automatic_function_calling_history[1]\n",
|
|
"print(\"\\nFunction Call 1:\", hist_turn.parts[1].function_call.name)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cab7dabb",
|
|
"metadata": {
|
|
"id": "XWJqKKVNTEHD"
|
|
},
|
|
"source": [
|
|
"### Example B: Manual Function Calling (Explicit Thought Preservation)\n",
|
|
"\n",
|
|
"Here, we manually run the tool and pass the signature block back in the history array to maintain the model's reasoning chain."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e10d4389",
|
|
"metadata": {
|
|
"id": "EVXwCkQET4qQ"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# 1. Define your tool\n",
|
|
"get_weather_declaration = types.FunctionDeclaration(\n",
|
|
" name=\"get_weather\",\n",
|
|
" description=\"Gets the current weather temperature for a given location.\",\n",
|
|
" parameters={\n",
|
|
" \"type\": \"object\",\n",
|
|
" \"properties\": {\"location\": {\"type\": \"string\"}},\n",
|
|
" \"required\": [\"location\"],\n",
|
|
" },\n",
|
|
")\n",
|
|
"get_weather_tool = types.Tool(function_declarations=[get_weather_declaration])\n",
|
|
"\n",
|
|
"# 2. Send a message that triggers the tool\n",
|
|
"prompt = \"What's the weather like in London?\"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=prompt,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[get_weather_tool],\n",
|
|
" thinking_config=types.ThinkingConfig(include_thoughts=True),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"# 3. Handle the function call\n",
|
|
"function_call = response.function_calls[0]\n",
|
|
"location = function_call.args[\"location\"]\n",
|
|
"print(f\"Model wants to call: {function_call.name}\")\n",
|
|
"\n",
|
|
"# Execute your tool (e.g., call an API)\n",
|
|
"# (This is a mock response for the example)\n",
|
|
"print(f\"Calling external tool for: {location}\")\n",
|
|
"function_response_data = {\n",
|
|
" \"location\": location,\n",
|
|
" \"temperature\": \"30C\",\n",
|
|
"}\n",
|
|
"\n",
|
|
"# 4. Send the tool's result back\n",
|
|
"# Append this turn's messages to history for a final response.\n",
|
|
"# The `content` object automatically attaches the required thought_signature behind the scenes.\n",
|
|
"history = [\n",
|
|
" types.Content(role=\"user\", parts=[types.Part(text=prompt)]),\n",
|
|
" response.candidates[0].content, # Signature preserved here\n",
|
|
" types.Content(\n",
|
|
" role=\"tool\",\n",
|
|
" parts=[\n",
|
|
" types.Part.from_function_response(\n",
|
|
" name=function_call.name,\n",
|
|
" response=function_response_data,\n",
|
|
" )\n",
|
|
" ],\n",
|
|
" ),\n",
|
|
"]\n",
|
|
"\n",
|
|
"response_2 = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=history,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[get_weather_tool],\n",
|
|
" thinking_config=types.ThinkingConfig(include_thoughts=True),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"# 5. Get the final, natural-language answer\n",
|
|
"print(f\"\\nFinal model response: {response_2.text}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a6ef3ccf",
|
|
"metadata": {
|
|
"id": "eNKS8D9Ofp9Y"
|
|
},
|
|
"source": [
|
|
"### Streaming Function Call Arguments\n",
|
|
"\n",
|
|
"Turn on partial tool argument streaming using `stream_function_call_arguments=True` to improve streaming performance when calling tools."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "094f551d",
|
|
"metadata": {
|
|
"id": "VY_Ir0OdID5v"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"get_weather_declaration = types.FunctionDeclaration(\n",
|
|
" name=\"get_weather\",\n",
|
|
" description=\"Gets the current weather temperature for a given location.\",\n",
|
|
" parameters={\n",
|
|
" \"type\": \"object\",\n",
|
|
" \"properties\": {\"location\": {\"type\": \"string\"}},\n",
|
|
" \"required\": [\"location\"],\n",
|
|
" },\n",
|
|
")\n",
|
|
"get_weather_tool = types.Tool(function_declarations=[get_weather_declaration])\n",
|
|
"\n",
|
|
"\n",
|
|
"for chunk in client.models.generate_content_stream(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"What's the weather in London and New York?\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[get_weather_tool],\n",
|
|
" tool_config=types.ToolConfig(\n",
|
|
" function_calling_config=types.FunctionCallingConfig(\n",
|
|
" mode=types.FunctionCallingConfigMode.AUTO,\n",
|
|
" stream_function_call_arguments=True,\n",
|
|
" )\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
"):\n",
|
|
" function_call = chunk.function_calls[0]\n",
|
|
" if function_call and function_call.name:\n",
|
|
" print(f\"{function_call.name}\")\n",
|
|
" print(f\"will_continue={function_call.will_continue}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d96ef8b2",
|
|
"metadata": {
|
|
"id": "gyoTiy1Cfp9Y"
|
|
},
|
|
"source": [
|
|
"### Multimodal Function Responses\n",
|
|
"\n",
|
|
"Return structured multimodal file references directly inside function call tool responses."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "6189f128",
|
|
"metadata": {
|
|
"id": "nX7EDrhaPkVh"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# 1. Define the function tool\n",
|
|
"get_image_declaration = types.FunctionDeclaration(\n",
|
|
" name=\"get_image\",\n",
|
|
" description=\"Retrieves the image file reference for a specific order item.\",\n",
|
|
" parameters={\n",
|
|
" \"type\": \"object\",\n",
|
|
" \"properties\": {\n",
|
|
" \"item_name\": {\n",
|
|
" \"type\": \"string\",\n",
|
|
" \"description\": \"The name or description of the item ordered (e.g., 'green shirt').\",\n",
|
|
" }\n",
|
|
" },\n",
|
|
" \"required\": [\"item_name\"],\n",
|
|
" },\n",
|
|
")\n",
|
|
"tool_config = types.Tool(function_declarations=[get_image_declaration])\n",
|
|
"\n",
|
|
"# 2. Send a message that triggers the tool\n",
|
|
"prompt = \"Show me the green shirt I ordered last month.\"\n",
|
|
"response_1 = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=[prompt],\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[tool_config],\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"# 3. Handle the function call\n",
|
|
"function_call = response_1.function_calls[0]\n",
|
|
"requested_item = function_call.args[\"item_name\"]\n",
|
|
"print(f\"Model wants to call: {function_call.name}\")\n",
|
|
"\n",
|
|
"# Execute your tool (e.g., call an API)\n",
|
|
"# (This is a mock response for the example)\n",
|
|
"print(f\"Calling external tool for: {requested_item}\")\n",
|
|
"\n",
|
|
"function_response_data = {\n",
|
|
" \"image_ref\": {\"$ref\": \"dress.jpg\"},\n",
|
|
"}\n",
|
|
"\n",
|
|
"function_response_multimodal_data = types.FunctionResponsePart(\n",
|
|
" file_data=types.FunctionResponseFileData(\n",
|
|
" mime_type=\"image/png\",\n",
|
|
" display_name=\"dress.jpg\",\n",
|
|
" file_uri=\"gs://cloud-samples-data/generative-ai/image/dress.jpg\",\n",
|
|
" )\n",
|
|
")\n",
|
|
"\n",
|
|
"# 4. Send the tool's result back\n",
|
|
"# Append this turn's messages to history for a final response.\n",
|
|
"history = [\n",
|
|
" types.Content(role=\"user\", parts=[types.Part(text=prompt)]),\n",
|
|
" response_1.candidates[0].content,\n",
|
|
" types.Content(\n",
|
|
" role=\"tool\",\n",
|
|
" parts=[\n",
|
|
" types.Part.from_function_response(\n",
|
|
" name=function_call.name,\n",
|
|
" response=function_response_data,\n",
|
|
" parts=[function_response_multimodal_data],\n",
|
|
" )\n",
|
|
" ],\n",
|
|
" ),\n",
|
|
"]\n",
|
|
"\n",
|
|
"response_2 = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=history,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[tool_config], thinking_config=types.ThinkingConfig(include_thoughts=True)\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"print(f\"\\nFinal model response: {response_2.text}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "102b3b25",
|
|
"metadata": {
|
|
"id": "MELl7Nmpfp9Z"
|
|
},
|
|
"source": [
|
|
"## 🚉 Platform Features & Model Configurations"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d39a6856",
|
|
"metadata": {
|
|
"id": "MELl7Nmpfp9Z_sub"
|
|
},
|
|
"source": [
|
|
"### ✅ Set System Instructions\n",
|
|
"\n",
|
|
"Establish a durable role, persona, or behavioral rule set that persists across all turns in a generation session."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "3b8f502c",
|
|
"metadata": {
|
|
"id": "GBCwUvdTfp9Z"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"system_instruction = \"\"\"\n",
|
|
" You are a helpful language translator.\n",
|
|
" Your mission is to translate text in English to Spanish.\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"prompt = \"\"\"\n",
|
|
" User input: I like bagels.\n",
|
|
" Answer:\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=prompt,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" system_instruction=system_instruction,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d7bca235",
|
|
"metadata": {
|
|
"id": "ddMgdnZ3fp9Z"
|
|
},
|
|
"source": [
|
|
"### ✅ Configure Model Parameters\n",
|
|
"\n",
|
|
"Define hyper-parameters like token constraints.\n",
|
|
"\n",
|
|
"> ⚠️ **Pro-tip:** For Gemini 3+, we strongly recommend keeping `temperature`, `top_p`, `top_k` at their default values. The advanced reasoning layers are calibrated and optimized specifically around the default."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "10e15aca",
|
|
"metadata": {
|
|
"id": "MMiH7-ilfp9Z"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"Tell me how the internet works, but pretend I'm a puppy who only understands squeaky toys.\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" max_output_tokens=500,\n",
|
|
" thinking_config=types.ThinkingConfig(\n",
|
|
" thinking_level=types.ThinkingLevel.MINIMAL,\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "95bba82f",
|
|
"metadata": {
|
|
"id": "NykdwhDHfp9b"
|
|
},
|
|
"source": [
|
|
"### ✅ Structured JSON Output\n",
|
|
"\n",
|
|
"Enforce a schema constraint to guarantee that the model outputs structured JSON matching a predefined format."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2c1dc164",
|
|
"metadata": {
|
|
"id": "O9NhOqV-fzfN"
|
|
},
|
|
"source": [
|
|
"#### Option A: Pydantic Model Schema Support"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "277040cf",
|
|
"metadata": {
|
|
"id": "IWx_KWGyfp9b"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"class CountryInfo(BaseModel):\n",
|
|
" name: str\n",
|
|
" population: int\n",
|
|
" capital: str\n",
|
|
" continent: str\n",
|
|
" gdp: int\n",
|
|
" official_language: str\n",
|
|
" total_area_sq_mi: int\n",
|
|
"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"Give me information for the United States.\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" response_mime_type=\"application/json\",\n",
|
|
" response_schema=CountryInfo,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"# Response as JSON\n",
|
|
"print(response.text)\n",
|
|
"# Response as Pydantic object\n",
|
|
"print(response.parsed)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a22ca9a0",
|
|
"metadata": {
|
|
"id": "IrKldRiPgPr2"
|
|
},
|
|
"source": [
|
|
"#### Option B: OpenAPI Schema Dictionary Support"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4ee89b43",
|
|
"metadata": {
|
|
"id": "bRgilziNgCyo"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response_schema = {\n",
|
|
" \"required\": [\n",
|
|
" \"name\",\n",
|
|
" \"population\",\n",
|
|
" \"capital\",\n",
|
|
" \"continent\",\n",
|
|
" \"gdp\",\n",
|
|
" \"official_language\",\n",
|
|
" \"total_area_sq_mi\",\n",
|
|
" ],\n",
|
|
" \"properties\": {\n",
|
|
" \"name\": {\"type\": \"STRING\"},\n",
|
|
" \"population\": {\"type\": \"INTEGER\"},\n",
|
|
" \"capital\": {\"type\": \"STRING\"},\n",
|
|
" \"continent\": {\"type\": \"STRING\"},\n",
|
|
" \"gdp\": {\"type\": \"INTEGER\"},\n",
|
|
" \"official_language\": {\"type\": \"STRING\"},\n",
|
|
" \"total_area_sq_mi\": {\"type\": \"INTEGER\"},\n",
|
|
" },\n",
|
|
" \"type\": \"OBJECT\",\n",
|
|
"}\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"Give me information for the United States.\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" response_mime_type=\"application/json\",\n",
|
|
" response_schema=response_schema,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"# As JSON\n",
|
|
"print(response.text)\n",
|
|
"# As Dict\n",
|
|
"print(response.parsed)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "07ee4041",
|
|
"metadata": {
|
|
"id": "tkwGORmQfp9b"
|
|
},
|
|
"source": [
|
|
"### ✅ Google Search Grounding (Search as a Tool)\n",
|
|
"\n",
|
|
"Ground responses in real-time internet search results to eliminate hallucinations regarding current events."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e36a0b3f",
|
|
"metadata": {
|
|
"id": "SnTT3W6Ifp9b"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"Where will the next FIFA World Cup be held?\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[types.Tool(google_search=types.GoogleSearch())],\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))\n",
|
|
"print(response.candidates[0].grounding_metadata.grounding_chunks)\n",
|
|
"display(\n",
|
|
" HTML(response.candidates[0].grounding_metadata.search_entry_point.rendered_content)\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "73fd2bed",
|
|
"metadata": {
|
|
"id": "CjVbPV9wfp9b"
|
|
},
|
|
"source": [
|
|
"### ✅ Python Code Execution Environment\n",
|
|
"\n",
|
|
"Give the model access to an isolated, secure Python sandbox to calculate exact values, verify algorithms, and eliminate mathematical calculation errors."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "8cece312",
|
|
"metadata": {
|
|
"id": "0rDZz6tdfp9b"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Define code execution tool\n",
|
|
"code_execution_tool = types.Tool(code_execution=types.ToolCodeExecution())\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"Calculate 20th fibonacci number. Then find the nearest palindrome to it.\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[code_execution_tool],\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(\n",
|
|
" Markdown(\n",
|
|
" f\"\"\"\n",
|
|
"## Code\n",
|
|
"\n",
|
|
"```py\n",
|
|
"{response.executable_code}\n",
|
|
"```\n",
|
|
"\n",
|
|
"### Output\n",
|
|
"\n",
|
|
"```\n",
|
|
"{response.code_execution_result}\n",
|
|
"```\n",
|
|
"\"\"\"\n",
|
|
" )\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "08df603b",
|
|
"metadata": {
|
|
"id": "WSfbJod3fp9b"
|
|
},
|
|
"source": [
|
|
"### ✅ URL Context (Website Grounding)\n",
|
|
"\n",
|
|
"Provide specific, long-form open web pages directly to the model as context."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "49a7c9a9",
|
|
"metadata": {
|
|
"id": "AbA6x42lfp9c"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Define the Url context tool\n",
|
|
"url_context_tool = types.Tool(url_context=types.UrlContext)\n",
|
|
"\n",
|
|
"url = \"https://blog.google/technology/developers/introducing-gemini-cli-open-source-ai-agent/\"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=f\"Summarize this document: {url}\",\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" tools=[url_context_tool],\n",
|
|
" thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.LOW),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown(response.text))\n",
|
|
"print(response.candidates[0].grounding_metadata)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a63590f1",
|
|
"metadata": {
|
|
"id": "yjoQryztjBx2"
|
|
},
|
|
"source": [
|
|
"### ✅ Token Management\n",
|
|
"\n",
|
|
"Count and compute prompt token volumes before dispatching requests."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "403a49d2",
|
|
"metadata": {
|
|
"id": "yOxaKemujDfQ"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Count tokens\n",
|
|
"response = client.models.count_tokens(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"why is the sky blue?\",\n",
|
|
")\n",
|
|
"\n",
|
|
"print(response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ef25b531",
|
|
"metadata": {
|
|
"id": "SrHw5Ip1lerP"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Compute tokens\n",
|
|
"response = client.models.compute_tokens(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=\"why is the sky blue?\",\n",
|
|
")\n",
|
|
"\n",
|
|
"print(response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "99538c9e",
|
|
"metadata": {
|
|
"id": "dn_pfhEyfp9a"
|
|
},
|
|
"source": [
|
|
"### ✅ Safety Filters\n",
|
|
"\n",
|
|
"Dynamically adjust threshold limits across multiple hazard and content categories."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "758d8a81",
|
|
"metadata": {
|
|
"id": "yud7PFzafp9a"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"system_instruction = \"Be as mean and hateful as possible.\"\n",
|
|
"\n",
|
|
"prompt = \"\"\"\n",
|
|
"Write a list of 5 disrespectful things that I might say to the universe after stubbing my toe in the dark.\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"safety_settings = [\n",
|
|
" types.SafetySetting(\n",
|
|
" category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,\n",
|
|
" threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
|
|
" ),\n",
|
|
" types.SafetySetting(\n",
|
|
" category=types.HarmCategory.HARM_CATEGORY_HARASSMENT,\n",
|
|
" threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
|
|
" ),\n",
|
|
" types.SafetySetting(\n",
|
|
" category=types.HarmCategory.HARM_CATEGORY_HATE_SPEECH,\n",
|
|
" threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
|
|
" ),\n",
|
|
" types.SafetySetting(\n",
|
|
" category=types.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,\n",
|
|
" threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
|
|
" ),\n",
|
|
" types.SafetySetting(\n",
|
|
" category=types.HarmCategory.HARM_CATEGORY_JAILBREAK,\n",
|
|
" threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
|
|
" ),\n",
|
|
"]\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=MODEL_ID,\n",
|
|
" contents=prompt,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" system_instruction=system_instruction,\n",
|
|
" safety_settings=safety_settings,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"# Response will be `None` if it is blocked.\n",
|
|
"print(response.text)\n",
|
|
"# Finish Reason will be `SAFETY` if it is blocked.\n",
|
|
"print(response.candidates[0].finish_reason)\n",
|
|
"# Safety Ratings show the levels for each filter.\n",
|
|
"for safety_rating in response.candidates[0].safety_ratings:\n",
|
|
" if safety_rating.blocked:\n",
|
|
" print(safety_rating)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"name": "intro_gemini_3_5_flash.ipynb",
|
|
"toc_visible": true
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"name": "python3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
}
|