wehub-resource-sync
5211 lines
332 KiB
Plaintext
5211 lines
332 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "cMQCs0oQf5Jo"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Copyright 2025 Google LLC\n",
|
|
"#\n",
|
|
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
|
"# you may not use this file except in compliance with the License.\n",
|
|
"# You may obtain a copy of the License at\n",
|
|
"#\n",
|
|
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
|
"#\n",
|
|
"# Unless required by applicable law or agreed to in writing, software\n",
|
|
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
|
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
|
"# See the License for the specific language governing permissions and\n",
|
|
"# limitations under the License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "iR0jdheRGG89"
|
|
},
|
|
"source": [
|
|
"# Analyze Multimodal Data in BigQuery"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "td9kx9LVgSve"
|
|
},
|
|
"source": [
|
|
"<table align=\"left\">\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fuse-cases%2Fapplying-llms-to-data%2Fmultimodal-analysis-bigquery%2Fanalyze_multimodal_data_bigquery.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\">\n",
|
|
" <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/bigquery/import?url=https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\">\n",
|
|
" <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/bigquery/v1/32px.svg\" alt=\"BigQuery Studio logo\"><br> Open in BigQuery Studio\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
"</table>\n",
|
|
"\n",
|
|
"<div style=\"clear: both;\"></div>\n",
|
|
"\n",
|
|
"<b>Share to:</b>\n",
|
|
"\n",
|
|
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/applying-llms-to-data/multimodal-analysis-bigquery/analyze_multimodal_data_bigquery.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
|
"</a>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "EiKsD6FM5i0m"
|
|
},
|
|
"source": [
|
|
"| Author |\n",
|
|
"| --- |\n",
|
|
"| [Jeff Nelson](https://github.com/jeffonelson) |"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "intro_md"
|
|
},
|
|
"source": [
|
|
"## Overview\n",
|
|
"\n",
|
|
"This notebook provides a hands-on example of BigQuery's powerful multimodal capabilities. You'll learn how to perform sophisticated, AI-driven analysis on your data - both structured and unstructured - right from your familiar SQL environment.\n",
|
|
"\n",
|
|
"Organizations typically store data in separate locations:\n",
|
|
"* There's **structured data** stored in neat rows and columns, like in [BigQuery tables](https://cloud.google.com/bigquery/docs/introduction)\n",
|
|
"* And there's **unstructured data**, which includes images, audio, video, and more. This generally lives in a cloud object store like [Google Cloud Storage (GCS)](https://cloud.google.com/storage/docs/introduction).\n",
|
|
"\n",
|
|
"The difficulty is querying both types of data together. For example, asking a question about customer satisfaction by analyzing the *audio* of a support call alongside customer support history.\n",
|
|
"\n",
|
|
"Fortunately, [BigQuery's multimodal](https://cloud.google.com/bigquery/docs/analyze-multimodal-data) and [generative AI capabilities](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-remote-model) cover these use cases, making it possible to analyze both structured in unstrctured data in a single query."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "SwQLISjqGTHd"
|
|
},
|
|
"source": [
|
|
"### Objectives\n",
|
|
"\n",
|
|
"You will learn to:\n",
|
|
"\n",
|
|
"* Understand what an [`ObjectRef`](https://cloud.google.com/bigquery/docs/analyze-multimodal-data#objectref_values) is, and how it bridges the gap between data in BigQuery and files in Google Cloud Storage (GCS).\n",
|
|
"* Create `ObjectRef`s in two different ways.\n",
|
|
"* Combine structured tables with unstructured files to create a single, unified view.\n",
|
|
"* Run generative AI models over your new multimodal tables to extract insights."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "costs_md"
|
|
},
|
|
"source": [
|
|
"### Services and Costs\n",
|
|
"\n",
|
|
"This tutorial uses the following billable components of Google Cloud:\n",
|
|
"\n",
|
|
"* **BigQuery**: [Pricing](https://cloud.google.com/bigquery/pricing)\n",
|
|
"\n",
|
|
"* **BigQuery ML**: [Pricing](https://cloud.google.com/bigquery/pricing#bqml)\n",
|
|
"\n",
|
|
"* **Vertex AI**: [Pricing](https://cloud.google.com/vertex-ai/generative-ai/pricing)\n",
|
|
"\n",
|
|
"You can use the [Pricing Calculator](https://cloud.google.com/products/calculator) to generate a cost estimate based on your projected usage."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "nkoCFoFVSPii"
|
|
},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"## Before you begin"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "setup_md_1"
|
|
},
|
|
"source": [
|
|
"### Set up your Google Cloud project\n",
|
|
"**The following steps are required, regardless of your notebook environment.**\n",
|
|
"\n",
|
|
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
|
|
"\n",
|
|
"2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
|
|
"\n",
|
|
"3. [Enable the BigQuery, BigQuery Connection, and Vertex AI APIs](https://console.cloud.google.com/flows/enableapi?apiid=bigquery.googleapis.com,bigqueryconnection.googleapis.com,aiplatform.googleapis.com).\n",
|
|
"\n",
|
|
"4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "YQ3g-h7uTaSf"
|
|
},
|
|
"source": [
|
|
"### Set your project ID"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "set_project_id"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"PROJECT_ID = \"YOUR-PROJECT-ID\" # @param {type:\"string\"}\n",
|
|
"\n",
|
|
"# Set the project id\n",
|
|
"! gcloud config set project {PROJECT_ID}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "auth_md"
|
|
},
|
|
"source": [
|
|
"### Authenticate to your Google Cloud account\n",
|
|
"\n",
|
|
"Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "V6NjZRCXU5Ro"
|
|
},
|
|
"source": [
|
|
"**1. Colab Enterprise in BigQuery Studio or Vertex AI**\n",
|
|
"* Do nothing as you are already authenticated."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "l0dV1hvAU1ed"
|
|
},
|
|
"source": [
|
|
"**2. Colab Consumer - uncomment and run the following:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "auth_code"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from google.colab import auth\n",
|
|
"auth.authenticate_user()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "4Pp4LAJ3UyRP"
|
|
},
|
|
"source": [
|
|
"**3. Local JupyterLab instance, uncomment and run the following:**\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "AzU7S3fMVDkW"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ! gcloud auth login"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "conn_md"
|
|
},
|
|
"source": [
|
|
"### Create BigQuery Cloud resource connection\n",
|
|
"\n",
|
|
"You will need to create a [Cloud resource connection](https://cloud.google.com/bigquery/docs/create-cloud-resource-connection) to enable BigQuery to interact with Vertex AI services."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "conn_code"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"!bq mk --connection --location=us \\\n",
|
|
" --connection_type=CLOUD_RESOURCE test_connection"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "perms_md"
|
|
},
|
|
"source": [
|
|
"### Set permissions for Service Account\n",
|
|
"\n",
|
|
"The resource connection service account requires certain project-level permissions to interact with Vertex AI and Google Cloud Storage."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "6OqpZNY953xR"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"SERVICE_ACCT = !bq show --format=prettyjson --connection us.test_connection | grep \"serviceAccountId\" | cut -d '\"' -f 4\n",
|
|
"SERVICE_ACCT_EMAIL = SERVICE_ACCT[-1]\n",
|
|
"print(SERVICE_ACCT_EMAIL)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "perms_code"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import time\n",
|
|
"\n",
|
|
"!gcloud projects add-iam-policy-binding --format=none $PROJECT_ID --member=serviceAccount:$SERVICE_ACCT_EMAIL --role='roles/storage.objectViewer'\n",
|
|
"!gcloud projects add-iam-policy-binding --format=none $PROJECT_ID --member=serviceAccount:$SERVICE_ACCT_EMAIL --role='roles/aiplatform.user'\n",
|
|
"\n",
|
|
"# Wait ~60 seconds, to give IAM updates time to propagate. Otherwise, subsequent cells may fail.\n",
|
|
"time.sleep(60)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "-fhqCNYQOLCq"
|
|
},
|
|
"source": [
|
|
"#### Create a BigQuery Dataset\n",
|
|
"\n",
|
|
"Running the following query creates a [BigQuery dataset](https://cloud.google.com/bigquery/docs/datasets-intro) called **`bq_mm_tutorial`** to house any tables or remote models for this tutorial:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "ZR9S5lgpOGDr"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"CREATE SCHEMA `bq_mm_tutorial` OPTIONS (location = 'US');"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "VfM1EL2NOTVR"
|
|
},
|
|
"source": [
|
|
"### Set Colab display options\n",
|
|
"\n",
|
|
"Colab includes the `google.colab.data_table` package that can be used to display large pandas dataframes as an interactive data table. It can be enabled with:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"id": "iSYRP39jOQ7R"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%load_ext google.colab.data_table"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "generate_md_1"
|
|
},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"## What is an `ObjectRef`?\n",
|
|
"\n",
|
|
"[`ObjectRef`](https://cloud.google.com/bigquery/docs/analyze-multimodal-data#objectref_values) is the foundation of multimodal analysis in BigQuery. Think of it as a secure pointer to a file (image, PDF, video, etc.) in GCS. It doesn't hold the file's data itself, but it contains the information BigQuery needs to find and access the file during a query.\n",
|
|
"\n",
|
|
"An `ObjectRef` is stored as a `STRUCT` and contains details like the file's URI path (i.e. its direct GCS \"address\"), a secure authorizer, and other metadata. By using `ObjectRef` columns in your BigQuery tables, you can effectively query against GCS objects right alongside your structured data.\n",
|
|
"\n",
|
|
"### Creating ObjectRefs:\n",
|
|
"\n",
|
|
"There are two primary ways to create tables with `ObjectRef` columns. We'll detail them in two short demo scenarios.\n",
|
|
"* Scenario 1: Use an [object table](https://cloud.google.com/bigquery/docs/object-table-introduction) to automatically generate an `ObjectRef` for every file in a GCS bucket\n",
|
|
"* Scenario 2: Use [built-in SQL functions](https://cloud.google.com/bigquery/docs/reference/standard-sql/objectref_functions) like `OBJ.MAKE_REF()` to create `ObjectRef`s programmatically from URIs in an existing table"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "K9_iiZJWTQXG"
|
|
},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"## Scenario 1: Analyze Customer Service Calls"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "a2xG-u0qaLuG"
|
|
},
|
|
"source": [
|
|
"### 1. Load structured data\n",
|
|
"\n",
|
|
"Let's begin with a practical example. Imagine we have a standard BigQuery table with details about customer support calls. The table contains a `call_id` column, which is a unique identifier for a call. Let's load this sample data into BigQuery."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 154,
|
|
"referenced_widgets": [
|
|
"b8ef66e84d794d77916d9836532ab1f1",
|
|
"2f3fb9fa82cc46c4950c8b64f5826446",
|
|
"937cf5ab5ad24b9bb8e4b8b1202af2b8",
|
|
"f50b56de18eb421191c7ce50637fb8d4",
|
|
"61f68bac11794c5a80f523d61b427ca8",
|
|
"e57e801c23ab4f56854c349e1d5f9dc5",
|
|
"bc55e1a7c8814b668b52bab40537fc61",
|
|
"b39b6fabac0143eb9cbab48dc650e804",
|
|
"90c29b09df5347a08df528b120b65a6a",
|
|
"a6286c3f439c400dbca934df88ff4c5b",
|
|
"e1ba640e598b4e849c78aa735b168694"
|
|
]
|
|
},
|
|
"id": "1JY3WhVBKT8c",
|
|
"outputId": "ae845812-8639-4f89-ea3b-938a46268baf"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "b8ef66e84d794d77916d9836532ab1f1",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 0,\n \"fields\": []\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [],\n columns: [[\"number\", \"index\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-741b7198-20e2-4ace-8509-b64bf44786ba\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-741b7198-20e2-4ace-8509-b64bf44786ba')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-741b7198-20e2-4ace-8509-b64bf44786ba button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-ccc6a058-8abe-4413-b025-eab0d64289fa\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ccc6a058-8abe-4413-b025-eab0d64289fa')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-ccc6a058-8abe-4413-b025-eab0d64289fa button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-ccc6a058-8abe-4413-b025-eab0d64289fa');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
"Empty DataFrame\n",
|
|
"Columns: []\n",
|
|
"Index: []"
|
|
]
|
|
},
|
|
"execution_count": 16,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"LOAD DATA OVERWRITE bq_mm_tutorial.calls\n",
|
|
"FROM FILES (\n",
|
|
" uris = ['gs://sample-data-and-media/customer-support/tables/calls'],\n",
|
|
" format = 'PARQUET'\n",
|
|
");"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "F_bbC8thoHGU"
|
|
},
|
|
"source": [
|
|
"Taking a peek at the table, we can see the `call_id` column alongside other call attributes."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 293,
|
|
"referenced_widgets": [
|
|
"d7090bcee20e409cb2d7413f37fd9912",
|
|
"2c5f64dc17024b20ad5e74bb3925dab9",
|
|
"c8cbc42966a04facb2b21b5073c6b2c2",
|
|
"6ba1b399479d49db88bc42a740fa7bbe",
|
|
"14d5e40d69f54c3c93c86bafb24246d8",
|
|
"45399fa005eb4628aabeec06da3b8bfa",
|
|
"d3f1e33c657347e5ab9b0aac598b9b3b",
|
|
"a5ac996c9e9c4b2190cba1c8b548d8d9",
|
|
"90a436b5e0b9413fa477652753d5b4b2",
|
|
"46816b0077744ddf827985ba2a286ca9",
|
|
"460a8a94caf14c4892c6bfe23f8c514f",
|
|
"c38f9bedc9784442a793f069221dfb3e",
|
|
"6f27f2c33d754f1f9532cc0001e83409",
|
|
"cbc61c8d15c543609f0f81a331ba033a",
|
|
"e5de7b2893e14d21963cd45a05123db1",
|
|
"4fb6c5de06514f838cd552d5d1a14ec6",
|
|
"644cdd45b8a2496fb2f67f5165393508",
|
|
"5797e98f6e3648b1a03d27563a578d97",
|
|
"57363affda8f469099e70e9195ee8af8",
|
|
"65142791887e438e94f4c2743dfb9a6d",
|
|
"d1a4a1a201d440b79f9525ef50cf0ec6",
|
|
"d89fbda5f4284a36b833bef145f8d1e7"
|
|
]
|
|
},
|
|
"id": "YLqRk757KT3O",
|
|
"outputId": "0ba6a364-583b-4f51-e15a-d18b8adf2862"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "d7090bcee20e409cb2d7413f37fd9912",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "c38f9bedc9784442a793f069221dfb3e",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"call_id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"call_3_20250411045541\",\n \"call_4_20250411045544\",\n \"call_5_20250411045547\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"company_name\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"Synergy Corp\",\n \"Global Tech Enterprises\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"company_revenue\",\n \"properties\": {\n \"dtype\": \"Int64\",\n \"num_unique_values\": 5,\n \"samples\": [\n 9800000,\n 20000000\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"customer_name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Robert Jones\",\n \"Emily Carter\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"product_name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Global Secure VPN\",\n \"Synergy Project Manager\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"call_reason\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"VPN service disconnecting frequently\",\n \"Praise for excellent customer support\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"call_1_20250411045535\",\n\"Global Tech Enterprises\",\n{\n 'v': 12500000,\n 'f': \"12500000\",\n },\n\"Sarah Miller\",\n\"Global Shield Antivirus\",\n\"Antivirus subscription activation issue\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"call_3_20250411045541\",\n\"Global Tech Enterprises\",\n{\n 'v': 9800000,\n 'f': \"9800000\",\n },\n\"Robert Jones\",\n\"Global Secure VPN\",\n\"VPN service disconnecting frequently\"],\n [{\n 'v': 2,\n 'f': \"2\",\n },\n\"call_5_20250411045547\",\n\"Global Tech Enterprises\",\n{\n 'v': 12000000,\n 'f': \"12000000\",\n },\n\"Michael Brown\",\n\"Global Protect Data Backup\",\n\"Need to upgrade data backup plan\"],\n [{\n 'v': 3,\n 'f': \"3\",\n },\n\"call_2_20250411045538\",\n\"Synergy Corp\",\n{\n 'v': 18000000,\n 'f': \"18000000\",\n },\n\"David Martin\",\n\"Synergy Cloud Storage\",\n\"Inquiry about enterprise cloud storage pricing\"],\n [{\n 'v': 4,\n 'f': \"4\",\n },\n\"call_4_20250411045544\",\n\"Synergy Corp\",\n{\n 'v': 20000000,\n 'f': \"20000000\",\n },\n\"Emily Carter\",\n\"Synergy Project Manager\",\n\"Praise for excellent customer support\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"call_id\"], [\"string\", \"company_name\"], [\"number\", \"company_revenue\"], [\"string\", \"customer_name\"], [\"string\", \"product_name\"], [\"string\", \"call_reason\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-28c205aa-5f43-492a-a1f8-69ff5fd0f8c3\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-28c205aa-5f43-492a-a1f8-69ff5fd0f8c3')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-28c205aa-5f43-492a-a1f8-69ff5fd0f8c3 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-d591f203-8043-4221-ace4-1af5ebef2018\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>call_id</th>\n",
|
|
" <th>company_name</th>\n",
|
|
" <th>company_revenue</th>\n",
|
|
" <th>customer_name</th>\n",
|
|
" <th>product_name</th>\n",
|
|
" <th>call_reason</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>call_1_20250411045535</td>\n",
|
|
" <td>Global Tech Enterprises</td>\n",
|
|
" <td>12500000</td>\n",
|
|
" <td>Sarah Miller</td>\n",
|
|
" <td>Global Shield Antivirus</td>\n",
|
|
" <td>Antivirus subscription activation issue</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>call_3_20250411045541</td>\n",
|
|
" <td>Global Tech Enterprises</td>\n",
|
|
" <td>9800000</td>\n",
|
|
" <td>Robert Jones</td>\n",
|
|
" <td>Global Secure VPN</td>\n",
|
|
" <td>VPN service disconnecting frequently</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>call_5_20250411045547</td>\n",
|
|
" <td>Global Tech Enterprises</td>\n",
|
|
" <td>12000000</td>\n",
|
|
" <td>Michael Brown</td>\n",
|
|
" <td>Global Protect Data Backup</td>\n",
|
|
" <td>Need to upgrade data backup plan</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>call_2_20250411045538</td>\n",
|
|
" <td>Synergy Corp</td>\n",
|
|
" <td>18000000</td>\n",
|
|
" <td>David Martin</td>\n",
|
|
" <td>Synergy Cloud Storage</td>\n",
|
|
" <td>Inquiry about enterprise cloud storage pricing</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>call_4_20250411045544</td>\n",
|
|
" <td>Synergy Corp</td>\n",
|
|
" <td>20000000</td>\n",
|
|
" <td>Emily Carter</td>\n",
|
|
" <td>Synergy Project Manager</td>\n",
|
|
" <td>Praise for excellent customer support</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d591f203-8043-4221-ace4-1af5ebef2018')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-d591f203-8043-4221-ace4-1af5ebef2018 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-d591f203-8043-4221-ace4-1af5ebef2018');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" <div id=\"df-c284a3bc-e0f1-4f2b-92fe-4adf0bd99228\">\n",
|
|
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c284a3bc-e0f1-4f2b-92fe-4adf0bd99228')\"\n",
|
|
" title=\"Suggest charts\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
|
|
" width=\"24px\">\n",
|
|
" <g>\n",
|
|
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
|
|
" </g>\n",
|
|
"</svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
"<style>\n",
|
|
" .colab-df-quickchart {\n",
|
|
" --bg-color: #E8F0FE;\n",
|
|
" --fill-color: #1967D2;\n",
|
|
" --hover-bg-color: #E2EBFA;\n",
|
|
" --hover-fill-color: #174EA6;\n",
|
|
" --disabled-fill-color: #AAA;\n",
|
|
" --disabled-bg-color: #DDD;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-quickchart {\n",
|
|
" --bg-color: #3B4455;\n",
|
|
" --fill-color: #D2E3FC;\n",
|
|
" --hover-bg-color: #434B5C;\n",
|
|
" --hover-fill-color: #FFFFFF;\n",
|
|
" --disabled-bg-color: #3B4455;\n",
|
|
" --disabled-fill-color: #666;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart {\n",
|
|
" background-color: var(--bg-color);\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: var(--fill-color);\n",
|
|
" height: 32px;\n",
|
|
" padding: 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart:hover {\n",
|
|
" background-color: var(--hover-bg-color);\n",
|
|
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: var(--button-hover-fill-color);\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart-complete:disabled,\n",
|
|
" .colab-df-quickchart-complete:disabled:hover {\n",
|
|
" background-color: var(--disabled-bg-color);\n",
|
|
" fill: var(--disabled-fill-color);\n",
|
|
" box-shadow: none;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-spinner {\n",
|
|
" border: 2px solid var(--fill-color);\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" animation:\n",
|
|
" spin 1s steps(1) infinite;\n",
|
|
" }\n",
|
|
"\n",
|
|
" @keyframes spin {\n",
|
|
" 0% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 20% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 30% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 40% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 60% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 80% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 90% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" async function quickchart(key) {\n",
|
|
" const quickchartButtonEl =\n",
|
|
" document.querySelector('#' + key + ' button');\n",
|
|
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
|
|
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
|
|
" try {\n",
|
|
" const charts = await google.colab.kernel.invokeFunction(\n",
|
|
" 'suggestCharts', [key], {});\n",
|
|
" } catch (error) {\n",
|
|
" console.error('Error during call to suggestCharts:', error);\n",
|
|
" }\n",
|
|
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
|
|
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
|
|
" }\n",
|
|
" (() => {\n",
|
|
" let quickchartButtonEl =\n",
|
|
" document.querySelector('#df-c284a3bc-e0f1-4f2b-92fe-4adf0bd99228 button');\n",
|
|
" quickchartButtonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
" })();\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" call_id company_name company_revenue \\\n",
|
|
"0 call_1_20250411045535 Global Tech Enterprises 12500000 \n",
|
|
"1 call_3_20250411045541 Global Tech Enterprises 9800000 \n",
|
|
"2 call_5_20250411045547 Global Tech Enterprises 12000000 \n",
|
|
"3 call_2_20250411045538 Synergy Corp 18000000 \n",
|
|
"4 call_4_20250411045544 Synergy Corp 20000000 \n",
|
|
"\n",
|
|
" customer_name product_name \\\n",
|
|
"0 Sarah Miller Global Shield Antivirus \n",
|
|
"1 Robert Jones Global Secure VPN \n",
|
|
"2 Michael Brown Global Protect Data Backup \n",
|
|
"3 David Martin Synergy Cloud Storage \n",
|
|
"4 Emily Carter Synergy Project Manager \n",
|
|
"\n",
|
|
" call_reason \n",
|
|
"0 Antivirus subscription activation issue \n",
|
|
"1 VPN service disconnecting frequently \n",
|
|
"2 Need to upgrade data backup plan \n",
|
|
"3 Inquiry about enterprise cloud storage pricing \n",
|
|
"4 Praise for excellent customer support "
|
|
]
|
|
},
|
|
"execution_count": 17,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT * FROM bq_mm_tutorial.calls;"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "CA6lCP22oTj8"
|
|
},
|
|
"source": [
|
|
"### 2. Create an Object Table\n",
|
|
"\n",
|
|
"We also have audio files stored in GCS corresponding to these customer support calls. The following creates an [object table](https://cloud.google.com/bigquery/docs/object-table-introduction) over a bucket containing customer service calls.\n",
|
|
"\n",
|
|
"The object table is a read-only BigQuery table that mirrors the contents of a GCS directly and [automatically generates an `ObjectRef`](https://cloud.google.com/bigquery/docs/analyze-multimodal-data#object_tables) for each file."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 154,
|
|
"referenced_widgets": [
|
|
"805b7cdd6292460b849297f22d2a049b",
|
|
"4f99d69cf78d480abe044073bece8e3c",
|
|
"0d0827342e45461bbcd954ed8a4496eb",
|
|
"23388c42f6e24c85bbd993139c71d53b",
|
|
"199b650704f64310a2e954b03340d4bb",
|
|
"c38322312b6349b68c6e55fa1427c590",
|
|
"eafb59d7bf99480a9de7902fb2f5b020",
|
|
"c6ba2126765c44f6b9fc5c599f508a02",
|
|
"a7aa3d31bb5f475db331d2a0f47bf1b6",
|
|
"490d2c50a16348c59d667dcdd733b058",
|
|
"4eddcc005af241f19ee910296bc16217"
|
|
]
|
|
},
|
|
"id": "stlUACEaLx73",
|
|
"outputId": "90ee7882-6975-49db-9950-4aaa0485592f"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "805b7cdd6292460b849297f22d2a049b",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 0,\n \"fields\": []\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [],\n columns: [[\"number\", \"index\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-a5d40930-888c-4a26-8f7d-093d4a4a47d3\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a5d40930-888c-4a26-8f7d-093d4a4a47d3')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-a5d40930-888c-4a26-8f7d-093d4a4a47d3 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-ea419da9-a9f7-4136-993e-16e79086ff23\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ea419da9-a9f7-4136-993e-16e79086ff23')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-ea419da9-a9f7-4136-993e-16e79086ff23 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-ea419da9-a9f7-4136-993e-16e79086ff23');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
"Empty DataFrame\n",
|
|
"Columns: []\n",
|
|
"Index: []"
|
|
]
|
|
},
|
|
"execution_count": 29,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"CREATE OR REPLACE EXTERNAL TABLE `bq_mm_tutorial.object_table`\n",
|
|
"WITH CONNECTION `us.test_connection`\n",
|
|
"OPTIONS (\n",
|
|
" object_metadata = 'SIMPLE',\n",
|
|
" uris = ['gs://sample-data-and-media/customer-support/calls/*.mp3']\n",
|
|
");"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "ybC6djJipNoV"
|
|
},
|
|
"source": [
|
|
"Let's check the table contents. Notice the `ref` column, which is an `ObjectRef` that we can use."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 471,
|
|
"referenced_widgets": [
|
|
"1bca622aa4ca4b9998751522fa0eeb16",
|
|
"83b86260252b42ee9acaf67b76eb1b18",
|
|
"22bbe4e1883840248af78f020fb1b12f",
|
|
"57c3e0e7cef944438a54edae5042a801",
|
|
"8798189b79c541008e96c9d4e343d98d",
|
|
"11439efc9f114571979f55a797de0246",
|
|
"28136705f7cb4fae983962fee6e11f85",
|
|
"06f75175ed854d11bcdd2942901e96fc",
|
|
"fdc57743864d4be5ab270ff8a41f54b1",
|
|
"d04d3060f6b8465e93ede0a2bb86e680",
|
|
"851c27b84a4b4fe28d6984a0df223043",
|
|
"65f74e6fb5274cbf8d0da8c8d8d9f7a1",
|
|
"15da5dbcba9049a3bf3429484e2ff861",
|
|
"16b9dfdff73e47c58e06b2ddd2d9bdb9",
|
|
"d5227fb6f9a54e75b6f3e7e00146f0ed",
|
|
"c418f8417d6740b9b2859a38185db8d8",
|
|
"9e96f19df9e84cdea61774119614257f",
|
|
"9fbf11468b694b29bf7b7e1ae4ef78f9",
|
|
"fbf8cbe3140c4dd588261b6203560030",
|
|
"2e360f2805884a9d96121412920e62cc",
|
|
"9db849c8e85d416bb2508612148c6550",
|
|
"d5c25f3cbacb4ef187ce6da624959245"
|
|
]
|
|
},
|
|
"id": "GD-DhrXiKT0p",
|
|
"outputId": "dd044ab1-9ed0-457c-da93-273df96ab10a"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "1bca622aa4ca4b9998751522fa0eeb16",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "65f74e6fb5274cbf8d0da8c8d8d9f7a1",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"uri\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"gs://sample-data-and-media/customer-support/calls/call_2_20250411045538.mp3\",\n \"gs://sample-data-and-media/customer-support/calls/call_5_20250411045547.mp3\",\n \"gs://sample-data-and-media/customer-support/calls/call_3_20250411045541.mp3\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"generation\",\n \"properties\": {\n \"dtype\": \"Int64\",\n \"num_unique_values\": 5,\n \"samples\": [\n 1753936794657505,\n 1753936795493499,\n 1753936794931315\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"content_type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"audio/mpeg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"size\",\n \"properties\": {\n \"dtype\": \"Int64\",\n \"num_unique_values\": 5,\n \"samples\": [\n 65376\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"md5_hash\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"c8c575ce7cf437c840c28cbb84174e43\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"updated\",\n \"properties\": {\n \"dtype\": \"date\",\n \"min\": \"2025-07-31 04:39:54.327000+00:00\",\n \"max\": \"2025-07-31 04:39:55.523000+00:00\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"2025-07-31 04:39:54.689000+00:00\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metadata\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ref\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"gs://sample-data-and-media/customer-support/calls/call_1_20250411045535.mp3\",\n{\n 'v': 1753936794296133,\n 'f': \"1753936794296133\",\n },\n\"audio/mpeg\",\n{\n 'v': 65952,\n 'f': \"65952\",\n },\n\"1e99258238c0830c9a59a1a45d5dd095\",\n\"2025-07-31 04:39:54.327000+00:00\",\n\"[]\",\n\"{'uri': 'gs://sample-data-and-media/customer-support/calls/call_1_20250411045535.mp3', 'version': '1753936794296133', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"1e99258238c0830c9a59a1a45d5dd095\\\",\\\"size\\\":65952,\\\"updated\\\":1753936794327000}}'}\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"gs://sample-data-and-media/customer-support/calls/call_2_20250411045538.mp3\",\n{\n 'v': 1753936794657505,\n 'f': \"1753936794657505\",\n },\n\"audio/mpeg\",\n{\n 'v': 65376,\n 'f': \"65376\",\n },\n\"c8c575ce7cf437c840c28cbb84174e43\",\n\"2025-07-31 04:39:54.689000+00:00\",\n\"[]\",\n\"{'uri': 'gs://sample-data-and-media/customer-support/calls/call_2_20250411045538.mp3', 'version': '1753936794657505', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"c8c575ce7cf437c840c28cbb84174e43\\\",\\\"size\\\":65376,\\\"updated\\\":1753936794689000}}'}\"],\n [{\n 'v': 2,\n 'f': \"2\",\n },\n\"gs://sample-data-and-media/customer-support/calls/call_3_20250411045541.mp3\",\n{\n 'v': 1753936794931315,\n 'f': \"1753936794931315\",\n },\n\"audio/mpeg\",\n{\n 'v': 75744,\n 'f': \"75744\",\n },\n\"de6342b8cea147fdba8a3e3d1edcdeee\",\n\"2025-07-31 04:39:54.963000+00:00\",\n\"[]\",\n\"{'uri': 'gs://sample-data-and-media/customer-support/calls/call_3_20250411045541.mp3', 'version': '1753936794931315', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"de6342b8cea147fdba8a3e3d1edcdeee\\\",\\\"size\\\":75744,\\\"updated\\\":1753936794963000}}'}\"],\n [{\n 'v': 3,\n 'f': \"3\",\n },\n\"gs://sample-data-and-media/customer-support/calls/call_4_20250411045544.mp3\",\n{\n 'v': 1753936795174045,\n 'f': \"1753936795174045\",\n },\n\"audio/mpeg\",\n{\n 'v': 65184,\n 'f': \"65184\",\n },\n\"960ca9e91bca3335b18c2ec1e37c84f2\",\n\"2025-07-31 04:39:55.204000+00:00\",\n\"[]\",\n\"{'uri': 'gs://sample-data-and-media/customer-support/calls/call_4_20250411045544.mp3', 'version': '1753936795174045', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"960ca9e91bca3335b18c2ec1e37c84f2\\\",\\\"size\\\":65184,\\\"updated\\\":1753936795204000}}'}\"],\n [{\n 'v': 4,\n 'f': \"4\",\n },\n\"gs://sample-data-and-media/customer-support/calls/call_5_20250411045547.mp3\",\n{\n 'v': 1753936795493499,\n 'f': \"1753936795493499\",\n },\n\"audio/mpeg\",\n{\n 'v': 63552,\n 'f': \"63552\",\n },\n\"4b846ecfda4b7e57761b55ed4b8b1e2b\",\n\"2025-07-31 04:39:55.523000+00:00\",\n\"[]\",\n\"{'uri': 'gs://sample-data-and-media/customer-support/calls/call_5_20250411045547.mp3', 'version': '1753936795493499', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"4b846ecfda4b7e57761b55ed4b8b1e2b\\\",\\\"size\\\":63552,\\\"updated\\\":1753936795523000}}'}\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"uri\"], [\"number\", \"generation\"], [\"string\", \"content_type\"], [\"number\", \"size\"], [\"string\", \"md5_hash\"], [\"string\", \"updated\"], [\"string\", \"metadata\"], [\"string\", \"ref\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-2bc93c82-fbab-4a1c-a70a-31b5742f014e\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-2bc93c82-fbab-4a1c-a70a-31b5742f014e')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-2bc93c82-fbab-4a1c-a70a-31b5742f014e button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-9f58cc7f-b3e7-4376-8f9f-f074a9357941\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>uri</th>\n",
|
|
" <th>generation</th>\n",
|
|
" <th>content_type</th>\n",
|
|
" <th>size</th>\n",
|
|
" <th>md5_hash</th>\n",
|
|
" <th>updated</th>\n",
|
|
" <th>metadata</th>\n",
|
|
" <th>ref</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>gs://sample-data-and-media/customer-support/ca...</td>\n",
|
|
" <td>1753936794296133</td>\n",
|
|
" <td>audio/mpeg</td>\n",
|
|
" <td>65952</td>\n",
|
|
" <td>1e99258238c0830c9a59a1a45d5dd095</td>\n",
|
|
" <td>2025-07-31 04:39:54.327000+00:00</td>\n",
|
|
" <td>[]</td>\n",
|
|
" <td>{'uri': 'gs://sample-data-and-media/customer-s...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>gs://sample-data-and-media/customer-support/ca...</td>\n",
|
|
" <td>1753936794657505</td>\n",
|
|
" <td>audio/mpeg</td>\n",
|
|
" <td>65376</td>\n",
|
|
" <td>c8c575ce7cf437c840c28cbb84174e43</td>\n",
|
|
" <td>2025-07-31 04:39:54.689000+00:00</td>\n",
|
|
" <td>[]</td>\n",
|
|
" <td>{'uri': 'gs://sample-data-and-media/customer-s...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>gs://sample-data-and-media/customer-support/ca...</td>\n",
|
|
" <td>1753936794931315</td>\n",
|
|
" <td>audio/mpeg</td>\n",
|
|
" <td>75744</td>\n",
|
|
" <td>de6342b8cea147fdba8a3e3d1edcdeee</td>\n",
|
|
" <td>2025-07-31 04:39:54.963000+00:00</td>\n",
|
|
" <td>[]</td>\n",
|
|
" <td>{'uri': 'gs://sample-data-and-media/customer-s...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>gs://sample-data-and-media/customer-support/ca...</td>\n",
|
|
" <td>1753936795174045</td>\n",
|
|
" <td>audio/mpeg</td>\n",
|
|
" <td>65184</td>\n",
|
|
" <td>960ca9e91bca3335b18c2ec1e37c84f2</td>\n",
|
|
" <td>2025-07-31 04:39:55.204000+00:00</td>\n",
|
|
" <td>[]</td>\n",
|
|
" <td>{'uri': 'gs://sample-data-and-media/customer-s...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>gs://sample-data-and-media/customer-support/ca...</td>\n",
|
|
" <td>1753936795493499</td>\n",
|
|
" <td>audio/mpeg</td>\n",
|
|
" <td>63552</td>\n",
|
|
" <td>4b846ecfda4b7e57761b55ed4b8b1e2b</td>\n",
|
|
" <td>2025-07-31 04:39:55.523000+00:00</td>\n",
|
|
" <td>[]</td>\n",
|
|
" <td>{'uri': 'gs://sample-data-and-media/customer-s...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9f58cc7f-b3e7-4376-8f9f-f074a9357941')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-9f58cc7f-b3e7-4376-8f9f-f074a9357941 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-9f58cc7f-b3e7-4376-8f9f-f074a9357941');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" <div id=\"df-4f05ab13-862a-4805-865e-bfcfe2f6a2d1\">\n",
|
|
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4f05ab13-862a-4805-865e-bfcfe2f6a2d1')\"\n",
|
|
" title=\"Suggest charts\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
|
|
" width=\"24px\">\n",
|
|
" <g>\n",
|
|
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
|
|
" </g>\n",
|
|
"</svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
"<style>\n",
|
|
" .colab-df-quickchart {\n",
|
|
" --bg-color: #E8F0FE;\n",
|
|
" --fill-color: #1967D2;\n",
|
|
" --hover-bg-color: #E2EBFA;\n",
|
|
" --hover-fill-color: #174EA6;\n",
|
|
" --disabled-fill-color: #AAA;\n",
|
|
" --disabled-bg-color: #DDD;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-quickchart {\n",
|
|
" --bg-color: #3B4455;\n",
|
|
" --fill-color: #D2E3FC;\n",
|
|
" --hover-bg-color: #434B5C;\n",
|
|
" --hover-fill-color: #FFFFFF;\n",
|
|
" --disabled-bg-color: #3B4455;\n",
|
|
" --disabled-fill-color: #666;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart {\n",
|
|
" background-color: var(--bg-color);\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: var(--fill-color);\n",
|
|
" height: 32px;\n",
|
|
" padding: 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart:hover {\n",
|
|
" background-color: var(--hover-bg-color);\n",
|
|
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: var(--button-hover-fill-color);\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart-complete:disabled,\n",
|
|
" .colab-df-quickchart-complete:disabled:hover {\n",
|
|
" background-color: var(--disabled-bg-color);\n",
|
|
" fill: var(--disabled-fill-color);\n",
|
|
" box-shadow: none;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-spinner {\n",
|
|
" border: 2px solid var(--fill-color);\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" animation:\n",
|
|
" spin 1s steps(1) infinite;\n",
|
|
" }\n",
|
|
"\n",
|
|
" @keyframes spin {\n",
|
|
" 0% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 20% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 30% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 40% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 60% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 80% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 90% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" async function quickchart(key) {\n",
|
|
" const quickchartButtonEl =\n",
|
|
" document.querySelector('#' + key + ' button');\n",
|
|
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
|
|
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
|
|
" try {\n",
|
|
" const charts = await google.colab.kernel.invokeFunction(\n",
|
|
" 'suggestCharts', [key], {});\n",
|
|
" } catch (error) {\n",
|
|
" console.error('Error during call to suggestCharts:', error);\n",
|
|
" }\n",
|
|
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
|
|
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
|
|
" }\n",
|
|
" (() => {\n",
|
|
" let quickchartButtonEl =\n",
|
|
" document.querySelector('#df-4f05ab13-862a-4805-865e-bfcfe2f6a2d1 button');\n",
|
|
" quickchartButtonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
" })();\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" uri generation \\\n",
|
|
"0 gs://sample-data-and-media/customer-support/ca... 1753936794296133 \n",
|
|
"1 gs://sample-data-and-media/customer-support/ca... 1753936794657505 \n",
|
|
"2 gs://sample-data-and-media/customer-support/ca... 1753936794931315 \n",
|
|
"3 gs://sample-data-and-media/customer-support/ca... 1753936795174045 \n",
|
|
"4 gs://sample-data-and-media/customer-support/ca... 1753936795493499 \n",
|
|
"\n",
|
|
" content_type size md5_hash \\\n",
|
|
"0 audio/mpeg 65952 1e99258238c0830c9a59a1a45d5dd095 \n",
|
|
"1 audio/mpeg 65376 c8c575ce7cf437c840c28cbb84174e43 \n",
|
|
"2 audio/mpeg 75744 de6342b8cea147fdba8a3e3d1edcdeee \n",
|
|
"3 audio/mpeg 65184 960ca9e91bca3335b18c2ec1e37c84f2 \n",
|
|
"4 audio/mpeg 63552 4b846ecfda4b7e57761b55ed4b8b1e2b \n",
|
|
"\n",
|
|
" updated metadata \\\n",
|
|
"0 2025-07-31 04:39:54.327000+00:00 [] \n",
|
|
"1 2025-07-31 04:39:54.689000+00:00 [] \n",
|
|
"2 2025-07-31 04:39:54.963000+00:00 [] \n",
|
|
"3 2025-07-31 04:39:55.204000+00:00 [] \n",
|
|
"4 2025-07-31 04:39:55.523000+00:00 [] \n",
|
|
"\n",
|
|
" ref \n",
|
|
"0 {'uri': 'gs://sample-data-and-media/customer-s... \n",
|
|
"1 {'uri': 'gs://sample-data-and-media/customer-s... \n",
|
|
"2 {'uri': 'gs://sample-data-and-media/customer-s... \n",
|
|
"3 {'uri': 'gs://sample-data-and-media/customer-s... \n",
|
|
"4 {'uri': 'gs://sample-data-and-media/customer-s... "
|
|
]
|
|
},
|
|
"execution_count": 30,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT * FROM `bq_mm_tutorial.object_table`;"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "odlYhyPmpToE"
|
|
},
|
|
"source": [
|
|
"### 3. Create a multimodal table\n",
|
|
"\n",
|
|
"We'll first join the `calls` table that contains structured attributes to the `object_table` table, which contains the `ObjectRef` column, `ref`. We'll refer to this as a \"multimodal table\" because it contains structured and unstructured fields."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 154,
|
|
"referenced_widgets": [
|
|
"8ede8c802af8468fb5c3be63f42ff3e1",
|
|
"ccc6ce2603e84141ac601b0d5c49a380",
|
|
"2ceec39a87b547188784d7b871ff9564",
|
|
"9e8609071a1c408b8d3f590233e54ccf",
|
|
"9e5ffe49d77d4b4f9aa791d6ae71b0c8",
|
|
"c47579c600474f0bab1d89cc29bfe6c2",
|
|
"69f9961815944587b9aa12ed08f191c2",
|
|
"42b50ebce8954111b3980eef454d8fe8",
|
|
"c29db691f5124d6ca9e7ab6c85375565",
|
|
"cde40bfcf9964bb3b4814b43f70294ba",
|
|
"1cd3e783ca604bb894fb09c21fb80b05"
|
|
]
|
|
},
|
|
"id": "wPepSxh2KTs1",
|
|
"outputId": "ea086dcf-2e95-4972-ce81-417ab5304d7f"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "8ede8c802af8468fb5c3be63f42ff3e1",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 0,\n \"fields\": []\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [],\n columns: [[\"number\", \"index\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-32450056-f978-4053-a877-9343322c6af3\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-32450056-f978-4053-a877-9343322c6af3')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-32450056-f978-4053-a877-9343322c6af3 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-e827b8a5-e5b5-48ac-903c-58c676932921\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e827b8a5-e5b5-48ac-903c-58c676932921')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-e827b8a5-e5b5-48ac-903c-58c676932921 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-e827b8a5-e5b5-48ac-903c-58c676932921');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
"Empty DataFrame\n",
|
|
"Columns: []\n",
|
|
"Index: []"
|
|
]
|
|
},
|
|
"execution_count": 31,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"CREATE OR REPLACE TABLE `bq_mm_tutorial.calls_combined` AS\n",
|
|
"SELECT\n",
|
|
" c.*,\n",
|
|
" o.ref\n",
|
|
"FROM\n",
|
|
" `bq_mm_tutorial.calls` AS c\n",
|
|
"LEFT JOIN\n",
|
|
" `bq_mm_tutorial.object_table` AS o\n",
|
|
"ON\n",
|
|
" c.call_id = REGEXP_EXTRACT(o.uri, r'calls/([^.]+)')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "Q2qhFXw1pqpi"
|
|
},
|
|
"source": [
|
|
"We now have a single table, `calls_combined` that contains both traditional structured data like `company_revenue` and `product_name` alongside a reference to unstructured customer support calls.\n",
|
|
"\n",
|
|
"We can run a single query against both data types."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "IeP5EoJZt05z"
|
|
},
|
|
"source": [
|
|
"### 4. Run a [multimodal query](https://cloud.google.com/bigquery/docs/analyze-multimodal-data#generative_ai_functions)\n",
|
|
"\n",
|
|
"Since `calls_combined` contains structured data and pointers to audio in GCS, we can filter the table on both conditions.\n",
|
|
"\n",
|
|
"In this case, we'll look for \"high value\" companies with audio indicating they're looking to purchase something USING [`AI.GENERATE_BOOL`](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-ai-generate-bool)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 229,
|
|
"referenced_widgets": [
|
|
"686da3cfdc4048e9acf5831fa5a2aad8",
|
|
"28342d3501f04f12bb63a12ab74acacf",
|
|
"034d032bedd14f3aba2446f6742af293",
|
|
"624dee9d1e3b491bb039bbbe59b304e8",
|
|
"2abe7e6721014b32a886ac517e67e929",
|
|
"1c59d05c23f7442dbb54457624f3d17a",
|
|
"63ed9bea712c4f3fab1cbe94bf409bd1",
|
|
"fab215c996c44201a1f62ea3f3b90642",
|
|
"b19ba465cbdc45259320be3c80b44e83",
|
|
"36a7136ce005462097fcdee4621456c8",
|
|
"460bace6250d463fa779b364bb84c210",
|
|
"ef01b1a0839a4fd2ac76c910fea8fb76",
|
|
"bc95da919c2e40b2a5d1397c3ccba0b4",
|
|
"d8599a119a244cefbd8fffb0eaded349",
|
|
"97036a798f164c30a8d3e6fa6da1807c",
|
|
"9286b683ed67409798cb40a279b1c41f",
|
|
"95d9aa5a25814f26bad5d4f806e401e8",
|
|
"d0b84a1a224a46f29cf238d1060ae0c6",
|
|
"75ac6046214b4066a73724ea897c7a23",
|
|
"ffd831e5406646c99916defe41d5f887",
|
|
"3066e0f0e70c4ec8b41def853d25c5fe",
|
|
"e0032930338440f4a01ddf44f45e791e"
|
|
]
|
|
},
|
|
"id": "QxrWwWiGtfud",
|
|
"outputId": "e30569ff-f8f5-4232-cfe6-28846d14be2c"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "686da3cfdc4048e9acf5831fa5a2aad8",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "ef01b1a0839a4fd2ac76c910fea8fb76",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 2,\n \"fields\": [\n {\n \"column\": \"company_name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"Synergy Corp\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"customer_name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"Emily Carter\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"product_name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"Synergy Project Manager\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"Synergy Corp\",\n\"David Martin\",\n\"Synergy Cloud Storage\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"Synergy Corp\",\n\"Emily Carter\",\n\"Synergy Project Manager\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"company_name\"], [\"string\", \"customer_name\"], [\"string\", \"product_name\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-d6652b0c-e1b5-4a9a-aac1-9d0b5cbe32b6\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d6652b0c-e1b5-4a9a-aac1-9d0b5cbe32b6')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-d6652b0c-e1b5-4a9a-aac1-9d0b5cbe32b6 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-548b675b-b2d8-450b-a0c5-113d28e600c8\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>company_name</th>\n",
|
|
" <th>customer_name</th>\n",
|
|
" <th>product_name</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>Synergy Corp</td>\n",
|
|
" <td>David Martin</td>\n",
|
|
" <td>Synergy Cloud Storage</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>Synergy Corp</td>\n",
|
|
" <td>Emily Carter</td>\n",
|
|
" <td>Synergy Project Manager</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-548b675b-b2d8-450b-a0c5-113d28e600c8')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-548b675b-b2d8-450b-a0c5-113d28e600c8 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-548b675b-b2d8-450b-a0c5-113d28e600c8');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" <div id=\"df-d4eb4d04-09fd-4d5d-9584-7ff14bf9fbfa\">\n",
|
|
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d4eb4d04-09fd-4d5d-9584-7ff14bf9fbfa')\"\n",
|
|
" title=\"Suggest charts\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
|
|
" width=\"24px\">\n",
|
|
" <g>\n",
|
|
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
|
|
" </g>\n",
|
|
"</svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
"<style>\n",
|
|
" .colab-df-quickchart {\n",
|
|
" --bg-color: #E8F0FE;\n",
|
|
" --fill-color: #1967D2;\n",
|
|
" --hover-bg-color: #E2EBFA;\n",
|
|
" --hover-fill-color: #174EA6;\n",
|
|
" --disabled-fill-color: #AAA;\n",
|
|
" --disabled-bg-color: #DDD;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-quickchart {\n",
|
|
" --bg-color: #3B4455;\n",
|
|
" --fill-color: #D2E3FC;\n",
|
|
" --hover-bg-color: #434B5C;\n",
|
|
" --hover-fill-color: #FFFFFF;\n",
|
|
" --disabled-bg-color: #3B4455;\n",
|
|
" --disabled-fill-color: #666;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart {\n",
|
|
" background-color: var(--bg-color);\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: var(--fill-color);\n",
|
|
" height: 32px;\n",
|
|
" padding: 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart:hover {\n",
|
|
" background-color: var(--hover-bg-color);\n",
|
|
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: var(--button-hover-fill-color);\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart-complete:disabled,\n",
|
|
" .colab-df-quickchart-complete:disabled:hover {\n",
|
|
" background-color: var(--disabled-bg-color);\n",
|
|
" fill: var(--disabled-fill-color);\n",
|
|
" box-shadow: none;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-spinner {\n",
|
|
" border: 2px solid var(--fill-color);\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" animation:\n",
|
|
" spin 1s steps(1) infinite;\n",
|
|
" }\n",
|
|
"\n",
|
|
" @keyframes spin {\n",
|
|
" 0% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 20% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 30% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 40% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 60% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 80% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 90% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" async function quickchart(key) {\n",
|
|
" const quickchartButtonEl =\n",
|
|
" document.querySelector('#' + key + ' button');\n",
|
|
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
|
|
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
|
|
" try {\n",
|
|
" const charts = await google.colab.kernel.invokeFunction(\n",
|
|
" 'suggestCharts', [key], {});\n",
|
|
" } catch (error) {\n",
|
|
" console.error('Error during call to suggestCharts:', error);\n",
|
|
" }\n",
|
|
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
|
|
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
|
|
" }\n",
|
|
" (() => {\n",
|
|
" let quickchartButtonEl =\n",
|
|
" document.querySelector('#df-d4eb4d04-09fd-4d5d-9584-7ff14bf9fbfa button');\n",
|
|
" quickchartButtonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
" })();\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" company_name customer_name product_name\n",
|
|
"0 Synergy Corp David Martin Synergy Cloud Storage\n",
|
|
"1 Synergy Corp Emily Carter Synergy Project Manager"
|
|
]
|
|
},
|
|
"execution_count": 35,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT company_name, customer_name, product_name\n",
|
|
"FROM `bq_mm_tutorial.calls_combined`\n",
|
|
"WHERE company_revenue > 15000000\n",
|
|
"AND\n",
|
|
" AI.GENERATE_BOOL(\n",
|
|
" prompt => (\"Wants to buy something\", ref),\n",
|
|
" connection_id => \"us.test_connection\").result\n",
|
|
";"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "qslXv2wXt2zs"
|
|
},
|
|
"source": [
|
|
"### 5. Run a multimodal query on an ARRAY of `ObjectRef`s\n",
|
|
"\n",
|
|
"Because an `ObjectRef` is a `STRUCT` data type, it can also be nested as an array and passed to Gemini for inference.\n",
|
|
"\n",
|
|
"In this example, we generate high-level themes customers are calling about, grouped by `company_name` using [`AI.GENERATE`](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-ai-generate)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 245,
|
|
"referenced_widgets": [
|
|
"5cad28074be84d29a4cde869bec779a2",
|
|
"65f107bf6c4948f0bded28f268f99308",
|
|
"343d1a19bc3c427e80f1c5af008cc88c",
|
|
"8f67587019714bfea93695c847f36a53",
|
|
"2198e1de24dd4ceabcbe3517adeba7ac",
|
|
"880a9a5982c94b62b9e460539b605883",
|
|
"1dd44c53f6b5478bb266f2ed2b20b4da",
|
|
"4ae1c2064441402eb2f5a561afcb8f5e",
|
|
"3c0f0169c5484d29879ec229e605ac17",
|
|
"ad003a8f494d48c29bf1b27d163fdd1d",
|
|
"c85a1f6b529f4b4f811d2e82a1f14d60",
|
|
"20df373ee81c4a5cbcea085b982679fd",
|
|
"f17e1b9f1fd34a9186cf7d5228519014",
|
|
"061ed8babda04387bff0bf6c829d565b",
|
|
"3fbabbc07fee4ae49ba768d98d2bd8ab",
|
|
"1773cfd4048f4e54ae50161292e7bd30",
|
|
"b6c900858dff4fe8bdd2e32589adabbd",
|
|
"708dc79209a6490ca8c3a6abef3d445a",
|
|
"b9d86186597d47a99cf1a938a107255e",
|
|
"77bb4dbef9944074ae456bc597136aed",
|
|
"d86f828be9ac4646866ccf66f28e6bcb",
|
|
"581b246d5e6449679873a67a9cad9589"
|
|
]
|
|
},
|
|
"id": "DFmwwe1yuL6F",
|
|
"outputId": "ca69a3e2-fec0-47ac-83a2-47d1174889a5"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "5cad28074be84d29a4cde869bec779a2",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "20df373ee81c4a5cbcea085b982679fd",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 2,\n \"fields\": [\n {\n \"column\": \"company_name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"Global Tech Enterprises\",\n \"Synergy Corp\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"themes\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"Synergy Corp\",\n\"['Customer Service' 'Cloud Storage']\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"Global Tech Enterprises\",\n\"['VPN' 'Disconnect' 'Refund' 'Data Backup' 'Storage' 'Upgrade'\\n 'Website Issue' 'Antivirus' 'Activation' 'Pop-ups']\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"company_name\"], [\"string\", \"themes\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-68b202da-944a-4ac5-a088-d050d72f9c82\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-68b202da-944a-4ac5-a088-d050d72f9c82')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-68b202da-944a-4ac5-a088-d050d72f9c82 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-1cf5a78c-16dd-4a3c-9ac8-aa02b1935599\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>company_name</th>\n",
|
|
" <th>themes</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>Synergy Corp</td>\n",
|
|
" <td>[Customer Service, Cloud Storage]</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>Global Tech Enterprises</td>\n",
|
|
" <td>[VPN, Disconnect, Refund, Data Backup, Storage...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1cf5a78c-16dd-4a3c-9ac8-aa02b1935599')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-1cf5a78c-16dd-4a3c-9ac8-aa02b1935599 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-1cf5a78c-16dd-4a3c-9ac8-aa02b1935599');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" <div id=\"df-1c63627e-abd7-444e-bef3-05aa667557c9\">\n",
|
|
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-1c63627e-abd7-444e-bef3-05aa667557c9')\"\n",
|
|
" title=\"Suggest charts\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
|
|
" width=\"24px\">\n",
|
|
" <g>\n",
|
|
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
|
|
" </g>\n",
|
|
"</svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
"<style>\n",
|
|
" .colab-df-quickchart {\n",
|
|
" --bg-color: #E8F0FE;\n",
|
|
" --fill-color: #1967D2;\n",
|
|
" --hover-bg-color: #E2EBFA;\n",
|
|
" --hover-fill-color: #174EA6;\n",
|
|
" --disabled-fill-color: #AAA;\n",
|
|
" --disabled-bg-color: #DDD;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-quickchart {\n",
|
|
" --bg-color: #3B4455;\n",
|
|
" --fill-color: #D2E3FC;\n",
|
|
" --hover-bg-color: #434B5C;\n",
|
|
" --hover-fill-color: #FFFFFF;\n",
|
|
" --disabled-bg-color: #3B4455;\n",
|
|
" --disabled-fill-color: #666;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart {\n",
|
|
" background-color: var(--bg-color);\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: var(--fill-color);\n",
|
|
" height: 32px;\n",
|
|
" padding: 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart:hover {\n",
|
|
" background-color: var(--hover-bg-color);\n",
|
|
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: var(--button-hover-fill-color);\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart-complete:disabled,\n",
|
|
" .colab-df-quickchart-complete:disabled:hover {\n",
|
|
" background-color: var(--disabled-bg-color);\n",
|
|
" fill: var(--disabled-fill-color);\n",
|
|
" box-shadow: none;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-spinner {\n",
|
|
" border: 2px solid var(--fill-color);\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" animation:\n",
|
|
" spin 1s steps(1) infinite;\n",
|
|
" }\n",
|
|
"\n",
|
|
" @keyframes spin {\n",
|
|
" 0% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 20% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 30% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 40% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 60% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 80% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 90% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" async function quickchart(key) {\n",
|
|
" const quickchartButtonEl =\n",
|
|
" document.querySelector('#' + key + ' button');\n",
|
|
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
|
|
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
|
|
" try {\n",
|
|
" const charts = await google.colab.kernel.invokeFunction(\n",
|
|
" 'suggestCharts', [key], {});\n",
|
|
" } catch (error) {\n",
|
|
" console.error('Error during call to suggestCharts:', error);\n",
|
|
" }\n",
|
|
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
|
|
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
|
|
" }\n",
|
|
" (() => {\n",
|
|
" let quickchartButtonEl =\n",
|
|
" document.querySelector('#df-1c63627e-abd7-444e-bef3-05aa667557c9 button');\n",
|
|
" quickchartButtonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
" })();\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" company_name themes\n",
|
|
"0 Synergy Corp [Customer Service, Cloud Storage]\n",
|
|
"1 Global Tech Enterprises [VPN, Disconnect, Refund, Data Backup, Storage..."
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT\n",
|
|
" company_name,\n",
|
|
" AI.GENERATE(\n",
|
|
" ('Give 1-2 word themes customers are calling about', refs),\n",
|
|
" connection_id => 'us.test_connection',\n",
|
|
" endpoint => 'gemini-2.5-flash',\n",
|
|
" output_schema => 'themes ARRAY<STRING>').themes\n",
|
|
"FROM (SELECT company_name, ARRAY_AGG(ref) AS refs FROM `bq_mm_tutorial.calls_combined` GROUP BY company_name);\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "JxM1hI7X5RXU"
|
|
},
|
|
"source": [
|
|
"### Recap\n",
|
|
"\n",
|
|
"In this scenario we:\n",
|
|
"* Began with a table called `calls` containing structured data\n",
|
|
"* Created an object table over audio files in GCS. This contained an `ObjectRef` column\n",
|
|
"* Created a multimodal table called `calls_combined`\n",
|
|
"* Ran a query against the `calls_combined` table using structured *and* unstructured data in a `WHERE` clause\n",
|
|
"* Used an array of `ObjectRef`s to understand key themes by `company_name`\n",
|
|
"\n",
|
|
"This scenario used an object table to create our `ObjectRef` column. In the next scenario, we'll create it programmatically."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "v6ibKGHPuayE"
|
|
},
|
|
"source": [
|
|
"\n",
|
|
"\n",
|
|
"---\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "nmHS_hmFuckJ"
|
|
},
|
|
"source": [
|
|
"## Scenario 2: City 311 Response\n",
|
|
"\n",
|
|
"In this scenario, we'll work with a common public sector dataset: [311 reports](https://en.wikipedia.org/wiki/311_(telephone_number)). Citizens submit issues (like potholes or graffiti) that include text descriptions and often upload supporting media like photos, audio recordings, or videos.\n",
|
|
"\n",
|
|
"Our goal is to use multimodal analysis to automatically triage these reports, assess their urgency, and route them to the correct city department."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "R0jTjJfC-NfD"
|
|
},
|
|
"source": [
|
|
"### 1. Load structured data\n",
|
|
"\n",
|
|
"We'll first begin by adding two tables to our BigQuery environment:\n",
|
|
"* **`reports`** contains 311 report tabular data, like the `ticket_id`, `location`, `text_description` and more.\n",
|
|
"* **`media`** contains `uri` fields corresponding to any media associated with a 311 ticket (e.g. images, audio)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 154,
|
|
"referenced_widgets": [
|
|
"18ef8fc44f44409dbf8e8aa84c3f52ad",
|
|
"978e297b1e00476384c974d274b7e735",
|
|
"c5487ec63239476cbe0b4c2eafbca798",
|
|
"a4fa0932219b4ed2aedd1bfb2c30e913",
|
|
"f6a9ed3959a7477e8512e030ac80790d",
|
|
"f859be1f377549c2a2e5581677316a05",
|
|
"f5ce1137975c4abba2b14db8130ef1ed",
|
|
"5b0739ef14da4091b45bfb914c906df4",
|
|
"0848d04abdce4976a4a138ebedf0289b",
|
|
"b18bdf1e286144079cf4b24d2a6b3234",
|
|
"a73f21ae668f46be995e3703c6f1835b"
|
|
]
|
|
},
|
|
"id": "42xL93VL1Buh",
|
|
"outputId": "f1349304-03e9-434a-9dd9-b840a01e11b2"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "18ef8fc44f44409dbf8e8aa84c3f52ad",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 0,\n \"fields\": []\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [],\n columns: [[\"number\", \"index\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-92eac144-6885-475c-a2d6-cee9799d5d71\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-92eac144-6885-475c-a2d6-cee9799d5d71')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-92eac144-6885-475c-a2d6-cee9799d5d71 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-76de3751-620f-44e5-be54-17268085a3b5\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-76de3751-620f-44e5-be54-17268085a3b5')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-76de3751-620f-44e5-be54-17268085a3b5 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-76de3751-620f-44e5-be54-17268085a3b5');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
"Empty DataFrame\n",
|
|
"Columns: []\n",
|
|
"Index: []"
|
|
]
|
|
},
|
|
"execution_count": 37,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"LOAD DATA OVERWRITE bq_mm_tutorial.reports\n",
|
|
"FROM FILES (\n",
|
|
" uris = ['gs://sample-data-and-media/311-demo/tables/city_311_reports'],\n",
|
|
" format = 'PARQUET'\n",
|
|
");\n",
|
|
"\n",
|
|
"LOAD DATA OVERWRITE bq_mm_tutorial.media\n",
|
|
"FROM FILES (\n",
|
|
" uris = ['gs://sample-data-and-media/311-demo/tables/city_311_media'],\n",
|
|
" format = 'PARQUET'\n",
|
|
");"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "YIE0ZeOzukua"
|
|
},
|
|
"source": [
|
|
"### 2. Programmatically create an `ObjectRef`\n",
|
|
"\n",
|
|
"In **Scenario 1**, you learned to create an `ObjectRef` using [object tables](https://cloud.google.com/bigquery/docs/analyze-multimodal-data#object_tables).\n",
|
|
"\n",
|
|
"In this scenario, you'll use a more dynamic workflow, where you create `ObjectRef`s using a set of built-in SQL functions.\n",
|
|
"\n",
|
|
"* [`OBJ.MAKE_REF`](https://cloud.google.com/bigquery/docs/reference/standard-sql/objectref_functions#objmake_ref): This function takes the string from the `uri` column and converts it into an `ObjectRef`\n",
|
|
"\n",
|
|
"* [`OBJ.FETCH_METADATA`](https://cloud.google.com/bigquery/docs/reference/standard-sql/objectref_functions#objfetch_metadata) populates the `ObjectRef` with important file metadata from GCS\n",
|
|
"\n",
|
|
"Here's an example with `OBJ.MAKE_REF` that references a single object with a connection:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 207,
|
|
"referenced_widgets": [
|
|
"65a79b101e774d3ebc70b11595c56b17",
|
|
"c79d4b6e13294795a7586e14a63255ea",
|
|
"0e2cf603458e486db98de941df3af34b",
|
|
"53ffd2a507dd4cfe8626c72736c6b35f",
|
|
"6e0eae13e8f9405dab2142a28b4143a7",
|
|
"7c33beb9ba2d4b4399db4ef2457a6e3b",
|
|
"68b06a2c95314dd7bbee7497a455ea4f",
|
|
"e7f0a26b0f5d4f78bd62c8af876e1b22",
|
|
"860d1619667a4532971f5430d811b827",
|
|
"9fd692dadc6d435da2698afb073758c4",
|
|
"47b24af3837f4ec89e5b608b5d6c58b9",
|
|
"e553f3c326104035916463e9dd94fbf2",
|
|
"ebaa2ccf7dcc47c88b80d3448918a25e",
|
|
"57ca18529ebc49d09c7fff98326816b6",
|
|
"ef974197c9864aad82d0ad78f0388d6a",
|
|
"86990d96210f4d5eb9dadcffc6eb8c0c",
|
|
"5389deee286c474f865a6f47ab995bd2",
|
|
"3bf11f2792c744f69f47e753337a52f7",
|
|
"d35cf9b20cb343a0b89299d71d586e67",
|
|
"7fff81ab90cf45798ab3d9fbe21e88e8",
|
|
"79411d421c014f4eb23dac0084a5c2b0",
|
|
"ba29c683005e4eeb9f50dacbf1983a6d"
|
|
]
|
|
},
|
|
"id": "QHdnc7hcu0k2",
|
|
"outputId": "c019c905-0fee-4f1e-ee39-5636aa33af3c"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "65a79b101e774d3ebc70b11595c56b17",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "e553f3c326104035916463e9dd94fbf2",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 1,\n \"fields\": [\n {\n \"column\": \"image_ref\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"{'uri': 'gs://sample-data-and-media/311-demo/images/311-20250409-1515EF.png', 'version': None, 'authorizer': 'data-demo-n25.us.test_connection', 'details': None}\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"image_ref\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-b2dc4353-7713-4db4-994c-5e1b97e8fe68\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b2dc4353-7713-4db4-994c-5e1b97e8fe68')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-b2dc4353-7713-4db4-994c-5e1b97e8fe68 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-fd8decdb-3400-46a0-b3b5-e38c7432e8d1\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>image_ref</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>{'uri': 'gs://sample-data-and-media/311-demo/i...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fd8decdb-3400-46a0-b3b5-e38c7432e8d1')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-fd8decdb-3400-46a0-b3b5-e38c7432e8d1 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-fd8decdb-3400-46a0-b3b5-e38c7432e8d1');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" image_ref\n",
|
|
"0 {'uri': 'gs://sample-data-and-media/311-demo/i..."
|
|
]
|
|
},
|
|
"execution_count": 38,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT OBJ.MAKE_REF('gs://sample-data-and-media/311-demo/images/311-20250409-1515EF.png', 'us.test_connection') AS image_ref;"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "nTaOjfIb06TI"
|
|
},
|
|
"source": [
|
|
"We can then get [GCS metadata](https://cloud.google.com/storage/docs/metadata) for the object by wrapping the `OBJ.FETCH_METADATA` function around the prior query.\n",
|
|
"\n",
|
|
"This provides additional metadata, like a GCS object version, the object type, when it was last updated, and more."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 224,
|
|
"referenced_widgets": [
|
|
"ba71adb40d0c4f93ba4d24293c3837dc",
|
|
"b0d4be6ee7064916b807b54e53bd233d",
|
|
"89711047bae74967860f75e348f058a8",
|
|
"df00a456ccdf410fab7f06f3ad845c98",
|
|
"6341733d9b4b47629d4ea14b210dcf3b",
|
|
"561d86e8001549138dc5aa4b2cc1e6e9",
|
|
"00fd5605c58746c2a8b7eac315062e2a",
|
|
"280b9c31d968456fa8bf49962a2141fc",
|
|
"1934aa5c02e346b98d75aa8187f16880",
|
|
"9ef6d136f75a4a4e8038f264a2ee252d",
|
|
"05f76663c67140c584189ea0be14cb4c",
|
|
"0e43461dda8447e68bcac400a2e3b5f7",
|
|
"c2df60b0eac349a8b6a7fbf5ca6300ad",
|
|
"d260ec4e8df94523a0ed165f015cbb63",
|
|
"8ddb8b981e7c4f90804d295ef64227dd",
|
|
"a4a975511e4b45aa9239f9aae0373a54",
|
|
"59145f4a708448819dec5c4513efdee9",
|
|
"1f774daa4b3c4467bc9aaba52f1a0894",
|
|
"44ea649320774071b52bfac20931ade6",
|
|
"b5d63777f49745c7b82953000f4ca854",
|
|
"8600cf07284749fd86c915731d5a0ce5",
|
|
"b1d86680f91a44c4a2d3cbd38ff6c0fe"
|
|
]
|
|
},
|
|
"id": "wkolFoSXu0Xz",
|
|
"outputId": "323501b1-44f5-4930-eee0-aac6c55ec833"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "ba71adb40d0c4f93ba4d24293c3837dc",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "0e43461dda8447e68bcac400a2e3b5f7",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 1,\n \"fields\": [\n {\n \"column\": \"image_ref\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"{'uri': 'gs://sample-data-and-media/311-demo/images/311-20250409-1515EF.png', 'version': '1753891211822957', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"image/png\\\",\\\"md5_hash\\\":\\\"6b50c5266a530079e836bc32c43b5c0e\\\",\\\"size\\\":2217041,\\\"updated\\\":1753891211000000}}'}\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"image_ref\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-37f9f814-1f0b-41b5-8abb-ebcdb39285b2\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-37f9f814-1f0b-41b5-8abb-ebcdb39285b2')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-37f9f814-1f0b-41b5-8abb-ebcdb39285b2 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-b48bb0f1-f078-4060-a732-180d190c0b83\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>image_ref</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>{'uri': 'gs://sample-data-and-media/311-demo/i...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b48bb0f1-f078-4060-a732-180d190c0b83')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-b48bb0f1-f078-4060-a732-180d190c0b83 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-b48bb0f1-f078-4060-a732-180d190c0b83');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" image_ref\n",
|
|
"0 {'uri': 'gs://sample-data-and-media/311-demo/i..."
|
|
]
|
|
},
|
|
"execution_count": 39,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT OBJ.FETCH_METADATA(OBJ.MAKE_REF('gs://sample-data-and-media/311-demo/images/311-20250409-1515EF.png', 'us.test_connection')) AS image_ref;"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "UsymB7TvAliK"
|
|
},
|
|
"source": [
|
|
"### 3. Create a multimodal table\n",
|
|
"\n",
|
|
"To create our multimodal table, `reports_mm` we first join the `reports` and `media` table and also convert the `audio_uri`, `image_uri`, and `video_uri` fields to `ObjectRef` fields using the `OBJ.` functions.\n",
|
|
"\n",
|
|
"Note that a multimodal table can have multiple `ObjectRef` columns and you can alias these columns."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 867,
|
|
"referenced_widgets": [
|
|
"8906a505d0764a308693e43a6e74d0b1",
|
|
"33ff4ba94a4f49cbb41317168773bb63",
|
|
"54eadb7847784de28baee69c04ec2730",
|
|
"a93a9fa0561e43daa15fa9d0cc1dac47",
|
|
"d5595568695e4649b1f65bb113841520",
|
|
"46941be814dd4bc08ee831b686ecbebe",
|
|
"1a6a53a23c11474ea1490f044e00fc90",
|
|
"ca4104787ba94939b7e0a56383353b1a",
|
|
"d98a1e7bc53e4085b9f752520cb29d54",
|
|
"f3ec2581dfed4739a2970d6ef7ec8a16",
|
|
"049e79c247d04bb7b84a99436a380f70",
|
|
"cbfd84f13b024610b76d58edd9a4859d",
|
|
"adda6f62d05e4a019e085a6f88b45397",
|
|
"1aef208b7fcb4f98a5af24f35ca7b60f",
|
|
"c0be490ef6e940658e0c5481ebc696b4",
|
|
"cc4a0f778a3b4291b8e182d3128e93ce",
|
|
"a833cdda5d3b4d94852ef4f4bde7c9b3",
|
|
"6492cc2410b14a38a3144ad27c47baf4",
|
|
"ba5b6d8fbdce456cab1de49f916e9651",
|
|
"dc1b836b4d73400aa7a256d439fe8d8c",
|
|
"719b3da5b8a74cf58c5b8e8352d29b60",
|
|
"f47785db50134dfb96eb7532f8fe7ebc"
|
|
]
|
|
},
|
|
"id": "FbnyBYuRAmdP",
|
|
"outputId": "f165ee31-d734-4e8c-81c0-42102844fffb"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "8906a505d0764a308693e43a6e74d0b1",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "cbfd84f13b024610b76d58edd9a4859d",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"ticket_id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"311-20250409-A57DB4\",\n \"311-20250409-60290D\",\n \"311-20250409-59911E\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"timestamp\",\n \"properties\": {\n \"dtype\": \"date\",\n \"min\": \"2025-04-09 17:16:21+00:00\",\n \"max\": \"2025-04-09 17:35:21+00:00\",\n \"num_unique_values\": 4,\n \"samples\": [\n \"2025-04-09 17:29:21+00:00\",\n \"2025-04-09 17:16:21+00:00\",\n \"2025-04-09 17:28:21+00:00\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"reporter_id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Citizen-77E1\",\n \"Citizen-4B3B\",\n \"Citizen-69DD\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"location\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"POINT(-118.33 34.09)\",\n \"POINT(-118.35 34.055)\",\n \"POINT(-118.385 34.06)\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"location_description\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Leaking fire hydrant on Highland Ave near the movie theater west of Hollywood Blvd\",\n \"Large sinkhole on the sidewalk near apartment buildings on La Brea Ave south of Wilshire Blvd\",\n \"Car parked on the sidewalk on Beverly Hills Blvd near the cafe east of La Cienega Blvd\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"category_reported\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Leaking Fire Hydrant\",\n \"Major Sinkhole\",\n \"Abandoned Vehicle\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"description\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"A leaky fire hydrant that has been creating a puddle\",\n \"A huge sinkhole that looks like it will eat a car. FIX SOON\",\n \"Someone drove their car on the sidewalk and abandoned it. It needs to be towed. Its in the way of pedestrians!\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"status\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"Open\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"district\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"South\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"audio_ref\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"image_ref\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"video_ref\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"311-20250409-52F19E\",\n\"2025-04-09 17:28:21+00:00\",\n\"Citizen-4A2E\",\n\"POINT(-118.245 34.052)\",\n\"Large trash pile in the alley behind the building on West 2nd St between Main St and Hope St\",\n\"Illegal Dumping/Trash Pile\",\n\"There is a huge pile of trash in the alley and I cannot walk through it. It gets bigger every day.\",\n\"Open\",\n\"North\",\n\"{'uri': 'gs://n25-311-demo/calls/311-20250409-52F19E.mp3', 'version': '1744220287159290', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"4e19319d8ada9401fedd4dbfc471b435\\\",\\\"size\\\":48576,\\\"updated\\\":1744220287000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/images/311-20250409-52F19E.png', 'version': '1744223317884648', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"image/png\\\",\\\"md5_hash\\\":\\\"eb8eb813edda4f3faaa9331410577977\\\",\\\"size\\\":1692711,\\\"updated\\\":1744223317000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/videos/311-20250409-52F19E.mp4', 'version': '1744238753544507', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"video/mp4\\\",\\\"md5_hash\\\":\\\"eb537faabffdd3c198234ed47d25d0aa\\\",\\\"size\\\":4205301,\\\"updated\\\":1744238753000000}}'}\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"311-20250409-A57DB4\",\n\"2025-04-09 17:29:21+00:00\",\n\"Citizen-77E1\",\n\"POINT(-118.33 34.09)\",\n\"Leaking fire hydrant on Highland Ave near the movie theater west of Hollywood Blvd\",\n\"Leaking Fire Hydrant\",\n\"A leaky fire hydrant that has been creating a puddle\",\n\"Open\",\n\"North\",\n\"{'uri': 'gs://n25-311-demo/calls/311-20250409-A57DB4.mp3', 'version': '1744220289163179', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"d961d77277ede4c52a7fcbe53777fcd3\\\",\\\"size\\\":40032,\\\"updated\\\":1744220289000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/images/311-20250409-A57DB4.png', 'version': '1744239339712496', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"image/png\\\",\\\"md5_hash\\\":\\\"00a987cfa97a9b09879bdd0d373a95c8\\\",\\\"size\\\":1600367,\\\"updated\\\":1744239339000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/videos/311-20250409-A57DB4.mp4', 'version': '1744239317334568', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"video/mp4\\\",\\\"md5_hash\\\":\\\"f92f5ff69a6228c21dd3c18ff15a8f9a\\\",\\\"size\\\":1335816,\\\"updated\\\":1744239317000000}}'}\"],\n [{\n 'v': 2,\n 'f': \"2\",\n },\n\"311-20250409-59911E\",\n\"2025-04-09 17:28:21+00:00\",\n\"Citizen-69DD\",\n\"POINT(-118.385 34.06)\",\n\"Car parked on the sidewalk on Beverly Hills Blvd near the cafe east of La Cienega Blvd\",\n\"Abandoned Vehicle\",\n\"Someone drove their car on the sidewalk and abandoned it. It needs to be towed. Its in the way of pedestrians!\",\n\"Open\",\n\"North\",\n\"{'uri': 'gs://n25-311-demo/calls/311-20250409-59911E.mp3', 'version': '1744220291265142', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"cea743e87b873554e6faf55c89db2aad\\\",\\\"size\\\":41856,\\\"updated\\\":1744220291000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/images/311-20250409-59911E.png', 'version': '1744240096289336', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"image/png\\\",\\\"md5_hash\\\":\\\"0e53d8ffe485532436822e9c5e8159ce\\\",\\\"size\\\":1945756,\\\"updated\\\":1744240096000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/videos/311-20250409-59911E.mp4', 'version': '1744240085395689', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"video/mp4\\\",\\\"md5_hash\\\":\\\"147422cb23771f97b9fad4f4ef440227\\\",\\\"size\\\":1689108,\\\"updated\\\":1744240085000000}}'}\"],\n [{\n 'v': 3,\n 'f': \"3\",\n },\n\"311-20250409-1515EF\",\n\"2025-04-09 17:35:21+00:00\",\n\"Citizen-5DFC\",\n\"POINT(-118.341 34.063)\",\n\"Large pothole on Sunset Blvd near the bus stop east of La Brea Ave\",\n\"Pothole\",\n\"There is a small pothole near the bus stop that should be filled. It is a little bit of a hazard.\",\n\"Open\",\n\"South\",\n\"{'uri': 'gs://n25-311-demo/calls/311-20250409-1515EF.mp3', 'version': '1744220284822352', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"0e1e2551947e47dc603dea1b144b6b09\\\",\\\"size\\\":38592,\\\"updated\\\":1744220284000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/images/311-20250409-1515EF.png', 'version': '1744238597874740', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"image/png\\\",\\\"md5_hash\\\":\\\"6b50c5266a530079e836bc32c43b5c0e\\\",\\\"size\\\":2217041,\\\"updated\\\":1744238597000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/videos/311-20250409-1515EF.mp4', 'version': '1744238755038594', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"video/mp4\\\",\\\"md5_hash\\\":\\\"f190671c4c52dee225ecc89f6a8b97e5\\\",\\\"size\\\":8157657,\\\"updated\\\":1744238755000000}}'}\"],\n [{\n 'v': 4,\n 'f': \"4\",\n },\n\"311-20250409-60290D\",\n\"2025-04-09 17:16:21+00:00\",\n\"Citizen-4B3B\",\n\"POINT(-118.35 34.055)\",\n\"Large sinkhole on the sidewalk near apartment buildings on La Brea Ave south of Wilshire Blvd\",\n\"Major Sinkhole\",\n\"A huge sinkhole that looks like it will eat a car. FIX SOON\",\n\"Open\",\n\"South\",\n\"{'uri': 'gs://n25-311-demo/calls/311-20250409-60290D.mp3', 'version': '1744220293329858', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"audio/mpeg\\\",\\\"md5_hash\\\":\\\"87194fd410abd637e86f0fb822bb459c\\\",\\\"size\\\":41184,\\\"updated\\\":1744220293000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/images/311-20250409-60290D.png', 'version': '1744239339727794', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"image/png\\\",\\\"md5_hash\\\":\\\"1a9369188045c5475cc14aa9b7e6af47\\\",\\\"size\\\":1825501,\\\"updated\\\":1744239339000000}}'}\",\n\"{'uri': 'gs://n25-311-demo/videos/311-20250409-60290D.mp4', 'version': '1744239208881322', 'authorizer': 'data-demo-n25.us.test_connection', 'details': '{\\\"gcs_metadata\\\":{\\\"content_type\\\":\\\"video/mp4\\\",\\\"md5_hash\\\":\\\"d51a19210029014d075fb9197eaf27f9\\\",\\\"size\\\":5407865,\\\"updated\\\":1744239208000000}}'}\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"ticket_id\"], [\"string\", \"timestamp\"], [\"string\", \"reporter_id\"], [\"string\", \"location\"], [\"string\", \"location_description\"], [\"string\", \"category_reported\"], [\"string\", \"description\"], [\"string\", \"status\"], [\"string\", \"district\"], [\"string\", \"audio_ref\"], [\"string\", \"image_ref\"], [\"string\", \"video_ref\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-bb74b008-2a36-4786-94e6-6c7ae15e11e1\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-bb74b008-2a36-4786-94e6-6c7ae15e11e1')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-bb74b008-2a36-4786-94e6-6c7ae15e11e1 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-5bcf0c16-8c07-44ab-84c8-7a6ddb43a0b7\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>ticket_id</th>\n",
|
|
" <th>timestamp</th>\n",
|
|
" <th>reporter_id</th>\n",
|
|
" <th>location</th>\n",
|
|
" <th>location_description</th>\n",
|
|
" <th>category_reported</th>\n",
|
|
" <th>description</th>\n",
|
|
" <th>status</th>\n",
|
|
" <th>district</th>\n",
|
|
" <th>audio_ref</th>\n",
|
|
" <th>image_ref</th>\n",
|
|
" <th>video_ref</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>311-20250409-52F19E</td>\n",
|
|
" <td>2025-04-09 17:28:21+00:00</td>\n",
|
|
" <td>Citizen-4A2E</td>\n",
|
|
" <td>POINT(-118.245 34.052)</td>\n",
|
|
" <td>Large trash pile in the alley behind the build...</td>\n",
|
|
" <td>Illegal Dumping/Trash Pile</td>\n",
|
|
" <td>There is a huge pile of trash in the alley and...</td>\n",
|
|
" <td>Open</td>\n",
|
|
" <td>North</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/calls/311-20250409-...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/images/311-20250409...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/videos/311-20250409...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>311-20250409-A57DB4</td>\n",
|
|
" <td>2025-04-09 17:29:21+00:00</td>\n",
|
|
" <td>Citizen-77E1</td>\n",
|
|
" <td>POINT(-118.33 34.09)</td>\n",
|
|
" <td>Leaking fire hydrant on Highland Ave near the ...</td>\n",
|
|
" <td>Leaking Fire Hydrant</td>\n",
|
|
" <td>A leaky fire hydrant that has been creating a ...</td>\n",
|
|
" <td>Open</td>\n",
|
|
" <td>North</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/calls/311-20250409-...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/images/311-20250409...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/videos/311-20250409...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>311-20250409-59911E</td>\n",
|
|
" <td>2025-04-09 17:28:21+00:00</td>\n",
|
|
" <td>Citizen-69DD</td>\n",
|
|
" <td>POINT(-118.385 34.06)</td>\n",
|
|
" <td>Car parked on the sidewalk on Beverly Hills Bl...</td>\n",
|
|
" <td>Abandoned Vehicle</td>\n",
|
|
" <td>Someone drove their car on the sidewalk and ab...</td>\n",
|
|
" <td>Open</td>\n",
|
|
" <td>North</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/calls/311-20250409-...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/images/311-20250409...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/videos/311-20250409...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>311-20250409-1515EF</td>\n",
|
|
" <td>2025-04-09 17:35:21+00:00</td>\n",
|
|
" <td>Citizen-5DFC</td>\n",
|
|
" <td>POINT(-118.341 34.063)</td>\n",
|
|
" <td>Large pothole on Sunset Blvd near the bus stop...</td>\n",
|
|
" <td>Pothole</td>\n",
|
|
" <td>There is a small pothole near the bus stop tha...</td>\n",
|
|
" <td>Open</td>\n",
|
|
" <td>South</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/calls/311-20250409-...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/images/311-20250409...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/videos/311-20250409...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>311-20250409-60290D</td>\n",
|
|
" <td>2025-04-09 17:16:21+00:00</td>\n",
|
|
" <td>Citizen-4B3B</td>\n",
|
|
" <td>POINT(-118.35 34.055)</td>\n",
|
|
" <td>Large sinkhole on the sidewalk near apartment ...</td>\n",
|
|
" <td>Major Sinkhole</td>\n",
|
|
" <td>A huge sinkhole that looks like it will eat a ...</td>\n",
|
|
" <td>Open</td>\n",
|
|
" <td>South</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/calls/311-20250409-...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/images/311-20250409...</td>\n",
|
|
" <td>{'uri': 'gs://n25-311-demo/videos/311-20250409...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5bcf0c16-8c07-44ab-84c8-7a6ddb43a0b7')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-5bcf0c16-8c07-44ab-84c8-7a6ddb43a0b7 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-5bcf0c16-8c07-44ab-84c8-7a6ddb43a0b7');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" <div id=\"df-e5840cf9-823c-4cf0-886e-4736342fd11f\">\n",
|
|
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e5840cf9-823c-4cf0-886e-4736342fd11f')\"\n",
|
|
" title=\"Suggest charts\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
|
|
" width=\"24px\">\n",
|
|
" <g>\n",
|
|
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
|
|
" </g>\n",
|
|
"</svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
"<style>\n",
|
|
" .colab-df-quickchart {\n",
|
|
" --bg-color: #E8F0FE;\n",
|
|
" --fill-color: #1967D2;\n",
|
|
" --hover-bg-color: #E2EBFA;\n",
|
|
" --hover-fill-color: #174EA6;\n",
|
|
" --disabled-fill-color: #AAA;\n",
|
|
" --disabled-bg-color: #DDD;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-quickchart {\n",
|
|
" --bg-color: #3B4455;\n",
|
|
" --fill-color: #D2E3FC;\n",
|
|
" --hover-bg-color: #434B5C;\n",
|
|
" --hover-fill-color: #FFFFFF;\n",
|
|
" --disabled-bg-color: #3B4455;\n",
|
|
" --disabled-fill-color: #666;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart {\n",
|
|
" background-color: var(--bg-color);\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: var(--fill-color);\n",
|
|
" height: 32px;\n",
|
|
" padding: 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart:hover {\n",
|
|
" background-color: var(--hover-bg-color);\n",
|
|
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: var(--button-hover-fill-color);\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart-complete:disabled,\n",
|
|
" .colab-df-quickchart-complete:disabled:hover {\n",
|
|
" background-color: var(--disabled-bg-color);\n",
|
|
" fill: var(--disabled-fill-color);\n",
|
|
" box-shadow: none;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-spinner {\n",
|
|
" border: 2px solid var(--fill-color);\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" animation:\n",
|
|
" spin 1s steps(1) infinite;\n",
|
|
" }\n",
|
|
"\n",
|
|
" @keyframes spin {\n",
|
|
" 0% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 20% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 30% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 40% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 60% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 80% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 90% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" async function quickchart(key) {\n",
|
|
" const quickchartButtonEl =\n",
|
|
" document.querySelector('#' + key + ' button');\n",
|
|
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
|
|
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
|
|
" try {\n",
|
|
" const charts = await google.colab.kernel.invokeFunction(\n",
|
|
" 'suggestCharts', [key], {});\n",
|
|
" } catch (error) {\n",
|
|
" console.error('Error during call to suggestCharts:', error);\n",
|
|
" }\n",
|
|
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
|
|
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
|
|
" }\n",
|
|
" (() => {\n",
|
|
" let quickchartButtonEl =\n",
|
|
" document.querySelector('#df-e5840cf9-823c-4cf0-886e-4736342fd11f button');\n",
|
|
" quickchartButtonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
" })();\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" ticket_id timestamp reporter_id \\\n",
|
|
"0 311-20250409-52F19E 2025-04-09 17:28:21+00:00 Citizen-4A2E \n",
|
|
"1 311-20250409-A57DB4 2025-04-09 17:29:21+00:00 Citizen-77E1 \n",
|
|
"2 311-20250409-59911E 2025-04-09 17:28:21+00:00 Citizen-69DD \n",
|
|
"3 311-20250409-1515EF 2025-04-09 17:35:21+00:00 Citizen-5DFC \n",
|
|
"4 311-20250409-60290D 2025-04-09 17:16:21+00:00 Citizen-4B3B \n",
|
|
"\n",
|
|
" location location_description \\\n",
|
|
"0 POINT(-118.245 34.052) Large trash pile in the alley behind the build... \n",
|
|
"1 POINT(-118.33 34.09) Leaking fire hydrant on Highland Ave near the ... \n",
|
|
"2 POINT(-118.385 34.06) Car parked on the sidewalk on Beverly Hills Bl... \n",
|
|
"3 POINT(-118.341 34.063) Large pothole on Sunset Blvd near the bus stop... \n",
|
|
"4 POINT(-118.35 34.055) Large sinkhole on the sidewalk near apartment ... \n",
|
|
"\n",
|
|
" category_reported \\\n",
|
|
"0 Illegal Dumping/Trash Pile \n",
|
|
"1 Leaking Fire Hydrant \n",
|
|
"2 Abandoned Vehicle \n",
|
|
"3 Pothole \n",
|
|
"4 Major Sinkhole \n",
|
|
"\n",
|
|
" description status district \\\n",
|
|
"0 There is a huge pile of trash in the alley and... Open North \n",
|
|
"1 A leaky fire hydrant that has been creating a ... Open North \n",
|
|
"2 Someone drove their car on the sidewalk and ab... Open North \n",
|
|
"3 There is a small pothole near the bus stop tha... Open South \n",
|
|
"4 A huge sinkhole that looks like it will eat a ... Open South \n",
|
|
"\n",
|
|
" audio_ref \\\n",
|
|
"0 {'uri': 'gs://n25-311-demo/calls/311-20250409-... \n",
|
|
"1 {'uri': 'gs://n25-311-demo/calls/311-20250409-... \n",
|
|
"2 {'uri': 'gs://n25-311-demo/calls/311-20250409-... \n",
|
|
"3 {'uri': 'gs://n25-311-demo/calls/311-20250409-... \n",
|
|
"4 {'uri': 'gs://n25-311-demo/calls/311-20250409-... \n",
|
|
"\n",
|
|
" image_ref \\\n",
|
|
"0 {'uri': 'gs://n25-311-demo/images/311-20250409... \n",
|
|
"1 {'uri': 'gs://n25-311-demo/images/311-20250409... \n",
|
|
"2 {'uri': 'gs://n25-311-demo/images/311-20250409... \n",
|
|
"3 {'uri': 'gs://n25-311-demo/images/311-20250409... \n",
|
|
"4 {'uri': 'gs://n25-311-demo/images/311-20250409... \n",
|
|
"\n",
|
|
" video_ref \n",
|
|
"0 {'uri': 'gs://n25-311-demo/videos/311-20250409... \n",
|
|
"1 {'uri': 'gs://n25-311-demo/videos/311-20250409... \n",
|
|
"2 {'uri': 'gs://n25-311-demo/videos/311-20250409... \n",
|
|
"3 {'uri': 'gs://n25-311-demo/videos/311-20250409... \n",
|
|
"4 {'uri': 'gs://n25-311-demo/videos/311-20250409... "
|
|
]
|
|
},
|
|
"execution_count": 40,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"CREATE OR REPLACE TABLE `bq_mm_tutorial.reports_mm` AS\n",
|
|
"SELECT\n",
|
|
" r.*,\n",
|
|
" OBJ.FETCH_METADATA(OBJ.MAKE_REF(m.audio_uri, 'us.test_connection')) as audio_ref,\n",
|
|
" OBJ.FETCH_METADATA(OBJ.MAKE_REF(m.image_uri, 'us.test_connection')) as image_ref,\n",
|
|
" OBJ.FETCH_METADATA(OBJ.MAKE_REF(m.video_uri, 'us.test_connection')) as video_ref\n",
|
|
"FROM `bq_mm_tutorial.reports` r\n",
|
|
"LEFT JOIN `bq_mm_tutorial.media` m\n",
|
|
"ON r.ticket_id = m.ticket_id;\n",
|
|
"\n",
|
|
"SELECT * FROM `bq_mm_tutorial.reports_mm`;"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "xqzgf0fASE6U"
|
|
},
|
|
"source": [
|
|
"### 4. Run a multimodal query with on `ObjectRef`\n",
|
|
"\n",
|
|
"Our `reports_mm` table has column called `description`, which provides a text description of a reported incident.\n",
|
|
"\n",
|
|
"However, there may be additional, secondary issues that AI can help infer from provided media.\n",
|
|
"\n",
|
|
"In this example, we pass the text `category_reported` alongside a prompt and then append `image_ref`. In this way, the [`AI.GENERATE_TABLE`](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-generate-table) function passes text **and** an image in the same Gemini model call before returning the results.\n",
|
|
"\n",
|
|
"But for BigQuery's `AI.GENERATE_TABLE` to call Gemini to parse unstructured data, we first need to create a [Remote Model](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-remote-model). We'll create that and move to our multimodal query."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 154,
|
|
"referenced_widgets": [
|
|
"efa7bed09a614513949c89bcc01764ff",
|
|
"40ae635e06e743e4bfbd852203da640d",
|
|
"d317ebcf563546209b6bdbb07247b50b",
|
|
"1606ec0a800549b7b0ac80bae1490ca7",
|
|
"54d5e3f9c2fd46059cbbdadc0fbb6397",
|
|
"f897fc1656854a5ab3a0a6dac941d3cf",
|
|
"63a16a5138104e3699980274d1a6a346",
|
|
"d09e04b34451415eb2083ac4a399efd0",
|
|
"068c95f20fb7448e8c2330227b24afed",
|
|
"18f60ba1083345b7a78518d77e7fe45d",
|
|
"80c473eebb5f40e3a66644d2c214055d"
|
|
]
|
|
},
|
|
"id": "g-1Lny9oeN9D",
|
|
"outputId": "06a8a033-c9af-47b9-f920-5d3aec1dccc0"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "efa7bed09a614513949c89bcc01764ff",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 0,\n \"fields\": []\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [],\n columns: [[\"number\", \"index\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-c937d751-8859-4630-b0ff-377e15f43a8b\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c937d751-8859-4630-b0ff-377e15f43a8b')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-c937d751-8859-4630-b0ff-377e15f43a8b button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-4cc23372-3649-4301-ac89-d8fe8c0c6f3f\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4cc23372-3649-4301-ac89-d8fe8c0c6f3f')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-4cc23372-3649-4301-ac89-d8fe8c0c6f3f button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-4cc23372-3649-4301-ac89-d8fe8c0c6f3f');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
"Empty DataFrame\n",
|
|
"Columns: []\n",
|
|
"Index: []"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"CREATE OR REPLACE MODEL `bq_mm_tutorial.gemini`\n",
|
|
"REMOTE WITH CONNECTION `us.test_connection`\n",
|
|
" OPTIONS(ENDPOINT = 'gemini-2.5-flash');"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 293,
|
|
"referenced_widgets": [
|
|
"bc3ccd30484e495ca31411dad7ef0a58",
|
|
"668aa95aae7443c197c0b0a5b5eb1245",
|
|
"01cea4f298a9425190fae5ccfac104d5",
|
|
"d7794f774ff84cfd8b25cf362baeb4ac",
|
|
"f805e0d5073941a78d4e1f8ec210a485",
|
|
"b56b5e01b7d940c5b1395a0644c1caca",
|
|
"ee6eb5747c0c4fc9beca5271f371ac90",
|
|
"a6b8e054c09b4747a84a666327348783",
|
|
"7e9557b4b0b443149dbe2255e1b3c73c",
|
|
"28f99e97ed584ec685e16a5c43261f81",
|
|
"cd73d9917e6c4d83943cb03f7438aec4",
|
|
"93129eed185c45aeb80d6eabe320a585",
|
|
"470bc1aa7b44486b9e80f5aa762efdaa",
|
|
"eace7c28638e478ca9397676d9c24e04",
|
|
"06f81c65251f4ec2b66913ba4039cf41",
|
|
"e770eb58bfa144888fabcaf30738756c",
|
|
"b52d7c0f6b654c78a7757fe137b2b189",
|
|
"2f5c17ffe1d14b719f36672eb35806fc",
|
|
"20c3ce0e41fa44eaad482c6b7418a7ce",
|
|
"eaa9ea3a207e487b831fd7bb84e26178",
|
|
"e9d84d732ba74ba487668cf9c85accab",
|
|
"9fb87c004c494d7ba38462afc7db4f39"
|
|
]
|
|
},
|
|
"id": "wdayCH4ISWts",
|
|
"outputId": "e5906084-b5ec-4458-a2eb-cb3d6fdd1726"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "bc3ccd30484e495ca31411dad7ef0a58",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "93129eed185c45aeb80d6eabe320a585",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"ticket_id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"311-20250409-60290D\",\n \"311-20250409-A57DB4\",\n \"311-20250409-52F19E\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"category_reported\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Major Sinkhole\",\n \"Leaking Fire Hydrant\",\n \"Illegal Dumping/Trash Pile\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"secondary_category\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"description\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"A huge sinkhole that looks like it will eat a car. FIX SOON\",\n \"A leaky fire hydrant that has been creating a puddle\",\n \"There is a huge pile of trash in the alley and I cannot walk through it. It gets bigger every day.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"311-20250409-59911E\",\n\"Abandoned Vehicle\",\n\"['Damaged Tree']\",\n\"Someone drove their car on the sidewalk and abandoned it. It needs to be towed. Its in the way of pedestrians!\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"311-20250409-60290D\",\n\"Major Sinkhole\",\n\"['Widespread pavement cracks']\",\n\"A huge sinkhole that looks like it will eat a car. FIX SOON\"],\n [{\n 'v': 2,\n 'f': \"2\",\n },\n\"311-20250409-52F19E\",\n\"Illegal Dumping/Trash Pile\",\n\"['Graffiti']\",\n\"There is a huge pile of trash in the alley and I cannot walk through it. It gets bigger every day.\"],\n [{\n 'v': 3,\n 'f': \"3\",\n },\n\"311-20250409-1515EF\",\n\"Pothole\",\n\"['Cracked Road']\",\n\"There is a small pothole near the bus stop that should be filled. It is a little bit of a hazard.\"],\n [{\n 'v': 4,\n 'f': \"4\",\n },\n\"311-20250409-A57DB4\",\n\"Leaking Fire Hydrant\",\n\"['Slipping Hazard']\",\n\"A leaky fire hydrant that has been creating a puddle\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"ticket_id\"], [\"string\", \"category_reported\"], [\"string\", \"secondary_category\"], [\"string\", \"description\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-6f3c5c03-af91-4564-8d7e-34eeb9589190\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-6f3c5c03-af91-4564-8d7e-34eeb9589190')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-6f3c5c03-af91-4564-8d7e-34eeb9589190 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-45750514-8eff-42d1-8d05-47e66833400e\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>ticket_id</th>\n",
|
|
" <th>category_reported</th>\n",
|
|
" <th>secondary_category</th>\n",
|
|
" <th>description</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>311-20250409-59911E</td>\n",
|
|
" <td>Abandoned Vehicle</td>\n",
|
|
" <td>[Damaged Tree]</td>\n",
|
|
" <td>Someone drove their car on the sidewalk and ab...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>311-20250409-60290D</td>\n",
|
|
" <td>Major Sinkhole</td>\n",
|
|
" <td>[Widespread pavement cracks]</td>\n",
|
|
" <td>A huge sinkhole that looks like it will eat a ...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>311-20250409-52F19E</td>\n",
|
|
" <td>Illegal Dumping/Trash Pile</td>\n",
|
|
" <td>[Graffiti]</td>\n",
|
|
" <td>There is a huge pile of trash in the alley and...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>311-20250409-1515EF</td>\n",
|
|
" <td>Pothole</td>\n",
|
|
" <td>[Cracked Road]</td>\n",
|
|
" <td>There is a small pothole near the bus stop tha...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>311-20250409-A57DB4</td>\n",
|
|
" <td>Leaking Fire Hydrant</td>\n",
|
|
" <td>[Slipping Hazard]</td>\n",
|
|
" <td>A leaky fire hydrant that has been creating a ...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-45750514-8eff-42d1-8d05-47e66833400e')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-45750514-8eff-42d1-8d05-47e66833400e button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-45750514-8eff-42d1-8d05-47e66833400e');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" <div id=\"df-3eb5e529-c481-42e2-b930-6875c50b961e\">\n",
|
|
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-3eb5e529-c481-42e2-b930-6875c50b961e')\"\n",
|
|
" title=\"Suggest charts\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
|
|
" width=\"24px\">\n",
|
|
" <g>\n",
|
|
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
|
|
" </g>\n",
|
|
"</svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
"<style>\n",
|
|
" .colab-df-quickchart {\n",
|
|
" --bg-color: #E8F0FE;\n",
|
|
" --fill-color: #1967D2;\n",
|
|
" --hover-bg-color: #E2EBFA;\n",
|
|
" --hover-fill-color: #174EA6;\n",
|
|
" --disabled-fill-color: #AAA;\n",
|
|
" --disabled-bg-color: #DDD;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-quickchart {\n",
|
|
" --bg-color: #3B4455;\n",
|
|
" --fill-color: #D2E3FC;\n",
|
|
" --hover-bg-color: #434B5C;\n",
|
|
" --hover-fill-color: #FFFFFF;\n",
|
|
" --disabled-bg-color: #3B4455;\n",
|
|
" --disabled-fill-color: #666;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart {\n",
|
|
" background-color: var(--bg-color);\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: var(--fill-color);\n",
|
|
" height: 32px;\n",
|
|
" padding: 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart:hover {\n",
|
|
" background-color: var(--hover-bg-color);\n",
|
|
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: var(--button-hover-fill-color);\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart-complete:disabled,\n",
|
|
" .colab-df-quickchart-complete:disabled:hover {\n",
|
|
" background-color: var(--disabled-bg-color);\n",
|
|
" fill: var(--disabled-fill-color);\n",
|
|
" box-shadow: none;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-spinner {\n",
|
|
" border: 2px solid var(--fill-color);\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" animation:\n",
|
|
" spin 1s steps(1) infinite;\n",
|
|
" }\n",
|
|
"\n",
|
|
" @keyframes spin {\n",
|
|
" 0% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 20% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 30% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 40% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 60% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 80% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 90% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" async function quickchart(key) {\n",
|
|
" const quickchartButtonEl =\n",
|
|
" document.querySelector('#' + key + ' button');\n",
|
|
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
|
|
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
|
|
" try {\n",
|
|
" const charts = await google.colab.kernel.invokeFunction(\n",
|
|
" 'suggestCharts', [key], {});\n",
|
|
" } catch (error) {\n",
|
|
" console.error('Error during call to suggestCharts:', error);\n",
|
|
" }\n",
|
|
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
|
|
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
|
|
" }\n",
|
|
" (() => {\n",
|
|
" let quickchartButtonEl =\n",
|
|
" document.querySelector('#df-3eb5e529-c481-42e2-b930-6875c50b961e button');\n",
|
|
" quickchartButtonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
" })();\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" ticket_id category_reported \\\n",
|
|
"0 311-20250409-59911E Abandoned Vehicle \n",
|
|
"1 311-20250409-60290D Major Sinkhole \n",
|
|
"2 311-20250409-52F19E Illegal Dumping/Trash Pile \n",
|
|
"3 311-20250409-1515EF Pothole \n",
|
|
"4 311-20250409-A57DB4 Leaking Fire Hydrant \n",
|
|
"\n",
|
|
" secondary_category \\\n",
|
|
"0 [Damaged Tree] \n",
|
|
"1 [Widespread pavement cracks] \n",
|
|
"2 [Graffiti] \n",
|
|
"3 [Cracked Road] \n",
|
|
"4 [Slipping Hazard] \n",
|
|
"\n",
|
|
" description \n",
|
|
"0 Someone drove their car on the sidewalk and ab... \n",
|
|
"1 A huge sinkhole that looks like it will eat a ... \n",
|
|
"2 There is a huge pile of trash in the alley and... \n",
|
|
"3 There is a small pothole near the bus stop tha... \n",
|
|
"4 A leaky fire hydrant that has been creating a ... "
|
|
]
|
|
},
|
|
"execution_count": 41,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT\n",
|
|
" ticket_id,\n",
|
|
" category_reported,\n",
|
|
" secondary_category,\n",
|
|
" description\n",
|
|
"FROM AI.GENERATE_TABLE(\n",
|
|
" MODEL `bq_mm_tutorial.gemini`,\n",
|
|
" (\n",
|
|
" SELECT (\n",
|
|
" category_reported, ' is the primary issue.'\n",
|
|
" 'Note a secondary issue, only if you find something severe needing attention.'\n",
|
|
" 'Inspect the background and in the object.'\n",
|
|
" , image_ref\n",
|
|
" ) AS prompt,\n",
|
|
" ticket_id,\n",
|
|
" category_reported,\n",
|
|
" description\n",
|
|
" FROM `bq_mm_tutorial.reports_mm`\n",
|
|
" ),\n",
|
|
" STRUCT(\n",
|
|
" \"secondary_category ARRAY<STRING>\" AS output_schema\n",
|
|
" )\n",
|
|
");"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "BzbLQ7fuVAsz"
|
|
},
|
|
"source": [
|
|
"### 5. Run a multimodal query with multiple `ObjectRef`s\n",
|
|
"\n",
|
|
"We can pass **multiple** `ObjectRef` fields in the same call to a Gemini model. In this example, we pass:\n",
|
|
"* The text description a user reported\n",
|
|
"* An image (`image_ref`)\n",
|
|
"* Call audio (`audio_ref`)\n",
|
|
"* Video (`video_ref`)\n",
|
|
"\n",
|
|
"Providing all of this multimodal data in the same function call to the Gemini model allows it to perform a complete analysis that takes into account all four pieces of information at once. The output helps a 311 Operator prioritize which tickets are highest urgency and need immediate assistance."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 624,
|
|
"referenced_widgets": [
|
|
"1ade78ecd6f649bfbc1aa07232920c8f",
|
|
"226564842aa34efe8e9e8fe7ddcddb30",
|
|
"d141ac9dc37843179c12eec324e9ab02",
|
|
"4a142df34b4b44a7a32ea954780fe6e4",
|
|
"50f7dbade42141c3a3db985b492d358f",
|
|
"c17c2fd2dd634300b492636fea3d0c22",
|
|
"689573d49df0476cb21e3bb748444189",
|
|
"0cd8da74b1c14958be15fe32ac68e1dc",
|
|
"be18dd05a2734d5e8b6b9f40ebe0d3ec",
|
|
"0e6f6b1cdcc44b6d99fc5ca5ce2815d1",
|
|
"76dda31c62e642eabe81d86e8d78e898",
|
|
"c0cfa49f540a498e91b196e83a9157d1",
|
|
"105b1f83272f4dbe98c16e97d4c252d7",
|
|
"1cdaa139b96c407280876aee31d240dc",
|
|
"0dad6963c40c4aa3b86c5083c9f8f3dd",
|
|
"c9f8549fc54640f3985f2c8d1f897b84",
|
|
"094e5a41b51544dd82d0c95cc324336e",
|
|
"bf0fa1e09e8b4f1d8eda6b1a6b0f1af4",
|
|
"aaf378ff4188447fb6bfeb34c7fc2e24",
|
|
"7e0e4c90626e4bec9d7cfff2c58d8641",
|
|
"90fc08c4e95d4ba688c893814c0f26f6",
|
|
"d9c37d7f1b714c4ca1153436d414c7e3"
|
|
]
|
|
},
|
|
"id": "A-PzHlJ1U1zo",
|
|
"outputId": "9d7b1a81-7684-4202-8323-3fa01542aed4"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "1ade78ecd6f649bfbc1aa07232920c8f",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "c0cfa49f540a498e91b196e83a9157d1",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"ticket_id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"311-20250409-60290D\",\n \"311-20250409-A57DB4\",\n \"311-20250409-1515EF\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"issue\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Massive sidewalk sinkhole\",\n \"Leaking fire hydrant\",\n \"Deep road pothole\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"secondary_issue\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 4,\n \"samples\": [\n \"Extensive pavement cracking\",\n \"Abandoned vehicle\",\n \"Graffiti, fire hazard\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"original_description\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"A huge sinkhole that looks like it will eat a car. FIX SOON\",\n \"A leaky fire hydrant that has been creating a puddle\",\n \"There is a small pothole near the bus stop that should be filled. It is a little bit of a hazard.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ai_summary_description\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"A substantial and rapidly expanding sinkhole has formed on a paved urban sidewalk, characterized by a deep cavity with visible rubble and extensive radial cracking of the surrounding concrete. The accompanying audio confirms the location on La Brea Avenue near an apartment, emphasizes its growing size, and expresses concern about potential falls, highlighting an immediate safety hazard.\",\n \"A red fire hydrant located on Highland Avenue near a movie theater is actively leaking water onto the sidewalk, creating a growing puddle. The leak has reportedly been ongoing for several hours, causing significant water waste and potentially creating a slip hazard for pedestrians.\",\n \"A significant and deep pothole is present in the middle of a busy residential street, surrounded by radiating cracks and a longer central crack, posing a considerable traffic hazard as evidenced by passing vehicles swerving to avoid it. Its location near a bus stop further increases safety concerns for both drivers and pedestrians.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"recommended_action\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Dispatch emergency crews immediately to cordon off the area, assess the full extent of the damage, and initiate urgent repairs to the sidewalk sinkhole.\",\n \"Dispatch a crew to Highland Avenue to address a leaking fire hydrant that is creating a significant puddle.\",\n \"Dispatch a crew to immediately repair the large and hazardous pothole on Sunset Boulevard near the bus stop to prevent accidents and further road damage.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"urgency_score\",\n \"properties\": {\n \"dtype\": \"Int64\",\n \"num_unique_values\": 3,\n \"samples\": [\n 10,\n 9,\n 7\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"city_response_department\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Public Works\",\n \"Water Department\",\n \"Roads\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"311-20250409-52F19E\",\n\"Alleyway blocked by trash\",\n\"Graffiti, fire hazard\",\n\"There is a huge pile of trash in the alley and I cannot walk through it. It gets bigger every day.\",\n\"The scene depicts an alleyway completely obstructed by an enormous accumulation of diverse refuse, including large furniture pieces, numerous black garbage bags, and various discarded items. This significant trash pile prevents passage and is reported to be growing, attracting pests, and emitting foul odors. Critically, during the observation, an active fire has ignited within the trash, posing an immediate danger and escalating the severity of the situation significantly. The alley walls also show signs of graffiti.\",\n\"Dispatch Sanitation and Fire Departments immediately to clear the massive trash pile and extinguish the active fire in the alley.\",\n{\n 'v': 10,\n 'f': \"10\",\n },\n\"Fire\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"311-20250409-60290D\",\n\"Massive sidewalk sinkhole\",\n\"Extensive pavement cracking\",\n\"A huge sinkhole that looks like it will eat a car. FIX SOON\",\n\"A substantial and rapidly expanding sinkhole has formed on a paved urban sidewalk, characterized by a deep cavity with visible rubble and extensive radial cracking of the surrounding concrete. The accompanying audio confirms the location on La Brea Avenue near an apartment, emphasizes its growing size, and expresses concern about potential falls, highlighting an immediate safety hazard.\",\n\"Dispatch emergency crews immediately to cordon off the area, assess the full extent of the damage, and initiate urgent repairs to the sidewalk sinkhole.\",\n{\n 'v': 9,\n 'f': \"9\",\n },\n\"Public Works\"],\n [{\n 'v': 2,\n 'f': \"2\",\n },\n\"311-20250409-1515EF\",\n\"Deep road pothole\",\n\"Extensive road cracking\",\n\"There is a small pothole near the bus stop that should be filled. It is a little bit of a hazard.\",\n\"A significant and deep pothole is present in the middle of a busy residential street, surrounded by radiating cracks and a longer central crack, posing a considerable traffic hazard as evidenced by passing vehicles swerving to avoid it. Its location near a bus stop further increases safety concerns for both drivers and pedestrians.\",\n\"Dispatch a crew to immediately repair the large and hazardous pothole on Sunset Boulevard near the bus stop to prevent accidents and further road damage.\",\n{\n 'v': 9,\n 'f': \"9\",\n },\n\"Roads\"],\n [{\n 'v': 3,\n 'f': \"3\",\n },\n\"311-20250409-59911E\",\n\"Car blocking sidewalk\",\n\"Abandoned vehicle\",\n\"Someone drove their car on the sidewalk and abandoned it. It needs to be towed. Its in the way of pedestrians!\",\n\"A silver car is illegally parked on a bustling urban sidewalk next to an outdoor cafe, significantly obstructing pedestrian flow. The vehicle appears to be abandoned, as it has been stationary for an extended period, creating a nuisance and potential safety hazard in a high-traffic area.\",\n\"Dispatch a tow truck to remove the illegally parked and abandoned vehicle from the sidewalk.\",\n{\n 'v': 7,\n 'f': \"7\",\n },\n\"Police\"],\n [{\n 'v': 4,\n 'f': \"4\",\n },\n\"311-20250409-A57DB4\",\n\"Leaking fire hydrant\",\nnull,\n\"A leaky fire hydrant that has been creating a puddle\",\n\"A red fire hydrant located on Highland Avenue near a movie theater is actively leaking water onto the sidewalk, creating a growing puddle. The leak has reportedly been ongoing for several hours, causing significant water waste and potentially creating a slip hazard for pedestrians.\",\n\"Dispatch a crew to Highland Avenue to address a leaking fire hydrant that is creating a significant puddle.\",\n{\n 'v': 7,\n 'f': \"7\",\n },\n\"Water Department\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"ticket_id\"], [\"string\", \"issue\"], [\"string\", \"secondary_issue\"], [\"string\", \"original_description\"], [\"string\", \"ai_summary_description\"], [\"string\", \"recommended_action\"], [\"number\", \"urgency_score\"], [\"string\", \"city_response_department\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-3a8fa99f-ef3e-4f8c-8668-759e2872938c\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-3a8fa99f-ef3e-4f8c-8668-759e2872938c')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-3a8fa99f-ef3e-4f8c-8668-759e2872938c button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-3c34a010-7dd7-440d-a0e3-adba595e48bf\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>ticket_id</th>\n",
|
|
" <th>issue</th>\n",
|
|
" <th>secondary_issue</th>\n",
|
|
" <th>original_description</th>\n",
|
|
" <th>ai_summary_description</th>\n",
|
|
" <th>recommended_action</th>\n",
|
|
" <th>urgency_score</th>\n",
|
|
" <th>city_response_department</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>311-20250409-52F19E</td>\n",
|
|
" <td>Alleyway blocked by trash</td>\n",
|
|
" <td>Graffiti, fire hazard</td>\n",
|
|
" <td>There is a huge pile of trash in the alley and...</td>\n",
|
|
" <td>The scene depicts an alleyway completely obstr...</td>\n",
|
|
" <td>Dispatch Sanitation and Fire Departments immed...</td>\n",
|
|
" <td>10</td>\n",
|
|
" <td>Fire</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>311-20250409-60290D</td>\n",
|
|
" <td>Massive sidewalk sinkhole</td>\n",
|
|
" <td>Extensive pavement cracking</td>\n",
|
|
" <td>A huge sinkhole that looks like it will eat a ...</td>\n",
|
|
" <td>A substantial and rapidly expanding sinkhole h...</td>\n",
|
|
" <td>Dispatch emergency crews immediately to cordon...</td>\n",
|
|
" <td>9</td>\n",
|
|
" <td>Public Works</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>311-20250409-1515EF</td>\n",
|
|
" <td>Deep road pothole</td>\n",
|
|
" <td>Extensive road cracking</td>\n",
|
|
" <td>There is a small pothole near the bus stop tha...</td>\n",
|
|
" <td>A significant and deep pothole is present in t...</td>\n",
|
|
" <td>Dispatch a crew to immediately repair the larg...</td>\n",
|
|
" <td>9</td>\n",
|
|
" <td>Roads</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>311-20250409-59911E</td>\n",
|
|
" <td>Car blocking sidewalk</td>\n",
|
|
" <td>Abandoned vehicle</td>\n",
|
|
" <td>Someone drove their car on the sidewalk and ab...</td>\n",
|
|
" <td>A silver car is illegally parked on a bustling...</td>\n",
|
|
" <td>Dispatch a tow truck to remove the illegally p...</td>\n",
|
|
" <td>7</td>\n",
|
|
" <td>Police</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>311-20250409-A57DB4</td>\n",
|
|
" <td>Leaking fire hydrant</td>\n",
|
|
" <td>None</td>\n",
|
|
" <td>A leaky fire hydrant that has been creating a ...</td>\n",
|
|
" <td>A red fire hydrant located on Highland Avenue ...</td>\n",
|
|
" <td>Dispatch a crew to Highland Avenue to address ...</td>\n",
|
|
" <td>7</td>\n",
|
|
" <td>Water Department</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3c34a010-7dd7-440d-a0e3-adba595e48bf')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-3c34a010-7dd7-440d-a0e3-adba595e48bf button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-3c34a010-7dd7-440d-a0e3-adba595e48bf');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" <div id=\"df-16715455-edd0-43f4-8607-44a2fde93f38\">\n",
|
|
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-16715455-edd0-43f4-8607-44a2fde93f38')\"\n",
|
|
" title=\"Suggest charts\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
|
|
" width=\"24px\">\n",
|
|
" <g>\n",
|
|
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
|
|
" </g>\n",
|
|
"</svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
"<style>\n",
|
|
" .colab-df-quickchart {\n",
|
|
" --bg-color: #E8F0FE;\n",
|
|
" --fill-color: #1967D2;\n",
|
|
" --hover-bg-color: #E2EBFA;\n",
|
|
" --hover-fill-color: #174EA6;\n",
|
|
" --disabled-fill-color: #AAA;\n",
|
|
" --disabled-bg-color: #DDD;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-quickchart {\n",
|
|
" --bg-color: #3B4455;\n",
|
|
" --fill-color: #D2E3FC;\n",
|
|
" --hover-bg-color: #434B5C;\n",
|
|
" --hover-fill-color: #FFFFFF;\n",
|
|
" --disabled-bg-color: #3B4455;\n",
|
|
" --disabled-fill-color: #666;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart {\n",
|
|
" background-color: var(--bg-color);\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: var(--fill-color);\n",
|
|
" height: 32px;\n",
|
|
" padding: 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart:hover {\n",
|
|
" background-color: var(--hover-bg-color);\n",
|
|
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: var(--button-hover-fill-color);\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart-complete:disabled,\n",
|
|
" .colab-df-quickchart-complete:disabled:hover {\n",
|
|
" background-color: var(--disabled-bg-color);\n",
|
|
" fill: var(--disabled-fill-color);\n",
|
|
" box-shadow: none;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-spinner {\n",
|
|
" border: 2px solid var(--fill-color);\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" animation:\n",
|
|
" spin 1s steps(1) infinite;\n",
|
|
" }\n",
|
|
"\n",
|
|
" @keyframes spin {\n",
|
|
" 0% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 20% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 30% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 40% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 60% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 80% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 90% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" async function quickchart(key) {\n",
|
|
" const quickchartButtonEl =\n",
|
|
" document.querySelector('#' + key + ' button');\n",
|
|
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
|
|
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
|
|
" try {\n",
|
|
" const charts = await google.colab.kernel.invokeFunction(\n",
|
|
" 'suggestCharts', [key], {});\n",
|
|
" } catch (error) {\n",
|
|
" console.error('Error during call to suggestCharts:', error);\n",
|
|
" }\n",
|
|
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
|
|
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
|
|
" }\n",
|
|
" (() => {\n",
|
|
" let quickchartButtonEl =\n",
|
|
" document.querySelector('#df-16715455-edd0-43f4-8607-44a2fde93f38 button');\n",
|
|
" quickchartButtonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
" })();\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" ticket_id issue \\\n",
|
|
"0 311-20250409-52F19E Alleyway blocked by trash \n",
|
|
"1 311-20250409-60290D Massive sidewalk sinkhole \n",
|
|
"2 311-20250409-1515EF Deep road pothole \n",
|
|
"3 311-20250409-59911E Car blocking sidewalk \n",
|
|
"4 311-20250409-A57DB4 Leaking fire hydrant \n",
|
|
"\n",
|
|
" secondary_issue \\\n",
|
|
"0 Graffiti, fire hazard \n",
|
|
"1 Extensive pavement cracking \n",
|
|
"2 Extensive road cracking \n",
|
|
"3 Abandoned vehicle \n",
|
|
"4 None \n",
|
|
"\n",
|
|
" original_description \\\n",
|
|
"0 There is a huge pile of trash in the alley and... \n",
|
|
"1 A huge sinkhole that looks like it will eat a ... \n",
|
|
"2 There is a small pothole near the bus stop tha... \n",
|
|
"3 Someone drove their car on the sidewalk and ab... \n",
|
|
"4 A leaky fire hydrant that has been creating a ... \n",
|
|
"\n",
|
|
" ai_summary_description \\\n",
|
|
"0 The scene depicts an alleyway completely obstr... \n",
|
|
"1 A substantial and rapidly expanding sinkhole h... \n",
|
|
"2 A significant and deep pothole is present in t... \n",
|
|
"3 A silver car is illegally parked on a bustling... \n",
|
|
"4 A red fire hydrant located on Highland Avenue ... \n",
|
|
"\n",
|
|
" recommended_action urgency_score \\\n",
|
|
"0 Dispatch Sanitation and Fire Departments immed... 10 \n",
|
|
"1 Dispatch emergency crews immediately to cordon... 9 \n",
|
|
"2 Dispatch a crew to immediately repair the larg... 9 \n",
|
|
"3 Dispatch a tow truck to remove the illegally p... 7 \n",
|
|
"4 Dispatch a crew to Highland Avenue to address ... 7 \n",
|
|
"\n",
|
|
" city_response_department \n",
|
|
"0 Fire \n",
|
|
"1 Public Works \n",
|
|
"2 Roads \n",
|
|
"3 Police \n",
|
|
"4 Water Department "
|
|
]
|
|
},
|
|
"execution_count": 42,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT\n",
|
|
" ticket_id,\n",
|
|
" issue,\n",
|
|
" secondary_issue,\n",
|
|
" description AS original_description,\n",
|
|
" ai_summary_description,\n",
|
|
" recommended_action,\n",
|
|
" urgency_score,\n",
|
|
" city_response_department\n",
|
|
"FROM AI.GENERATE_TABLE(\n",
|
|
" MODEL `bq_mm_tutorial.gemini`,\n",
|
|
" (\n",
|
|
" SELECT (\n",
|
|
" 'Describe the primary issue in the photo in 2-3 words. Be descriptive.'\n",
|
|
" 'Only if you notice a secondary issue, list it too. Inspect the background and everywhere.'\n",
|
|
" 'Rate the urgency score for the city to respond from 1-10 where 1 is low, 10 is absolutely critical to safety.'\n",
|
|
" 'Write an AI generated description of the issue taking into account the text, image, audio, and video.'\n",
|
|
" 'Write a 1 sentence description to city dispatch for a recommended action.'\n",
|
|
" 'Assign a single city response department (e.g. Roads, Sanitation, Parks, Fire)'\n",
|
|
" , description, image_ref, audio_ref, video_ref\n",
|
|
" ) AS prompt,\n",
|
|
" ticket_id,\n",
|
|
" description\n",
|
|
" FROM `bq_mm_tutorial.reports_mm`\n",
|
|
" ),\n",
|
|
" STRUCT(\n",
|
|
" \"issue STRING, secondary_issue STRING, urgency_score INT64, ai_summary_description STRING, recommended_action STRING, city_response_department STRING\" AS output_schema\n",
|
|
" )\n",
|
|
")\n",
|
|
"ORDER BY urgency_score DESC;"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "WL-v0ztHV59I"
|
|
},
|
|
"source": [
|
|
"### 6. Run a multimodal query on an ARRAY of `ObjectRef`s\n",
|
|
"\n",
|
|
"Of course, we can also aggregate `STRING` and `ObjectRef` fields as an array and pass them to a Gemini model too.\n",
|
|
"\n",
|
|
"In this example, we'll return the top issues that need attention in the North and South districts of the city. This maybe useful for citywide reporting."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/",
|
|
"height": 245,
|
|
"referenced_widgets": [
|
|
"3fe1652ecc8647e791bb92b359083c7b",
|
|
"2333507379c94604b0085c82d1dee027",
|
|
"c656c3e7e3674cfcbd6da8aedd6ab8ea",
|
|
"9f5f022f32354990a37d8792e9e5835c",
|
|
"33c78c57e7af4206a719a36fc40dcd51",
|
|
"5255edd87c294e76833fcf92b435fe14",
|
|
"e1e41d0228a44c1f863c148b1767d884",
|
|
"16c0578e35ca41598b32ce04cb41ec7e",
|
|
"606a489da1fc463583afe11c0507ef23",
|
|
"56d9703bee29498992ae0ab38971cef5",
|
|
"b55dd25b8cad49b999700f012dbeb204",
|
|
"4c6e8534504d45dcb9725ebd4930030b",
|
|
"498b73aebe394b6082564b8d7142ffad",
|
|
"d4b3eb88a9b6401ab0bf3715dc8dda0f",
|
|
"2d91ea4991f243af8f3df17b68067bcd",
|
|
"bdb7e257379c4d589775f8d70b8e3256",
|
|
"11c3c34bc07940f6bb3072a79cc9631f",
|
|
"a20b904217474868bdbadd00fd44c608",
|
|
"ad8c27470b924b47bda49ce9a7108bbe",
|
|
"6fe605d131a14af3b688dd830dd27cfe",
|
|
"758c1d812bca41b186c2e78cb808ee75",
|
|
"f765af49ffa14f98a3c23da874ac1994"
|
|
]
|
|
},
|
|
"id": "jotBZrmyV7TD",
|
|
"outputId": "5216d7f4-2871-47d0-c789-cfbd4e285a85"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "3fe1652ecc8647e791bb92b359083c7b",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Query is running: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "4c6e8534504d45dcb9725ebd4930030b",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Downloading: 0%| |"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.google.colaboratory.intrinsic+json": {
|
|
"summary": "{\n \"name\": \"get_ipython()\",\n \"rows\": 2,\n \"fields\": [\n {\n \"column\": \"district\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"North\",\n \"South\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"summary\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
|
|
"type": "dataframe"
|
|
},
|
|
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/9e65f7085e7ffcb7/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"South\",\n\"['A small pothole near the bus stop needs to be filled as it poses a hazard.'\\n 'A huge sinkhole on the road, described as potentially car-eating, needs immediate repair.'\\n 'A really big sinkhole is growing on the sidewalk near an apartment, raising concerns about people falling in.']\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"North\",\n\"['Massive pile of trash in the alley causing obstruction and attracting pests'\\n 'Leaky fire hydrant causing a puddle'\\n 'Abandoned car blocking the sidewalk and pedestrian access']\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"district\"], [\"string\", \"summary\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n <div id=\"df-f7ea97e0-e5d0-433a-b786-05b72c5ac67c\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f7ea97e0-e5d0-433a-b786-05b72c5ac67c')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n\n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n\n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-f7ea97e0-e5d0-433a-b786-05b72c5ac67c button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n </div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
|
|
"text/html": [
|
|
"\n",
|
|
" <div id=\"df-a5998f9f-e01f-44f0-90af-df53fd063933\" class=\"colab-df-container\">\n",
|
|
" <div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>district</th>\n",
|
|
" <th>summary</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>South</td>\n",
|
|
" <td>[A small pothole near the bus stop needs to be...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>North</td>\n",
|
|
" <td>[Massive pile of trash in the alley causing ob...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>\n",
|
|
" <div class=\"colab-df-buttons\">\n",
|
|
"\n",
|
|
" <div class=\"colab-df-container\">\n",
|
|
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a5998f9f-e01f-44f0-90af-df53fd063933')\"\n",
|
|
" title=\"Convert this dataframe to an interactive table.\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
|
|
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
|
|
" </svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
" <style>\n",
|
|
" .colab-df-container {\n",
|
|
" display:flex;\n",
|
|
" gap: 12px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert {\n",
|
|
" background-color: #E8F0FE;\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: #1967D2;\n",
|
|
" height: 32px;\n",
|
|
" padding: 0 0 0 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-convert:hover {\n",
|
|
" background-color: #E2EBFA;\n",
|
|
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: #174EA6;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-buttons div {\n",
|
|
" margin-bottom: 4px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert {\n",
|
|
" background-color: #3B4455;\n",
|
|
" fill: #D2E3FC;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-convert:hover {\n",
|
|
" background-color: #434B5C;\n",
|
|
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
|
|
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
|
|
" fill: #FFFFFF;\n",
|
|
" }\n",
|
|
" </style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" const buttonEl =\n",
|
|
" document.querySelector('#df-a5998f9f-e01f-44f0-90af-df53fd063933 button.colab-df-convert');\n",
|
|
" buttonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
"\n",
|
|
" async function convertToInteractive(key) {\n",
|
|
" const element = document.querySelector('#df-a5998f9f-e01f-44f0-90af-df53fd063933');\n",
|
|
" const dataTable =\n",
|
|
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
|
|
" [key], {});\n",
|
|
" if (!dataTable) return;\n",
|
|
"\n",
|
|
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
|
|
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
|
|
" + ' to learn more about interactive tables.';\n",
|
|
" element.innerHTML = '';\n",
|
|
" dataTable['output_type'] = 'display_data';\n",
|
|
" await google.colab.output.renderOutput(dataTable, element);\n",
|
|
" const docLink = document.createElement('div');\n",
|
|
" docLink.innerHTML = docLinkHtml;\n",
|
|
" element.appendChild(docLink);\n",
|
|
" }\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
"\n",
|
|
" <div id=\"df-fff0c8dc-324b-4a6d-a26f-cd9478e86e22\">\n",
|
|
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-fff0c8dc-324b-4a6d-a26f-cd9478e86e22')\"\n",
|
|
" title=\"Suggest charts\"\n",
|
|
" style=\"display:none;\">\n",
|
|
"\n",
|
|
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
|
|
" width=\"24px\">\n",
|
|
" <g>\n",
|
|
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
|
|
" </g>\n",
|
|
"</svg>\n",
|
|
" </button>\n",
|
|
"\n",
|
|
"<style>\n",
|
|
" .colab-df-quickchart {\n",
|
|
" --bg-color: #E8F0FE;\n",
|
|
" --fill-color: #1967D2;\n",
|
|
" --hover-bg-color: #E2EBFA;\n",
|
|
" --hover-fill-color: #174EA6;\n",
|
|
" --disabled-fill-color: #AAA;\n",
|
|
" --disabled-bg-color: #DDD;\n",
|
|
" }\n",
|
|
"\n",
|
|
" [theme=dark] .colab-df-quickchart {\n",
|
|
" --bg-color: #3B4455;\n",
|
|
" --fill-color: #D2E3FC;\n",
|
|
" --hover-bg-color: #434B5C;\n",
|
|
" --hover-fill-color: #FFFFFF;\n",
|
|
" --disabled-bg-color: #3B4455;\n",
|
|
" --disabled-fill-color: #666;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart {\n",
|
|
" background-color: var(--bg-color);\n",
|
|
" border: none;\n",
|
|
" border-radius: 50%;\n",
|
|
" cursor: pointer;\n",
|
|
" display: none;\n",
|
|
" fill: var(--fill-color);\n",
|
|
" height: 32px;\n",
|
|
" padding: 0;\n",
|
|
" width: 32px;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart:hover {\n",
|
|
" background-color: var(--hover-bg-color);\n",
|
|
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
|
|
" fill: var(--button-hover-fill-color);\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-quickchart-complete:disabled,\n",
|
|
" .colab-df-quickchart-complete:disabled:hover {\n",
|
|
" background-color: var(--disabled-bg-color);\n",
|
|
" fill: var(--disabled-fill-color);\n",
|
|
" box-shadow: none;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .colab-df-spinner {\n",
|
|
" border: 2px solid var(--fill-color);\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" animation:\n",
|
|
" spin 1s steps(1) infinite;\n",
|
|
" }\n",
|
|
"\n",
|
|
" @keyframes spin {\n",
|
|
" 0% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 20% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 30% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-left-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 40% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-top-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 60% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 80% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-right-color: var(--fill-color);\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" 90% {\n",
|
|
" border-color: transparent;\n",
|
|
" border-bottom-color: var(--fill-color);\n",
|
|
" }\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"\n",
|
|
" <script>\n",
|
|
" async function quickchart(key) {\n",
|
|
" const quickchartButtonEl =\n",
|
|
" document.querySelector('#' + key + ' button');\n",
|
|
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
|
|
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
|
|
" try {\n",
|
|
" const charts = await google.colab.kernel.invokeFunction(\n",
|
|
" 'suggestCharts', [key], {});\n",
|
|
" } catch (error) {\n",
|
|
" console.error('Error during call to suggestCharts:', error);\n",
|
|
" }\n",
|
|
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
|
|
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
|
|
" }\n",
|
|
" (() => {\n",
|
|
" let quickchartButtonEl =\n",
|
|
" document.querySelector('#df-fff0c8dc-324b-4a6d-a26f-cd9478e86e22 button');\n",
|
|
" quickchartButtonEl.style.display =\n",
|
|
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
|
|
" })();\n",
|
|
" </script>\n",
|
|
" </div>\n",
|
|
"\n",
|
|
" </div>\n",
|
|
" </div>\n"
|
|
],
|
|
"text/plain": [
|
|
" district summary\n",
|
|
"0 South [A small pothole near the bus stop needs to be...\n",
|
|
"1 North [Massive pile of trash in the alley causing ob..."
|
|
]
|
|
},
|
|
"execution_count": 43,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bigquery --project {PROJECT_ID}\n",
|
|
"\n",
|
|
"SELECT\n",
|
|
" district,\n",
|
|
" summary\n",
|
|
"FROM AI.GENERATE_TABLE(\n",
|
|
" MODEL `bq_mm_tutorial.gemini`,\n",
|
|
" (\n",
|
|
" SELECT (\n",
|
|
" 'Describe the primary issues that need to be fixed in this district'\n",
|
|
" , ARRAY_AGG(description), ARRAY_AGG(image_ref), ARRAY_AGG(video_ref), ARRAY_AGG(audio_ref)\n",
|
|
" ) AS prompt,\n",
|
|
" district\n",
|
|
" FROM `bq_mm_tutorial.reports_mm`\n",
|
|
" GROUP BY district\n",
|
|
" ),\n",
|
|
" STRUCT(\n",
|
|
" \"summary ARRAY<STRING>\" AS output_schema\n",
|
|
" )\n",
|
|
");"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "izKiUVyzbsvA"
|
|
},
|
|
"source": [
|
|
"### Recap\n",
|
|
"\n",
|
|
"In this scenario we:\n",
|
|
"* Began with two tables, `reports` and `media` containing structured data and `uri` columns pointing to GCS objects\n",
|
|
"* Explored how to programmatically create an `ObjectRef` using `OBJ.` functions.\n",
|
|
"* Created a multimodal table called `reports_mm`\n",
|
|
"* Ran several queries against the `reports_mm` table using structured data and one *or more* `ObjectRef` columns\n",
|
|
"* Ran a multimodal query against an ARRAY of `ObjectRef`s to produce aggregated insights"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "q_A4zzWousGx"
|
|
},
|
|
"source": [
|
|
"\n",
|
|
"---\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "cleanup_md"
|
|
},
|
|
"source": [
|
|
"# Cleaning Up\n",
|
|
"\n",
|
|
"To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
|
|
"\n",
|
|
"Otherwise, you can delete the individual resources you created in this tutorial:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "yRzpOQf_Wqqi"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Delete the BigQuery tables\n",
|
|
"! bq rm --table -f bq_mm_tutorial.object_table\n",
|
|
"! bq rm --table -f bq_mm_tutorial.calls\n",
|
|
"! bq rm --table -f bq_mm_tutorial.calls_combined\n",
|
|
"! bq rm --table -f bq_mm_tutorial.media\n",
|
|
"! bq rm --table -f bq_mm_tutorial.reports\n",
|
|
"! bq rm --table -f bq_mm_tutorial.reports_mm\n",
|
|
"\n",
|
|
"# Delete the remote model\n",
|
|
"! bq rm --model -f bq_mm_tutorial.gemini\n",
|
|
"\n",
|
|
"# Delete the remote connection\n",
|
|
"! bq rm --connection --project_id=$PROJECT_ID --location=us test_connection\n",
|
|
"\n",
|
|
"# Delete the BigQuery dataset\n",
|
|
"! bq rm -r -f $PROJECT_ID:bq_mm_tutorial"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"collapsed_sections": [
|
|
"td9kx9LVgSve"
|
|
],
|
|
"provenance": [],
|
|
"toc_visible": true
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"name": "python"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
}
|