wehub-resource-sync
785 lines
28 KiB
Plaintext
785 lines
28 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "ur8xi4C7S06n"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Copyright 2024 Google LLC\n",
|
||
"#\n",
|
||
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
||
"# you may not use this file except in compliance with the License.\n",
|
||
"# You may obtain a copy of the License at\n",
|
||
"#\n",
|
||
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
||
"#\n",
|
||
"# Unless required by applicable law or agreed to in writing, software\n",
|
||
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
||
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
||
"# See the License for the specific language governing permissions and\n",
|
||
"# limitations under the License."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "JAPoU8Sm5E6e"
|
||
},
|
||
"source": [
|
||
"# Guess who or what app using Hugging Face Deep Learning container model on Vertex AI\n",
|
||
"\n",
|
||
"<table align=\"left\">\n",
|
||
" <td style=\"text-align: center\">\n",
|
||
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/open-models/use-cases/guess_app.ipynb\">\n",
|
||
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
||
" </a>\n",
|
||
" </td>\n",
|
||
" <td style=\"text-align: center\">\n",
|
||
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fopen-models%2Fuse-cases%2Fguess_app.ipynb\">\n",
|
||
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
||
" </a>\n",
|
||
" </td>\n",
|
||
" <td style=\"text-align: center\">\n",
|
||
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/open-models/use-cases/guess_app.ipynb\">\n",
|
||
" <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
|
||
" </a>\n",
|
||
" </td>\n",
|
||
" <td style=\"text-align: center\">\n",
|
||
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/open-models/use-cases/guess_app.ipynb\">\n",
|
||
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
||
" </a>\n",
|
||
" </td>\n",
|
||
"</table>\n",
|
||
"\n",
|
||
"<div style=\"clear: both;\"></div>\n",
|
||
"\n",
|
||
"<b>Share to:</b>\n",
|
||
"\n",
|
||
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/open-models/use-cases/guess_app.ipynb\" target=\"_blank\">\n",
|
||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
||
"</a>\n",
|
||
"\n",
|
||
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/open-models/use-cases/guess_app.ipynb\" target=\"_blank\">\n",
|
||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
||
"</a>\n",
|
||
"\n",
|
||
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/open-models/use-cases/guess_app.ipynb\" target=\"_blank\">\n",
|
||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
||
"</a>\n",
|
||
"\n",
|
||
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/open-models/use-cases/guess_app.ipynb\" target=\"_blank\">\n",
|
||
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
||
"</a>\n",
|
||
"\n",
|
||
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/open-models/use-cases/guess_app.ipynb\" target=\"_blank\">\n",
|
||
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
||
"</a>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "84f0f73a0f76"
|
||
},
|
||
"source": [
|
||
"| | |\n",
|
||
"|-|-|\n",
|
||
"| Author(s) | [Ivan Nardini](https://github.com/inardini) |"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "tvgnzT1CKxrO"
|
||
},
|
||
"source": [
|
||
"## Overview\n",
|
||
"\n",
|
||
"This notebook shows how you can use Vertex AI and Hugging Face Deep Learning container to create a simple \"Guess who or what\" application which combines an image generation open model with Gemini to answer and represent subjects of Guess who or what riddles."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "61RBz8LLbxCR"
|
||
},
|
||
"source": [
|
||
"## Get started"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "No17Cw5hgx12"
|
||
},
|
||
"source": [
|
||
"### Install Vertex AI SDK and other required packages\n",
|
||
"\n",
|
||
"To run this example, you will only need the [`google-cloud-aiplatform`](https://github.com/googleapis/python-aiplatform) Python SDK and the [`huggingface_hub`](https://github.com/huggingface/huggingface_hub) Python package."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "tFy3H3aPgx12"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"%pip install --upgrade --user --quiet google-cloud-aiplatform huggingface_hub gradio"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "R5Xep4W9lq-Z"
|
||
},
|
||
"source": [
|
||
"### Restart runtime (Colab only)\n",
|
||
"\n",
|
||
"To use the newly installed packages in this Jupyter environment, if you are on Colab you must restart the runtime. You can do this by running the cell below, which restarts the current kernel. The restart might take a minute or longer. After it's restarted, continue to the next step."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "XRvKdaPDTznN"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Automatically restart kernel after installs so that your environment can access the new packages\n",
|
||
"# import IPython\n",
|
||
"\n",
|
||
"# app = IPython.Application.instance()\n",
|
||
"# app.kernel.do_shutdown(True)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "SbmM4z7FOBpM"
|
||
},
|
||
"source": [
|
||
"<div class=\"alert alert-block alert-warning\">\n",
|
||
"<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
|
||
"</div>\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "dmWOrTJ3gx13"
|
||
},
|
||
"source": [
|
||
"### Authenticate your Google Cloud account\n",
|
||
"\n",
|
||
"Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "9cc332aee5fd"
|
||
},
|
||
"source": [
|
||
"**1. Vertex AI Workbench**\n",
|
||
"\n",
|
||
"* Do nothing as you are already authenticated."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "54c81e1a63e6"
|
||
},
|
||
"source": [
|
||
"**2. Local JupyterLab instance, uncomment and run:**"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "c93d96bc9c43"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# !gcloud auth login"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "d3e571ce6c56"
|
||
},
|
||
"source": [
|
||
"**3. Colab, uncomment and run:**"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "984a0526fb68"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"from google.colab import auth\n",
|
||
"\n",
|
||
"auth.authenticate_user()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "f9af3e57f89a"
|
||
},
|
||
"source": [
|
||
"### Authenticate your Hugging Face account"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "4c31c7272804"
|
||
},
|
||
"source": [
|
||
"Then you can install the `huggingface_hub` that comes with a CLI that will be used for the authentication with the token generated in advance. So that then the token can be safely retrieved via `huggingface_hub.get_token`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "8d836e0210fe"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"from huggingface_hub import interpreter_login\n",
|
||
"\n",
|
||
"interpreter_login()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "c71a4314c250"
|
||
},
|
||
"source": [
|
||
"Read more about [Hugging Face Security](https://huggingface.co/docs/hub/en/security), specifically about [Hugging Face User Access Tokens](https://huggingface.co/docs/hub/en/security-tokens)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "DF4l8DTdWgPY"
|
||
},
|
||
"source": [
|
||
"### Set Google Cloud project information and initialize Vertex AI SDK\n",
|
||
"\n",
|
||
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com), if not enabled already.\n",
|
||
"\n",
|
||
"Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "Nqwi-5ufWp_B"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os\n",
|
||
"\n",
|
||
"from google.cloud import aiplatform\n",
|
||
"\n",
|
||
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\", isTemplate: true}\n",
|
||
"if PROJECT_ID == \"[your-project-id]\":\n",
|
||
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
||
"\n",
|
||
"LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
|
||
"\n",
|
||
"aiplatform.init(project=PROJECT_ID, location=LOCATION)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "ee37e1544281"
|
||
},
|
||
"source": [
|
||
"### Requirements"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "877cd3fb2dce"
|
||
},
|
||
"source": [
|
||
"To successfully deploy your Hugging Face model on Vertex AI, you need to have the following IAM roles set:\n",
|
||
"\n",
|
||
"- Artifact Registry Reader (`roles/artifactregistry.reader`)\n",
|
||
"- Vertex AI User (`roles/aiplatform.user`)\n",
|
||
"\n",
|
||
"For more information about granting roles, see [Manage access](https://cloud.google.com/iam/docs/granting-changing-revoking-access).\n",
|
||
"\n",
|
||
"---\n",
|
||
"\n",
|
||
"You also need to enable the following APIs (if not enabled already):\n",
|
||
"\n",
|
||
"- Vertex AI API (`aiplatform.googleapis.com`)\n",
|
||
"- Artifact Registry API (`artifactregistry.googleapis.com`)\n",
|
||
"\n",
|
||
"For more information about API enablement, see [Enabling APIs](https://cloud.google.com/apis/docs/getting-started#enabling_apis).\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "2pNoqLHVWG6X"
|
||
},
|
||
"source": [
|
||
"### Import libraries"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "V41joDcRWIzY"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"import base64\n",
|
||
"import io\n",
|
||
"import os\n",
|
||
"\n",
|
||
"from IPython.display import display\n",
|
||
"from PIL import Image\n",
|
||
"from google.cloud.aiplatform import Endpoint\n",
|
||
"import gradio as gr\n",
|
||
"from huggingface_hub import get_token\n",
|
||
"import vertexai\n",
|
||
"from vertexai.generative_models import GenerationConfig, GenerativeModel, SafetySetting"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "2qxTWPNu1p-d"
|
||
},
|
||
"source": [
|
||
"# Build a \"Guess what or who\" app\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "dJsGNCv4T6R-"
|
||
},
|
||
"source": [
|
||
"## Register FLUX on Vertex AI\n",
|
||
"\n",
|
||
"To deploy a \"Guess what or who\" application using a Hugging Face model like [FLUX](https://console.cloud.google.com/vertex-ai/publishers/black-forest-labs/model-garden/flux1-schnell) for image generation on Vertex AI, you first register the chosen model within the [Vertex AI Model Registry](https://cloud.google.com/vertex-ai/docs/model-registry/introduction). This registry serves as a central repository for managing your models on Vertex AI.\n",
|
||
"\n",
|
||
"Registering a model involves specifying a serving container image and corresponding environment variables, which vary depending on the chosen model. For instance, for FLUX, you'll use a regular PyTorch Inference Deep Learning Container. You can find the appropriate container URI and further details in the [Google Cloud Deep Learning Containers documentation](https://huggingface.co/docs/google-cloud/en/containers/introduction). "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "Pc-Owc_7WiZb"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"flux_model = aiplatform.Model.upload(\n",
|
||
" display_name=\"flux--generate\",\n",
|
||
" serving_container_image_uri=\"us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu121.2-2.ubuntu2204.py310\",\n",
|
||
" serving_container_environment_variables={\n",
|
||
" \"HF_MODEL_ID\": \"black-forest-labs/FLUX.1-dev\",\n",
|
||
" \"HF_TASK\": \"text-to-image\",\n",
|
||
" \"HF_TOKEN\": get_token(),\n",
|
||
" },\n",
|
||
")\n",
|
||
"flux_model.wait()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "c427c0a87016"
|
||
},
|
||
"source": [
|
||
"## Deploy Flux on Vertex AI\n",
|
||
"\n",
|
||
"After successfully registering the model, you can then deploy it to a Vertex AI Endpoint according to your preferred deployment configuration, making it ready for image generation.\n",
|
||
"\n",
|
||
"This deployment creates a new instance of the FLUX model in Vertex AI Prediction, the managed inference service on Vertex AI."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "sxI3SsHxVYn4"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"endpoint = aiplatform.Endpoint.create(display_name=\"flux--generate-endpoint\")\n",
|
||
"\n",
|
||
"deployed_flux_model = flux_model.deploy(\n",
|
||
" endpoint=endpoint,\n",
|
||
" machine_type=\"g2-standard-48\",\n",
|
||
" accelerator_type=\"NVIDIA_L4\",\n",
|
||
" accelerator_count=4,\n",
|
||
" sync=False,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "8lIr8uko04T_"
|
||
},
|
||
"source": [
|
||
"## Generate predictions\n",
|
||
"\n",
|
||
"After the model gets successfully deployed, you can test it by submitting a prediction request."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "ERBKKCPylaTt"
|
||
},
|
||
"source": [
|
||
"### Generate an image using FLUX"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "Hxw6otB3SBZE"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"response = deployed_flux_model.predict(\n",
|
||
" instances=[\"a image of a cat riding a horse in illustration style\"],\n",
|
||
" parameters={\n",
|
||
" \"width\": 512,\n",
|
||
" \"height\": 512,\n",
|
||
" \"num_inference_steps\": 8,\n",
|
||
" \"guidance_scale\": 3.5,\n",
|
||
" },\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "wAGzijoxOqvr"
|
||
},
|
||
"source": [
|
||
"### Get the generated image"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "-9vR2U6b12sK"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"image = Image.open(io.BytesIO(base64.b64decode(response.predictions[0])))\n",
|
||
"display(image)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "9b3fd1898241"
|
||
},
|
||
"source": [
|
||
"## Play \"Guess who or what\"\n",
|
||
"\n",
|
||
"With the image generation open model deployed in a Vertex AI Endpoint, you are now able to build your \"Guess who or what\" Gen AI application.\n",
|
||
"\n",
|
||
"In this scenario, you use Gradio to quickly build a web application for your Gen AI models."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "wkBl-jwH0_AJ"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Get enviroment variables\n",
|
||
"PROJECT_ID = os.environ.get(\"PROJECT_ID\", PROJECT_ID)\n",
|
||
"LOCATION = os.environ.get(\"LOCATION\", LOCATION)\n",
|
||
"MODEL_ID = os.environ.get(\"MODEL_ID\", \"gemini-2.0-flash\")\n",
|
||
"ENDPOINT_NAME = os.environ.get(\"ENDPOINT\", endpoint.resource_name)\n",
|
||
"\n",
|
||
"# Initialize Vertex AI SDK\n",
|
||
"aiplatform.init(project=PROJECT_ID, location=LOCATION)\n",
|
||
"vertexai.init(project=PROJECT_ID, location=LOCATION)\n",
|
||
"\n",
|
||
"# Initialize Gemini model\n",
|
||
"MODEL = GenerativeModel(\"gemini-2.0-flash\")\n",
|
||
"ENDPOINT = Endpoint(ENDPOINT_NAME)\n",
|
||
"\n",
|
||
"# Common generation config and safety settings\n",
|
||
"GENERATION_CONFIG = GenerationConfig(\n",
|
||
" candidate_count=1, max_output_tokens=8192, temperature=0\n",
|
||
")\n",
|
||
"\n",
|
||
"SAFETY_SETTINGS = [\n",
|
||
" SafetySetting(\n",
|
||
" category=SafetySetting.HarmCategory.HARM_CATEGORY_HATE_SPEECH,\n",
|
||
" threshold=SafetySetting.HarmBlockThreshold.OFF,\n",
|
||
" ),\n",
|
||
" SafetySetting(\n",
|
||
" category=SafetySetting.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,\n",
|
||
" threshold=SafetySetting.HarmBlockThreshold.OFF,\n",
|
||
" ),\n",
|
||
" SafetySetting(\n",
|
||
" category=SafetySetting.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,\n",
|
||
" threshold=SafetySetting.HarmBlockThreshold.OFF,\n",
|
||
" ),\n",
|
||
" SafetySetting(\n",
|
||
" category=SafetySetting.HarmCategory.HARM_CATEGORY_HARASSMENT,\n",
|
||
" threshold=SafetySetting.HarmBlockThreshold.OFF,\n",
|
||
" ),\n",
|
||
"]\n",
|
||
"\n",
|
||
"\n",
|
||
"def generate_gemini_content(prompt_template: str, **kwargs) -> str:\n",
|
||
" \"\"\"Generate content using Gemini model with given prompt template.\"\"\"\n",
|
||
" prompt = prompt_template.format(**kwargs)\n",
|
||
" response = MODEL.generate_content(\n",
|
||
" [prompt],\n",
|
||
" generation_config=GENERATION_CONFIG,\n",
|
||
" safety_settings=SAFETY_SETTINGS,\n",
|
||
" stream=False,\n",
|
||
" )\n",
|
||
" return response.text\n",
|
||
"\n",
|
||
"\n",
|
||
"def generate_subject(riddle: str) -> str:\n",
|
||
" \"\"\"Extract the subject/answer from a given riddle using Gemini model.\"\"\"\n",
|
||
" riddle_solver_prompt_template = \"\"\"\n",
|
||
" You are the best riddle solver. Given a riddle, your goal is solve it and only indicate the subject of the riddle.\n",
|
||
" RIDDLE: {riddle}\n",
|
||
" SUBJECT:\n",
|
||
" \"\"\"\n",
|
||
" subject = generate_gemini_content(riddle_solver_prompt_template, riddle=riddle)\n",
|
||
" return subject.replace(\"SUBJECT:\", \"\").strip()\n",
|
||
"\n",
|
||
"\n",
|
||
"def generate_prompt(subject: str) -> str:\n",
|
||
" \"\"\"Generate an image generation prompt for a given subject.\"\"\"\n",
|
||
" image_gen_prompt_template = \"\"\"\n",
|
||
" You are a professional prompt engineer. Given a subject, prototype the most appropriate prompt to best visualize the subject.\n",
|
||
" Only return the preferred prompt.\n",
|
||
" SUBJECT: {subject}\n",
|
||
" PROMPT:\n",
|
||
" \"\"\"\n",
|
||
" return generate_gemini_content(image_gen_prompt_template, subject=subject)\n",
|
||
"\n",
|
||
"\n",
|
||
"def generate_image(image_gen_prompt: str) -> Image.Image:\n",
|
||
" \"\"\"Generate an image based on the provided prompt.\"\"\"\n",
|
||
" response = ENDPOINT.predict(\n",
|
||
" instances=[image_gen_prompt],\n",
|
||
" parameters={\n",
|
||
" \"width\": 512,\n",
|
||
" \"height\": 512,\n",
|
||
" \"num_inference_steps\": 8,\n",
|
||
" \"guidance_scale\": 3.5,\n",
|
||
" },\n",
|
||
" )\n",
|
||
" return Image.open(io.BytesIO(base64.b64decode(response.predictions[0])))\n",
|
||
"\n",
|
||
"\n",
|
||
"def guess_game(riddle: str) -> tuple[Image.Image, str, str]:\n",
|
||
" \"\"\"Run the complete riddle-to-image game flow.\"\"\"\n",
|
||
" answer = generate_subject(riddle)\n",
|
||
" prompt = generate_prompt(answer)\n",
|
||
" image = generate_image(prompt)\n",
|
||
" return image, answer, prompt\n",
|
||
"\n",
|
||
"\n",
|
||
"def increment_counter(counter: int) -> tuple[int, Image.Image | None, str, str, str]:\n",
|
||
" \"\"\"Increment the game counter\"\"\"\n",
|
||
" return counter + 1, None, \"\", \"\", \"\"\n",
|
||
"\n",
|
||
"\n",
|
||
"def reset_game(counter: int) -> tuple[int, Image.Image | None, str, str, str]:\n",
|
||
" \"\"\"Reset the game state.\"\"\"\n",
|
||
" return counter, None, \"\", \"\", \"\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "5vzru9pIXxIT"
|
||
},
|
||
"source": [
|
||
"## Build the app\n",
|
||
"\n",
|
||
"Put together functions you define to build your Gradio app."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "OpZ0mbmzXirY"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Create main application block with Ocean theme\n",
|
||
"with gr.Blocks(theme=gr.themes.Ocean()) as app:\n",
|
||
"\n",
|
||
" # Header row containing title and counter state\n",
|
||
" with gr.Row():\n",
|
||
" gr.Markdown(\"# Guess What Game ❓\")\n",
|
||
" counter_state = gr.State(value=0)\n",
|
||
"\n",
|
||
" # Input row for user's riddle/description\n",
|
||
" with gr.Row():\n",
|
||
" prompt_input = gr.Textbox(label=\"Describe someone or something 💬 \")\n",
|
||
"\n",
|
||
" # Submit button to generate image and answer\n",
|
||
" submit_btn = gr.Button(\"Submit\")\n",
|
||
"\n",
|
||
" # Output row displaying generated content\n",
|
||
" with gr.Row():\n",
|
||
" image_prompt = gr.Textbox(\n",
|
||
" label=\"Generated Image Prompt with Gemini 🎨 \", visible=True\n",
|
||
" )\n",
|
||
" image_output = gr.Image(label=\"Generated Image 🖼️ \")\n",
|
||
" answer_output = gr.Textbox(\n",
|
||
" label=\"Generated Answer with Gemini 🌌 \", interactive=False\n",
|
||
" )\n",
|
||
"\n",
|
||
" # Game control buttons row\n",
|
||
" with gr.Row():\n",
|
||
" correct_btn = gr.Button(\"+1 Correct\")\n",
|
||
" reset_btn = gr.Button(\"Reset\")\n",
|
||
"\n",
|
||
" # Display for tracking correct guesses\n",
|
||
" counter_display = gr.Number(value=0, label=\"Correct Guesses 👍\", interactive=False)\n",
|
||
"\n",
|
||
" # Event handler for submit button\n",
|
||
" submit_btn.click(\n",
|
||
" guess_game,\n",
|
||
" inputs=[prompt_input],\n",
|
||
" outputs=[image_output, answer_output, image_prompt],\n",
|
||
" )\n",
|
||
"\n",
|
||
" # Event handler for correct button\n",
|
||
" correct_btn.click(\n",
|
||
" increment_counter,\n",
|
||
" inputs=[counter_state],\n",
|
||
" outputs=[\n",
|
||
" counter_state,\n",
|
||
" image_output,\n",
|
||
" answer_output,\n",
|
||
" image_prompt,\n",
|
||
" prompt_input,\n",
|
||
" ],\n",
|
||
" ).then(lambda x: x, inputs=[counter_state], outputs=[counter_display])\n",
|
||
"\n",
|
||
" # Event handler for reset button\n",
|
||
" reset_btn.click(\n",
|
||
" reset_game,\n",
|
||
" inputs=[counter_state],\n",
|
||
" outputs=[\n",
|
||
" counter_state,\n",
|
||
" image_output,\n",
|
||
" answer_output,\n",
|
||
" image_prompt,\n",
|
||
" prompt_input,\n",
|
||
" ],\n",
|
||
" ).then(lambda x: x, inputs=[counter_state], outputs=[counter_display])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "sbSdpVjoOSgD"
|
||
},
|
||
"source": [
|
||
"### Launch the app\n",
|
||
"\n",
|
||
"You are now ready to launch your \"Guess who or what\" Gradio app."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "cwUjdcP23Qmw"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"app.launch()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "2a4e033321ad"
|
||
},
|
||
"source": [
|
||
"## Cleaning up"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "iRfRH5fQc1IL"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"delete_app = True\n",
|
||
"delete_endpoint = True\n",
|
||
"delete_model = True\n",
|
||
"\n",
|
||
"if delete_app:\n",
|
||
" app.close()\n",
|
||
"\n",
|
||
"if delete_endpoint:\n",
|
||
" deployed_flux_model.delete()\n",
|
||
"\n",
|
||
"if delete_model:\n",
|
||
" flux_model.delete()"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"colab": {
|
||
"name": "guess_app.ipynb",
|
||
"toc_visible": true
|
||
},
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"name": "python3"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 0
|
||
}
|