wehub-resource-sync
740 lines
30 KiB
Plaintext
740 lines
30 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "KjTHAV8FgEza"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Copyright 2025 Google LLC\n",
|
|
"#\n",
|
|
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
|
"# you may not use this file except in compliance with the License.\n",
|
|
"# You may obtain a copy of the License at\n",
|
|
"#\n",
|
|
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
|
"#\n",
|
|
"# Unless required by applicable law or agreed to in writing, software\n",
|
|
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
|
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
|
"# See the License for the specific language governing permissions and\n",
|
|
"# limitations under the License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "UdL4uvQQs76x"
|
|
},
|
|
"source": [
|
|
"# Veo 3.1 Video Generation\n",
|
|
"\n",
|
|
"<table align=\"left\">\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo3_video_generation.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fvision%2Fgetting-started%2Fveo3_video_generation.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/vision/getting-started/veo3_video_generation.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo3_video_generation.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
"</table>\n",
|
|
"\n",
|
|
"<div style=\"clear: both;\"></div>\n",
|
|
"\n",
|
|
"<p>\n",
|
|
"<b>Share to:</b>\n",
|
|
"\n",
|
|
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo3_video_generation.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo3_video_generation.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo3_video_generation.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo3_video_generation.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo3_video_generation.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
|
"</a>\n",
|
|
"</p>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "lUCUMoTmN_lJ"
|
|
},
|
|
"source": [
|
|
"| | |\n",
|
|
"|-|-|\n",
|
|
"|Author(s) | [Katie Nguyen](https://github.com/katiemn) |"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "rDjAqcgigwdX"
|
|
},
|
|
"source": [
|
|
"## Overview\n",
|
|
"\n",
|
|
"### Veo 3.1\n",
|
|
"\n",
|
|
"Veo 3.1 on Agent Platform gives application developers access to Google's cutting-edge video generation. This model creates videos with stunning detail and realistic physics across a wide array of visual styles. Veo 3.1 enhances video quality from text and image prompts, and now includes dialogue and audio generation.\n",
|
|
"\n",
|
|
"In this tutorial, you will learn how to use the Google Gen AI SDK for Python to interact with Veo 3.1 and generate new videos with audio from:\n",
|
|
"- Enhanced text prompts\n",
|
|
"- Starting input images"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "v2_iOv5uhXVg"
|
|
},
|
|
"source": [
|
|
"## Get started"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "K4uerc9Xhf1f"
|
|
},
|
|
"source": [
|
|
"### Install Google Gen AI SDK for Python"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "rJyFNKoQhiwF"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install --upgrade --quiet google-genai"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "GWYnCW0-h6HI"
|
|
},
|
|
"source": [
|
|
"### Authenticate your notebook environment (Colab only)\n",
|
|
"\n",
|
|
"If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"id": "bqz5LUG6h8fA"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import sys\n",
|
|
"\n",
|
|
"if \"google.colab\" in sys.modules:\n",
|
|
" from google.colab import auth\n",
|
|
"\n",
|
|
" auth.authenticate_user()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "LVrasKoriKZn"
|
|
},
|
|
"source": [
|
|
"### Import libraries"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "yxBkUEqdiB1g"
|
|
},
|
|
"source": [
|
|
"### Set Google Cloud project information\n",
|
|
"\n",
|
|
"To get started using Agent Platform, you must have an existing Google Cloud project and [enable the Agent Platform API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
|
"\n",
|
|
"Learn more about [setting up a project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects) and a [development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"id": "GtjPBmYHiEfx"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import time\n",
|
|
"\n",
|
|
"import matplotlib.image as img\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"from IPython.display import Markdown, Video, display\n",
|
|
"from google import genai\n",
|
|
"from google.genai import types\n",
|
|
"\n",
|
|
"# fmt: off\n",
|
|
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
|
"# fmt: on\n",
|
|
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
|
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
|
"\n",
|
|
"LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
|
|
"\n",
|
|
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)\n",
|
|
"\n",
|
|
"gemini_client = genai.Client(enterprise=True, project=PROJECT_ID, location=\"global\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "qD_bwA9hiMzL"
|
|
},
|
|
"source": [
|
|
"### Define a helper function to display media"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"id": "GUrEwbvFiPhJ"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def show_video(video):\n",
|
|
" if isinstance(video, str):\n",
|
|
" file_name = video.split(\"/\")[-1]\n",
|
|
" !gcloud storage cp {video} {file_name}\n",
|
|
" display(Video(file_name, embed=True, width=600))\n",
|
|
" else:\n",
|
|
" with open(\"sample.mp4\", \"wb\") as out_file:\n",
|
|
" out_file.write(video)\n",
|
|
" display(Video(\"sample.mp4\", embed=True, width=600))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "2jaSOOadiUj6"
|
|
},
|
|
"source": [
|
|
"### Load the video generation model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"id": "APRfTklCiYR2"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"video_model = \"veo-3.1-generate-001\"\n",
|
|
"video_model_fast = \"veo-3.1-fast-generate-001\"\n",
|
|
"video_model_lite = \"veo-3.1-lite-generate-001\"\n",
|
|
"gemini_model = \"gemini-3.5-flash\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "H_R1_Y76i4QB"
|
|
},
|
|
"source": [
|
|
"## Generate videos\n",
|
|
"Now, you'll generate videos from text and/or image prompts. You can get started with your own prompts or complete the next section to optimize your prompts with some established best practices.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "v9ERu5b3EEdp"
|
|
},
|
|
"source": [
|
|
"### [Optional] Optimize your prompt: Text-to-video\n",
|
|
"\n",
|
|
"By considering the following options in your prompt, you can use Veo to create higher quality videos that more closely resemble your desired outcome. Learn more about advanced [prompting techniques for Veo 3.1](https://cloud.google.com/blog/products/ai-machine-learning/ultimate-prompting-guide-for-veo-3-1). To get started specify the following, or leave them as `None` if they don't align with your specific goals.\n",
|
|
"- **Subject:** The \"who\" or \"what\" of your video\n",
|
|
"- **Action:** Describe movements, interactions, etc.\n",
|
|
"- **Scene:** The \"where\" and \"when\" of your video\n",
|
|
"- **Camera angles:** The shot's viewpoint\n",
|
|
"- **Camera movements:** For a more cinematic/dynamic experience\n",
|
|
"- **Lens effects:** How the camera \"sees\" the world\n",
|
|
"- **Style:** The video's artistic filter\n",
|
|
"- **Temporal elements:** To imply changes in time\n",
|
|
"- **Audio:** Various sound effects or dialogue that guides the visuals through sound"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"id": "yx7DKfqri3Vf"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"subject = \"a detective\" # @param {type: 'string'}\n",
|
|
"action = \"interrogating a rubber duck\" # @param {type: 'string'}\n",
|
|
"scene = \"in a dark interview room\" # @param {type: 'string'}\n",
|
|
"\n",
|
|
"# fmt: off\n",
|
|
"camera_angle = \"Over-the-Shoulder Shot\" # @param [\"None\", \"Eye-Level Shot\", \"Low-Angle Shot\", \"High-Angle Shot\", \"Bird's-Eye View\", \"Top-Down Shot\", \"Worm's-Eye View\", \"Dutch Angle\", \"Canted Angle\", \"Close-Up\", \"Extreme Close-Up\", \"Medium Shot\", \"Full Shot\", \"Long Shot\", \"Wide Shot\", \"Establishing Shot\", \"Over-the-Shoulder Shot\", \"Point-of-View (POV) Shot\"]\n",
|
|
"camera_movement = \"Zoom (In)\" # @param [\"None\", \"Static Shot (or fixed)\", \"Pan (left)\", \"Pan (right)\", \"Tilt (up)\", \"Tilt (down)\", \"Dolly (In)\", \"Dolly (Out)\", \"Zoom (In)\", \"Zoom (Out)\", \"Truck (Left)\", \"Truck (Right)\", \"Pedestal (Up)\", \"Pedestal (Down)\", \"Crane Shot\", \"Aerial Shot\", \"Drone Shot\", \"Handheld\", \"Shaky Cam\", \"Whip Pan\", \"Arc Shot\"]\n",
|
|
"lens_effects = \"None\" # @param [\"None\", \"Wide-Angle Lens (e.g., 24mm)\", \"Telephoto Lens (e.g., 85mm)\", \"Shallow Depth of Field\", \"Bokeh\", \"Deep Depth of Field\", \"Lens Flare\", \"Rack Focus\", \"Fisheye Lens Effect\", \"Vertigo Effect (Dolly Zoom)\"]\n",
|
|
"style = \"Cinematic\" # @param [\"None\", \"Photorealistic\", \"Cinematic\", \"Vintage\", \"Japanese anime style\", \"Claymation style\", \"Stop-motion animation\", \"In the style of Van Gogh\", \"Surrealist painting\", \"Monochromatic black and white\", \"Vibrant and saturated\", \"Film noir style\", \"High-key lighting\", \"Low-key lighting\", \"Golden hour glow\", \"Volumetric lighting\", \"Backlighting to create a silhouette\"]\n",
|
|
"temporal_elements = \"None\" # @param [\"None\", \"Slow-motion\", \"Fast-paced action\", \"Time-lapse\", \"Hyperlapse\", \"Pulsating light\", \"Rhythmic movement\"]\n",
|
|
"\n",
|
|
"sound_effects = \"Ticking clock\" # @param [\"None\", \"Sound of a phone ringing\", \"Water splashing\", \"Soft house sounds\", \"Ticking clock\", \"City traffic and sirens\", \"Waves crashing\", \"Quiet office hum\"]\n",
|
|
"# fmt: on\n",
|
|
"dialogue = \"Where were you last night?\" # @param {type: 'string'}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "PMFj078dLG0D"
|
|
},
|
|
"source": [
|
|
"Now, you'll use Gemini to take all of these keywords and combine them into a detailed Veo prompt."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "f3jvksummytE"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\n",
|
|
"\n",
|
|
"keywords = [subject, action, scene]\n",
|
|
"optional_keywords = [\n",
|
|
" camera_angle,\n",
|
|
" camera_movement,\n",
|
|
" lens_effects,\n",
|
|
" style,\n",
|
|
" temporal_elements,\n",
|
|
" sound_effects,\n",
|
|
"]\n",
|
|
"for keyword in optional_keywords:\n",
|
|
" if keyword != \"None\":\n",
|
|
" keywords.append(keyword)\n",
|
|
"if dialogue != \"\":\n",
|
|
" keywords.append(dialogue)\n",
|
|
"\n",
|
|
"gemini_prompt = f\"\"\"\n",
|
|
"You are an expert video prompt engineer for Google's Veo model. Your task is to construct the most effective and optimal prompt string using the following keywords. Every single keyword MUST be included. Synthesize them into a single, cohesive, and cinematic instruction. Do not add any new core concepts. Output ONLY the final prompt string, without any introduction or explanation. Mandatory Keywords: {\",\".join(keywords)}\n",
|
|
"\"\"\"\n",
|
|
"response = gemini_client.models.generate_content(\n",
|
|
" model=gemini_model,\n",
|
|
" contents=gemini_prompt,\n",
|
|
")\n",
|
|
"\n",
|
|
"# Set Gemini's response in a prompt variable\n",
|
|
"prompt = response.text\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "aDaTx8WCidRG"
|
|
},
|
|
"source": [
|
|
"### Generate videos from a text prompt\n",
|
|
"\n",
|
|
"With Veo 3.1, you have the option to generate videos from a text prompt. In order to generate a video in the following sample, specify the following info:\n",
|
|
"- **Prompt:** A detailed description of the video you would like to see. Only edit the prompt if you didn't generate a detailed prompt with Gemini in the previous section.\n",
|
|
"- **Audio generation:** Set `generate_audio` to True if you'd like audio to be included in the output video.\n",
|
|
"- **Aspect ratio:** 16:9 or 9:16.\n",
|
|
"- **Number of videos:** Set this value to 1 or 2.\n",
|
|
"- **Video duration:** 4, 6, or 8 seconds.\n",
|
|
"- **Resolution:** Can be 1080p, 720p or 4k.\n",
|
|
"\n",
|
|
"**NOTE:** Generating a video in 4k will introduce increased latency up to several minutes."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"id": "W_iqdJNGl2Sa"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"if prompt == \"\":\n",
|
|
" # fmt: off\n",
|
|
" prompt = \"a garden gnome singing a pop song in a whimsical outdoor garden\" # @param {type: 'string'}\n",
|
|
"# fmt: on"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "tygfLLlWyTo_"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"print(prompt)\n",
|
|
"\n",
|
|
"operation = client.models.generate_videos(\n",
|
|
" model=video_model,\n",
|
|
" prompt=prompt,\n",
|
|
" config=types.GenerateVideosConfig(\n",
|
|
" aspect_ratio=\"16:9\",\n",
|
|
" number_of_videos=1,\n",
|
|
" duration_seconds=6,\n",
|
|
" resolution=\"1080p\",\n",
|
|
" person_generation=\"allow_adult\",\n",
|
|
" generate_audio=True,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"while not operation.done:\n",
|
|
" time.sleep(15)\n",
|
|
" operation = client.operations.get(operation)\n",
|
|
" print(operation)\n",
|
|
"\n",
|
|
"if operation.response:\n",
|
|
" show_video(operation.result.generated_videos[0].video.video_bytes)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "688nb6GEwqR4"
|
|
},
|
|
"source": [
|
|
"- **Veo 3.1 Fast:** For this next example, you'll use Veo 3.1 Fast. This model is a great option for use cases where latency is a priority.\n",
|
|
"- **File location:** In this request, the video will be stored in Cloud Storage once video generation is complete. Specify the bucket path where you would like this video to be stored in the `output_gcs` field.\n",
|
|
"- **Person generation:** When generating videos of people you can also set the `person_generation` parameter accordingly: `allow_adult`, `dont_allow`"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "sp0K0WYUwxLJ"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# fmt: off\n",
|
|
"prompt = \"sculpting a bowl on a pottery wheel while the person sculpting remarks: wow, I love this one\" # @param {type: 'string'}\n",
|
|
"# fmt: on\n",
|
|
"output_gcs = \"gs://[your-bucket-path]\" # @param {type: 'string'}\n",
|
|
"\n",
|
|
"\n",
|
|
"operation = client.models.generate_videos(\n",
|
|
" model=video_model_fast,\n",
|
|
" prompt=prompt,\n",
|
|
" config=types.GenerateVideosConfig(\n",
|
|
" aspect_ratio=\"9:16\",\n",
|
|
" output_gcs_uri=output_gcs,\n",
|
|
" number_of_videos=1,\n",
|
|
" duration_seconds=8,\n",
|
|
" resolution=\"720p\",\n",
|
|
" person_generation=\"allow_adult\",\n",
|
|
" generate_audio=True,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"while not operation.done:\n",
|
|
" time.sleep(15)\n",
|
|
" operation = client.operations.get(operation)\n",
|
|
" print(operation)\n",
|
|
"\n",
|
|
"if operation.response:\n",
|
|
" show_video(operation.result.generated_videos[0].video.uri)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "8494d440c7d4"
|
|
},
|
|
"source": [
|
|
"- **Veo 3.1 Lite:** In the next example, you'll use Veo 3.1 Lite, which is ideal for rapid prototyping and iteration."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "dc95738712eb"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\"\n",
|
|
"A sleek, modern white swivel chair begins to rotate smoothly in a full 360-degree circle. As it spins, a vibrant purple glow gradually washes over the fabric, transitioning the chair from pure white to a deep royal purple by the time it completes its rotation.\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"operation = client.models.generate_videos(\n",
|
|
" model=video_model_lite,\n",
|
|
" prompt=prompt,\n",
|
|
" config=types.GenerateVideosConfig(\n",
|
|
" aspect_ratio=\"16:9\",\n",
|
|
" number_of_videos=1,\n",
|
|
" duration_seconds=8,\n",
|
|
" resolution=\"720p\",\n",
|
|
" person_generation=\"allow_adult\",\n",
|
|
" generate_audio=True,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"while not operation.done:\n",
|
|
" time.sleep(10)\n",
|
|
" operation = client.operations.get(operation)\n",
|
|
" print(operation)\n",
|
|
"\n",
|
|
"if operation.response:\n",
|
|
" show_video(operation.result.generated_videos[0].video.video_bytes)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "YysYLyiVj8Zd"
|
|
},
|
|
"source": [
|
|
"### Generate videos from an image\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "X-NdoBONKpJD"
|
|
},
|
|
"source": [
|
|
"#### Download the starting image\n",
|
|
"\n",
|
|
"You can also generate a video by starting with an input image. In this example, you'll locally download an image that's stored in Google Cloud Storage. If you'd like, you can add the URL of the image you'd like to use to display it below. If you have a local image you'd like to use, you can specify that in the following steps."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "h_X5dS1Rye2Q"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/flowers.png"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "WGtJsg-MPSXi"
|
|
},
|
|
"source": [
|
|
"If you'd like to use a different local image, modify the file name in `starting_image`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "cl_lTdvEIt8k"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"starting_image = \"flowers.png\" # @param {type: 'string'}\n",
|
|
"\n",
|
|
"# Display the image\n",
|
|
"fig, axis = plt.subplots(1, 1, figsize=(12, 6))\n",
|
|
"axis.imshow(img.imread(starting_image))\n",
|
|
"axis.set_title(\"Starting Frame\")\n",
|
|
"axis.axis(\"off\")\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "FsHmRk-_NYya"
|
|
},
|
|
"source": [
|
|
"#### [Optional] Optimize your prompt: Image-to-video\n",
|
|
"\n",
|
|
"By considering the following options in your prompt, you can use Veo to create higher quality videos that more closely resemble your desired outcome. Learn more about advanced [prompting techniques for Veo 3.1](https://cloud.google.com/blog/products/ai-machine-learning/ultimate-prompting-guide-for-veo-3-1). To get started specify the following, or leave them as `None` if they don't align with your specific goals.\n",
|
|
"- **Camera motion:** The camera movement that occurs while the rest of the scene remains static\n",
|
|
"- **Subject animation:** Main character or object movement \n",
|
|
"- **Environmental animation:** Background or atmosphere movement\n",
|
|
"- **Audio:** Various sound effects or dialogue that guides the visuals through sound"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "6UE7AfftqSM5"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# fmt: off\n",
|
|
"camera_motion = \"Zoom (In)\" # @param [\"None\", \"Eye-Level Shot\", \"Low-Angle Shot\", \"High-Angle Shot\", \"Bird's-Eye View\", \"Top-Down Shot\", \"Worm's-Eye View\", \"Dutch Angle\", \"Canted Angle\", \"Close-Up\", \"Extreme Close-Up\", \"Medium Shot\", \"Full Shot\", \"Long Shot\", \"Wide Shot\", \"Establishing Shot\", \"Over-the-Shoulder Shot\", \"Point-of-View (POV) Shot\", \"Static Shot (or fixed)\", \"Pan (left)\", \"Pan (right)\", \"Tilt (up)\", \"Tilt (down)\", \"Dolly (In)\", \"Dolly (Out)\", \"Zoom (In)\", \"Zoom (Out)\", \"Truck (Left)\", \"Truck (Right)\", \"Pedestal (Up)\", \"Pedestal (Down)\", \"Crane Shot\", \"Aerial Shot\", \"Drone Shot\", \"Handheld\", \"Shaky Cam\", \"Whip Pan\", \"Arc Shot\", \"Wide-Angle Lens (e.g., 24mm)\", \"Telephoto Lens (e.g., 85mm)\", \"Shallow Depth of Field\", \"Bokeh\", \"Deep Depth of Field\", \"Lens Flare\", \"Rack Focus\", \"Fisheye Lens Effect\", \"Vertigo Effect (Dolly Zoom)\"]\n",
|
|
"\n",
|
|
"subject_animation = \"None\" # @param [\"None\", \"The subject's head turns slowly\", \"The subject blinks slowly\", \"The subject's hair and clothes flutter gently in the wind\", \"A subtle smile appears on the subject's face\"]\n",
|
|
"environmental_animation = \"Light changes subtly\" # @param [\"None\", \"Fog rolls in slowly\", \"Rain starts to fall gently\", \"Leaves rustle in the wind\", \"Light changes subtly\", \"Reflections move on water\"]\n",
|
|
"\n",
|
|
"sound_effects = \"None\" # @param [\"None\", \"Sound of a phone ringing\", \"Water splashing\", \"Soft house sounds\", \"Ticking clock\", \"City traffic and sirens\", \"Waves crashing\", \"Quiet office hum\"]\n",
|
|
"# fmt: on\n",
|
|
"dialogue = \"\" # @param {type: 'string'}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "nG4tknxOR70c"
|
|
},
|
|
"source": [
|
|
"Now, you'll use Gemini to take all of these keywords and combine them into a detailed Veo prompt."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "a1NvfHEHrx2m"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\n",
|
|
"\n",
|
|
"keywords = []\n",
|
|
"optional_keywords = [\n",
|
|
" camera_motion,\n",
|
|
" subject_animation,\n",
|
|
" environmental_animation,\n",
|
|
" sound_effects,\n",
|
|
"]\n",
|
|
"for keyword in optional_keywords:\n",
|
|
" if keyword != \"None\":\n",
|
|
" keywords.append(keyword)\n",
|
|
"if dialogue != \"\":\n",
|
|
" keywords.append(dialogue)\n",
|
|
"\n",
|
|
"gemini_prompt = f\"\"\"\n",
|
|
"You are an expert prompt engineer for Google's Veo model. Analyze the provided image and combine its content with the following motion and audio keywords to generate a single, cohesive, and cinematic prompt. Integrate the image's subject and scene with the requested motion and audio effects. The final output must be ONLY the prompt itself, with no preamble. Mandatory Keywords: {\",\".join(keywords)}\n",
|
|
"\"\"\"\n",
|
|
"with open(starting_image, \"rb\") as f:\n",
|
|
" image = f.read()\n",
|
|
"\n",
|
|
"response = gemini_client.models.generate_content(\n",
|
|
" model=gemini_model,\n",
|
|
" contents=[gemini_prompt, types.Part.from_bytes(data=image, mime_type=\"image/png\")],\n",
|
|
")\n",
|
|
"\n",
|
|
"# Set Gemini's response in a prompt variable\n",
|
|
"prompt = response.text\n",
|
|
"display(Markdown(response.text))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "mCc77j59KjGN"
|
|
},
|
|
"source": [
|
|
"#### Send the video request\n",
|
|
"\n",
|
|
"If you're generating a video from an image you don't need to provide a prompt. The model will simply add motion to your image; however, a detailed prompt will help with video quality.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "thPYb0eNtwxN"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"if prompt == \"\":\n",
|
|
" # fmt: off\n",
|
|
" prompt = \"zoom out of the flower field, play whimsical music\" # @param {type: 'string'}\n",
|
|
"# fmt: on"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "tFgQYeXHnidx"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"print(prompt)\n",
|
|
"\n",
|
|
"operation = client.models.generate_videos(\n",
|
|
" model=video_model,\n",
|
|
" prompt=prompt,\n",
|
|
" image=types.Image.from_file(location=starting_image),\n",
|
|
" config=types.GenerateVideosConfig(\n",
|
|
" aspect_ratio=\"16:9\",\n",
|
|
" number_of_videos=1,\n",
|
|
" duration_seconds=6,\n",
|
|
" resolution=\"1080p\",\n",
|
|
" person_generation=\"allow_adult\",\n",
|
|
" generate_audio=True,\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"while not operation.done:\n",
|
|
" time.sleep(15)\n",
|
|
" operation = client.operations.get(operation)\n",
|
|
" print(operation)\n",
|
|
"\n",
|
|
"if operation.response:\n",
|
|
" show_video(operation.result.generated_videos[0].video.video_bytes)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"name": "veo3_video_generation.ipynb",
|
|
"toc_visible": true
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"name": "python3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
}
|