wehub/googlecloudplatform--generative-ai
1
0
Fork 0
Code Issues Pull Requests Actions 1 Deployments Models Agent Notes Packages Projects Releases Wiki Activity
Files
main
googlecloudplatform--genera…/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb
T
wehub-resource-sync 914fea506e chore: import upstream snapshot with attribution
2026-07-13 13:30:30 +08:00

1180 lines
59 KiB
Plaintext
Raw Permalink Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ur8xi4C7S06n"
},
"outputs": [],
"source": [
"# Copyright 2025 Google LLC\n",
"#\n",
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JAPoU8Sm5E6e"
},
"source": [
"# Data Curation Pipeline: Splitting and Transcoding \n",
"\n",
"<table align=\"left\">\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb\">\n",
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fuse-cases%2Fmultimodal-data-curation%2Fsplitting_and_transcoding.ipynb\">\n",
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb\">\n",
" <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb\">\n",
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
" </a>\n",
" </td>\n",
"</table>\n",
"\n",
"<div style=\"clear: both;\"></div>\n",
"\n",
"<b>Share to:</b>\n",
"\n",
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/multimodal-data-curation/splitting_and_transcoding.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
"</a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "84f0f73a0f76"
},
"source": [
"| Author(s) |\n",
"| --- |\n",
"| [Noa Ben-Efraim](https://github.com/noabenefraim) |"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tvgnzT1CKxrO"
},
"source": [
"## Overview\n",
"\n",
"Video data presents a unique challenge and opportunity for multi-modal learning. Its sheer scale and temporal redundancy necessitate careful consideration when building datasets for Video Language Models (VLMs). Unlike static images, videos often contain lengthy sequences with minimal informational variance or semantic significance. Training foundation models on such raw data can lead to computational bottlenecks and hinder the model's ability to learn meaningful associations between visual and linguistic cues.\n",
"\n",
"This notebook explores the critical step of video splitting and transcoding involved in constructing effective data curation pipelines tailored for VLM pre-training. By focusing on extracting the most information-dense and semantically relevant segments, we can achieve significant gains in computational efficiency, optimize memory utilization during training, and empower models to capture more fine-grained temporal relationships. We will use a subset VIDGEN-1M to illustrate these steps in the data curation pipeline. The following sections will outline practical approaches to building a VLM data curation pipeline, covering aspects from initial video processing to advanced techniques for identifying and isolating key visual narratives:\n",
"\n",
"+ Data overview\n",
"+ Why video splitting?\n",
"+ Video splitting strategies \n",
"+ Transcoding overview"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "61RBz8LLbxCR"
},
"source": [
"## Get started"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "No17Cw5hgx12"
},
"source": [
"### Install Google Gen AI SDK and other required packages\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "tFy3H3aPgx12"
},
"outputs": [],
"source": [
"%pip install --upgrade --quiet google-genai"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dmWOrTJ3gx13"
},
"source": [
"### Authenticate your notebook environment (Colab only)\n",
"\n",
"If you're running this notebook on Google Colab, run the cell below to authenticate your environment."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NyKGtVQjgx13"
},
"outputs": [],
"source": [
"import sys\n",
"\n",
"if \"google.colab\" in sys.modules:\n",
" from google.colab import auth\n",
"\n",
" auth.authenticate_user()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DF4l8DTdWgPY"
},
"source": [
"### Set Google Cloud project information\n",
"\n",
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
"\n",
"Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Nqwi-5ufWp_B"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/usr/local/google/home/noabe/projects/data_curation/video-data-curation/.venv_data_curation/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"# Use the environment variable if the user doesn't provide Project ID.\n",
"import os\n",
"\n",
"from google import genai\n",
"from google.cloud import storage\n",
"\n",
"# fmt: off\n",
"PROJECT_ID = \"\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
"# fmt: on\n",
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
"\n",
"LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
"\n",
"bucket = \"vidgen-1m\"\n",
"storage_client = storage.Client()\n",
"\n",
"client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5303c05f7aa6"
},
"source": [
"### Import libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "6fc324893334"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/usr/local/google/home/noabe/projects/data_curation/video-data-curation/.venv_data_curation/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"import glob\n",
"import io\n",
"import json\n",
"import os\n",
"import shutil\n",
"import subprocess\n",
"import tempfile\n",
"\n",
"from google.api_core.exceptions import NotFound\n",
"from google.cloud import storage\n",
"from google.cloud.transcoder_v1.services.transcoder_service import (\n",
" TranscoderServiceClient,\n",
")\n",
"from scenedetect import SceneManager, open_video\n",
"from scenedetect.detectors import ContentDetector"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c98ed44596e4"
},
"source": [
"## Metadata Extraction and Filtering\n",
"\n",
"Metadata extraction is an important step to extract essential metadata (duration, resolution, format) using ffmpeg or a similar library. Then we can filter out videos that do not meet the criteria for VLM training.\n",
"\n",
"* Example: Minimum duration of 10 seconds.\n",
"* Example: Minimum resolution of 240p.\n",
"\n",
"In this notebook, we are introducing the library __ffmpeg__. FFmpeg is a command-line tool and library suite for handling multimedia data. Its general uses include transcoding between various video/audio formats, resizing and scaling resolutions, extracting components like audio streams or individual frames, and performing basic editing operations such as cutting or concatenating video segments. In a video data curation pipeline, its role is critical for robust transcoding, ensuring raw video files are converted into consistent, compatible formats, and for precise splitting and manipulation, which are fundamental steps before quality filtering and captioning can begin."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1bbf302acc2a"
},
"source": [
"### Load Video Data\n",
"\n",
"This notebook will use a subset of videos from the [VidGen-1M dataset](https://arxiv.org/abs/2408.02629).\n",
"\n",
"First we will load the videos from the GCS bucket."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "738f0e37d178"
},
"outputs": [],
"source": [
"def list_videos_in_bucket(bucket_name, prefix):\n",
" \"\"\"Lists video files in a Google Cloud Storage bucket/\n",
"\n",
" Args:\n",
" bucket_name (str): The name of the Google Cloud Storage bucket.\n",
" \"\"\"\n",
" blobs = storage_client.bucket(bucket_name).list_blobs(prefix=prefix)\n",
" video_files = [blob.name for blob in blobs if blob.name.lower().endswith(\".mp4\")]\n",
"\n",
" if not video_files:\n",
" print(\"No video files found in the specified bucket (or with the prefix).\")\n",
" return []\n",
" print(f\"Found {len(video_files)} video file(s).\")\n",
" return video_files"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1c2d2a229fa3"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 492 video file(s).\n"
]
}
],
"source": [
"# TODO larger video files\n",
"video_files = list_videos_in_bucket(bucket_name=\"vidgen-1m\", prefix=\"VidGen_video_0\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "01e9a9580d29"
},
"source": [
"Now, that we have our video files. We perform the following steps:\n",
"\n",
"1. Metadata Extraction: The get_video_metadata function uses ffprobe to extract essential information from video files stored in Google Cloud Storage, such as duration, format, and stream details (codec, resolution). \n",
"\n",
"2. Video Filtering: The filter_videos function takes the extracted metadata and filters the video files based on user-defined criteria, specifically minimum duration and resolution."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "e47fdf88d121"
},
"outputs": [],
"source": [
"def get_video_metadata(bucket_name, blob_name):\n",
" \"\"\"Retrieves video metadata from a video file stored in Google Cloud Storage.\n",
"\n",
" Args:\n",
" bucket_name (str): The name of the Google Cloud Storage bucket.\n",
" blob_name (str): The name of the video file (blob) within the bucket.\n",
"\n",
" Returns:\n",
" dict: A dictionary containing video metadata, including:\n",
" - 'duration' (float): The length of the video in seconds.\n",
" - 'streams' (list): A list of stream dictionaries, containing information\n",
" about video, audio, and other streams. Useful for codec info,\n",
" resolution, etc.\n",
" - 'format' (dict): Contains overall format information about the\n",
" video file.\n",
" Returns None if any error occurs.\n",
" \"\"\"\n",
" storage_client = storage.Client()\n",
" bucket = storage_client.bucket(bucket_name)\n",
" blob = bucket.blob(blob_name)\n",
"\n",
" try:\n",
" video_data = blob.download_as_bytes()\n",
" video_stream = io.BytesIO(video_data)\n",
"\n",
" command = [\n",
" \"ffprobe\",\n",
" \"-i\",\n",
" \"pipe:0\",\n",
" \"-analyzeduration\",\n",
" \"2\",\n",
" \"-probesize\",\n",
" \"32\",\n",
" \"-show_format\",\n",
" \"-show_streams\",\n",
" \"-print_format\",\n",
" \"json\",\n",
" ]\n",
"\n",
" process = subprocess.Popen(\n",
" command,\n",
" stdin=subprocess.PIPE,\n",
" stdout=subprocess.PIPE,\n",
" stderr=subprocess.PIPE,\n",
" )\n",
" output, error = process.communicate(input=video_stream.getvalue())\n",
"\n",
" if error:\n",
" error_string = error.decode(\"utf-8\")\n",
" if \"partial file\" in error_string:\n",
" print(\n",
" f\"ffprobe error (partial file): {error_string}. Attempting to extract partial metadata.\"\n",
" )\n",
" else:\n",
" print(f\"ffprobe error: {error_string}\")\n",
" return None\n",
"\n",
" try:\n",
" metadata = json.loads(output.decode(\"utf-8\"))\n",
" duration = None\n",
" if \"format\" in metadata and \"duration\" in metadata[\"format\"]:\n",
" duration = float(metadata[\"format\"][\"duration\"])\n",
" metadata_dict = {\n",
" \"duration\": duration,\n",
" \"streams\": metadata.get(\"streams\", []),\n",
" \"format\": metadata.get(\"format\", {}),\n",
" \"filename\": blob_name,\n",
" }\n",
" return metadata_dict\n",
" except json.JSONDecodeError:\n",
" print(f\"Error decoding ffprobe output for {blob_name}. Skipping file.\")\n",
" return None\n",
"\n",
" except NotFound:\n",
" print(f\"File not found: {blob_name} in bucket {bucket_name}\")\n",
" return None\n",
" except Exception as e:\n",
" print(f\"Error processing {blob_name}: {e}\")\n",
" return None\n",
"\n",
"\n",
"def filter_videos(video_files, bucket_name, min_duration=10, min_resolution_height=240):\n",
" \"\"\"Filters video files based on duration and resolution, using metadata obtained\n",
" from get_video_metadata.\n",
"\n",
" Args:\n",
" video_files (list): A list of video file names (blobs) in the bucket.\n",
" bucket_name (str): The name of the Google Cloud Storage bucket.\n",
" min_duration (int, optional): The minimum duration in seconds.\n",
" min_resolution_height (int, optional): The minimum resolution height\n",
" in pixels.\n",
"\n",
" Returns:\n",
" list: A list of video file names that meet the criteria.\n",
" \"\"\"\n",
" filtered_videos = []\n",
" for video_file in video_files:\n",
" metadata = get_video_metadata(bucket_name, video_file)\n",
" if metadata:\n",
" duration = metadata.get(\"duration\")\n",
" streams = metadata.get(\"streams\", [])\n",
"\n",
" if duration is not None and duration >= min_duration:\n",
" meets_resolution = False\n",
" for stream in streams:\n",
" if stream.get(\"codec_type\") == \"video\":\n",
" height = stream.get(\"height\")\n",
" if height is not None and height >= min_resolution_height:\n",
" meets_resolution = True\n",
" break\n",
"\n",
" if meets_resolution:\n",
" filtered_videos.append(video_file)\n",
" else:\n",
" print(\n",
" f\"Video {video_file} does not meet minimum resolution criteria.\"\n",
" )\n",
" else:\n",
" print(f\"Video {video_file} does not meet minimum duration criteria.\")\n",
" else:\n",
" print(f\"Skipping {video_file} due to metadata extraction failure.\")\n",
" return filtered_videos"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cfb9df161c92"
},
"outputs": [],
"source": [
"video_files_small = video_files[:10]\n",
"\n",
"filtered_video_files = filter_videos(\n",
" video_files_small, bucket, min_duration=10, min_resolution_height=240\n",
")\n",
"\n",
"if filtered_video_files:\n",
" print(\"Filtered Videos:\")\n",
" for video_file in filtered_video_files:\n",
" print(video_file)\n",
"else:\n",
" print(\"No videos found that meet the criteria.\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "12919db7a864"
},
"source": [
"## Defining Split Points\n",
"\n",
"The process of defining split points is crucial for segmenting video data effectively for subsequent analysis, processing, or storage. The \"optimal\" split point strategy is highly dependent on the intended use case of the segmented data. Different approaches offer varying levels of granularity and rely on distinct characteristics of the video content. Choosing the right method can significantly impact the efficiency and relevance of downstream tasks.\n",
"\n",
"__Fixed-Length Segments__\n",
"This is perhaps the simplest method, involving splitting a video into clips of a predetermined, fixed duration (e.g., every 5 seconds). This approach is computationally inexpensive and easy to implement. It's suitable for scenarios where a consistent time-based segmentation is required, such as creating uniform chunks for distributed processing or generating short previews. However, it disregards the video's content, potentially cutting through crucial actions, dialogue, or scene transitions, which can lead to semantically incoherent segments. The fixed length must be chosen carefully based on the expected content and the goals of the data curation.\n",
"\n",
"__Scene Detection Algorithms__\n",
"Scene detection algorithms aim to identify transitions between distinct scenes within a video. These algorithms typically analyze visual differences between consecutive frames, such as changes in color histograms, pixel intensity, or edge detection. More advanced methods can also consider temporal information and machine learning to improve accuracy and differentiate between abrupt cuts, fades, dissolves, and other types of transitions. Splitting by scene creates semantically meaningful segments that represent coherent narrative units, which is beneficial for content-based indexing, retrieval, and analysis. However, the accuracy of these algorithms can vary depending on the video quality, editing style, and the complexity of the visual content.\n",
"\n",
"__Content-Aware Methods__\n",
"Content-aware methods go beyond simple visual differences to understand the underlying content and structure of the video. This category includes techniques like shot boundary detection and keyframe analysis. Shot boundary detection is a more refined form of scene detection, focusing on identifying the immediate transitions between individual camera shots. Keyframe analysis involves identifying the most representative frames within a shot or a longer segment, often based on visual saliency, information content, or diversity. Splitting or annotating based on these methods allows for a more granular and semantically rich representation of the video content, facilitating tasks such as video summarization, activity recognition, and object tracking. These methods often require more computational resources and can be more complex to implement than fixed-length splitting.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6f8f0e23447b"
},
"source": [
"#### Fixed Length Segments\n",
"\n",
"This method offers a straightforward, time-based strategy for segmenting video, commonly implemented using Python libraries like MoviePy which abstract low-level details. The process involves loading the video clip, mathematically determining segment start and end times based on a fixed duration, and programmatically extracting each subclip. The result is a series of video files of uniform length, providing simple partitioning at the cost of disregarding the video's inherent content or scene structure.\n",
"\n",
"Based on our experience, using direct FFmpeg via ```subprocess``` for fixed-length splitting proved more robust and reliable, particularly for handling the final, potentially short segment, leveraging FFmpeg's purpose-built segment mixer and efficient stream copying (```-c copy```). In contrast, while MoviePy offers a more pythonic and user-friendly API, it encountered a specific bug during the re-encoding process for short segments, making it less dependable for this particular task despite its ease of use. Therefore, for critical data curation pipelines prioritizing reliability and performance in splitting, the direct FFmpeg method holds an advantage even with its less abstract command-line interface.\n",
"\n",
"Therefore, to demonstrate fixed length segment splitting we will use FFmpeg."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "e3c2aba3231f"
},
"outputs": [],
"source": [
"bucket_name = \"vidgen-1m\" # Insert your GCS bucket name\n",
"source_blob_name = \"YOUR _BLOB_NAME\" # Example GCS blob name\n",
"\n",
"temp_dir = tempfile.mkdtemp()\n",
"downloaded_file_path = os.path.join(temp_dir, os.path.basename(source_blob_name))\n",
"\n",
"segment_duration_seconds = 5\n",
"output_directory = \"ffmpeg_segments_output\"\n",
"output_filename_pattern = \"segment_%04d.mp4\"\n",
"\n",
"\n",
"def download_blob(bucket_name, source_blob_name, destination_file_name):\n",
" \"\"\"Downloads a blob from the bucket to a local file using authenticated client.\"\"\"\n",
" try:\n",
" storage_client = storage.Client()\n",
" bucket = storage_client.bucket(bucket_name)\n",
" blob = bucket.blob(source_blob_name)\n",
"\n",
" print(\n",
" f\"Attempting authenticated download gs://{bucket_name}/{source_blob_name} to {destination_file_name}\"\n",
" )\n",
"\n",
" if not blob.exists():\n",
" print(\n",
" f\"Error: Blob gs://{bucket_name}/{source_blob_name} does not exist or is not accessible with current credentials.\"\n",
" )\n",
" return False\n",
"\n",
" blob.download_to_filename(destination_file_name)\n",
" print(f\"Download complete: {destination_file_name}\")\n",
" return True\n",
"\n",
" except Exception as e:\n",
" print(f\"An error occurred during authenticated download: {e}\")\n",
" return False\n",
"\n",
"\n",
"def get_video_duration(file_path):\n",
" \"\"\"Gets video duration in seconds using ffprobe.\"\"\"\n",
" ffprobe_command = [\n",
" \"ffprobe\",\n",
" \"-v\",\n",
" \"error\",\n",
" \"-show_entries\",\n",
" \"format=duration\",\n",
" \"-of\",\n",
" \"default=noprint_wrappers=1:nokey=1\",\n",
" file_path,\n",
" ]\n",
" try:\n",
" result = subprocess.run(\n",
" ffprobe_command, check=True, capture_output=True, text=True\n",
" )\n",
" duration_str = result.stdout.strip()\n",
" return float(duration_str)\n",
" except (\n",
" FileNotFoundError,\n",
" subprocess.CalledProcessError,\n",
" ValueError,\n",
" Exception,\n",
" ) as e:\n",
" print(f\"Warning: Could not get duration for {file_path} using ffprobe: {e}\")\n",
" return None\n",
"\n",
"\n",
"def ffmpeg_call():\n",
" \"\"\"Constructs and executes ffmpeg command for fixed length video segmenting.\"\"\"\n",
" ffmpeg_command = [\n",
" \"ffmpeg\",\n",
" \"-i\",\n",
" downloaded_file_path,\n",
" \"-map\",\n",
" \"0\",\n",
" \"-c\",\n",
" \"copy\",\n",
" \"-f\",\n",
" \"segment\",\n",
" \"-segment_time\",\n",
" str(segment_duration_seconds),\n",
" \"-reset_timestamps\",\n",
" \"1\",\n",
" \"-loglevel\",\n",
" \"error\",\n",
" os.path.join(output_directory, output_filename_pattern),\n",
" ]\n",
"\n",
" print(\"Executing FFmpeg command...\")\n",
" subprocess.run(ffmpeg_command, check=True, capture_output=True, text=True)\n",
"\n",
" print(\"FFmpeg command executed successfully.\")\n",
"\n",
" print(\"\\nVerifying output segment files:\")\n",
" try:\n",
" expected_prefix = output_filename_pattern.split(\"%\")[0]\n",
" expected_suffix = \".\" + output_filename_pattern.split(\".\")[-1]\n",
"\n",
" output_items = os.listdir(output_directory)\n",
" segment_files = [\n",
" item\n",
" for item in output_items\n",
" if item.startswith(expected_prefix)\n",
" and item.endswith(expected_suffix)\n",
" and os.path.isfile(os.path.join(output_directory, item))\n",
" ]\n",
"\n",
" if segment_files:\n",
" print(\n",
" f\"Detected {len(segment_files)} segment file(s) written to '{output_directory}':\"\n",
" )\n",
"\n",
" for seg_file in sorted(segment_files):\n",
" full_file_path = os.path.join(output_directory, seg_file)\n",
"\n",
" segment_duration = get_video_duration(full_file_path)\n",
" duration_display = (\n",
" f\"{segment_duration:.2f}s\"\n",
" if segment_duration is not None\n",
" else \"N/A (ffprobe failed)\"\n",
" )\n",
" print(f\"- {seg_file} (Duration: {duration_display})\")\n",
"\n",
" else:\n",
" print(\n",
" f\"No files matching the pattern '{expected_prefix}* {expected_suffix}' found in '{output_directory}'.\"\n",
" )\n",
" print(\n",
" \"This might indicate FFmpeg ran but produced no output, or the output pattern is incorrect.\"\n",
" )\n",
"\n",
" except Exception as e:\n",
" print(f\"Error listing or verifying output files: {e}\")\n",
"\n",
"\n",
"temp_dir_to_cleanup = temp_dir\n",
"input_video_duration = None\n",
"\n",
"try:\n",
" # 1. Download the video from GCS (This step requires authentication)\n",
" download_success = download_blob(\n",
" bucket_name, source_blob_name, downloaded_file_path\n",
" )\n",
"\n",
" if not download_success:\n",
" print(\"Authenticated video download failed. Exiting.\")\n",
" exit()\n",
"\n",
" input_video_duration = get_video_duration(downloaded_file_path)\n",
" if input_video_duration is None:\n",
" print(\n",
" f\"Error: Could not get duration of input video {downloaded_file_path}. Cannot proceed.\"\n",
" )\n",
" exit()\n",
" else:\n",
" print(f\"\\nInput video duration: {input_video_duration:.2f} seconds.\")\n",
"\n",
" # 2. Ensure the output directory for segments exists\n",
" os.makedirs(output_directory, exist_ok=True)\n",
" print(f\"\\nOutput directory '{output_directory}' ensured.\")\n",
"\n",
" # 3. Construct and Execute FFmpeg Command for Fixed-Length Splitting\n",
" print(\n",
" f\"\\nStarting FFmpeg fixed-length splitting ({segment_duration_seconds}s segments) for {downloaded_file_path}...\"\n",
" )\n",
" ffmpeg_call()\n",
"\n",
"\n",
"except FileNotFoundError:\n",
" print(\"Error: FFmpeg or ffprobe command not found.\")\n",
" print(\n",
" \"Please ensure FFmpeg (which includes ffprobe) is installed and in your system's PATH.\"\n",
" )\n",
"except subprocess.CalledProcessError as e:\n",
" print(\"Error during FFmpeg execution:\")\n",
" print(f\"Command: {' '.join(e.cmd)}\")\n",
" print(f\"Return Code: {e.returncode}\")\n",
" print(f\"stdout:\\n{e.stdout}\")\n",
" print(f\"stderr:\\n{e.stderr}\")\n",
"except Exception as e:\n",
" print(f\"\\nAn unexpected error occurred during processing: {e}\")\n",
"\n",
"\n",
"finally:\n",
" # Clean up the temporary local file and the temporary directory\n",
" print(\"\\n--- Cleaning up temporary files ---\")\n",
" if os.path.exists(downloaded_file_path):\n",
" try:\n",
" os.remove(downloaded_file_path)\n",
" print(f\"Removed temporary file: {downloaded_file_path}\")\n",
" except OSError as e:\n",
" print(f\"Error removing file {downloaded_file_path}: {e}\")\n",
"\n",
" # This removes the temporary directory the input file was in (e.g., /tmp/tmp...)\n",
" if os.path.exists(temp_dir_to_cleanup):\n",
" try:\n",
" shutil.rmtree(temp_dir_to_cleanup)\n",
" print(f\"Removed temporary directory: {temp_dir_to_cleanup}\")\n",
" except OSError as e:\n",
" print(f\"Error removing temporary directory {temp_dir_to_cleanup}: {e}.\")\n",
"\n",
"\n",
"print(\"\\nFixed-length splitting process finished.\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4575f17c40e9"
},
"source": [
"#### Scene Detection\n",
"\n",
"For this approach, we are using PySceneDetect. This is a command-line tool and a Python library specifically designed for detecting scene changes in videos. The process utilizes the `open_video` function to correctly load the video source, preparing it for analysis by the scene detection algorithms. A `SceneManager` is then configured with specific detectors, such as the `ContentDetector`, which analyzes frame-to-frame visual changes to pinpoint potential scene transitions. For the `ContentDetector`, the `threshold` parameter is critical as it determines the sensitivity to visual changes, requiring careful tuning based on the video's characteristics. Finally, executing the detection process on the video object yields a structured list of detected scene boundaries, providing precise split points for data curation.\n",
"\n",
"In this example, we will show an example script for how to perform scene detection based splitting on one video of the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1d30c9b63395"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Attempting authenticated download gs://vidgen-1m/VidGen_video_0/-EGK4gGtz44-Scene-0398.mp4 to /tmp/tmpws4ba8nm/-EGK4gGtz44-Scene-0398.mp4\n",
"Download complete: /tmp/tmpws4ba8nm/-EGK4gGtz44-Scene-0398.mp4\n",
"\n",
"Processing local temporary file: /tmp/tmpws4ba8nm/-EGK4gGtz44-Scene-0398.mp4\n",
"Video source opened successfully with open_video.\n",
"Starting scene detection...\n",
"Scene detection finished.\n",
"\n",
"Detected 2 scenes for VidGen_video_0/-EGK4gGtz44-Scene-0398.mp4.\n",
" Scene 1: Start 00:00:00.000 - End 00:00:04.200\n",
" Scene 2: Start 00:00:04.200 - End 00:00:04.367\n",
"\n",
"--- Cleaning up temporary files ---\n",
"Removed temporary file: /tmp/tmpws4ba8nm/-EGK4gGtz44-Scene-0398.mp4\n",
"Removed temporary directory: /tmp/tmpws4ba8nm\n"
]
}
],
"source": [
"bucket_name = \"vidgen-1m\"\n",
"source_blob_name = \"VidGen_video_0/-EGK4gGtz44-Scene-0398.mp4\"\n",
"\n",
"\n",
"# Create a temporary directory to store the downloaded file\n",
"temp_dir = tempfile.mkdtemp()\n",
"\n",
"destination_file_name = os.path.join(temp_dir, os.path.basename(source_blob_name))\n",
"\n",
"video_object = None\n",
"temp_dir_to_cleanup = temp_dir\n",
"\n",
"try:\n",
" # 1. Download the video from GCS (Requires Authentication)\n",
" download_success = download_blob(\n",
" bucket_name, source_blob_name, destination_file_name\n",
" )\n",
"\n",
" if not download_success:\n",
" print(\"Authenticated video download failed. Exiting.\")\n",
" exit()\n",
"\n",
" video_source_path = destination_file_name\n",
" print(f\"\\nProcessing local temporary file: {video_source_path}\")\n",
"\n",
" # 2. Open the video source using the recommended open_video function\n",
" video_object = open_video(video_source_path)\n",
" print(\"Video source opened successfully with open_video.\")\n",
"\n",
" # 3. Create SceneManager and add detectors\n",
" scene_manager = SceneManager(\n",
" stats_manager=None\n",
" ) # Pass stats_manager=None if not using\n",
" scene_manager.add_detector(\n",
" ContentDetector(threshold=20)\n",
" ) # Adjust threshold as needed\n",
"\n",
" # 4. Perform scene detection, passing the video object directly\n",
" print(\"Starting scene detection...\")\n",
" scene_manager.detect_scenes(video_object) # Pass the result of open_video here\n",
" print(\"Scene detection finished.\")\n",
"\n",
" # 5. Retrieve list of scenes\n",
" scene_list = scene_manager.get_scene_list()\n",
"\n",
" print(f\"\\nDetected {len(scene_list)} scenes for {source_blob_name}.\")\n",
"\n",
" # 6. Print scene timestamps\n",
" for i, scene in enumerate(scene_list):\n",
" print(\n",
" f\" Scene {i + 1}: Start {scene[0].get_timecode()} - End {scene[1].get_timecode()}\"\n",
" )\n",
"\n",
"\n",
"except Exception as e:\n",
" print(f\"\\nAn error occurred during scene processing: {e}\")\n",
"\n",
"finally:\n",
" # Clean up the temporary local file and the temporary directory\n",
" print(\"\\n--- Cleaning up temporary files ---\")\n",
" if \"destination_file_name\" in locals() and os.path.exists(destination_file_name):\n",
" try:\n",
" os.remove(destination_file_name)\n",
" print(f\"Removed temporary file: {destination_file_name}\")\n",
" except OSError as e:\n",
" print(f\"Error removing file {destination_file_name}: {e}\")\n",
"\n",
" if \"temp_dir_to_cleanup\" in locals() and os.path.exists(temp_dir_to_cleanup):\n",
" try:\n",
" shutil.rmtree(temp_dir_to_cleanup)\n",
" print(f\"Removed temporary directory: {temp_dir_to_cleanup}\")\n",
" except OSError as e:\n",
" print(\n",
" f\"Error removing temporary directory {temp_dir_to_cleanup}: {e}. It might not be empty.\"\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "153f9c9b4996"
},
"source": [
"## Transcoding \n",
"\n",
"Now that you have your video split into segments, the next step in a data curation pipeline is to transcode them. Transcoding is the process of converting a video file from one format (codec, container, resolution, bitrate, etc.) to another. This is often done to standardize the video format for analysis, reduce file size, or ensure compatibility with specific tools or platforms.   \n",
"\n",
"Similar to splitting, the most common and powerful tool for transcoding is FFmpeg. You can perform transcoding on each segment by calling FFmpeg via Python's ```subprocess``` module.   \n",
"\n",
"Determining the \"best\" target transcoding parameters for a video curation pipeline depends heavily on the specific goals and constraints of that pipeline, such as the required quality for analysis, storage limitations, compatibility with downstream tools, and processing speed requirements. However, here are some common and recommended parameters that strike a good balance for many curation purposes:\n",
"\n",
"1. Container Format (e.g., MP4): MP4 (.mp4) is the most widely compatible container format. It's well-supported across various operating systems, software libraries, and hardware, making it an excellent choice for ensuring your curated data is easily accessible and usable by different tools in the pipeline.   \n",
"\n",
"2. Video Codec (e.g., H.264 or H.265):\n",
"\n",
" H.264 (AVC) (libx264 in FFmpeg): This is the most compatible and still offers a very good balance between compression efficiency and quality. It's the safest default as almost all video software and hardware can decode it efficiently.   \n",
"\n",
" H.265 (HEVC) (libx265 in FFmpeg): Offers significantly better compression than H.264 for the same quality, resulting in smaller file sizes. Choose this if storage or bandwidth is a major concern, but be aware it requires more processing power to encode/decode and has less universal compatibility than H.264.   \n",
"\n",
"3. Video Quality/Bitrate (e.g., CRF 20-23):\n",
"\n",
" Constant Rate Factor (CRF) for H.264/H.265: Using CRF is often preferred over specifying a fixed bitrate. It tells the encoder to maintain a consistent perceptual quality throughout the video, adjusting the bitrate as needed for complex scenes. A CRF value between 18 and 23 typically provides a good balance, where 18 is often considered visually near-lossless and 23 offers significant file size reduction with minimal noticeable quality loss for many purposes.   \n",
"\n",
" Fixed Bitrate (-b:v): Use this if you have strict file size targets, but quality will vary depending on the complexity of the video content.\n",
"\n",
" Resolution: The ideal resolution depends on the detail needed for downstream analysis or display.\n",
"\n",
" Keep Original: If full detail is essential.\n",
"\n",
" Downscale: Common choices are standard HD resolutions like 1920x1080 (1080p) or 1280x720 (720p). Downscaling reduces file size and standardizes dimensions, which can be beneficial for consistent input to models or analysis tools. Use FFmpeg's scale video filter (-vf scale=1280:720 or -vf scale=-2:720 to maintain aspect ratio).   \n",
"\n",
"4. Frame Rate (FPS):\n",
"\n",
" Keep Original: Generally recommended unless standardization of temporal sampling is required.\n",
"\n",
" Standardize: Choosing a common frame rate like 25 or 30 fps can be useful if your analysis or models require consistent time steps. Use FFmpeg's -r option.\n",
"\n",
"5. Audio Codec and Bitrate (e.g., AAC 128k-192k):\n",
"\n",
" Audio Codec (e.g., AAC): AAC (aac in FFmpeg) is the standard audio codec for MP4 containers and offers good compression with decent quality.   \n",
" \n",
" Audio Bitrate (-b:a): A bitrate between 128k and 192k is typically sufficient for most curation purposes, providing clear audio without excessively increasing file size.\n",
" \n",
"\n",
"A solid starting point for a general-purpose curation pipeline, balancing compatibility, quality, and manageability, would be:\n",
"\n",
"+ Container: MP4 (.mp4)\n",
"+ Video Codec: H.264 (libx264)\n",
"+ Video Quality: CRF 21-23 (Adjust based on acceptable quality vs. size)\n",
"+ Resolution: 1920x1080 or 1280x720 (if downscaling is acceptable, choose based on need)\n",
"+ Frame Rate: Keep original (omit -r) or standardize to 25/30.\n",
"+ Audio Codec: AAC (aac)\n",
"+ Audio Bitrate: 128k or 192k (-b:a 128k)\n",
"\n",
"Always test your chosen parameters on a sample of your video data to ensure the output quality, file size, and processing time meet your pipeline's requirements.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "451df7a9d818"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Transcoded output directory 'transcoded_segments' ensured.\n",
"\n",
"Starting transcoding for 2 files in ffmpeg_segments_output...\n",
"\n",
"Processing segment_0000.mp4 -> segment_0000_transcoded.mp4\n",
" Executing FFmpeg command for segment_0000.mp4...\n",
" Transcoding successful for segment_0000.mp4.\n",
"\n",
"Processing segment_0001.mp4 -> segment_0001_transcoded.mp4\n",
" Executing FFmpeg command for segment_0001.mp4...\n",
" Transcoding successful for segment_0001.mp4.\n",
"\n",
"Transcoding process finished for all files.\n"
]
}
],
"source": [
"input_directory = \"ffmpeg_segments_output\"\n",
"transcoding_output_directory = \"transcoded_segments\"\n",
"\n",
"# Define target transcoding parameters (adjust as needed)\n",
"target_video_codec = \"libx264\" # Example: H.264\n",
"target_audio_codec = \"aac\" # Example: AAC\n",
"target_resolution = (\n",
" \"1280x720\" # Example resolution or None to keep original, or 'scale=-2:720' etc.\n",
")\n",
"target_video_bitrate = \"1500k\" # Example bitrate or None\n",
"target_audio_bitrate = \"128k\" # Example bitrate or None\n",
"target_fps = \"25\" # Example frame rate or None to keep original\n",
"\n",
"if not os.path.isdir(input_directory):\n",
" print(f\"Error: Input directory not found at {input_directory}\")\n",
" exit()\n",
"\n",
"segment_files_to_transcode = glob.glob(os.path.join(input_directory, \"*.mp4\"))\n",
"\n",
"if not segment_files_to_transcode:\n",
" print(f\"No MP4 files found in the input directory: {input_directory}\")\n",
" exit()\n",
"\n",
"# Ensure the output directory exists\n",
"os.makedirs(transcoding_output_directory, exist_ok=True)\n",
"print(f\"Transcoded output directory '{transcoding_output_directory}' ensured.\")\n",
"\n",
"print(\n",
" f\"\\nStarting transcoding for {len(segment_files_to_transcode)} files in {input_directory}...\"\n",
")\n",
"\n",
"\n",
"def ffmpeg_transcode_segment(input_segment_path):\n",
" \"\"\"Constructs the ffmpeg command for transcoding segments with transcoding parameters.\"\"\"\n",
" input_segment_filename = os.path.basename(input_segment_path)\n",
" base_filename, file_extension = os.path.splitext(input_segment_filename)\n",
" output_extension = \".mp4\"\n",
" output_transcoded_filename = f\"{base_filename}_transcoded{output_extension}\"\n",
" output_transcoded_path = os.path.join(\n",
" transcoding_output_directory, output_transcoded_filename\n",
" )\n",
"\n",
" print(f\"\\nProcessing {input_segment_filename} -> {output_transcoded_filename}\")\n",
"\n",
" ffmpeg_command = [\n",
" \"ffmpeg\",\n",
" \"-i\",\n",
" input_segment_path, # Input file\n",
" \"-c:v\",\n",
" target_video_codec, # Set video codec\n",
" \"-b:v\",\n",
" target_video_bitrate, # Set video bitrate\n",
" \"-c:a\",\n",
" target_audio_codec, # Set audio codec\n",
" \"-b:a\",\n",
" target_audio_bitrate, # Set audio bitrate\n",
" \"-vf\",\n",
" f\"scale={target_resolution}\", # Scale video filter\n",
" \"-r\",\n",
" target_fps, # Set frame rate\n",
" \"-y\", # Overwrite output file without asking\n",
" \"-loglevel\",\n",
" \"error\", # Suppress verbose FFmpeg output, show errors only\n",
" output_transcoded_path, # Output file\n",
" ]\n",
"\n",
" try:\n",
" print(f\" Executing FFmpeg command for {input_segment_filename}...\")\n",
" subprocess.run(ffmpeg_command, check=True, capture_output=True, text=True)\n",
" print(f\" Transcoding successful for {input_segment_filename}.\")\n",
"\n",
" except FileNotFoundError:\n",
" print(f\" Error: FFmpeg command not found. Skipping {input_segment_filename}.\")\n",
" except subprocess.CalledProcessError as e:\n",
" print(f\" Error during FFmpeg execution for {input_segment_filename}:\")\n",
" print(f\" Command: {' '.join(e.cmd)}\")\n",
" print(f\" Return Code: {e.returncode}\")\n",
" print(f\" stdout:\\n{e.stdout}\")\n",
" print(f\" stderr:\\n{e.stderr}\")\n",
" except Exception as e:\n",
" print(f\" An unexpected error occurred for {input_segment_filename}: {e}\")\n",
"\n",
"\n",
"# Sort the files for consistent processing order\n",
"for input_segment_path in sorted(segment_files_to_transcode):\n",
" ffmpeg_transcode_segment(input_segment_path)\n",
"\n",
"print(\"\\nTranscoding process finished for all files.\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3144cdac492a"
},
"source": [
"### Transcoding with the Google Cloud Transcoder API\n",
"\n",
"The current implementation uses `ffmpeg`, which is the industry standard. However this requires constructing complex command line arguments. By contrast, Google Cloud Transcoder API is a fully managed service that simplifies complex video conversion, removing the need to operate and maintain your own encoding infrastructure. It specializes in producing adaptive bitrate formats like HLS and MPEG-DASH, ensuring a smooth, high-quality viewing experience for users on any device and network condition. Beyond simple transcoding, the API enables sophisticated content manipulation, including the ability to stitch multiple clips together, overlay watermarks, and embed captions. It's asynchronous, job-based workflow allows applications to submit a detailed processing request and receive notifications upon completion without being blocked. Ultimately, this service streamlines professional media pipelines by offloading heavy processing, which empowers developers to build advanced video features faster.\n",
"\n",
"Below is an example code block with how to use the Transcoder API."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1adbf41bc5d1"
},
"outputs": [],
"source": [
"gcs_input_uri = \"gs://your-input-bucket/path/to/your-video.mp4\"\n",
"gcs_output_uri_prefix = \"gs://your-output-bucket/path/for-transcoded-segments/\"\n",
"\n",
"# Define the start and end times for the clip you want to extract.\n",
"# This example will create a 10-second clip starting at the 5-second mark.\n",
"start_time_seconds = 5.0\n",
"end_time_seconds = 15.0\n",
"\n",
"# Instantiate the Transcoder API client\n",
"transcoder_client = TranscoderServiceClient()\n",
"\n",
"# The parent resource for the API call\n",
"parent = f\"projects/{PROJECT_ID}/locations/{LOCATION}\"\n",
"\n",
"# Define a unique name for the output segment\n",
"output_segment_filename = (\n",
" f\"video_segment_{int(start_time_seconds)}s_to_{int(end_time_seconds)}s.mp4\"\n",
")\n",
"gcs_output_uri = f\"{gcs_output_uri_prefix}{output_segment_filename}\"\n",
"\n",
"# Create the job configuration dictionary. This structure tells the Transcoder API\n",
"# exactly what to do.\n",
"job = {\n",
" \"input_uri\": gcs_input_uri,\n",
" \"output_uri\": gcs_output_uri,\n",
" \"config\": {\n",
" # 'edit_list' is used to define splits or stitch together clips.\n",
" # Here we define one \"atom\" that points to our input video and\n",
" # specifies the start and end times.\n",
" \"edit_list\": [\n",
" {\n",
" \"key\": \"atom0\",\n",
" \"inputs\": [{\"key\": \"input0\", \"uri\": gcs_input_uri}],\n",
" \"start_time_offset\": {\"seconds\": int(start_time_seconds)},\n",
" \"end_time_offset\": {\"seconds\": int(end_time_seconds)},\n",
" }\n",
" ],\n",
" # 'elementary_streams' define the video and audio encoding settings.\n",
" \"elementary_streams\": [\n",
" {\n",
" \"key\": \"video-stream0\",\n",
" \"video_stream\": {\n",
" \"h264\": {\n",
" \"height_pixels\": 720,\n",
" \"width_pixels\": 1280,\n",
" \"bitrate_bps\": 2500000,\n",
" \"frame_rate\": 30,\n",
" }\n",
" },\n",
" },\n",
" {\n",
" \"key\": \"audio-stream0\",\n",
" \"audio_stream\": {\"codec\": \"aac\", \"bitrate_bps\": 128000},\n",
" },\n",
" ],\n",
" # 'mux_streams' combine the elementary streams into an output file container.\n",
" \"mux_streams\": [\n",
" {\n",
" \"key\": \"output-mp4\",\n",
" \"container\": \"mp4\",\n",
" \"elementary_streams\": [\"video-stream0\", \"audio-stream0\"],\n",
" }\n",
" ],\n",
" },\n",
"}\n",
"\n",
"try:\n",
" print(f\"Submitting transcoding job to split '{gcs_input_uri}'...\")\n",
"\n",
" # Use the client to create the job\n",
" response = transcoder_client.create_job(parent=parent, job=job)\n",
"\n",
" print(\"\\nAPI Response:\")\n",
" print(f\"Job Name: {response.name}\")\n",
" print(f\"Creation Time: {response.create_time}\")\n",
" print(f\"Status: {response.state}\")\n",
" print(\"\\nThe job has been submitted and is processing asynchronously.\")\n",
" print(f\"Check the GCS output path for results: {gcs_output_uri}\")\n",
"\n",
"except Exception as e:\n",
" print(f\"\\nAn error occurred: {e}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "48696dbe3e42"
},
"source": [
"## Conclusion"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "412096908949"
},
"source": [
"This notebook demonstrates foundational steps in a video data curation pipeline, essential for transforming raw video sources into a format suitable for analysis or machine learning tasks. A key initial process is video splitting, which breaks down large video files into smaller, more manageable segments based on time or content boundaries. Following splitting, transcoding standardizes the format, resolution, and other technical parameters of these segments, ensuring compatibility and efficiency for consistent downstream processing. By completing these crucial splitting and transcoding stages, the video dataset becomes a standardized collection of clips, prepared for subsequent quality filtering and in-depth analysis.\n"
]
}
],
"metadata": {
"colab": {
"name": "splitting_and_transcoding.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Reference in New Issue View Git Blame Copy Permalink