wehub-resource-sync
794 lines
31 KiB
Plaintext
794 lines
31 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "NcIyYxlxZ7AW"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Copyright 2026 Google LLC\n",
|
|
"#\n",
|
|
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
|
"# you may not use this file except in compliance with the License.\n",
|
|
"# You may obtain a copy of the License at\n",
|
|
"#\n",
|
|
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
|
"#\n",
|
|
"# Unless required by applicable law or agreed to in writing, software\n",
|
|
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
|
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
|
"# See the License for the specific language governing permissions and\n",
|
|
"# limitations under the License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "0sB0minZaO0T"
|
|
},
|
|
"source": [
|
|
"# Gemini 3.1 Flash Text-to-Speech Generation\n",
|
|
"\n",
|
|
"<table align=\"left\">\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/audio/speech/getting-started/gemini_3_1_flash_tts.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/agent-platform/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Faudio%2Fspeech%2Fgetting-started%2Fgemini_3_1_flash_tts.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/agent-platform/workbench/instances?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/audio/speech/getting-started/gemini_3_1_flash_tts.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://storage.googleapis.com/github-repo/workbench-icon.svg\" alt=\"Workbench logo\"><br> Open in Workbench\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/speech/getting-started/gemini_3_1_flash_tts.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
"</table>\n",
|
|
"\n",
|
|
"<div style=\"clear: both;\"></div>\n",
|
|
"\n",
|
|
"<p>\n",
|
|
"<b>Share to:</b>\n",
|
|
"\n",
|
|
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/speech/getting-started/gemini_3_1_flash_tts.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/speech/getting-started/gemini_3_1_flash_tts.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/speech/getting-started/gemini_3_1_flash_tts.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/speech/getting-started/gemini_3_1_flash_tts.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/speech/getting-started/gemini_3_1_flash_tts.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
|
"</a>\n",
|
|
"</p>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "2XpbkyEbb-U3"
|
|
},
|
|
"source": [
|
|
"| Author |\n",
|
|
"| --- |\n",
|
|
"| [Katie Nguyen](https://github.com/katiemn) |"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "PM7AnicdcFg0"
|
|
},
|
|
"source": [
|
|
"## Overview\n",
|
|
"\n",
|
|
"### Gemini 3.1 Flash TTS\n",
|
|
"\n",
|
|
"Gemini 3.1 Flash TTS on Agent Platform gives application developers access to Google's cutting-edge speech generation. This model provides powerful, low-latency speech generation with expressive audio tags for carefully crafted narration.\n",
|
|
"\n",
|
|
"In this tutorial, you will learn how to use the Google Gen AI SDK for Python to interact with Gemini 3.1 Flash TTS to:\n",
|
|
"- Convert text to an audio output with prebuilt voices\n",
|
|
"- Generate speech clips in different languages\n",
|
|
"- Define multiple speakers\n",
|
|
"- Convert text to speech with audio tags"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "DvNFe-xxsced"
|
|
},
|
|
"source": [
|
|
"## Get started"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "gyIP8rb3sgOR"
|
|
},
|
|
"source": [
|
|
"### Install Google Gen AI SDK for Python"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "JLl1jyeJsOPp"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install --upgrade --quiet google-genai"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "vy23JbZps1Jm"
|
|
},
|
|
"source": [
|
|
"### Authenticate your notebook environment (Colab only)\n",
|
|
"\n",
|
|
"If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"id": "svjU0gJVsdzu"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import sys\n",
|
|
"\n",
|
|
"if \"google.colab\" in sys.modules:\n",
|
|
" from google.colab import auth\n",
|
|
"\n",
|
|
" auth.authenticate_user()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "dL81amVRtI89"
|
|
},
|
|
"source": [
|
|
"### Import libraries"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {
|
|
"id": "Gfo9egZHshrZ"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"\n",
|
|
"import numpy as np\n",
|
|
"from IPython.display import Audio, Markdown, display\n",
|
|
"from google import genai\n",
|
|
"from google.genai import types"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "wikdtLdjtVQT"
|
|
},
|
|
"source": [
|
|
"### Set Google Cloud project information\n",
|
|
"\n",
|
|
"To get started using Agent Platform, you must have an existing Google Cloud project and [enable the Agent Platform API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
|
"\n",
|
|
"Learn more about [setting up a project](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects) and a [development environment](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"id": "hqwZDw6Esj3z"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# fmt: off\n",
|
|
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
|
|
"# fmt: on\n",
|
|
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
|
|
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
|
|
"\n",
|
|
"LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"global\")\n",
|
|
"\n",
|
|
"client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "Mfb2XfgYuV7c"
|
|
},
|
|
"source": [
|
|
"### Define a helper function to play audio"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"id": "Ka_sWHxbsmfb"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def play_audio(response):\n",
|
|
" audio_bytes = response.candidates[0].content.parts[0].inline_data.data\n",
|
|
" audio_array = np.frombuffer(audio_bytes, dtype=\"<i2\")\n",
|
|
" display(Audio(data=audio_array, rate=24000))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "-67nb2b_x8eP"
|
|
},
|
|
"source": [
|
|
"### Load the models"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"id": "6KUUn1chsow0"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"speech_model = \"gemini-3.1-flash-tts-preview\"\n",
|
|
"gemini_model = \"gemini-3.5-flash\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "O2TiFc8O4JtP"
|
|
},
|
|
"source": [
|
|
"## Convert Text-to-Speech\n",
|
|
"\n",
|
|
"When converting text-to-speech, you can choose from a list of [voice options](https://docs.cloud.google.com/text-to-speech/docs/gemini-tts#voice_options), which you'll set through the `VoiceConfig`.\n",
|
|
"\n",
|
|
"By default, all audio generated with Gemini 3.1 Flash TTS utilizes [SynthID](https://deepmind.google/technologies/synthid/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "wApkg_7OAY-h"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\"\n",
|
|
"Hey there! How are you doing today?\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=speech_model,\n",
|
|
" contents=prompt,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" response_modalities=[\"AUDIO\"],\n",
|
|
" speech_config=types.SpeechConfig(\n",
|
|
" voice_config=types.VoiceConfig(\n",
|
|
" prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name=\"Fenrir\")\n",
|
|
" )\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"play_audio(response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "n3aRIjeNnr7h"
|
|
},
|
|
"source": [
|
|
"## Generate speech in different languages\n",
|
|
"\n",
|
|
"The TTS models detect the input language automatically. Check the documentation to learn more about the list of [supported languages](https://docs.cloud.google.com/text-to-speech/docs/gemini-tts#available_languages)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "kQryZG6RZrZl"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\"\n",
|
|
"¡Hola! ¿Quieres hacer algo hoy?\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=speech_model,\n",
|
|
" contents=prompt,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" response_modalities=[\"AUDIO\"],\n",
|
|
" speech_config=types.SpeechConfig(\n",
|
|
" voice_config=types.VoiceConfig(\n",
|
|
" prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name=\"Despina\")\n",
|
|
" )\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"play_audio(response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "ZrQLAflLolcc"
|
|
},
|
|
"source": [
|
|
"## Define multiple speakers\n",
|
|
"\n",
|
|
"For multi-speaker audio clips, you'll need to include a `MultiSpeakerVoiceConfig` in your request. You can include up to 2 different speakers, each with their own `SpeakerVoiceConfig` that includes the `speaker` name and `voice_name`. Make sure the `speaker` name is the same one used in the prompt."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "_K_aio7ouYP4"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\"\n",
|
|
"Ryan: How are you doing today Katie?\n",
|
|
"Katie: Not too bad, how about you?\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=speech_model,\n",
|
|
" contents=prompt,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" response_modalities=[\"AUDIO\"],\n",
|
|
" speech_config=types.SpeechConfig(\n",
|
|
" multi_speaker_voice_config=types.MultiSpeakerVoiceConfig(\n",
|
|
" speaker_voice_configs=[\n",
|
|
" types.SpeakerVoiceConfig(\n",
|
|
" speaker=\"Ryan\",\n",
|
|
" voice_config=types.VoiceConfig(\n",
|
|
" prebuilt_voice_config=types.PrebuiltVoiceConfig(\n",
|
|
" voice_name=\"Umbriel\",\n",
|
|
" )\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
" types.SpeakerVoiceConfig(\n",
|
|
" speaker=\"Katie\",\n",
|
|
" voice_config=types.VoiceConfig(\n",
|
|
" prebuilt_voice_config=types.PrebuiltVoiceConfig(\n",
|
|
" voice_name=\"Leda\",\n",
|
|
" )\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
" ]\n",
|
|
" )\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"play_audio(response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "ZKm_x60Sr5cp"
|
|
},
|
|
"source": [
|
|
"## Prompting with audio tags\n",
|
|
"\n",
|
|
"Audio tags let you insert modifiers directly into your prompt to help control speaker style, tone, or delivery. An audio tag is a word in square brackets that specifies how the text should be delivered, such as `[confusion]` or `[laughs]`.\n",
|
|
"\n",
|
|
"You can experiment with different tags. Here is a list of examples to get started:\n",
|
|
"\n",
|
|
"| Audio Tag | Audio Tag | Audio Tag | Audio Tag | Audio Tag |\n",
|
|
"| --- | --- | --- | --- | --- |\n",
|
|
"| [determination]| [enthusiasm] | [adoration] |[interest] | [awe] |\n",
|
|
"|[admiration] | [nervousness] | [frustration] | [excitement] | [curiosity] |\n",
|
|
"| [hope] | [annoyance] | [amusement] | [aggression] | [tension] |\n",
|
|
"| [agitation] | [confusion] | [anger] | [positive] | [neutral] |\n",
|
|
"| [negative] | [whispers] | [laughs] | | |\n",
|
|
"\n",
|
|
"**NOTE:** For best results, add audio tags in English, even if your transcript is in a different language."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "OgemQk62Zowg"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"prompt = \"\"\"\n",
|
|
"[confusion] I have absolutely no idea why this variable is returning as undefined, [agitation] and I've been staring at this same block of code until my eyes are literally crossing! [aggression] Just work already! [whispers] Wait... is it really just a missing semicolon right there on line forty-two? [laughs] I can't believe I spent two hours debugging a simple typo!\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"response = client.models.generate_content(\n",
|
|
" model=speech_model,\n",
|
|
" contents=prompt,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" response_modalities=[\"AUDIO\"],\n",
|
|
" speech_config=types.SpeechConfig(\n",
|
|
" voice_config=types.VoiceConfig(\n",
|
|
" prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name=\"Charon\")\n",
|
|
" )\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"play_audio(response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "RvcPT7El3o8m"
|
|
},
|
|
"source": [
|
|
"### Adding audio tags\n",
|
|
"\n",
|
|
"If you have a long transcript, it may be useful to have Gemini help add in audio tags. In this next example, you'll take a fictitious podcast transcript and give it to Gemini with a list of example audio tags that can be included."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"id": "SjQanjUYckuA"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"podcast = \"\"\"\n",
|
|
"If you walk through a local park on a Saturday morning, you are bound to see a familiar cast of characters. There is the floppy-eared Golden Retriever chasing a tennis ball, the stout French Bulldog trotting along the pavement, and almost certainly, the athletic Labrador Retriever leading the way. We think of these breeds as timeless icons of the domestic home, but the \"popularity\" of a dog is rarely an accident of nature. It is a fascinating mirror held up to human history, shifting based on our wars, our technology, and even our movies.\n",
|
|
"\n",
|
|
"For most of human history, a \"popular\" dog was simply the one that kept you alive. We didn't have breeds in the modern sense; we had jobs. There were dogs that pulled sleds, dogs that guarded livestock, and dogs that cleared vermin from granaries. If a dog was popular in the year 1500, it was because it was an elite employee. The idea of a dog as a \"member of the family\" with no specific chore to perform is a relatively new luxury, largely born out of the Victorian era.\n",
|
|
"\n",
|
|
"It was during the 19th century in England that dog ownership shifted from a functional necessity to a high-stakes hobby. This was the birth of the Kennel Club and the formalization of \"breeds.\" Suddenly, the physical consistency of a dog—the shape of its snout or the curl of its tail—became a status symbol. A popular breed wasn't just a good worker; it was a fashion statement. This is when the Beagle and the Fox Terrier began their rise.\n",
|
|
"\n",
|
|
"However, nothing influences what we choose to bring into our homes quite like the silver screen. We see this in what researchers call the \"Lassie Effect.\" When the movie Lassie Come Home was released in 1943, registrations for Rough Collies skyrocketed. It's a recurring theme in history: we fall in love with a character, and the breed becomes a trend.\n",
|
|
"\n",
|
|
"Then there is the king of the mountain: the Labrador Retriever. For thirty-one consecutive years, the Lab held the title of the most popular dog in America. Its rise to power was a perfect storm of timing. As the world moved into the suburbs after World War II, people wanted a dog that was \"versatile.\" The Lab was small enough to live indoors but sturdy enough to go hiking; it was gentle with children but capable of being a serious hunting partner. It became the \"all-purpose\" dog for the modern age.\n",
|
|
"\n",
|
|
"Today, we are seeing another shift. The French Bulldog recently dethroned the Labrador, ending its three-decade reign. This tells us something about how our lives have changed again. We live in smaller spaces, we have less time for long runs in the woods, and we value portability. The popularity of \"designer\" breeds like the various Doodles reflects a modern obsession with customization—wanting the personality of a Retriever with the hypoallergenic coat of a Poodle.\n",
|
|
"\n",
|
|
"When we look at the most popular breeds of today, we aren't just looking at dogs. We are looking at a map of our own desires. We choose the breeds that fit the lives we want to live, whether that's the rugged adventurer, the urban socialite, or the suburban family. The dogs change because we change.\n",
|
|
"\n",
|
|
"Thanks for joining me on The Everyday. Next time you see a dog on the street, take a look at the human on the other end of the leash—you might learn just as much about them as you do about the dog. I'll see you next time.\n",
|
|
"\"\"\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {
|
|
"id": "-qEsorzYdwdn"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"example_audio_tags = [\n",
|
|
" \"[acceptance]\",\n",
|
|
" \"[accomplishment]\",\n",
|
|
" \"[achievement]\",\n",
|
|
" \"[active]\",\n",
|
|
" \"[admiration]\",\n",
|
|
" \"[admonition]\",\n",
|
|
" \"[adoration]\",\n",
|
|
" \"[affection]\",\n",
|
|
" \"[aggression]\",\n",
|
|
" \"[agitation]\",\n",
|
|
" \"[alarm]\",\n",
|
|
" \"[amazement]\",\n",
|
|
" \"[ambivalence]\",\n",
|
|
" \"[amused]\",\n",
|
|
" \"[amusement]\",\n",
|
|
" \"[analysis]\",\n",
|
|
" \"[anger]\",\n",
|
|
" \"[animation]\",\n",
|
|
" \"[annoyance]\",\n",
|
|
" \"[anticipation]\",\n",
|
|
" \"[anxiety]\",\n",
|
|
" \"[apology]\",\n",
|
|
" \"[appreciation]\",\n",
|
|
" \"[apprehension]\",\n",
|
|
" \"[approval]\",\n",
|
|
" \"[arrogance]\",\n",
|
|
" \"[assertion]\",\n",
|
|
" \"[assertive]\",\n",
|
|
" \"[assertiveness]\",\n",
|
|
" \"[assurance]\",\n",
|
|
" \"[astonishment]\",\n",
|
|
" \"[aversion]\",\n",
|
|
" \"[awareness]\",\n",
|
|
" \"[awe]\",\n",
|
|
" \"[awkwardness]\",\n",
|
|
" \"[bargaining]\",\n",
|
|
" \"[boredom]\",\n",
|
|
" \"[caring]\",\n",
|
|
" \"[caution]\",\n",
|
|
" \"[cautious]\",\n",
|
|
" \"[certainty]\",\n",
|
|
" \"[challenging]\",\n",
|
|
" \"[comfort]\",\n",
|
|
" \"[compassion]\",\n",
|
|
" \"[concentration]\",\n",
|
|
" \"[concern]\",\n",
|
|
" \"[confidence]\",\n",
|
|
" \"[confident]\",\n",
|
|
" \"[confusion]\",\n",
|
|
" \"[contemplative]\",\n",
|
|
" \"[contempt]\",\n",
|
|
" \"[contentment]\",\n",
|
|
" \"[conviction]\",\n",
|
|
" \"[courage]\",\n",
|
|
" \"[craving]\",\n",
|
|
" \"[critical]\",\n",
|
|
" \"[criticism]\",\n",
|
|
" \"[curiosity]\",\n",
|
|
" \"[decision]\",\n",
|
|
" \"[defiance]\",\n",
|
|
" \"[demonstration]\",\n",
|
|
" \"[description]\",\n",
|
|
" \"[descriptive]\",\n",
|
|
" \"[desire]\",\n",
|
|
" \"[despair]\",\n",
|
|
" \"[desperation]\",\n",
|
|
" \"[despondency]\",\n",
|
|
" \"[determination]\",\n",
|
|
" \"[determined]\",\n",
|
|
" \"[devotion]\",\n",
|
|
" \"[directness]\",\n",
|
|
" \"[disagreement]\",\n",
|
|
" \"[disappointment]\",\n",
|
|
" \"[disapproval]\",\n",
|
|
" \"[disbelief]\",\n",
|
|
" \"[discernment]\",\n",
|
|
" \"[discomfort]\",\n",
|
|
" \"[disdain]\",\n",
|
|
" \"[disgust]\",\n",
|
|
" \"[disillusionment]\",\n",
|
|
" \"[dislike]\",\n",
|
|
" \"[dismissive]\",\n",
|
|
" \"[distress]\",\n",
|
|
" \"[doubt]\",\n",
|
|
" \"[dread]\",\n",
|
|
" \"[eagerness]\",\n",
|
|
" \"[effervescence]\",\n",
|
|
" \"[embarrassment]\",\n",
|
|
" \"[embitterment]\",\n",
|
|
" \"[embracement]\",\n",
|
|
" \"[empathy]\",\n",
|
|
" \"[emphasis]\",\n",
|
|
" \"[enchantment]\",\n",
|
|
" \"[encouraging]\",\n",
|
|
" \"[energetic]\",\n",
|
|
" \"[enjoyment]\",\n",
|
|
" \"[enthusiasm]\",\n",
|
|
" \"[enthusiastic]\",\n",
|
|
" \"[excitement]\",\n",
|
|
" \"[exhaustion]\",\n",
|
|
" \"[explaining]\",\n",
|
|
" \"[fascination]\",\n",
|
|
" \"[fast]\",\n",
|
|
" \"[fear]\",\n",
|
|
" \"[focus]\",\n",
|
|
" \"[fondness]\",\n",
|
|
" \"[friendly]\",\n",
|
|
" \"[frustration]\",\n",
|
|
" \"[gratification]\",\n",
|
|
" \"[gratitude]\",\n",
|
|
" \"[grief]\",\n",
|
|
" \"[guilt]\",\n",
|
|
" \"[happy]\",\n",
|
|
" \"[high energy]\",\n",
|
|
" \"[hope]\",\n",
|
|
" \"[horror]\",\n",
|
|
" \"[humor]\",\n",
|
|
" \"[hurt]\",\n",
|
|
" \"[incredulity]\",\n",
|
|
" \"[indifference]\",\n",
|
|
" \"[indignation]\",\n",
|
|
" \"[informative]\",\n",
|
|
" \"[instruction]\",\n",
|
|
" \"[interest]\",\n",
|
|
" \"[intrigue]\",\n",
|
|
" \"[invitation]\",\n",
|
|
" \"[joy]\",\n",
|
|
" \"[laughs]\",\n",
|
|
" \"[logical reasoning]\",\n",
|
|
" \"[long pause]\",\n",
|
|
" \"[love]\",\n",
|
|
" \"[low energy]\",\n",
|
|
" \"[melancholy]\",\n",
|
|
" \"[mixed]\",\n",
|
|
" \"[negative]\",\n",
|
|
" \"[negative surprise]\",\n",
|
|
" \"[nervousness]\",\n",
|
|
" \"[neutral]\",\n",
|
|
" \"[nostalgia]\",\n",
|
|
" \"[observation]\",\n",
|
|
" \"[offense]\",\n",
|
|
" \"[optimism]\",\n",
|
|
" \"[pain]\",\n",
|
|
" \"[panic]\",\n",
|
|
" \"[passion]\",\n",
|
|
" \"[passive]\",\n",
|
|
" \"[pensive]\",\n",
|
|
" \"[pessimism]\",\n",
|
|
" \"[pity]\",\n",
|
|
" \"[planning]\",\n",
|
|
" \"[playful]\",\n",
|
|
" \"[pleading]\",\n",
|
|
" \"[pleased]\",\n",
|
|
" \"[positive]\",\n",
|
|
" \"[positive surprise]\",\n",
|
|
" \"[praise]\",\n",
|
|
" \"[pride]\",\n",
|
|
" \"[realization]\",\n",
|
|
" \"[recognition]\",\n",
|
|
" \"[reflection]\",\n",
|
|
" \"[regret]\",\n",
|
|
" \"[relaxation]\",\n",
|
|
" \"[relief]\",\n",
|
|
" \"[reminiscence]\",\n",
|
|
" \"[resignation]\",\n",
|
|
" \"[sadness]\",\n",
|
|
" \"[sarcasm]\",\n",
|
|
" \"[satisfaction]\",\n",
|
|
" \"[self-deprecation]\",\n",
|
|
" \"[self-satisfaction]\",\n",
|
|
" \"[sentimentality]\",\n",
|
|
" \"[serenity]\",\n",
|
|
" \"[seriousness]\",\n",
|
|
" \"[shame]\",\n",
|
|
" \"[shock]\",\n",
|
|
" \"[short pause]\",\n",
|
|
" \"[skepticism]\",\n",
|
|
" \"[slight relief]\",\n",
|
|
" \"[smitten]\",\n",
|
|
" \"[solemnity]\",\n",
|
|
" \"[speculation]\",\n",
|
|
" \"[slow]\",\n",
|
|
" \"[strategizing]\",\n",
|
|
" \"[stress]\",\n",
|
|
" \"[struggle]\",\n",
|
|
" \"[success]\",\n",
|
|
" \"[suffering]\",\n",
|
|
" \"[suggestion]\",\n",
|
|
" \"[summary]\",\n",
|
|
" \"[surprise]\",\n",
|
|
" \"[suspicion]\",\n",
|
|
" \"[sympathy]\",\n",
|
|
" \"[tension]\",\n",
|
|
" \"[terror]\",\n",
|
|
" \"[thanks]\",\n",
|
|
" \"[thinking]\",\n",
|
|
" \"[thrill]\",\n",
|
|
" \"[tiredness]\",\n",
|
|
" \"[triumph]\",\n",
|
|
" \"[uncertainty]\",\n",
|
|
" \"[unclear]\",\n",
|
|
" \"[understanding]\",\n",
|
|
" \"[unease]\",\n",
|
|
" \"[urgency]\",\n",
|
|
" \"[victory]\",\n",
|
|
" \"[warning]\",\n",
|
|
" \"[weariness]\",\n",
|
|
" \"[whispers]\",\n",
|
|
" \"[wisdom]\",\n",
|
|
" \"[wistful]\",\n",
|
|
" \"[worry]\",\n",
|
|
" \"[yearning]\",\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "_cztl0y-4Yum"
|
|
},
|
|
"source": [
|
|
"Now, you'll prompt Gemini to take the podcast transcript and insert audio tags from the provided list."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "qOetTxFRdbys"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=gemini_model,\n",
|
|
" contents=f\"Please take the {podcast} provided and insert audio tags from {example_audio_tags}. Insert the tags immediately before the phrase or sentence they are meant to influence. Ensure the tag matches the emotional arc of the narrative. Avoid overusing tags, place them where a natural change in tone or pace would occur.\",\n",
|
|
")\n",
|
|
"\n",
|
|
"podcast_tags = response.text\n",
|
|
"display(Markdown(podcast_tags))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "k-X-R4zB4oSY"
|
|
},
|
|
"source": [
|
|
"Finally, you'll send the updated transcript with the audio tags to the TTS model to generate the audio with the updated modifiers."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "g2l6-5e9fpVw"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.models.generate_content(\n",
|
|
" model=speech_model,\n",
|
|
" contents=podcast_tags,\n",
|
|
" config=types.GenerateContentConfig(\n",
|
|
" response_modalities=[\"AUDIO\"],\n",
|
|
" speech_config=types.SpeechConfig(\n",
|
|
" voice_config=types.VoiceConfig(\n",
|
|
" prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name=\"Achernar\")\n",
|
|
" )\n",
|
|
" ),\n",
|
|
" ),\n",
|
|
")\n",
|
|
"\n",
|
|
"play_audio(response)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"name": "gemini_3_1_flash_tts.ipynb",
|
|
"toc_visible": true
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"name": "python3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
}
|