commit 9a6da7d40d8e637bb82ede5d3ac453707f468f5f Author: wehub-resource-sync Date: Mon Jul 13 12:03:55 2026 +0800 chore: import upstream snapshot with attribution diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..b376cdb --- /dev/null +++ b/.dockerignore @@ -0,0 +1,30 @@ +# Exclude common Python files and directories +venv/ +.venv/ +__pycache__/ +*.pyc +*.pyo +*.pyd +*.pyz +*.pyw +*.pyi +*.egg-info/ + +# Exclude development and local files +.env +.env.* +*.log +*.db +.DS_Store +.pytest_cache/ +.ruff_cache/ +.idea/ +logs/ + +# Exclude version control system files +.git/ +.gitignore +.svn/ + +storage/ +config.toml diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..18b57c2 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,16 @@ +# Auto detect text files and normalize line endings in the repository +* text=auto + +# Binary assets +*.mp3 binary +*.mp4 binary +*.png binary +*.jpg binary +*.jpeg binary +*.gif binary +*.ico binary +*.ttf binary +*.ttc binary +*.otf binary +*.woff binary +*.woff2 binary diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 0000000..3e0e95f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,87 @@ +name: 🐛 Bug | Bug Report +description: 报告错误或异常问题 | Report an error or unexpected behavior +title: "[Bug]: " +labels: + - bug + +body: + - type: markdown + attributes: + value: | + **提交问题前，请确保您已阅读以下文档：[Getting Started (English)](https://github.com/harry0703/MoneyPrinterTurbo/blob/main/README-en.md#system-requirements-) 或 [快速开始 (中文)](https://github.com/harry0703/MoneyPrinterTurbo/blob/main/README.md#%E5%BF%AB%E9%80%9F%E5%BC%80%E5%A7%8B-)。** + + **Before submitting an issue, please make sure you've read the following documentation: [Getting Started (English)](https://github.com/harry0703/MoneyPrinterTurbo/blob/main/README-en.md#system-requirements-) or [快速开始 (Chinese)](https://github.com/harry0703/MoneyPrinterTurbo/blob/main/README.md#%E5%BF%AB%E9%80%9F%E5%BC%80%E5%A7%8B-).** + + - type: textarea + attributes: + label: 问题描述 | Current Behavior + description: | + 描述您遇到的问题 + Describe the issue you're experiencing + placeholder: | + 当我执行...操作时，程序出现了...问题 + When I perform..., the program shows... + validations: + required: true + - type: textarea + attributes: + label: 重现步骤 | Steps to Reproduce + description: | + 详细描述如何重现此问题 + Describe in detail how to reproduce this issue + placeholder: | + 1. 打开... + 2. 点击... + 3. 出现错误... + + 1. Open... + 2. Click on... + 3. Error occurs... + validations: + required: true + - type: textarea + attributes: + label: 错误日志 | Error Logs + description: | + 请提供相关错误信息或日志（注意不要包含敏感信息） + Please provide any error messages or logs (be careful not to include sensitive information) + placeholder: | + 错误信息、日志或截图... + Error messages, logs, or screenshots... + validations: + required: true + - type: input + attributes: + label: Python 版本 | Python Version + description: | + 您使用的 Python 版本 + The Python version you're using + placeholder: v3.13.0, v3.10.0, etc. + validations: + required: true + - type: input + attributes: + label: 操作系统 | Operating System + description: | + 您的操作系统信息 + Your operating system information + placeholder: macOS 14.1, Windows 11, Ubuntu 22.04, etc. + validations: + required: true + - type: input + attributes: + label: MoneyPrinterTurbo 版本 | Version + description: | + 您使用的 MoneyPrinterTurbo 版本 + The version of MoneyPrinterTurbo you're using + placeholder: v1.2.2, etc. + validations: + required: true + - type: textarea + attributes: + label: 补充信息 | Additional Information + description: | + 其他对解决问题有帮助的信息（如截图、视频等） + Any other information that might help solve the issue (screenshots, videos, etc.) + validations: + required: false \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..ec4bb38 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1 @@ +blank_issues_enabled: false \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml new file mode 100644 index 0000000..abd3045 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -0,0 +1,29 @@ +name: ✨ 增加功能 | Feature Request +description: 为此项目提出一个新想法或建议 | Suggest a new idea for this project +title: "[Feature]: " +labels: + - enhancement + +body: + - type: textarea + attributes: + label: 需求描述 | Problem Statement + description: | + 请描述您希望解决的问题或需求 + Please describe the problem you want to solve + placeholder: | + 我在使用过程中遇到了... + I encountered... when using this project + validations: + required: true + - type: textarea + attributes: + label: 建议的解决方案 | Proposed Solution + description: | + 请描述您认为可行的解决方案或实现方式 + Please describe your suggested solution or implementation + placeholder: | + 可以考虑添加...功能来解决这个问题 + Consider adding... feature to address this issue + validations: + required: true \ No newline at end of file diff --git a/.github/SECURITY.md b/.github/SECURITY.md new file mode 100644 index 0000000..dd7135e --- /dev/null +++ b/.github/SECURITY.md @@ -0,0 +1,28 @@ +# Security Policy + +## Supported Versions + +Security fixes are applied on a best-effort basis to the latest `main` branch and the most recent published release line. + +## Reporting a Vulnerability + +Please do **not** disclose suspected vulnerabilities in public GitHub issues. + +Preferred process: + +1. Use GitHub private vulnerability reporting for this repository if it is available in the repository security settings. +2. If private reporting is not available, open a minimal public issue that only requests a private contact channel and does **not** include vulnerability details, proof-of-concept code, payloads, or sensitive file paths. +3. Wait for a maintainer response before sharing any technical details publicly. + +When reporting a vulnerability privately, include: + +- affected commit, tag, or release version +- attack surface or vulnerable endpoint +- impact summary +- reproduction conditions +- suggested remediation, if available + +## Disclosure Expectations + +- Please give maintainers reasonable time to investigate and prepare a fix before public disclosure. +- Once a fix is available, coordinated public disclosure is welcome. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..0201bcc --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,35 @@ +name: CI + +on: + pull_request: + push: + branches: + - main + +jobs: + tests: + name: Python tests + runs-on: ubuntu-latest + + steps: + - name: Check out repository + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install uv + uses: astral-sh/setup-uv@v6 + with: + enable-cache: true + + - name: Install dependencies + run: uv sync --frozen + + - name: Compile Python sources + run: uv run python -m compileall app cli.py main.py webui test + + - name: Run deterministic unit tests + run: uv run python -X utf8 -m unittest test.services.test_state test.services.test_task test.services.test_schema test.services.test_video_effects test.services.test_webui_i18n diff --git a/.github/workflows/docker-ghcr.yml b/.github/workflows/docker-ghcr.yml new file mode 100644 index 0000000..52375fd --- /dev/null +++ b/.github/workflows/docker-ghcr.yml @@ -0,0 +1,73 @@ +name: Publish Docker image + +on: + push: + branches: + - main + tags: + - "v*" + paths-ignore: + - "README*.md" + - "docs/**" + - ".github/workflows/docker-ghcr.yml" + workflow_dispatch: + +permissions: + contents: read + packages: write + +concurrency: + group: docker-ghcr-${{ github.ref }} + cancel-in-progress: true + +env: + IMAGE_NAME: ghcr.io/harry0703/moneyprinterturbo + +jobs: + publish: + name: Build and publish + runs-on: ubuntu-latest + timeout-minutes: 90 + + steps: + - name: Check out repository + uses: actions/checkout@v4 + + - name: Set up QEMU + uses: docker/setup-qemu-action@v3 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Log in to GHCR + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract Docker metadata + id: meta + uses: docker/metadata-action@v5 + with: + images: ${{ env.IMAGE_NAME }} + tags: | + type=ref,event=branch + type=ref,event=tag + type=semver,pattern={{version}} + type=raw,value=latest,enable={{is_default_branch}} + + - name: Build and push Docker image + uses: docker/build-push-action@v6 + with: + context: . + file: ./Dockerfile + push: true + platforms: linux/amd64,linux/arm64 + build-args: | + DOCKER_BUILD_MIRROR=default + PIP_USE_OFFICIAL=1 + tags: ${{ steps.meta.outputs.tags }} + labels: ${{ steps.meta.outputs.labels }} + cache-from: type=gha + cache-to: type=gha,mode=max diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..e1575a0 --- /dev/null +++ b/.gitignore @@ -0,0 +1,40 @@ +.DS_Store +/config.toml +/config.toml.bak +/storage/ +/.idea/ +/app/services/__pycache__ +/app/__pycache__/ +/app/config/__pycache__/ +/app/models/__pycache__/ +/app/utils/__pycache__/ +/*/__pycache__/* +.vscode +/**/.streamlit +__pycache__ +logs/ + +node_modules +# VuePress 默认临时文件目录 +/sites/docs/.vuepress/.temp +# VuePress 默认缓存目录 +/sites/docs/.vuepress/.cache +# VuePress 默认构建生成的静态文件目录 +/sites/docs/.vuepress/dist +# 模型目录 +/models/ +./models/* + +venv/ +.venv + +# Debug and test files +CLAUDE.md +debug/ +debug_*.py +test_*.py +!test/services/test_*.py +streamlit.log +.codegraph +/docs/superpowers +.understand-anything diff --git a/.python-version b/.python-version new file mode 100644 index 0000000..2c07333 --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.11 diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..8a0c46c --- /dev/null +++ b/Dockerfile @@ -0,0 +1,82 @@ +# Use an official Python runtime as a parent image +FROM python:3.11-slim-bullseye + +# Set the working directory in the container +WORKDIR /MoneyPrinterTurbo + +# 设置/MoneyPrinterTurbo目录权限为777 +RUN chmod 777 /MoneyPrinterTurbo + +ENV PYTHONPATH="/MoneyPrinterTurbo" + +# 本地用户默认继续优先使用国内镜像；GitHub Actions 发布 GHCR 镜像时使用 default， +# 避免海外 runner 访问国内镜像过慢导致镜像发布长时间卡住。 +ARG DOCKER_BUILD_MIRROR=china +ARG PIP_USE_OFFICIAL=0 + +# Install system dependencies with retry logic +RUN if [ "$DOCKER_BUILD_MIRROR" = "china" ]; then \ + echo "deb http://mirrors.aliyun.com/debian bullseye main" > /etc/apt/sources.list && \ + echo "deb http://mirrors.aliyun.com/debian-security bullseye-security main" >> /etc/apt/sources.list; \ + else \ + echo "Using default Debian mirrors"; \ + fi && \ + ( \ + for i in 1 2 3; do \ + echo "Attempt $i: installing system dependencies"; \ + apt-get update && apt-get install -y --no-install-recommends \ + git \ + ffmpeg && break || \ + echo "Attempt $i failed, retrying..."; \ + if [ "$DOCKER_BUILD_MIRROR" = "china" ] && [ $i -eq 3 ]; then \ + echo "Aliyun mirror failed, switching to Tsinghua mirror"; \ + sed -i 's/mirrors.aliyun.com/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list && \ + sed -i 's/mirrors.aliyun.com\/debian-security/mirrors.tuna.tsinghua.edu.cn\/debian-security/g' /etc/apt/sources.list && \ + ( \ + apt-get update && apt-get install -y --no-install-recommends \ + git \ + ffmpeg || \ + ( \ + echo "Tsinghua mirror failed, switching to default Debian mirror"; \ + sed -i 's/mirrors.tuna.tsinghua.edu.cn/deb.debian.org/g' /etc/apt/sources.list && \ + sed -i 's/mirrors.tuna.tsinghua.edu.cn\/debian-security/security.debian.org/g' /etc/apt/sources.list; \ + apt-get update && apt-get install -y --no-install-recommends \ + git \ + ffmpeg; \ + ); \ + ); \ + fi; \ + sleep 5; \ + done \ + ) && rm -rf /var/lib/apt/lists/* + +# Copy only the requirements.txt first to leverage Docker cache +COPY requirements.txt ./ + +# 本地默认优先国内 PyPI 镜像；GHCR 发布使用官方 PyPI，避免海外 runner 因跨境镜像访问变慢。 +RUN if [ "$PIP_USE_OFFICIAL" = "1" ]; then \ + pip install --no-cache-dir --retries 3 --timeout 60 -r requirements.txt; \ + else \ + pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com --retries 3 --timeout 60 -r requirements.txt || \ + pip install --no-cache-dir -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/ --trusted-host mirrors.tuna.tsinghua.edu.cn --retries 3 --timeout 60 -r requirements.txt || \ + pip install --no-cache-dir --retries 3 --timeout 60 -r requirements.txt; \ + fi + +# Now copy the rest of the codebase into the image +COPY . . + +# Expose the port the app runs on +EXPOSE 8501 + +# 容器内部必须监听 0.0.0.0，宿主机仍通过 docker 端口映射限制为 127.0.0.1。 +# browser.serverAddress 只决定浏览器展示的访问地址，不能替代 server.address。 +CMD ["streamlit", "run", "./webui/Main.py", "--server.address=0.0.0.0", "--server.port=8501", "--browser.serverAddress=127.0.0.1", "--server.enableCORS=True", "--browser.gatherUsageStats=False", "--client.toolbarMode=minimal", "--logger.hideWelcomeMessage=True", "--server.showEmailPrompt=False"] + +# 1. Build the Docker image using the following command +# docker build -t moneyprinterturbo . + +# 2. Run the Docker container using the following command +## For Linux or MacOS: +# docker run -v $(pwd)/config.toml:/MoneyPrinterTurbo/config.toml -v $(pwd)/storage:/MoneyPrinterTurbo/storage -p 127.0.0.1:8501:8501 moneyprinterturbo +## For Windows: +# docker run -v ${PWD}/config.toml:/MoneyPrinterTurbo/config.toml -v ${PWD}/storage:/MoneyPrinterTurbo/storage -p 127.0.0.1:8501:8501 moneyprinterturbo diff --git a/Dockerfile.gpu b/Dockerfile.gpu new file mode 100644 index 0000000..5a5b5b3 --- /dev/null +++ b/Dockerfile.gpu @@ -0,0 +1,52 @@ +# Use NVIDIA CUDA runtime as parent image (includes CUDA 12.1 + cuDNN 8) +FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 + +# Avoid interactive timezone prompt +ENV DEBIAN_FRONTEND=noninteractive + +# Set the working directory in the container +WORKDIR /MoneyPrinterTurbo +RUN chmod 777 /MoneyPrinterTurbo + +ENV PYTHONPATH="/MoneyPrinterTurbo" + +# Install Python 3.11 and system dependencies +RUN apt-get update && apt-get install -y --no-install-recommends \ + software-properties-common \ + git \ + ffmpeg \ + curl \ + && add-apt-repository ppa:deadsnakes/ppa \ + && apt-get update && apt-get install -y --no-install-recommends \ + python3.11 \ + python3.11-venv \ + python3.11-dev \ + python3-pip \ + && ln -sf /usr/bin/python3.11 /usr/bin/python3 \ + && ln -sf /usr/bin/python3.11 /usr/bin/python \ + && python3.11 -m pip install --upgrade pip \ + && apt-get remove -y python3-blinker || true \ + && rm -rf /var/lib/apt/lists/* + +# Copy only the requirements.txt first to leverage Docker cache +COPY requirements.txt ./ + +# Install Python dependencies +RUN python3 -m pip install --no-cache-dir \ + -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com \ + --retries 3 --timeout 60 -r requirements.txt || \ + python3 -m pip install --no-cache-dir \ + -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/ --trusted-host mirrors.tuna.tsinghua.edu.cn \ + --retries 3 --timeout 60 -r requirements.txt || \ + python3 -m pip install --no-cache-dir \ + --retries 3 --timeout 60 -r requirements.txt + +# Now copy the rest of the codebase into the image +COPY . . + +# Expose the port the app runs on +EXPOSE 8501 + +# 容器内部必须监听 0.0.0.0，宿主机仍通过 docker 端口映射限制为 127.0.0.1。 +# browser.serverAddress 只决定浏览器展示的访问地址，不能替代 server.address。 +CMD ["streamlit", "run", "./webui/Main.py", "--server.address=0.0.0.0", "--server.port=8501", "--browser.serverAddress=127.0.0.1", "--server.enableCORS=True", "--browser.gatherUsageStats=False", "--client.toolbarMode=minimal", "--logger.hideWelcomeMessage=True", "--server.showEmailPrompt=False"] diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..3b409c9 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2024 Harry + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/README-en.md b/README-en.md new file mode 100644 index 0000000..c92a1fc --- /dev/null +++ b/README-en.md @@ -0,0 +1,485 @@ +
+ +# MoneyPrinterTurbo 💸 + +### An All-in-One AI Short Video Generator + +Provide a video topic or keyword, and MoneyPrinterTurbo will generate the script, match footage, create subtitles and background music, and produce an HD short video. + +[![Version](https://img.shields.io/github/v/release/harry0703/MoneyPrinterTurbo?color=blue&label=version)](https://github.com/harry0703/MoneyPrinterTurbo/releases) +[![Platform](https://img.shields.io/badge/platform-Windows%20%7C%20macOS%20%7C%20Linux-lightgrey.svg)](https://github.com/harry0703/MoneyPrinterTurbo/releases/latest) +[![Python](https://img.shields.io/badge/python-3.11%2B-3776AB?logo=python&logoColor=white)](https://www.python.org/) +[![Downloads](https://img.shields.io/github/downloads/harry0703/MoneyPrinterTurbo/total)](https://github.com/harry0703/MoneyPrinterTurbo/releases/latest) + +harry0703%2FMoneyPrinterTurbo | Trendshift +Star History Rank + +English | [简体中文](README.md) | [Releases](https://github.com/harry0703/MoneyPrinterTurbo/releases) | [Issues](https://github.com/harry0703/MoneyPrinterTurbo/issues) + +
+ +## Screenshots 🖥️ + +

WebUI

+ +![](docs/webui-en.jpg) + +

API

+ +![](docs/api.jpg) + +## Special Thanks ❤️ + +
+ Kimi sponsors MoneyPrinterTurbo +
+ +Thanks to [Kimi](https://platform.kimi.ai?aff=MoneyPrinterTurbo) for sponsoring this project! [Kimi K2.7 Code](https://platform.kimi.ai/docs/guide/kimi-k2-7-code-quickstart) is an open-source, coding-focused agentic model developed by Moonshot AI, with substantial gains on real-world long-horizon coding tasks and higher end-to-end success across complex software engineering workflows. It also cuts thinking-token usage by approximately 30% compared with K2.6. Within MoneyPrinterTurbo, Kimi's LLM powers video creation: it writes the video script and extracts the search keywords that decide the final footage, so the sharper its understanding, the more on-topic the results. + +**MoneyPrinterTurbo already supports Kimi. Visit the [Kimi Open Platform](https://platform.kimi.ai?aff=MoneyPrinterTurbo) ([中文站](https://platform.kimi.com?aff=MoneyPrinterTurbo) | [Global](https://platform.kimi.ai?aff=MoneyPrinterTurbo)) to try the API, or explore the cost-effective [Coding Plan](https://www.kimi.com/code?aff=MoneyPrinterTurbo).** + +
+ + + + + + + + + + + + + + + + + + + + + + + + + +
+ BytePlus
+ BytePlus ModelArk + 		+ Thanks to Dola Seed for sponsoring this project! Dola Seed 2.0 is a full-modal general large model independently developed by ByteDance for the global market. Built on a unified multimodal architecture, it supports joint understanding and generation of text, images, audio, and video. It natively enables agent collaboration, with strong reasoning, long-task execution, tool integration, and coding capabilities. Register via this link to get 500,000 tokens of free inference quota per model. +
+ CCSub
+ CCSub + 		+ Thanks to CCSub for sponsoring this project! CCSub is a stable, affordable AI API relay platform — your drop-in replacement for a Claude.ai subscription. One API key gives you access to Claude Opus 4.8, Sonnet, Haiku, GPT-5, and Gemini at roughly 30% of direct API cost, with no VPN required from anywhere in the world. Compatible with Claude Code, Codex, Cursor, Cline, Continue, Windsurf, and all major AI coding tools. Register at www.ccsub.net and get $5 free credit on sign-up. +
+ Compshare
+ Compshare + 		+ Thanks to Compshare for sponsoring this project! Compshare is an AI cloud platform under UCloud that provides one-stop API access to mainstream Chinese and international models with a single key. Its CodingPlan package focuses on cost-effective Chinese models such as GLM5.2 and Deepseek-v4, while also offering stable official relay channels for overseas models across different development scenarios. It is compatible with Claude Code, Codex, and other mainstream AI coding tools and general API calls, with enterprise-grade high concurrency, 24/7 technical support, and self-service invoicing. Register now to receive up to ¥10 in free trial credits. +
+ Cubence
+ Cubence + 		+ Thanks to Cubence for supporting this project. Cubence is a platform focused on AI model API access, helping developers and teams call models in a stable and convenient way. Since its launch in September 2025, Cubence has supported API access scenarios for Claude Code, Codex, Gemini, and other AI models and developer tools, making it suitable for users who need unified management and access to multiple model capabilities. Cubence offers MoneyPrinterTurbo users an exclusive discount code: MPT. Use it on your first purchase to get 10% off. +
+ RecCloud
+ RecCloud + 		+ Due to the deployment and usage of this project, there is a certain threshold for some beginner users. We would like to express our special thanks to RecCloud (AI-Powered Multimedia Service Platform) for providing a free AI Video Generator service based on this project. It allows for online use without deployment, which is very convenient. +
+ Picwish
+ Picwish + 		+ Thanks to Picwish for supporting and sponsoring this project, enabling continuous updates and maintenance. Picwish focuses on the image processing field, providing a rich set of image processing tools that extremely simplify complex operations, truly making image processing easier. +
+ +## Features 🎯 + +- [x] Provides **AI Agent**, **WebUI**, **API**, and **CLI** workflows, with code organized by controller, service, and model responsibilities +- [x] Supports **AI-generated video scripts** and custom scripts +- [x] Supports various **high-definition video** sizes + - [x] Portrait 9:16, `1080x1920` + - [x] Landscape 16:9, `1920x1080` +- [x] Supports **batch video generation**, allowing the creation of multiple videos at once, then selecting the most + satisfactory one +- [x] Supports setting the **duration of video clips**, facilitating adjustments to material switching frequency +- [x] Supports **multilingual video script** generation +- [x] Supports **Edge TTS**, **Azure Speech**, **SiliconFlow**, **Google Gemini**, **Xiaomi MiMo**, **ElevenLabs**, and **Chatterbox** speech synthesis with real-time previews +- [x] Supports **subtitle generation** with configurable fonts, position, color, size, outline, and background styles +- [x] Supports random or custom **background music** with adjustable volume +- [x] Supports your own **local assets** and free-to-use HD footage from **Pexels**, **Pixabay**, and **Coverr** +- [x] Supports leading model providers including **Kimi / Moonshot AI**, **OpenAI**, **Google Gemini**, **DeepSeek**, **Alibaba Cloud Qwen**, **Microsoft Azure OpenAI**, **ByteDance VolcEngine Ark**, **xAI Grok**, **MiniMax**, and **Xiaomi MiMo**, plus unified gateways, aggregators, and local runtimes such as **Cloudflare AI Gateway**, **Alibaba ModelScope**, **AIHubMix**, **AIML API**, **EvoLink**, **Ollama**, **OneAPI**, **LiteLLM**, **Groq**, and **Pollinations AI** +- [x] Supports one-click **cross-platform publishing** to **TikTok**, **Instagram**, and **YouTube Shorts** after video generation + +## Gallery 🎬 + +All examples below were generated with MoneyPrinterTurbo. + +### Portrait 9:16 + + + + + + + + + + + + + + +
When the City Wakes
When the City Wakes
Chinese · 14 sec		The Future of Clean Energy
The Future of Clean Energy
Chinese · 24 sec		Why We Still Explore Space
Why We Still Explore Space
Chinese · 27 sec		A Seed's Journey
A Seed's Journey
Chinese · 44 sec
The Future of Everyday Robotics
The Future of Everyday Robotics
English · 21 sec		Small Habits, Lasting Change
Small Habits, Lasting Change
English · 19 sec		Making Space for Creative Work
Making Space for Creative Work
English · 20 sec		The Science Inside Coffee
The Science Inside Coffee
English · 23 sec
+ +### Landscape 16:9 + + + + + + + + + + + + + + +
Light in the Deep Ocean
Light in the Deep Ocean
Chinese · 23 sec		How Reading Shapes Us
How Reading Shapes Us
Chinese · 23 sec		The Details of Pour-Over Coffee
The Details of Pour-Over Coffee
Chinese · 23 sec		Spring Is Made for Travel
Spring Is Made for Travel
Chinese · 14 sec
Why Ocean Conservation Matters
Why Ocean Conservation Matters
English · 25 sec		Designing More Sustainable Cities
Designing More Sustainable Cities
English · 27 sec		What Mountains Teach Us
What Mountains Teach Us
English · 18 sec		A Brief History of Human Flight
A Brief History of Human Flight
English · 59 sec
+ +## System Requirements 📦 + +- Recommended platforms: Windows 10+, macOS 11+, or a mainstream Linux distribution +- Local deployment requires Python 3.11 or later; Python 3.11 is recommended +- A GPU is not required, but it is recommended if you want faster local transcription, faster video processing, or smoother batch generation + +| Item | Minimum | Recommended | Optimal | +| ---- | ------------ | ------------ | ---------- | +| CPU | 4 cores | 6 to 8 cores | 8+ cores | +| RAM | 4 GB | 8 GB | 16+ GB | +| GPU | Not required | 4+ GB VRAM | 8+ GB VRAM | + +- If you mainly rely on cloud LLMs, cloud TTS, and online material sources, CPU and RAM matter more than GPU +- If you use `faster-whisper`, batch generation, or heavier local processing, a GPU will improve throughput noticeably + +## Quick Start 🚀 + +### Recommended Paths + +- If you do not want to install or configure the project manually: generate videos with an AI Agent +- Windows users: use the one-click package first for the fastest local trial +- macOS / Linux users: use `uv` for the primary local setup path +- If you want a more isolated runtime: use Docker deployment + +### Generate Videos with an AI Agent + +If your AI Agent can read Skill documents and operate a local terminal, send it the prompt below. The Agent will install and configure MoneyPrinterTurbo, generate the video, and return the video file path. It will ask only for required API keys that are not already configured. This workflow currently supports macOS and Windows. + +```text +Use this Skill: https://raw.githubusercontent.com/harry0703/MoneyPrinterTurbo/main/docs/skill/SKILL.md +Create a video with the topic "How AI is changing everyday life." +``` + +### Run in Google Colab + +Want to try MoneyPrinterTurbo without setting up a local environment? Run it directly in Google Colab! + +[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/harry0703/MoneyPrinterTurbo/blob/main/docs/MoneyPrinterTurbo.ipynb) + +### Windows + +Download the latest Windows one-click package from GitHub Releases, then extract it directly. + +- GitHub Release: https://github.com/harry0703/MoneyPrinterTurbo/releases/latest + +After downloading, it is recommended to **double-click** `update.bat` first to update to the **latest code**, then double-click `start.bat` to launch + +After launching, the browser will open automatically (if it opens blank, it is recommended to use **Chrome** or **Edge**) + +### macOS / Linux + +Use the local setup or Docker instructions below. + +## Installation & Deployment 📥 + +### Prerequisites + +- Local deployment requires Python 3.11 or later +- On Windows, avoid project paths containing non-ASCII characters, special characters, or spaces + +#### ① Clone the Project + +```shell +git clone https://github.com/harry0703/MoneyPrinterTurbo.git +``` + +#### ② Configure the Project (Optional) + +On first launch, the project creates `config.toml` from `config.example.toml`. You can configure the LLM provider, footage source, and related API keys directly in the WebUI basic settings. + +### Docker Deployment 🐳 + +#### ① Launch the Docker Container + +If you haven't installed Docker, please install it first https://www.docker.com/products/docker-desktop/ +If you are using a Windows system, please refer to Microsoft's documentation: + +1. https://learn.microsoft.com/en-us/windows/wsl/install +2. https://learn.microsoft.com/en-us/windows/wsl/tutorials/wsl-containers + +```shell +cd MoneyPrinterTurbo +docker compose -f docker-compose.release.yml up +``` + +> The recommended default is `docker-compose.release.yml`, which pulls the prebuilt image from GitHub Container Registry: `ghcr.io/harry0703/moneyprinterturbo:latest`. +> If you need to build the image locally, you can still run `docker compose up`. +> Before the first start, copy `config.example.toml` to `config.toml` so it can be mounted into the containers. + +#### ② Access the WebUI + +Open your browser and visit http://127.0.0.1:8501 + +#### ③ Access the API Documentation + +Open your browser and visit http://127.0.0.1:8080/docs or http://127.0.0.1:8080/redoc + +### Manual Deployment 📦 + +#### ① Create a Python Virtual Environment + +Use [uv](https://docs.astral.sh/uv/) to manage the Python environment and dependencies. The project supports Python 3.11 or later; the example below uses Python 3.11. + +```shell +git clone https://github.com/harry0703/MoneyPrinterTurbo.git +cd MoneyPrinterTurbo +uv python install 3.11 +uv sync --frozen +``` + +If you are not using `uv` yet, you can still use `venv + pip`. + +```shell +python3.11 -m venv .venv +source .venv/bin/activate +pip install -r requirements.txt +``` + +Notes: + +- `pyproject.toml` is now the primary dependency manifest. +- `uv.lock` pins the resolved environment, so `uv sync --frozen` is recommended by default. +- `requirements.txt` is kept only for legacy `pip`-based installation. + +#### ② Launch the WebUI 🌐 + +Note that you need to execute the following commands in the `root directory` of the MoneyPrinterTurbo project + +###### Windows + +```powershell +.\webui.bat +``` + +You can also run `webui.bat` in CMD. +`webui.bat` prefers the project `.venv` or bundled Python from the portable package. If no project Python is found but `uv` is installed, it automatically falls back to `uv run streamlit`. +To allow other devices on your LAN to access the WebUI, run `set MPT_WEBUI_HOST=0.0.0.0` before running `webui.bat`. + +###### macOS or Linux + +```shell +sh webui.sh +``` + +The script automatically uses the project virtual environment or `uv` and selects an available local port. To allow access from other devices on your LAN, run: + +```shell +MPT_WEBUI_HOST=0.0.0.0 sh webui.sh +``` + +After launching, the browser will open automatically + +#### ③ Launch the API Service 🚀 + +```shell +uv run python main.py +``` + +If you have already activated the virtual environment manually, you can still run: + +```shell +python main.py +``` + +#### ④ Pure CLI Mode (No Browser) ⌨️ + +If you cannot use a browser or port forwarding, generate videos directly from the +command line. The simplest complete generation command is: + +```shell +uv run python cli.py --video-subject "How AI is changing everyday life" +``` + +For the complete command reference, parameter descriptions, and usage instructions, +run: + +```shell +uv run python cli.py --help +``` + +## Voice Synthesis 🗣 + +The default provider is the free **Edge TTS**, shown as **Azure TTS V1** in the WebUI. MoneyPrinterTurbo also supports **Azure TTS V2**, **SiliconFlow TTS**, **Google Gemini TTS**, **Xiaomi MiMo TTS**, **ElevenLabs TTS**, self-hosted **Chatterbox TTS**, and a no-voice mode. + +Select a provider and voice in the WebUI, then follow the on-screen instructions for any required credentials. Edge TTS does not require an API key; [Azure TTS V2](https://portal.azure.com/) and other cloud providers require credentials from their respective platforms. See the available Edge TTS voices in the [voice list](./docs/voice-list.txt). + +## Subtitle Generation 📜 + +Two subtitle generation modes are available: + +- **edge**: Uses TTS timestamps, runs quickly without a GPU, and is the default mode. +- **whisper**: Uses local `faster-whisper` transcription when a more accurate subtitle timeline is needed. The model is downloaded on first use. + +Set `subtitle_provider` in `config.toml` to switch modes. Whisper uses the approximately 3 GB `large-v3` model by default. To use the smaller and faster, approximately 1.6 GB `large-v3-turbo` model: + +```toml +[app] +subtitle_provider = "whisper" + +[whisper] +model_size = "large-v3-turbo" +``` + +> On first use, Whisper automatically downloads the model from Hugging Face. If the automatic download fails, download `whisper-large-v3` manually from [Hugging Face](https://huggingface.co/Systran/faster-whisper-large-v3). + +After extracting the model, place the entire directory in `.\MoneyPrinterTurbo\models`. The final path should be `.\MoneyPrinterTurbo\models\whisper-large-v3`: + +``` +MoneyPrinterTurbo + ├─models + │ └─whisper-large-v3 + │ config.json + │ model.bin + │ preprocessor_config.json + │ tokenizer.json + │ vocabulary.json +``` + +## Background Music 🎵 + +Background music for videos is located in the project's `resource/songs` directory. + +> The current project includes some default music from YouTube videos. If there are copyright issues, please delete +> them. + +## Subtitle Fonts 🅰 + +Fonts for rendering video subtitles are located in the project's `resource/fonts` directory, and you can also add your +own fonts. + +## Common Questions 🤔 + +
+How do I publish to TikTok, Instagram, or YouTube Shorts? + +Create an [Upload-Post](https://upload-post.com/) account and API key, then add the following settings under `[app]` in `config.toml`: + +```toml +[app] +upload_post_enabled = true +upload_post_api_key = "your-api-key" +upload_post_username = "your-username" +upload_post_platforms = ["tiktok", "instagram", "youtube"] +upload_post_auto_upload = true +upload_post_youtube_privacy_status = "public" +``` + +Restart the app after saving. Generated videos will then be published automatically to the configured platforms. YouTube privacy can be set to `public`, `unlisted`, or `private`. + +
+ +
+RuntimeError: No ffmpeg exe could be found + +Normally, ffmpeg will be automatically downloaded and detected. +However, if your environment has issues preventing automatic downloads, you may encounter the following error: + +``` +RuntimeError: No ffmpeg exe could be found. +Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable. +``` + +In this case, you can download ffmpeg from https://www.gyan.dev/ffmpeg/builds/, unzip it, and set `ffmpeg_path` to your +actual installation path. + +```toml +[app] +# Please set according to your actual path, note that Windows path separators are \\ +ffmpeg_path = "C:\\Users\\harry\\Downloads\\ffmpeg.exe" +``` + +
+ +
+OSError: [Errno 24] Too many open files + +This issue is caused by the system's limit on the number of open files. You can solve it by modifying the system's file open limit. + +Check the current limit: + +```shell +ulimit -n +``` + +If it's too low, you can increase it, for example: + +```shell +ulimit -n 10240 +``` + +
+ +
+Whisper model download failed + +``` +LocalEntryNotFoundError: Cannot find an appropriate cached snapshot folder for the specified revision on the local disk and +outgoing traffic has been disabled. +To enable repo look-ups and downloads online, pass 'local_files_only=False' as input. +``` + +or + +``` +An error occurred while synchronizing the model Systran/faster-whisper-large-v3 from the Hugging Face Hub: +An error happened while trying to locate the files on the Hub and we cannot find the appropriate snapshot folder for the +specified revision on the local disk. Please check your internet connection and try again. +Trying to load the model directly from the local cache, if it exists. +``` + +Solution: [See how to download the model manually from Hugging Face](#subtitle-generation-) + +
+ +## Feedback & Suggestions 📢 + +- You can submit an [issue](https://github.com/harry0703/MoneyPrinterTurbo/issues) or a [pull request](https://github.com/harry0703/MoneyPrinterTurbo/pulls). + +## License 📝 + +Click to view the [`LICENSE`](LICENSE) file + +## Star History + + + + + + Star History Chart + + diff --git a/README.md b/README.md new file mode 100644 index 0000000..913c8c5 --- /dev/null +++ b/README.md @@ -0,0 +1,481 @@ +
+ +# MoneyPrinterTurbo 💸 + +### 一站式 AI 短视频生成工具 + +只需提供视频主题或关键词，即可自动生成视频脚本、匹配素材、生成字幕和背景音乐，并合成高清短视频。 + +[![Version](https://img.shields.io/github/v/release/harry0703/MoneyPrinterTurbo?color=blue&label=version)](https://github.com/harry0703/MoneyPrinterTurbo/releases) +[![Platform](https://img.shields.io/badge/platform-Windows%20%7C%20macOS%20%7C%20Linux-lightgrey.svg)](https://github.com/harry0703/MoneyPrinterTurbo/releases/latest) +[![Python](https://img.shields.io/badge/python-3.11%2B-3776AB?logo=python&logoColor=white)](https://www.python.org/) +[![Downloads](https://img.shields.io/github/downloads/harry0703/MoneyPrinterTurbo/total)](https://github.com/harry0703/MoneyPrinterTurbo/releases/latest) + +harry0703%2FMoneyPrinterTurbo | Trendshift +Star History Rank + +简体中文 | [English](README-en.md) | [版本发布](https://github.com/harry0703/MoneyPrinterTurbo/releases) | [问题反馈](https://github.com/harry0703/MoneyPrinterTurbo/issues) + +
+ +## 界面预览 🖥️ + +

WebUI

+ +![](docs/webui.jpg) + +

API

+ +![](docs/api.jpg) + +## 特别感谢 ❤️ + +
+ Kimi 赞助 MoneyPrinterTurbo +
+ +感谢 [Kimi](https://platform.kimi.com?aff=MoneyPrinterTurbo) 赞助本项目！[Kimi K2.7 Code](https://platform.kimi.com/docs/guide/kimi-k2-7-code-quickstart) 是 Moonshot AI 推出的编程专用开源智能体模型，在真实长程编程与复杂软件工程工作流中显著提升端到端任务成功率，同时优化推理效率，相比 K2.6 平均减少约 30% 的推理 token 消耗。而在本项目中，Kimi 大模型能直接驱动视频创作，不仅撰写视频文案，还会提炼素材搜索关键词、决定成片画面，文案理解越精准，匹配到的素材就越贴题。 + +**本项目已接入 Kimi 大模型。前往 [Kimi 开放平台](https://platform.kimi.com?aff=MoneyPrinterTurbo)（[中文站](https://platform.kimi.com?aff=MoneyPrinterTurbo)｜[Global](https://platform.kimi.ai?aff=MoneyPrinterTurbo)）体验 API，或了解高性价比 [Coding Plan 套餐](https://www.kimi.com/code?aff=MoneyPrinterTurbo)。** +
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
+ 火山引擎
+ 火山引擎 + 		+ 感谢 火山引擎 赞助了本项目！方舟 Agent Plan 模型订阅套餐集成了包含 Doubao-Seed、Doubao-Seedance、Doubao-Seedream 等在内的字节跳动自研 SOTA 级模型，覆盖文本、代码、图像、视频等多模态任务。最新支持 MiniMax-M3、DeepSeek-V4 系列、GLM-5.2、Doubao-Seed-2.0 系列、Kimi-K2.7 等模型，工具不限。超全模态模型与 Harness 升级一步到位，深度支持 Agent 框架与 AI 编程工具。一次订阅，可以为不同任务切换合适的 AI 引擎。方舟 Agent Plan 限时 2.5 折订阅，点击链接抢购，名额有限，先到先得。 +
+ CCSub
+ CCSub + 		+ 感谢 CCSub 赞助本项目！CCSub 是稳定、实惠的 AI API 中转平台，是 Claude Code 官方订阅的超强平替。一个 API Key 即可调用 Claude Opus 4.8、Sonnet 4.6、Haiku 4.5、GPT-5、Gemini 等模型，价格约为官方直连的 1/3，全球直连无需梯子。兼容 Claude Code、Codex、Cursor、Cline、Continue、Windsurf 等所有主流 AI 编程工具。前往 www.ccsub.net 注册即送 $5 体验额度。 +
+ 优云智算
+ 优云智算 + 		+ 感谢 优云智算 赞助本项目！优云智算是 UCloud 旗下 AI 云平台，一站式提供国内外主流模型的 API 服务，一个 Key 即可调用所有模型。主打高性价比国产模型 CodingPlan 套餐（GLM5.2、Deepseek-v4等），同时提供官方转发的稳定海外模型通道，满足多场景开发需求。已兼容 Claude Code、Codex 等主流 AI 编程工具及通用 API 调用，支持企业级高并发、7×24 技术支持和自助开票。点击注册，最高可获得 ¥10 免费体验金。 +
+ Cubence
+ Cubence + 		+ 感谢 Cubence 对本项目的支持。Cubence 是一家专注于 AI 模型 API 接入服务的平台，致力于为开发者和团队提供稳定、便捷的模型调用体验。自 2025 年 9 月上线以来，Cubence 已支持 Claude Code、Codex、Gemini 等多种 AI 模型与开发工具相关的 API 接入场景，适合需要统一管理和调用多模型能力的用户使用。Cubence 为本开源项目用户提供了专属优惠码：MPT。首次购买时使用该优惠码，可享受 9 折优惠。 +
+ 录咖
+ 录咖 AI + 		+ 由于该项目的 部署 和 使用，对于一些小白用户来说，还是 有一定的门槛，在此特别感谢 录咖（AI智能 多媒体服务平台） 网站基于该项目，提供的免费 AI视频生成器 服务，可以不用部署，直接在线使用，非常方便。 +
+ 佐糖
+ 佐糖 + 		+ 感谢 佐糖 对该项目的支持和赞助，使得该项目能够持续的更新和维护。佐糖专注于图像处理领域，提供丰富的图像处理工具，将复杂操作极致简化，真正实现让图像处理更简单。 +
+ +## 功能特性 🎯 + +- [x] 提供 **AI Agent**、**WebUI**、**API** 和 **CLI** 四种使用方式，代码按控制器、服务和模型等职责分层 +- [x] 支持 **AI 自动生成视频脚本**，也可以使用自定义脚本 +- [x] 支持多种 **高清视频** 尺寸 + - [x] 竖屏 9:16，`1080x1920` + - [x] 横屏 16:9，`1920x1080` +- [x] 支持 **批量视频生成**，可以一次生成多个视频，然后选择一个最满意的 +- [x] 支持 **视频片段时长** 设置，方便调节素材切换频率 +- [x] 支持 **多语言视频脚本** 生成 +- [x] 支持 **Edge TTS**、**Azure Speech**、**SiliconFlow**、**Google Gemini**、**小米 MiMo**、**ElevenLabs** 和 **Chatterbox** 语音合成，可实时试听 +- [x] 支持 **字幕生成**，可调整字体、位置、颜色、大小、描边和背景样式 +- [x] 支持 **背景音乐**，可随机选择或使用指定音乐，并调整音量 +- [x] 支持使用自己的 **本地素材**，也可从 **Pexels**、**Pixabay** 和 **Coverr** 获取可免费使用的高清素材 +- [x] 支持 **Kimi / Moonshot AI**、**OpenAI**、**Google Gemini**、**DeepSeek**、**阿里云通义千问**、**Microsoft Azure OpenAI**、**火山引擎方舟**、**xAI Grok**、**MiniMax**、**小米 MiMo** 等主流模型服务，并兼容 **Cloudflare AI Gateway**、**魔搭 ModelScope**、**AIHubMix**、**AIML API**、**EvoLink**、**Ollama**、**OneAPI**、**LiteLLM**、**Groq**、**Pollinations AI** 等统一网关、聚合平台和本地运行环境 +- [x] 支持一键 **跨平台发布**，生成完成后可自动上传至 **TikTok**、**Instagram** 和 **YouTube Shorts** + +## 作品展示 🎬 + +以下示例均由 MoneyPrinterTurbo 实际生成。 + +### 竖屏 9:16 + + + + + + + + + + + + + + +
城市醒来的时刻
城市醒来的时刻
中文 · 14 秒		清洁能源的未来
清洁能源的未来
中文 · 24 秒		为什么我们仍要探索太空
为什么我们仍要探索太空
中文 · 27 秒		一粒种子的旅程
一粒种子的旅程
中文 · 44 秒
The Future of Everyday Robotics
The Future of Everyday Robotics
English · 21 sec		Small Habits, Lasting Change
Small Habits, Lasting Change
English · 19 sec		Making Space for Creative Work
Making Space for Creative Work
English · 20 sec		The Science Inside Coffee
The Science Inside Coffee
English · 23 sec
+ +### 横屏 16:9 + + + + + + + + + + + + + + +
深海里的微光
深海里的微光
中文 · 23 秒		阅读如何塑造我们
阅读如何塑造我们
中文 · 23 秒		一杯手冲咖啡的细节
一杯手冲咖啡的细节
中文 · 23 秒		春天适合出发
春天适合出发
中文 · 14 秒
Why Ocean Conservation Matters
Why Ocean Conservation Matters
English · 25 sec		Designing More Sustainable Cities
Designing More Sustainable Cities
English · 27 sec		What Mountains Teach Us
What Mountains Teach Us
English · 18 sec		A Brief History of Human Flight
A Brief History of Human Flight
English · 59 sec
+ +## 配置要求 📦 + +- 建议系统：Windows 10、macOS 11.0 或更高版本，以及主流 Linux 发行版 +- 本地部署需要 Python 3.11 或更高版本，推荐使用 Python 3.11 +- GPU 不是必需项，但如果你希望本地转录、更快的视频处理或更顺畅的批量生成体验，建议使用带显存的独立显卡 + +| 项目 | 最低配置 | 推荐配置 | 理想配置 | +| ---- | -------- | --------------- | --------------- | +| CPU | 4 核 | 6 到 8 核 | 8 核及以上 | +| RAM | 4 GB | 8 GB | 16 GB 及以上 | +| GPU | 非必须 | 4 GB 显存及以上 | 8 GB 显存及以上 | + +- 如果你主要依赖云端 LLM、云端 TTS 和在线素材源，CPU 与内存比 GPU 更重要 +- 如果你启用 `faster-whisper`、批量生成或更重的本地处理链路，GPU 会明显提升速度 + +## 快速开始 🚀 + +### 推荐使用方式 + +- 不想手动安装和配置：直接使用 AI Agent 生成视频 +- Windows 用户：优先使用一键启动包，适合快速体验 +- macOS / Linux 用户：优先使用 `uv` 进行本地部署 +- 想要隔离运行环境：优先使用 Docker 部署 + +### 使用 AI Agent 生成视频 + +如果你的 AI Agent 支持读取 Skill 文档并操作本地终端，可以直接发送下面这段话。Agent 会自动完成安装、配置和视频生成；只有缺少必要的 API Key 时才会向你询问，完成后会返回生成的视频文件路径。目前支持 macOS 和 Windows。 + +```text +使用这个 Skill：https://raw.githubusercontent.com/harry0703/MoneyPrinterTurbo/main/docs/skill/SKILL.md +帮我生成一个主题为“人工智能如何改变普通人的日常生活”的视频。 +``` + +### 在 Google Colab 中运行 + +免去本地环境配置，点击直接在 Google Colab 中快速体验 MoneyPrinterTurbo + +[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/harry0703/MoneyPrinterTurbo/blob/main/docs/MoneyPrinterTurbo.ipynb) + +### Windows 一键启动包 + +下载一键启动包，解压直接使用（路径不要有 **中文**、**特殊字符**、**空格**） + +- GitHub Releases：https://github.com/harry0703/MoneyPrinterTurbo/releases/latest + +下载后，建议先**双击执行** `update.bat` 更新到**最新代码**，然后双击 `start.bat` 启动 + +启动后，会自动打开浏览器（如果打开是空白，建议换成 **Chrome** 或者 **Edge** 打开） + +## 安装部署 📥 + +### 前提条件 + +- 本地部署需要 Python 3.11 或更高版本 +- Windows 用户建议避免使用包含中文、特殊字符或空格的项目路径 + +#### ① 克隆代码 + +```shell +git clone https://github.com/harry0703/MoneyPrinterTurbo.git +``` + +#### ② 配置项目（可选） + +首次启动时，项目会根据 `config.example.toml` 自动创建 `config.toml`。大模型 Provider、素材来源和相关 API Key 可以直接在 WebUI 的基础设置中配置。 + +### Docker 部署 🐳 + +#### ① 启动 Docker + +如果未安装 Docker，请先安装 https://www.docker.com/products/docker-desktop/ + +Windows 用户可以参考微软的文档： + +1. https://learn.microsoft.com/zh-cn/windows/wsl/install +2. https://learn.microsoft.com/zh-cn/windows/wsl/tutorials/wsl-containers + +```shell +cd MoneyPrinterTurbo +docker compose -f docker-compose.release.yml up +``` + +> 默认推荐使用 `docker-compose.release.yml`，它会直接拉取 GitHub Container Registry 上的预构建镜像：`ghcr.io/harry0703/moneyprinterturbo:latest`。 +> 如果你需要本地重新构建镜像，可以继续使用 `docker compose up`。 +> 首次启动前，请将 `config.example.toml` 复制为 `config.toml`，供容器挂载使用。 + +#### ② 访问 WebUI + +打开浏览器，访问 http://127.0.0.1:8501 + +#### ③ 访问 API 文档 + +打开浏览器，访问 http://127.0.0.1:8080/docs 或者 http://127.0.0.1:8080/redoc + +### 手动部署 📦 + +> 视频教程 + +- 完整的使用演示：https://v.douyin.com/iFhnwsKY/ +- 如何在 Windows 上部署：https://v.douyin.com/iFyjoW3M + +#### ① 创建虚拟环境 + +推荐使用 [uv](https://docs.astral.sh/uv/) 管理 Python 环境和依赖。项目支持 Python 3.11 或更高版本，以下示例使用 Python 3.11。 + +```shell +git clone https://github.com/harry0703/MoneyPrinterTurbo.git +cd MoneyPrinterTurbo +uv python install 3.11 +uv sync --frozen +``` + +如果你暂时不使用 `uv`，也可以继续使用 `venv + pip` + +```shell +python3.11 -m venv .venv +source .venv/bin/activate +pip install -r requirements.txt +``` + +说明： + +- `pyproject.toml` 是主依赖定义文件 +- `uv.lock` 是锁文件，建议默认执行 `uv sync --frozen` +- `requirements.txt` 仅保留给旧的 `pip` 安装方式兼容使用 + +#### ② 启动 WebUI 🌐 + +注意需要到 MoneyPrinterTurbo 项目 `根目录` 下执行以下命令 + +###### Windows + +```powershell +.\webui.bat +``` + +在 CMD 中也可以执行 `webui.bat`。 +`webui.bat` 会优先使用项目 `.venv` 或一键包内置 Python；如果没有找到项目 Python，但已安装 `uv`，会自动切换为 `uv run streamlit`。 +如需允许局域网内其他设备访问 WebUI，可以先执行 `set MPT_WEBUI_HOST=0.0.0.0`，再运行 `webui.bat`。 + +###### macOS 或 Linux + +```shell +sh webui.sh +``` + +脚本会自动使用项目虚拟环境或 `uv`，并选择可用的本地端口。如需允许局域网内其他设备访问，可以执行： + +```shell +MPT_WEBUI_HOST=0.0.0.0 sh webui.sh +``` + +启动后，会自动打开浏览器（如果打开是空白，建议换成 **Chrome** 或者 **Edge** 打开） + +#### ③ 启动 API 服务 🚀 + +```shell +uv run python main.py +``` + +如果你已经手动激活了虚拟环境，也可以直接执行： + +```shell +python main.py +``` + +#### ④ 纯命令行方式（无浏览器）⌨️ + +如果你无法使用浏览器或端口转发，可以直接在命令行生成视频。最简单的完整视频生成命令如下： + +```shell +uv run python cli.py --video-subject "人工智能如何改变日常生活" +``` + +如需查看完整命令、参数说明和使用方法，可以执行： + +```shell +uv run python cli.py --help +``` + +## 语音合成 🗣 + +默认使用免费的 **Edge TTS**，在 WebUI 中显示为 **Azure TTS V1**。项目同时支持 **Azure TTS V2**、**SiliconFlow TTS**、**Google Gemini TTS**、**小米 MiMo TTS**、**ElevenLabs TTS**、自托管 **Chatterbox TTS**，以及无配音模式。 + +可直接在 WebUI 中选择 Provider 和音色，并按照界面提示填写所需凭据。Edge TTS 不需要 API Key；[Azure TTS V2](https://portal.azure.com/) 及其他云端服务需要对应平台的凭据。Edge TTS 音色可查看：[音色列表](./docs/voice-list.txt)。 + +## 字幕生成 📜 + +当前支持两种字幕生成方式： + +- **edge**：使用 TTS 时间戳生成字幕，速度快，不需要 GPU，默认使用该模式。 +- **whisper**：使用本地 `faster-whisper` 转写音频，适用于需要更准确字幕时间轴的场景。首次使用时需要下载模型。 + +在 `config.toml` 中修改 `subtitle_provider` 即可切换模式。Whisper 默认使用约 3 GB 的 `large-v3`；如需更小、更快的模型，可以使用约 1.6 GB 的 `large-v3-turbo`： + +```toml +[app] +subtitle_provider = "whisper" + +[whisper] +model_size = "large-v3-turbo" +``` + +> 首次使用 Whisper 时，程序会自动从 Hugging Face 下载模型。如果当前网络无法自动下载，可以从 [Hugging Face](https://huggingface.co/Systran/faster-whisper-large-v3) 手动下载 `whisper-large-v3`。 + +下载并解压后，将整个目录放到 `.\MoneyPrinterTurbo\models`，最终路径应为 `.\MoneyPrinterTurbo\models\whisper-large-v3`： + +``` +MoneyPrinterTurbo + ├─models + │ └─whisper-large-v3 + │ config.json + │ model.bin + │ preprocessor_config.json + │ tokenizer.json + │ vocabulary.json +``` + +## 背景音乐 🎵 + +用于视频的背景音乐，位于项目的 `resource/songs` 目录下。 + +> 当前项目里面放了一些默认的音乐，来自于 YouTube 视频，如有侵权，请删除。 + +## 字幕字体 🅰 + +用于视频字幕的渲染，位于项目的 `resource/fonts` 目录下，你也可以放进去自己的字体。 + +## 常见问题 🤔 + +
+如何发布到 TikTok、Instagram 或 YouTube Shorts？ + +注册 [Upload-Post](https://upload-post.com/) 账号并获取 API Key，然后在 `config.toml` 的 `[app]` 下添加以下配置： + +```toml +[app] +upload_post_enabled = true +upload_post_api_key = "your-api-key" +upload_post_username = "your-username" +upload_post_platforms = ["tiktok", "instagram", "youtube"] +upload_post_auto_upload = true +upload_post_youtube_privacy_status = "public" +``` + +保存配置并重启项目。视频生成完成后，程序会自动发布到已配置的平台。YouTube 可见性可设置为 `public`、`unlisted` 或 `private`。 + +
+ +
+RuntimeError: No ffmpeg exe could be found + +通常情况下，ffmpeg 会被自动下载，并且会被自动检测到。 +但是如果你的环境有问题，无法自动下载，可能会遇到如下错误： + +``` +RuntimeError: No ffmpeg exe could be found. +Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable. +``` + +此时你可以从 https://www.gyan.dev/ffmpeg/builds/ 下载ffmpeg，解压后，设置 `ffmpeg_path` 为你的实际安装路径即可。 + +```toml +[app] +# 请根据你的实际路径设置，注意 Windows 路径分隔符为 \\ +ffmpeg_path = "C:\\Users\\harry\\Downloads\\ffmpeg.exe" +``` + +
+ +
+OSError: [Errno 24] Too many open files + +这个问题是由于系统打开文件数限制导致的，可以通过修改系统的文件打开数限制来解决。 + +查看当前限制 + +```shell +ulimit -n +``` + +如果过低，可以调高一些，比如 + +```shell +ulimit -n 10240 +``` + +
+ +
+Whisper 模型下载失败 + +``` +LocalEntryNotFoundError: Cannot find an appropriate cached snapshot folder for the specified revision on the local disk and +outgoing traffic has been disabled. +To enable repo look-ups and downloads online, pass 'local_files_only=False' as input. +``` + +或者 + +``` +An error occurred while synchronizing the model Systran/faster-whisper-large-v3 from the Hugging Face Hub: +An error happened while trying to locate the files on the Hub and we cannot find the appropriate snapshot folder for the +specified revision on the local disk. Please check your internet connection and try again. +Trying to load the model directly from the local cache, if it exists. +``` + +解决方法：[查看如何从 Hugging Face 手动下载模型](#%E5%AD%97%E5%B9%95%E7%94%9F%E6%88%90-) + +
+ +## 反馈建议 📢 + +- 可以提交 [issue](https://github.com/harry0703/MoneyPrinterTurbo/issues) 或者 [pull request](https://github.com/harry0703/MoneyPrinterTurbo/pulls)。 + +## 许可证 📝 + +点击查看 [`LICENSE`](LICENSE) 文件 + +## Star History + + + + + + Star History Chart + + diff --git a/README.wehub.md b/README.wehub.md new file mode 100644 index 0000000..3865328 --- /dev/null +++ b/README.wehub.md @@ -0,0 +1,7 @@ +# WeHub 来源说明 + +- 原始项目：`harry0703/MoneyPrinterTurbo` +- 原始仓库：https://github.com/harry0703/MoneyPrinterTurbo +- 导入方式：上游默认分支的最新快照 +- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准 +- 本文件仅用于记录来源，不代表 WeHub 是原项目作者 diff --git a/app/__init__.py b/app/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/asgi.py b/app/asgi.py new file mode 100644 index 0000000..01f8b7d --- /dev/null +++ b/app/asgi.py @@ -0,0 +1,82 @@ +"""Application implementation - ASGI.""" + +import os + +from fastapi import FastAPI, Request +from fastapi.exceptions import RequestValidationError +from fastapi.middleware.cors import CORSMiddleware +from fastapi.responses import JSONResponse +from fastapi.staticfiles import StaticFiles +from loguru import logger + +from app.config import config +from app.models.exception import HttpException +from app.router import root_api_router +from app.utils import utils + + +def exception_handler(request: Request, e: HttpException): + return JSONResponse( + status_code=e.status_code, + content=utils.get_response(e.status_code, e.data, e.message), + ) + + +def validation_exception_handler(request: Request, e: RequestValidationError): + return JSONResponse( + status_code=400, + content=utils.get_response( + status=400, data=e.errors(), message="field required" + ), + ) + + +def get_application() -> FastAPI: + """Initialize FastAPI application. + + Returns: + FastAPI: Application object instance. + + """ + instance = FastAPI( + title=config.project_name, + description=config.project_description, + version=config.project_version, + debug=False, + ) + instance.include_router(root_api_router) + instance.add_exception_handler(HttpException, exception_handler) + instance.add_exception_handler(RequestValidationError, validation_exception_handler) + return instance + + +app = get_application() + +# Configures the CORS middleware for the FastAPI app +cors_allowed_origins_str = os.getenv("CORS_ALLOWED_ORIGINS", "") +origins = cors_allowed_origins_str.split(",") if cors_allowed_origins_str else ["*"] +app.add_middleware( + CORSMiddleware, + allow_origins=origins, + allow_credentials=True, + allow_methods=["*"], + allow_headers=["*"], +) + +task_dir = utils.task_dir() +app.mount( + "/tasks", StaticFiles(directory=task_dir, html=True, follow_symlink=True), name="" +) + +public_dir = utils.public_dir() +app.mount("/", StaticFiles(directory=public_dir, html=True), name="") + + +@app.on_event("shutdown") +def shutdown_event(): + logger.info("shutdown event") + + +@app.on_event("startup") +def startup_event(): + logger.info("startup event") diff --git a/app/config/__init__.py b/app/config/__init__.py new file mode 100644 index 0000000..dd46812 --- /dev/null +++ b/app/config/__init__.py @@ -0,0 +1,56 @@ +import os +import sys + +from loguru import logger + +from app.config import config +from app.utils import utils + + +def __init_logger(): + # _log_file = utils.storage_dir("logs/server.log") + _lvl = config.log_level + root_dir = os.path.dirname( + os.path.dirname(os.path.dirname(os.path.realpath(__file__))) + ) + + def format_record(record): + # 获取日志记录中的文件全路径 + file_path = record["file"].path + # 将绝对路径转换为相对于项目根目录的路径 + relative_path = os.path.relpath(file_path, root_dir) + # 更新记录中的文件路径 + record["file"].path = f"./{relative_path}" + # 返回修改后的格式字符串 + # 您可以根据需要调整这里的格式 + _format = ( + "{time:%Y-%m-%d %H:%M:%S} | " + + "{level} | " + + '"{file.path}:{line}": {function} ' + + "- {message}" + + "\n" + ) + return _format + + logger.remove() + + logger.add( + sys.stdout, + level=_lvl, + format=format_record, + colorize=True, + ) + + # logger.add( + # _log_file, + # level=_lvl, + # format=format_record, + # rotation="00:00", + # retention="3 days", + # backtrace=True, + # diagnose=True, + # enqueue=True, + # ) + + +__init_logger() diff --git a/app/config/config.py b/app/config/config.py new file mode 100644 index 0000000..a37150a --- /dev/null +++ b/app/config/config.py @@ -0,0 +1,287 @@ +import os +import shutil +import socket +import tempfile +import threading +from contextlib import contextmanager + +import toml +from loguru import logger + +root_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))) +config_file = f"{root_dir}/config.toml" +_CONTAINER_CGROUP_MARKERS = ("docker", "containerd", "kubepods", "libpod", "podman") +_DOCKER_HOST_GATEWAY_NAME = "host.docker.internal" +_config_save_lock = threading.RLock() +_MISSING = object() + + +class _SynchronizedConfig(dict): + """保持 dict 使用方式不变，同时让运行期配置写操作服从同一把锁。""" + + def __setitem__(self, key, value): + with _config_save_lock: + super().__setitem__(key, value) + + def __delitem__(self, key): + with _config_save_lock: + super().__delitem__(key) + + def clear(self): + with _config_save_lock: + super().clear() + + def pop(self, key, default=_MISSING): + with _config_save_lock: + if default is _MISSING: + return super().pop(key) + return super().pop(key, default) + + def setdefault(self, key, default=None): + with _config_save_lock: + return super().setdefault(key, default) + + def update(self, *args, **kwargs): + with _config_save_lock: + super().update(*args, **kwargs) + + +@contextmanager +def runtime_config_lock(): + """ + 在一次依赖全局配置的完整操作期间阻止其它 WebUI 会话改写配置。 + + 当前项目默认绑定本地回环地址，配置仍然是单用户全局配置。这个轻量锁主要 + 保护生成、试听等长操作，避免另一个标签页在操作中途切换 Provider 或密钥。 + """ + with _config_save_lock: + yield + + +def is_running_in_container( + dockerenv_path: str = "/.dockerenv", + containerenv_path: str = "/run/.containerenv", + cgroup_path: str = "/proc/1/cgroup", +) -> bool: + """ + 判断当前进程是否运行在容器内。 + + 这个判断主要用于 Ollama 默认地址选择： + - 普通本机运行时，`localhost` 指向用户机器本身； + - Docker 容器内，`localhost` 指向容器自己，访问宿主机 Ollama + 通常需要使用 `host.docker.internal`。 + + 不能只判断 `/proc/1/cgroup` 是否存在，因为普通 Linux 也会有这个文件。 + 这里只在检测到明确的容器标记时返回 True，避免误伤非 Docker Linux 用户。 + 参数保留为可注入路径，便于单元测试覆盖不同运行环境。 + """ + if os.path.isfile(dockerenv_path) or os.path.isfile(containerenv_path): + return True + + try: + with open(cgroup_path, mode="r", encoding="utf-8") as fp: + cgroup_content = fp.read().lower() + except OSError: + return False + + return any(marker in cgroup_content for marker in _CONTAINER_CGROUP_MARKERS) + + +def _can_resolve_hostname(hostname: str) -> bool: + try: + socket.gethostbyname(hostname) + except OSError: + return False + return True + + +def _decode_linux_route_gateway(hex_gateway: str) -> str: + # /proc/net/route 里的 Gateway 是 16 进制小端序，例如 010011AC 表示 + # 172.17.0.1。这里单独解析，是为了在原生 Linux Docker 没有 + # host.docker.internal DNS 记录时，还能尝试访问容器默认网关上的宿主机。 + if len(hex_gateway) != 8: + raise ValueError("invalid gateway length") + + octets = [ + str(int(hex_gateway[index : index + 2], 16)) + for index in range(6, -1, -2) + ] + return ".".join(octets) + + +def get_container_default_gateway_ip(route_path: str = "/proc/net/route") -> str: + """ + 读取 Linux 容器里的默认网关 IP。 + + Docker Desktop 通常提供 `host.docker.internal`，但原生 Linux Docker + 默认不一定提供这个 DNS 名称。默认网关通常可以作为访问宿主机服务的 + 兜底地址；如果用户的 Ollama 只监听 127.0.0.1，则仍需要用户让 + Ollama 监听宿主机网卡或手动配置 `ollama_base_url`。 + """ + try: + with open(route_path, mode="r", encoding="utf-8") as fp: + route_lines = fp.readlines() + except OSError: + return "" + + for line in route_lines[1:]: + fields = line.strip().split() + if len(fields) < 3: + continue + + destination = fields[1] + gateway = fields[2] + if destination != "00000000" or gateway == "00000000": + continue + + try: + return _decode_linux_route_gateway(gateway) + except ValueError: + logger.warning(f"invalid container gateway route entry: {line.strip()}") + return "" + + return "" + + +def get_default_ollama_base_url() -> str: + """ + 返回 Ollama 的默认 OpenAI-compatible base_url。 + + 用户显式配置 `ollama_base_url` 时不会走这里；这里只处理“未配置时的 + 最佳默认值”。容器内默认指向宿主机，普通本机运行默认指向 localhost。 + """ + if not is_running_in_container(): + return "http://localhost:11434/v1" + + if _can_resolve_hostname(_DOCKER_HOST_GATEWAY_NAME): + return f"http://{_DOCKER_HOST_GATEWAY_NAME}:11434/v1" + + gateway_ip = get_container_default_gateway_ip() + if gateway_ip: + logger.info( + "host.docker.internal is not resolvable, fallback to container " + f"default gateway for Ollama: {gateway_ip}" + ) + return f"http://{gateway_ip}:11434/v1" + + logger.warning( + "failed to resolve host.docker.internal and container default gateway; " + "fallback to host.docker.internal for Ollama" + ) + return f"http://{_DOCKER_HOST_GATEWAY_NAME}:11434/v1" + + +def load_config(): + # fix: IsADirectoryError: [Errno 21] Is a directory: '/MoneyPrinterTurbo/config.toml' + if os.path.isdir(config_file): + shutil.rmtree(config_file) + + if not os.path.isfile(config_file): + example_file = f"{root_dir}/config.example.toml" + if os.path.isfile(example_file): + shutil.copyfile(example_file, config_file) + logger.info("copy config.example.toml to config.toml") + + logger.info(f"load config from file: {config_file}") + + try: + _config_ = toml.load(config_file) + except Exception as e: + logger.warning(f"load config failed: {str(e)}, try to load as utf-8-sig") + with open(config_file, mode="r", encoding="utf-8-sig") as fp: + _cfg_content = fp.read() + _config_ = toml.loads(_cfg_content) + return _config_ + + +def save_config(): + """ + 原子保存运行时配置。 + + Streamlit 的不同会话可能在相近时间触发配置保存。直接覆盖 config.toml 时， + 另一个线程可能读取到只写了一部分的 TOML 内容。这里使用进程内可重入锁串行化 + 保存，并先写入同目录临时文件，再通过 os.replace 原子替换目标文件。 + + 这仍然保留项目现有的单用户全局配置语义，不额外引入复杂的多用户配置系统； + 主要用于避免多标签页或快速 rerun 时损坏配置文件。 + """ + with _config_save_lock: + config_to_save = dict(_cfg) + config_to_save["app"] = dict(app) + config_to_save["azure"] = dict(azure) + config_to_save["siliconflow"] = dict(siliconflow) + config_to_save["elevenlabs"] = dict(elevenlabs) + config_to_save["chatterbox"] = dict(chatterbox) + config_to_save["ui"] = dict(ui) + serialized_config = toml.dumps(config_to_save) + + # WebUI 完整 rerun 结束时会调用保存。内容没有变化时直接返回，避免每次 + # 点击普通控件都产生一次磁盘写入和 fsync。 + try: + with open(config_file, mode="r", encoding="utf-8") as f: + if f.read() == serialized_config: + _cfg.clear() + _cfg.update(config_to_save) + return + except (OSError, UnicodeError): + pass + + temp_path = "" + try: + fd, temp_path = tempfile.mkstemp( + prefix=".config-", + suffix=".toml.tmp", + dir=root_dir, + ) + with os.fdopen(fd, mode="w", encoding="utf-8") as f: + f.write(serialized_config) + f.flush() + os.fsync(f.fileno()) + os.replace(temp_path, config_file) + _cfg.clear() + _cfg.update(config_to_save) + finally: + if temp_path and os.path.exists(temp_path): + os.remove(temp_path) + + +_cfg = load_config() +app = _SynchronizedConfig(_cfg.get("app", {})) +whisper = _cfg.get("whisper", {}) +proxy = _cfg.get("proxy", {}) +azure = _SynchronizedConfig(_cfg.get("azure", {})) +siliconflow = _SynchronizedConfig(_cfg.get("siliconflow", {})) +elevenlabs = _SynchronizedConfig(_cfg.get("elevenlabs", {})) +chatterbox = _SynchronizedConfig(_cfg.get("chatterbox", {})) +ui = _SynchronizedConfig( + _cfg.get( + "ui", + { + "hide_log": False, + }, + ) +) + +hostname = socket.gethostname() + +log_level = _cfg.get("log_level", "DEBUG") +listen_host = _cfg.get("listen_host", "0.0.0.0") +listen_port = _cfg.get("listen_port", 8080) +project_name = _cfg.get("project_name", "MoneyPrinterTurbo") +project_description = _cfg.get( + "project_description", + "https://github.com/harry0703/MoneyPrinterTurbo", +) +project_version = _cfg.get("project_version", "1.3.2") +reload_debug = False + +app["redis_host"] = os.getenv( + "MPT_APP_REDIS_HOST", + os.getenv("REDIS_HOST", app.get("redis_host", "localhost")), +) + +ffmpeg_path = app.get("ffmpeg_path", "") +if ffmpeg_path and os.path.isfile(ffmpeg_path): + os.environ["IMAGEIO_FFMPEG_EXE"] = ffmpeg_path + +logger.info(f"{project_name} v{project_version}") diff --git a/app/controllers/base.py b/app/controllers/base.py new file mode 100644 index 0000000..122e341 --- /dev/null +++ b/app/controllers/base.py @@ -0,0 +1,31 @@ +from uuid import uuid4 + +from fastapi import Request + +from app.config import config +from app.models.exception import HttpException + + +def get_task_id(request: Request): + task_id = request.headers.get("x-task-id") + if not task_id: + task_id = uuid4() + return str(task_id) + + +def get_api_key(request: Request): + api_key = request.headers.get("x-api-key") + return api_key + + +def verify_token(request: Request): + token = get_api_key(request) + if token != config.app.get("api_key", ""): + request_id = get_task_id(request) + request_url = request.url + user_agent = request.headers.get("user-agent") + raise HttpException( + task_id=request_id, + status_code=401, + message=f"invalid token: {request_url}, {user_agent}", + ) diff --git a/app/controllers/manager/base_manager.py b/app/controllers/manager/base_manager.py new file mode 100644 index 0000000..3480479 --- /dev/null +++ b/app/controllers/manager/base_manager.py @@ -0,0 +1,87 @@ +import threading +from typing import Any, Callable, Dict + +from loguru import logger + + +class TaskQueueFullError(ValueError): + pass + + +class TaskManager: + def __init__(self, max_concurrent_tasks: int, max_queued_tasks: int = 100): + self.max_concurrent_tasks = max_concurrent_tasks + self.max_queued_tasks = max_queued_tasks + self.current_tasks = 0 + self.lock = threading.Lock() + self.queue = self.create_queue() + + def create_queue(self): + raise NotImplementedError() + + def add_task(self, func: Callable, *args: Any, **kwargs: Any): + with self.lock: + if self.current_tasks < self.max_concurrent_tasks: + logger.info( + f"add task: {func.__name__}, current_tasks: {self.current_tasks}" + ) + self.execute_task(func, *args, **kwargs) + else: + queue_size = self.queue_size() + # 并发数已满时才进入排队。队列必须有上限，否则匿名接口可以持续 + # 堆积任务对象和请求参数，最终造成内存耗尽或第三方 API 成本失控。 + if queue_size >= self.max_queued_tasks: + logger.warning( + f"reject task: {func.__name__}, queue_size: {queue_size}, " + f"max_queued_tasks: {self.max_queued_tasks}" + ) + raise TaskQueueFullError("task queue is full, please try again later") + + logger.info( + f"enqueue task: {func.__name__}, current_tasks: {self.current_tasks}, " + f"queue_size: {queue_size}" + ) + self.enqueue({"func": func, "args": args, "kwargs": kwargs}) + + def execute_task(self, func: Callable, *args: Any, **kwargs: Any): + thread = threading.Thread( + target=self.run_task, args=(func, *args), kwargs=kwargs + ) + thread.start() + + def run_task(self, func: Callable, *args: Any, **kwargs: Any): + try: + with self.lock: + self.current_tasks += 1 + func(*args, **kwargs) # call the function here, passing *args and **kwargs. + finally: + self.task_done() + + def check_queue(self): + with self.lock: + if ( + self.current_tasks < self.max_concurrent_tasks + and not self.is_queue_empty() + ): + task_info = self.dequeue() + func = task_info["func"] + args = task_info.get("args", ()) + kwargs = task_info.get("kwargs", {}) + self.execute_task(func, *args, **kwargs) + + def task_done(self): + with self.lock: + self.current_tasks -= 1 + self.check_queue() + + def enqueue(self, task: Dict): + raise NotImplementedError() + + def dequeue(self): + raise NotImplementedError() + + def is_queue_empty(self): + raise NotImplementedError() + + def queue_size(self): + raise NotImplementedError() diff --git a/app/controllers/manager/memory_manager.py b/app/controllers/manager/memory_manager.py new file mode 100644 index 0000000..4a2612b --- /dev/null +++ b/app/controllers/manager/memory_manager.py @@ -0,0 +1,21 @@ +from queue import Queue +from typing import Dict + +from app.controllers.manager.base_manager import TaskManager + + +class InMemoryTaskManager(TaskManager): + def create_queue(self): + return Queue(maxsize=self.max_queued_tasks) + + def enqueue(self, task: Dict): + self.queue.put(task) + + def dequeue(self): + return self.queue.get() + + def is_queue_empty(self): + return self.queue.empty() + + def queue_size(self): + return self.queue.qsize() diff --git a/app/controllers/manager/redis_manager.py b/app/controllers/manager/redis_manager.py new file mode 100644 index 0000000..861af0f --- /dev/null +++ b/app/controllers/manager/redis_manager.py @@ -0,0 +1,64 @@ +import json +from typing import Dict + +import redis + +from app.controllers.manager.base_manager import TaskManager +from app.models.schema import VideoParams +from app.services import task as tm + +FUNC_MAP = { + "start": tm.start, + # 'start_test': tm.start_test +} + + +class RedisTaskManager(TaskManager): + def __init__( + self, + max_concurrent_tasks: int, + redis_url: str, + max_queued_tasks: int = 100, + ): + self.redis_client = redis.Redis.from_url(redis_url) + super().__init__(max_concurrent_tasks, max_queued_tasks=max_queued_tasks) + + def create_queue(self): + return "task_queue" + + def enqueue(self, task: Dict): + task_with_serializable_params = task.copy() + + if "params" in task["kwargs"] and isinstance( + task["kwargs"]["params"], VideoParams + ): + task_with_serializable_params["kwargs"]["params"] = task["kwargs"][ + "params" + ].dict() + + # 将函数对象转换为其名称 + task_with_serializable_params["func"] = task["func"].__name__ + self.redis_client.rpush(self.queue, json.dumps(task_with_serializable_params)) + + def dequeue(self): + task_json = self.redis_client.lpop(self.queue) + if task_json: + task_info = json.loads(task_json) + # 将函数名称转换回函数对象 + task_info["func"] = FUNC_MAP[task_info["func"]] + + if "params" in task_info["kwargs"] and isinstance( + task_info["kwargs"]["params"], dict + ): + task_info["kwargs"]["params"] = VideoParams( + **task_info["kwargs"]["params"] + ) + + return task_info + return None + + def is_queue_empty(self): + return self.redis_client.llen(self.queue) == 0 + + def queue_size(self): + return self.redis_client.llen(self.queue) diff --git a/app/controllers/ping.py b/app/controllers/ping.py new file mode 100644 index 0000000..073247b --- /dev/null +++ b/app/controllers/ping.py @@ -0,0 +1,13 @@ +from fastapi import APIRouter, Request + +router = APIRouter() + + +@router.get( + "/ping", + tags=["Health Check"], + description="检查服务可用性", + response_description="pong", +) +def ping(request: Request) -> str: + return "pong" diff --git a/app/controllers/v1/base.py b/app/controllers/v1/base.py new file mode 100644 index 0000000..1336e47 --- /dev/null +++ b/app/controllers/v1/base.py @@ -0,0 +1,11 @@ +from fastapi import APIRouter + + +def new_router(dependencies=None): + router = APIRouter() + router.tags = ["V1"] + router.prefix = "/api/v1" + # 将认证依赖项应用于所有路由 + if dependencies: + router.dependencies = dependencies + return router diff --git a/app/controllers/v1/llm.py b/app/controllers/v1/llm.py new file mode 100644 index 0000000..68d7bb7 --- /dev/null +++ b/app/controllers/v1/llm.py @@ -0,0 +1,67 @@ +from fastapi import Request + +from app.controllers.v1.base import new_router +from app.models.schema import ( + VideoScriptRequest, + VideoScriptResponse, + VideoSocialMetadataRequest, + VideoSocialMetadataResponse, + VideoTermsRequest, + VideoTermsResponse, +) +from app.services import llm +from app.utils import utils + +# authentication dependency +# router = new_router(dependencies=[Depends(base.verify_token)]) +router = new_router() + + +@router.post( + "/scripts", + response_model=VideoScriptResponse, + summary="Create a script for the video", +) +def generate_video_script(request: Request, body: VideoScriptRequest): + video_script = llm.generate_script( + video_subject=body.video_subject, + language=body.video_language, + paragraph_number=body.paragraph_number, + video_script_prompt=body.video_script_prompt, + custom_system_prompt=body.custom_system_prompt, + ) + response = {"video_script": video_script} + return utils.get_response(200, response) + + +@router.post( + "/terms", + response_model=VideoTermsResponse, + summary="Generate video terms based on the video script", +) +def generate_video_terms(request: Request, body: VideoTermsRequest): + video_terms = llm.generate_terms( + video_subject=body.video_subject, + video_script=body.video_script, + amount=body.amount, + match_script_order=body.match_materials_to_script, + ) + response = {"video_terms": video_terms} + return utils.get_response(200, response) + + +@router.post( + "/social-metadata", + response_model=VideoSocialMetadataResponse, + summary="Generate social publishing metadata", +) +def generate_video_social_metadata( + request: Request, body: VideoSocialMetadataRequest +): + metadata = llm.generate_social_metadata( + video_subject=body.video_subject, + video_script=body.video_script, + language=body.language, + platform=body.platform, + ) + return utils.get_response(200, metadata) diff --git a/app/controllers/v1/video.py b/app/controllers/v1/video.py new file mode 100644 index 0000000..8d561c6 --- /dev/null +++ b/app/controllers/v1/video.py @@ -0,0 +1,400 @@ +import glob +import os +import pathlib +import shutil +from typing import Union + +from fastapi import BackgroundTasks, Depends, Path, Query, Request, UploadFile +from fastapi.params import File +from fastapi.responses import FileResponse, StreamingResponse +from loguru import logger + +from app.config import config +from app.controllers import base +from app.controllers.manager.base_manager import TaskQueueFullError +from app.controllers.manager.memory_manager import InMemoryTaskManager +from app.controllers.manager.redis_manager import RedisTaskManager +from app.controllers.v1.base import new_router +from app.models.exception import HttpException +from app.models.schema import ( + AudioRequest, + BgmRetrieveResponse, + BgmUploadResponse, + SubtitleRequest, + TaskDeletionResponse, + TaskQueryRequest, + TaskQueryResponse, + TaskResponse, + TaskVideoRequest, + VideoMaterialUploadResponse, + VideoMaterialRetrieveResponse +) +from app.services import state as sm +from app.services import task as tm +from app.utils import file_security, utils + +# 认证依赖项 +# router = new_router(dependencies=[Depends(base.verify_token)]) +router = new_router() + +_enable_redis = config.app.get("enable_redis", False) +_redis_host = config.app.get("redis_host", "localhost") +_redis_port = config.app.get("redis_port", 6379) +_redis_db = config.app.get("redis_db", 0) +_redis_password = config.app.get("redis_password", None) +_max_concurrent_tasks = config.app.get("max_concurrent_tasks", 5) +_max_queued_tasks = config.app.get("max_queued_tasks", 100) + +redis_url = f"redis://:{_redis_password}@{_redis_host}:{_redis_port}/{_redis_db}" +# 根据配置选择合适的任务管理器 +if _enable_redis: + task_manager = RedisTaskManager( + max_concurrent_tasks=_max_concurrent_tasks, + redis_url=redis_url, + max_queued_tasks=_max_queued_tasks, + ) +else: + task_manager = InMemoryTaskManager( + max_concurrent_tasks=_max_concurrent_tasks, + max_queued_tasks=_max_queued_tasks, + ) + + +def _sanitize_upload_filename(filename: str, request_id: str) -> str: + # 浏览器或客户端有时会附带目录信息，甚至可能夹带 ../ 这类穿越片段。 + # 这里只保留纯文件名，避免上传接口把文件写到目标目录之外。 + normalized_name = (filename or "").replace("\\", "/").split("/")[-1].strip() + if not normalized_name or normalized_name in {".", ".."}: + raise HttpException( + task_id=request_id, + status_code=400, + message=f"{request_id}: invalid filename", + ) + return normalized_name + + +def _resolve_path_within_directory(base_dir: str, unsafe_path: str, request_id: str) -> str: + try: + return file_security.resolve_path_within_directory(base_dir, unsafe_path) + except ValueError as exc: + logger.warning( + f"reject unsafe file path, request_id: {request_id}, path: {unsafe_path}, " + f"error: {str(exc)}" + ) + raise HttpException( + task_id=request_id, + status_code=404 if str(exc) == "file does not exist" else 403, + message=f"{request_id}: invalid file path", + ) + +def _task_file_to_uri(file: str, endpoint: str, task_dir: str, request_id: str) -> str: + if not isinstance(file, str): + return file + + if file.startswith(("http://", "https://")): + return file + + try: + resolved_path = file_security.resolve_path_within_directory(task_dir, file) + except ValueError as exc: + # 任务状态理论上只应保存任务目录内的产物路径。这里不再继续拼接 URL， + # 避免把异常路径包装成可访问链接；同时保留原值，便于排查历史脏数据。 + logger.warning( + f"skip unsafe task output path, request_id: {request_id}, path: {file}, " + f"error: {str(exc)}" + ) + return file + + relative_path = os.path.relpath(resolved_path, task_dir).replace("\\", "/") + uri_path = f"tasks/{relative_path}" + if endpoint: + return f"{endpoint.rstrip('/')}/{uri_path}" + return f"/{uri_path}" + + +@router.post("/videos", response_model=TaskResponse, summary="Generate a short video") +def create_video( + background_tasks: BackgroundTasks, request: Request, body: TaskVideoRequest +): + return create_task(request, body, stop_at="video") + + +@router.post("/subtitle", response_model=TaskResponse, summary="Generate subtitle only") +def create_subtitle( + background_tasks: BackgroundTasks, request: Request, body: SubtitleRequest +): + return create_task(request, body, stop_at="subtitle") + + +@router.post("/audio", response_model=TaskResponse, summary="Generate audio only") +def create_audio( + background_tasks: BackgroundTasks, request: Request, body: AudioRequest +): + return create_task(request, body, stop_at="audio") + + +def create_task( + request: Request, + body: Union[TaskVideoRequest, SubtitleRequest, AudioRequest], + stop_at: str, +): + task_id = utils.get_uuid() + request_id = base.get_task_id(request) + try: + task = { + "task_id": task_id, + "request_id": request_id, + "params": body.model_dump(), + } + sm.state.update_task(task_id) + task_manager.add_task(tm.start, task_id=task_id, params=body, stop_at=stop_at) + logger.success(f"Task created: {utils.to_json(task)}") + return utils.get_response(200, task) + except TaskQueueFullError as e: + sm.state.delete_task(task_id) + logger.warning( + f"reject task because queue is full, request_id: {request_id}, task_id: {task_id}" + ) + raise HttpException( + task_id=task_id, status_code=429, message=f"{request_id}: {str(e)}" + ) + except ValueError as e: + raise HttpException( + task_id=task_id, status_code=400, message=f"{request_id}: {str(e)}" + ) + +@router.get("/tasks", response_model=TaskQueryResponse, summary="Get all tasks") +def get_all_tasks(request: Request, page: int = Query(1, ge=1), page_size: int = Query(10, ge=1)): + tasks, total = sm.state.get_all_tasks(page, page_size) + + response = { + "tasks": tasks, + "total": total, + "page": page, + "page_size": page_size, + } + return utils.get_response(200, response) + + + +@router.get( + "/tasks/{task_id}", response_model=TaskQueryResponse, summary="Query task status" +) +def get_task( + request: Request, + task_id: str = Path(..., description="Task ID"), + query: TaskQueryRequest = Depends(), +): + request_id = base.get_task_id(request) + endpoint = config.app.get("endpoint", "").rstrip("/") + task = sm.state.get_task(task_id) + if task: + task_dir = utils.task_dir() + response_task = dict(task) + + if "videos" in task: + response_task["videos"] = [ + _task_file_to_uri(v, endpoint, task_dir, request_id) + for v in task["videos"] + ] + if "combined_videos" in task: + response_task["combined_videos"] = [ + _task_file_to_uri(v, endpoint, task_dir, request_id) + for v in task["combined_videos"] + ] + return utils.get_response(200, response_task) + + raise HttpException( + task_id=task_id, status_code=404, message=f"{request_id}: task not found" + ) + + +@router.delete( + "/tasks/{task_id}", + response_model=TaskDeletionResponse, + summary="Delete a generated short video task", +) +def delete_video(request: Request, task_id: str = Path(..., description="Task ID")): + request_id = base.get_task_id(request) + task = sm.state.get_task(task_id) + if task: + tasks_dir = utils.task_dir() + current_task_dir = os.path.join(tasks_dir, task_id) + if os.path.exists(current_task_dir): + shutil.rmtree(current_task_dir) + + sm.state.delete_task(task_id) + logger.success(f"video deleted: {utils.to_json(task)}") + return utils.get_response(200) + + raise HttpException( + task_id=task_id, status_code=404, message=f"{request_id}: task not found" + ) + + +@router.get( + "/musics", response_model=BgmRetrieveResponse, summary="Retrieve local BGM files" +) +def get_bgm_list(request: Request): + suffix = "*.mp3" + song_dir = utils.song_dir() + files = glob.glob(os.path.join(song_dir, suffix)) + bgm_list = [] + for file in files: + filename = os.path.basename(file) + bgm_list.append( + { + "name": filename, + "size": os.path.getsize(file), + # 只返回文件名，避免把服务器绝对路径暴露给调用方。 + # 服务端后续会把该文件名解析回 songs 白名单目录。 + "file": filename, + } + ) + response = {"files": bgm_list} + return utils.get_response(200, response) + + +@router.post( + "/musics", + response_model=BgmUploadResponse, + summary="Upload the BGM file to the songs directory", +) +def upload_bgm_file(request: Request, file: UploadFile = File(...)): + request_id = base.get_task_id(request) + safe_filename = _sanitize_upload_filename(file.filename, request_id) + # check file ext + if safe_filename.lower().endswith("mp3"): + song_dir = utils.song_dir() + save_path = os.path.join(song_dir, safe_filename) + # save file + with open(save_path, "wb+") as buffer: + # If the file already exists, it will be overwritten + file.file.seek(0) + buffer.write(file.file.read()) + response = {"file": safe_filename} + return utils.get_response(200, response) + + raise HttpException( + "", status_code=400, message=f"{request_id}: Only *.mp3 files can be uploaded" + ) + +@router.get( + "/video_materials", response_model=VideoMaterialRetrieveResponse, summary="Retrieve local video materials" +) +def get_video_materials_list(request: Request): + allowed_suffixes = ("mp4", "mov", "avi", "flv", "mkv", "jpg", "jpeg", "png") + local_videos_dir = utils.storage_dir("local_videos", create=True) + files = [] + for suffix in allowed_suffixes: + files.extend(glob.glob(os.path.join(local_videos_dir, f"*.{suffix}"))) + # 文件系统枚举顺序不稳定，直接返回会导致“顺序拼接”在不同机器或不同 + # 时刻表现不一致。这里统一按文件名排序，至少保证服务端返回顺序可预测。 + files.sort(key=lambda file_path: os.path.basename(file_path).lower()) + video_materials_list = [] + for file in files: + filename = os.path.basename(file) + video_materials_list.append( + { + "name": filename, + "size": os.path.getsize(file), + # 与 BGM 一样，只返回文件名；创建任务时再在 local_videos + # 白名单目录内解析，避免 API 泄露宿主机绝对路径。 + "file": filename, + } + ) + response = {"files": video_materials_list} + return utils.get_response(200, response) + + +@router.post( + "/video_materials", + response_model=VideoMaterialUploadResponse, + summary="Upload the video material file to the local videos directory", +) +def upload_video_material_file(request: Request, file: UploadFile = File(...)): + request_id = base.get_task_id(request) + safe_filename = _sanitize_upload_filename(file.filename, request_id) + # check file ext + allowed_suffixes = ("mp4", "mov", "avi", "flv", "mkv", "jpg", "jpeg", "png") + normalized_filename = safe_filename.lower() + # 统一按小写扩展名校验，兼容 .MOV 这类大写后缀文件。 + if normalized_filename.endswith(allowed_suffixes): + local_videos_dir = utils.storage_dir("local_videos", create=True) + save_path = os.path.join(local_videos_dir, safe_filename) + # save file + with open(save_path, "wb+") as buffer: + # If the file already exists, it will be overwritten + file.file.seek(0) + buffer.write(file.file.read()) + response = {"file": safe_filename} + return utils.get_response(200, response) + + raise HttpException( + "", status_code=400, message=f"{request_id}: Only files with extensions {', '.join(allowed_suffixes)} can be uploaded" + ) + +@router.get("/stream/{file_path:path}") +async def stream_video(request: Request, file_path: str): + request_id = base.get_task_id(request) + tasks_dir = utils.task_dir() + video_path = _resolve_path_within_directory(tasks_dir, file_path, request_id) + range_header = request.headers.get("Range") + video_size = os.path.getsize(video_path) + start, end = 0, video_size - 1 + + length = video_size + if range_header: + range_ = range_header.split("bytes=")[1] + start, end = [int(part) if part else None for part in range_.split("-")] + if start is None: + start = video_size - end + end = video_size - 1 + if end is None: + end = video_size - 1 + length = end - start + 1 + + def file_iterator(file_path, offset=0, bytes_to_read=None): + with open(file_path, "rb") as f: + f.seek(offset, os.SEEK_SET) + remaining = bytes_to_read or video_size + while remaining > 0: + bytes_to_read = min(4096, remaining) + data = f.read(bytes_to_read) + if not data: + break + remaining -= len(data) + yield data + + response = StreamingResponse( + file_iterator(video_path, start, length), media_type="video/mp4" + ) + response.headers["Content-Range"] = f"bytes {start}-{end}/{video_size}" + response.headers["Accept-Ranges"] = "bytes" + response.headers["Content-Length"] = str(length) + response.status_code = 206 # Partial Content + + return response + + +@router.get("/download/{file_path:path}") +async def download_video(request: Request, file_path: str): + """ + download video + :param request: Request request + :param file_path: video file path, eg: /cd1727ed-3473-42a2-a7da-4faafafec72b/final-1.mp4 + :return: video file + """ + request_id = base.get_task_id(request) + tasks_dir = utils.task_dir() + video_path = _resolve_path_within_directory(tasks_dir, file_path, request_id) + file_path = pathlib.Path(video_path) + filename = file_path.stem + extension = file_path.suffix + headers = {"Content-Disposition": f"attachment; filename={filename}{extension}"} + return FileResponse( + path=video_path, + headers=headers, + filename=f"{filename}{extension}", + media_type=f"video/{extension[1:]}", + ) diff --git a/app/models/__init__.py b/app/models/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/models/const.py b/app/models/const.py new file mode 100644 index 0000000..b589274 --- /dev/null +++ b/app/models/const.py @@ -0,0 +1,30 @@ +PUNCTUATIONS = [ + "?", + ",", + ".", + "、", + ";", + ":", + "!", + "…", + "？", + "，", + "。", + "、", + "；", + "：", + "！", + "...", + # 阿拉伯语常用标点也应作为自然断句点，避免脚本文本和 edge-tts + # 返回的字幕停顿边界不一致，导致后续逐行匹配失败。 + "،", + "؛", + "؟", +] + +TASK_STATE_FAILED = -1 +TASK_STATE_COMPLETE = 1 +TASK_STATE_PROCESSING = 4 + +FILE_TYPE_VIDEOS = ["mp4", "mov", "mkv", "webm"] +FILE_TYPE_IMAGES = ["jpg", "jpeg", "png", "bmp"] diff --git a/app/models/exception.py b/app/models/exception.py new file mode 100644 index 0000000..d938cb5 --- /dev/null +++ b/app/models/exception.py @@ -0,0 +1,28 @@ +import traceback +from typing import Any + +from loguru import logger + + +class HttpException(Exception): + def __init__( + self, task_id: str, status_code: int, message: str = "", data: Any = None + ): + self.message = message + self.status_code = status_code + self.data = data + # Retrieve the exception stack trace information. + tb_str = traceback.format_exc().strip() + if not tb_str or tb_str == "NoneType: None": + msg = f"HttpException: {status_code}, {task_id}, {message}" + else: + msg = f"HttpException: {status_code}, {task_id}, {message}\n{tb_str}" + + if status_code == 400: + logger.warning(msg) + else: + logger.error(msg) + + +class FileNotFoundException(Exception): + pass diff --git a/app/models/llm_provider.py b/app/models/llm_provider.py new file mode 100644 index 0000000..59011e6 --- /dev/null +++ b/app/models/llm_provider.py @@ -0,0 +1,270 @@ +from dataclasses import dataclass + + +DEFAULT_LLM_PROVIDER_ID = "moonshot" + + +@dataclass(frozen=True, slots=True) +class LLMProviderField: + """描述 Provider 除 API Key、Base URL、模型名之外的额外配置字段。""" + + config_suffix: str + label_key: str + required: bool = False + secret: bool = False + default_value: str = "" + + +@dataclass(frozen=True, slots=True) +class LLMProviderSpec: + """ + LLM Provider 的集中声明。 + + 这里集中保存跨 WebUI、配置加载和服务调用都会使用的稳定元数据，包括默认 + 展示名称和 locale key，但不保存具体翻译文案，也不实现 API 请求。这样 + Provider 的“是什么”由 Registry 维护，“怎么调用”仍由服务层适配器负责。 + """ + + provider_id: str + default_label: str + adapter: str = "openai_compatible" + api_key_url: str = "" + default_model: str = "" + default_base_url: str = "" + requires_api_key: bool = True + requires_model_name: bool = True + requires_base_url: bool = True + show_api_key: bool = True + show_base_url: bool = True + deprecated_models: tuple[str, ...] = () + deprecated_base_urls: tuple[str, ...] = () + extra_fields: tuple[LLMProviderField, ...] = () + + @property + def label_key(self) -> str: + return f"llm_provider_label.{self.provider_id}" + + @property + def tips_key(self) -> str: + return f"llm_provider_tips.{self.provider_id}" + + def config_key(self, suffix: str) -> str: + return f"{self.provider_id}_{suffix}" + + def resolve_model_name(self, configured_model: str | None) -> str: + """将空值或已废弃的历史默认值统一解析为当前默认模型。""" + model_name = (configured_model or "").strip() + if not model_name or model_name in self.deprecated_models: + return self.default_model + return model_name + + def resolve_base_url(self, configured_base_url: str | None) -> str: + """解析 Base URL，并将已经停用的历史地址迁移到当前默认值。""" + base_url = (configured_base_url or "").strip() + deprecated_urls = {url.rstrip("/") for url in self.deprecated_base_urls} + if not base_url or base_url.rstrip("/") in deprecated_urls: + return self.default_base_url + return base_url + + +# 元组顺序就是 WebUI 下拉框顺序。新增普通 OpenAI-compatible Provider 时， +# 通常只需要在这里增加一项并补充 locale；只有协议不同的 Provider 才需要在 +# app/services/llm.py 中增加对应 adapter 实现。 +LLM_PROVIDER_REGISTRY = ( + # 推荐 Provider + LLMProviderSpec( + "moonshot", + "Kimi / Moonshot AI", + api_key_url="https://platform.kimi.com/console/api-keys?aff=MoneyPrinterTurbo", + default_model="kimi-k2.7-code", + default_base_url="https://api.moonshot.cn/v1", + ), + # 主流模型原厂与云厂商 + LLMProviderSpec( + "openai", + "OpenAI", + api_key_url="https://platform.openai.com/api-keys", + default_model="gpt-5.5", + default_base_url="https://api.openai.com/v1", + ), + LLMProviderSpec( + "gemini", + "Google Gemini", + adapter="gemini", + api_key_url="https://aistudio.google.com/app/apikey", + default_model="gemini-3.1-pro-preview", + requires_base_url=False, + show_base_url=False, + deprecated_models=("gemini-pro", "gemini-1.0-pro"), + ), + LLMProviderSpec( + "deepseek", + "DeepSeek", + api_key_url="https://platform.deepseek.com/api_keys", + default_model="deepseek-v4-pro", + default_base_url="https://api.deepseek.com", + ), + LLMProviderSpec( + "qwen", + "Alibaba Cloud Qwen", + adapter="qwen", + api_key_url="https://dashscope.console.aliyun.com/apiKey", + default_model="qwen-max", + requires_base_url=False, + show_base_url=False, + ), + LLMProviderSpec( + "azure", + "Microsoft Azure OpenAI", + adapter="azure", + api_key_url=( + "https://portal.azure.com/#view/" + "Microsoft_Azure_ProjectOxford/CognitiveServicesHub/~/OpenAI" + ), + default_model="gpt-35-turbo", + ), + LLMProviderSpec( + "volcengine", + "ByteDance VolcEngine Ark", + api_key_url=( + "https://www.volcengine.com/activity/ai618?utm_campaign=hw&" + "utm_content=hw&utm_medium=devrel_tool_web&utm_source=OWO&" + "utm_term=MoneyPrinterTurbo" + ), + default_model="doubao-seed-2-1-turbo-260628", + default_base_url="https://ark.cn-beijing.volces.com/api/v3", + ), + LLMProviderSpec( + "grok", + "xAI Grok", + api_key_url="https://console.x.ai/", + default_model="grok-4.3", + default_base_url="https://api.x.ai/v1", + ), + LLMProviderSpec( + "minimax", + "MiniMax", + api_key_url="https://platform.minimax.io/", + default_model="MiniMax-M3", + default_base_url="https://api.minimax.io/v1", + ), + LLMProviderSpec( + "mimo", + "Xiaomi MiMo", + api_key_url=( + "https://platform.xiaomimimo.com/docs/zh-CN/quick-start/first-api-call" + ), + default_model="mimo-v2.5-pro", + default_base_url="https://api.xiaomimimo.com/v1", + ), + # 聚合与统一接入平台 + LLMProviderSpec( + "cloudflare", + "Cloudflare AI Gateway", + adapter="cloudflare_ai_gateway", + api_key_url="https://dash.cloudflare.com/", + default_model="openai/gpt-4.1-mini", + requires_base_url=False, + show_base_url=False, + deprecated_models=("@cf/meta/llama-3.1-8b-instruct",), + extra_fields=( + LLMProviderField("account_id", "Account ID", required=True), + LLMProviderField( + "gateway_id", + "Gateway ID", + default_value="default", + ), + ), + ), + LLMProviderSpec( + "modelscope", + "Alibaba ModelScope", + adapter="modelscope", + api_key_url=("https://modelscope.cn/docs/model-service/API-Inference/intro"), + default_model="ZhipuAI/GLM-5.2", + default_base_url="https://api-inference.modelscope.cn/v1/", + ), + LLMProviderSpec( + "aihubmix", + "AIHubMix", + api_key_url="https://aihubmix.com/", + default_model="gpt-5.4-mini", + default_base_url="https://aihubmix.com/v1", + ), + LLMProviderSpec( + "aimlapi", + "AIML API", + api_key_url="https://aimlapi.com/app/keys", + default_model="openai/gpt-5-5", + default_base_url="https://api.aimlapi.com/v1", + ), + LLMProviderSpec( + "evolink", + "EvoLink", + api_key_url="https://evolink.ai/dashboard/keys", + default_model="gpt-5.5", + default_base_url="https://direct.evolink.ai/v1", + ), + # 本地部署与通用网关 + LLMProviderSpec( + "ollama", + "Ollama", + requires_api_key=False, + show_api_key=False, + ), + LLMProviderSpec( + "oneapi", + "OneAPI", + api_key_url="https://github.com/songquanpeng/one-api", + ), + LLMProviderSpec( + "litellm", + "LiteLLM", + adapter="litellm", + default_model="openai/gpt-4o-mini", + requires_api_key=False, + requires_base_url=False, + show_api_key=False, + show_base_url=False, + ), + # 其它推理与公共服务 + LLMProviderSpec( + "groq", + "Groq", + api_key_url="https://console.groq.com/keys", + default_model="llama-3.3-70b-versatile", + default_base_url="https://api.groq.com/openai/v1", + ), + LLMProviderSpec( + "pollinations", + "Pollinations AI", + api_key_url="https://enter.pollinations.ai/", + default_model="openai-fast", + default_base_url="https://gen.pollinations.ai/v1", + deprecated_models=("default",), + deprecated_base_urls=("https://text.pollinations.ai/openai",), + ), +) + +LLM_PROVIDERS = {provider.provider_id: provider for provider in LLM_PROVIDER_REGISTRY} + +if len(LLM_PROVIDERS) != len(LLM_PROVIDER_REGISTRY): + raise RuntimeError("duplicate LLM provider id in registry") + + +def get_llm_provider(provider_id: str) -> LLMProviderSpec | None: + return LLM_PROVIDERS.get((provider_id or "").lower()) + + +def normalize_provider_override(value: str | None, default_value: str | None) -> str: + """ + 只保留与 Registry 默认值不同的用户覆盖值。 + + WebUI 需要把默认值展示在输入框中，但不能因此把默认值固化到 config.toml； + 否则后续升级 Registry 默认模型或地址时，旧配置会继续覆盖新默认值。 + """ + normalized_value = (value or "").strip() + normalized_default = (default_value or "").strip() + if normalized_value == normalized_default: + return "" + return normalized_value diff --git a/app/models/schema.py b/app/models/schema.py new file mode 100644 index 0000000..f1e8451 --- /dev/null +++ b/app/models/schema.py @@ -0,0 +1,384 @@ +import warnings +from enum import Enum +from typing import Any, List, Optional, Union + +import pydantic +from pydantic import BaseModel, Field + +from app.config import config + +# 忽略 Pydantic 的特定警告 +warnings.filterwarnings( + "ignore", + category=UserWarning, + message="Field name.*shadows an attribute in parent.*", +) + + +class VideoConcatMode(str, Enum): + random = "random" + sequential = "sequential" + + +class VideoTransitionMode(str, Enum): + none = None + shuffle = "Shuffle" + fade_in = "FadeIn" + fade_out = "FadeOut" + slide_in = "SlideIn" + slide_out = "SlideOut" + zoom_in = "ZoomIn" + zoom_out = "ZoomOut" + + +class VideoAspect(str, Enum): + landscape = "16:9" + portrait = "9:16" + square = "1:1" + + def to_resolution(self): + if self == VideoAspect.landscape: + return 1920, 1080 + elif self == VideoAspect.portrait: + return 1080, 1920 + elif self == VideoAspect.square: + return 1080, 1080 + raise ValueError(f"unsupported video aspect: {self}") + + +class _Config: + arbitrary_types_allowed = True + + +@pydantic.dataclasses.dataclass(config=_Config) +class MaterialInfo: + provider: str = "pexels" + url: str = "" + duration: int = 0 + + +class VideoParams(BaseModel): + """ + { + "video_subject": "", + "video_aspect": "横屏 16:9（西瓜视频）", + "voice_name": "女生-晓晓", + "bgm_name": "random", + "font_name": "STHeitiMedium 黑体-中", + "text_color": "#FFFFFF", + "font_size": 60, + "stroke_color": "#000000", + "stroke_width": 1.5 + } + """ + + video_subject: str + video_script: str = "" # Script used to generate the video + video_terms: Optional[str | list] = None # Keywords used to generate the video + video_aspect: Optional[VideoAspect] = VideoAspect.portrait.value + video_concat_mode: Optional[VideoConcatMode] = VideoConcatMode.random.value + video_transition_mode: Optional[VideoTransitionMode] = None + video_clip_duration: Optional[int] = 5 + match_materials_to_script: bool = False + video_count: Optional[int] = 1 + + video_source: Optional[str] = "pexels" + video_materials: Optional[List[MaterialInfo]] = ( + None # Materials used to generate the video + ) + + custom_audio_file: Optional[str] = None # Custom audio file path, will ignore TTS and can still use Whisper subtitles + video_language: Optional[str] = "" # auto detect + + voice_name: Optional[str] = "" + voice_volume: Optional[float] = 1.0 + voice_rate: Optional[float] = 1.0 + bgm_type: Optional[str] = "random" + bgm_file: Optional[str] = "" + bgm_volume: Optional[float] = 0.2 + + subtitle_enabled: Optional[bool] = True + subtitle_position: Optional[str] = config.ui.get("subtitle_position", "bottom") # top, bottom, center, custom + custom_position: float = config.ui.get("custom_position", 70.0) + font_name: Optional[str] = "STHeitiMedium.ttc" + text_fore_color: Optional[str] = "#FFFFFF" + text_background_color: Union[bool, str] = False + rounded_subtitle_background: bool = False + + font_size: int = 60 + stroke_color: Optional[str] = "#000000" + stroke_width: float = 1.5 + n_threads: Optional[int] = 2 + paragraph_number: int = Field(default=1, ge=1, le=10) + video_script_prompt: str = Field(default="", max_length=2000) + custom_system_prompt: str = Field(default="", max_length=8000) + + +class SubtitleRequest(BaseModel): + video_script: str + video_language: Optional[str] = "" + voice_name: Optional[str] = "zh-CN-XiaoxiaoNeural-Female" + voice_volume: Optional[float] = 1.0 + voice_rate: Optional[float] = 1.2 + bgm_type: Optional[str] = "random" + bgm_file: Optional[str] = "" + bgm_volume: Optional[float] = 0.2 + subtitle_position: Optional[str] = config.ui.get("subtitle_position", "bottom") + font_name: Optional[str] = "STHeitiMedium.ttc" + text_fore_color: Optional[str] = "#FFFFFF" + text_background_color: Union[bool, str] = False + rounded_subtitle_background: bool = False + font_size: int = 60 + stroke_color: Optional[str] = "#000000" + stroke_width: float = 1.5 + video_source: Optional[str] = "local" + subtitle_enabled: Optional[str] = "true" + + +class AudioRequest(BaseModel): + video_script: str + video_language: Optional[str] = "" + voice_name: Optional[str] = "zh-CN-XiaoxiaoNeural-Female" + voice_volume: Optional[float] = 1.0 + voice_rate: Optional[float] = 1.2 + bgm_type: Optional[str] = "random" + bgm_file: Optional[str] = "" + bgm_volume: Optional[float] = 0.2 + video_source: Optional[str] = "local" + + +class VideoScriptParams: + """ + { + "video_subject": "春天的花海", + "video_language": "", + "paragraph_number": 1, + "video_script_prompt": "", + "custom_system_prompt": "" + } + """ + + video_subject: Optional[str] = "春天的花海" + video_language: Optional[str] = "" + paragraph_number: int = Field(default=1, ge=1, le=10) + video_script_prompt: str = Field(default="", max_length=2000) + custom_system_prompt: str = Field(default="", max_length=8000) + + +class VideoTermsParams: + """ + { + "video_subject": "", + "video_script": "", + "amount": 5, + "match_materials_to_script": false + } + """ + + video_subject: Optional[str] = "春天的花海" + video_script: Optional[str] = ( + "春天的花海，如诗如画般展现在眼前。万物复苏的季节里，大地披上了一袭绚丽多彩的盛装。金黄的迎春、粉嫩的樱花、洁白的梨花、艳丽的郁金香……" + ) + amount: Optional[int] = 5 + match_materials_to_script: bool = False + + +class VideoSocialMetadataParams: + """ + { + "video_subject": "A day in Shanghai", + "video_script": "", + "language": "auto", + "platform": "tiktok" + } + """ + + video_subject: Optional[str] = Field(default="A day in Shanghai", max_length=500) + video_script: Optional[str] = Field(default="", max_length=8000) + language: Optional[str] = Field(default="auto", max_length=64) + platform: Optional[str] = Field(default="tiktok", max_length=64) + + +class BaseResponse(BaseModel): + status: int = 200 + message: Optional[str] = "success" + data: Any = None + + +class TaskVideoRequest(VideoParams, BaseModel): + pass + + +class TaskQueryRequest(BaseModel): + pass + + +class VideoScriptRequest(VideoScriptParams, BaseModel): + pass + + +class VideoTermsRequest(VideoTermsParams, BaseModel): + pass + + +class VideoSocialMetadataRequest(VideoSocialMetadataParams, BaseModel): + pass + + +###################################################################################################### +###################################################################################################### +###################################################################################################### +###################################################################################################### +class TaskResponse(BaseResponse): + class TaskResponseData(BaseModel): + task_id: str + + data: TaskResponseData + + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": {"task_id": "6c85c8cc-a77a-42b9-bc30-947815aa0558"}, + }, + } + + +class TaskQueryResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": { + "state": 1, + "progress": 100, + "videos": [ + "http://127.0.0.1:8080/tasks/6c85c8cc-a77a-42b9-bc30-947815aa0558/final-1.mp4" + ], + "combined_videos": [ + "http://127.0.0.1:8080/tasks/6c85c8cc-a77a-42b9-bc30-947815aa0558/combined-1.mp4" + ], + }, + }, + } + + +class TaskDeletionResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": { + "state": 1, + "progress": 100, + "videos": [ + "http://127.0.0.1:8080/tasks/6c85c8cc-a77a-42b9-bc30-947815aa0558/final-1.mp4" + ], + "combined_videos": [ + "http://127.0.0.1:8080/tasks/6c85c8cc-a77a-42b9-bc30-947815aa0558/combined-1.mp4" + ], + }, + }, + } + + +class VideoScriptResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": { + "video_script": "春天的花海，是大自然的一幅美丽画卷。在这个季节里，大地复苏，万物生长，花朵争相绽放，形成了一片五彩斑斓的花海..." + }, + }, + } + + +class VideoTermsResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": {"video_terms": ["sky", "tree"]}, + }, + } + + +class VideoSocialMetadataResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": { + "title": "A Day in Shanghai You Should Not Miss", + "caption": "Save this quick Shanghai inspiration and follow for more short travel ideas.", + "hashtags": ["#shorts", "#travel", "#shanghai", "#viral", "#fyp"], + }, + }, + } + + +class BgmRetrieveResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": { + "files": [ + { + "name": "output013.mp3", + "size": 1891269, + "file": "/MoneyPrinterTurbo/resource/songs/output013.mp3", + } + ] + }, + }, + } + + +class BgmUploadResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": {"file": "/MoneyPrinterTurbo/resource/songs/example.mp3"}, + }, + } + +class VideoMaterialRetrieveResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": { + "files": [ + { + "name": "example.mp4", + "size": 12345678, + "file": "/MoneyPrinterTurbo/resource/videos/example.mp4", + } + ] + }, + }, + } + +class VideoMaterialUploadResponse(BaseResponse): + class Config: + json_schema_extra = { + "example": { + "status": 200, + "message": "success", + "data": { + "file": "/MoneyPrinterTurbo/resource/videos/example.mp4", + }, + }, + } diff --git a/app/router.py b/app/router.py new file mode 100644 index 0000000..cf84037 --- /dev/null +++ b/app/router.py @@ -0,0 +1,17 @@ +"""Application configuration - root APIRouter. + +Defines all FastAPI application endpoints. + +Resources: + 1. https://fastapi.tiangolo.com/tutorial/bigger-applications + +""" + +from fastapi import APIRouter + +from app.controllers.v1 import llm, video + +root_api_router = APIRouter() +# v1 +root_api_router.include_router(video.router) +root_api_router.include_router(llm.router) diff --git a/app/services/__init__.py b/app/services/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/services/cache_manager.py b/app/services/cache_manager.py new file mode 100644 index 0000000..6247589 --- /dev/null +++ b/app/services/cache_manager.py @@ -0,0 +1,215 @@ +"""视频素材缓存的统计、预览和清理服务。""" + +from __future__ import annotations + +import os +import re +import time +from dataclasses import dataclass +from typing import Iterator + +from loguru import logger + +from app.utils import utils + + +# 在线素材使用 URL 的 MD5 作为稳定文件名。缓存管理只接受该命名格式，避免把 +# 用户误放到目录中的视频、说明文件或其它业务文件当作缓存删除。 +_VIDEO_CACHE_FILE_PATTERN = re.compile(r"^vid-[0-9a-f]{32}\.mp4$") +_SECONDS_PER_DAY = 24 * 60 * 60 + + +@dataclass(frozen=True) +class VideoCacheStats: + """缓存目录的轻量统计结果，只包含文件系统元数据。""" + + file_count: int = 0 + total_size: int = 0 + oldest_mtime: float | None = None + newest_mtime: float | None = None + + +@dataclass(frozen=True) +class VideoCacheCleanupResult: + """一次清理的执行结果，允许部分文件删除失败。""" + + deleted_count: int = 0 + deleted_size: int = 0 + failed_count: int = 0 + + +@dataclass(frozen=True) +class _VideoCacheEntry: + """扫描阶段保存的最小文件信息，避免清理时打开或解析视频。""" + + path: str + name: str + size: int + mtime: float + + +def video_cache_dir() -> str: + """返回项目管理的默认视频缓存目录。""" + + return os.path.realpath(utils.storage_dir("cache_videos")) + + +def _iter_video_cache_entries() -> Iterator[_VideoCacheEntry]: + """ + 顺序扫描默认缓存目录第一层。 + + 使用 ``os.scandir`` 是为了在缓存达到数万文件时复用目录遍历返回的元数据， + 避免 ``Path.iterdir`` 后再次查询文件类型。这里不递归、不打开视频，也不调用 + FFmpeg，因此耗时主要与文件数量线性相关，而不是与视频总容量相关。 + """ + + cache_dir = video_cache_dir() + try: + entries = os.scandir(cache_dir) + except FileNotFoundError: + return + except OSError as exc: + logger.warning( + f"failed to scan video cache directory: path={cache_dir}, error={exc}" + ) + return + + with entries: + for entry in entries: + if not _VIDEO_CACHE_FILE_PATTERN.fullmatch(entry.name): + continue + + try: + # 不跟随符号链接，确保清理逻辑不会越过默认缓存目录边界。 + if not entry.is_file(follow_symlinks=False): + continue + stat_result = entry.stat(follow_symlinks=False) + except OSError as exc: + logger.warning( + f"failed to inspect video cache file: file={entry.name}, error={exc}" + ) + continue + + yield _VideoCacheEntry( + path=entry.path, + name=entry.name, + size=stat_result.st_size, + mtime=stat_result.st_mtime, + ) + + +def _is_cleanup_candidate( + entry: _VideoCacheEntry, + max_age_days: int | None, + now: float, +) -> bool: + if max_age_days is None: + return True + return entry.mtime < now - max_age_days * _SECONDS_PER_DAY + + +def _validate_max_age_days(max_age_days: int | None) -> None: + """即使缓存目录为空，也应稳定拒绝无效清理参数。""" + if max_age_days is None: + return + if ( + isinstance(max_age_days, bool) + or not isinstance(max_age_days, int) + or max_age_days <= 0 + ): + raise ValueError("max_age_days must be a positive integer or None") + + +def get_video_cache_stats(max_age_days: int | None = None) -> VideoCacheStats: + """ + 统计全部缓存，或预览修改时间早于指定天数的可清理缓存。 + + ``max_age_days=None`` 表示全部缓存。统计过程只读取目录项的大小和修改时间， + 不读取视频内容，因此即使缓存总容量很大也不会产生与容量成比例的 I/O。 + """ + + _validate_max_age_days(max_age_days) + now = time.time() + file_count = 0 + total_size = 0 + oldest_mtime = None + newest_mtime = None + + for entry in _iter_video_cache_entries(): + if not _is_cleanup_candidate(entry, max_age_days, now): + continue + file_count += 1 + total_size += entry.size + oldest_mtime = ( + entry.mtime if oldest_mtime is None else min(oldest_mtime, entry.mtime) + ) + newest_mtime = ( + entry.mtime if newest_mtime is None else max(newest_mtime, entry.mtime) + ) + + return VideoCacheStats( + file_count=file_count, + total_size=total_size, + oldest_mtime=oldest_mtime, + newest_mtime=newest_mtime, + ) + + +def clean_video_cache(max_age_days: int | None = None) -> VideoCacheCleanupResult: + """ + 清理默认视频缓存，并返回可向用户展示的汇总结果。 + + 页面预览与真正点击清理之间可能间隔较久，所以执行时必须重新扫描和判断， + 不能复用旧候选列表。删除采用逐文件容错：单个文件被占用或权限不足时记录 + 警告并继续，避免几百个文件中一个异常文件导致整次清理失败。 + """ + + _validate_max_age_days(max_age_days) + now = time.time() + logger.info( + f"start cleaning video cache: max_age_days={max_age_days}" + ) + + candidate_count = 0 + candidate_size = 0 + deleted_count = 0 + deleted_size = 0 + failed_count = 0 + cache_dir = video_cache_dir() + + # 边扫描边删除，不在内存中保留完整候选列表。即使目录增长到几十万个文件， + # 清理过程的额外内存仍保持常量级；执行时使用统一 now，避免长清理过程中 + # 截止时间不断移动而产生不可预测的候选范围。 + for entry in _iter_video_cache_entries(): + if not _is_cleanup_candidate(entry, max_age_days, now): + continue + candidate_count += 1 + candidate_size += entry.size + try: + # entry.path 来自默认目录的第一层 scandir；删除前再次校验父目录和 + # 文件名，防止未来修改扫描逻辑时意外扩大可删除范围。 + if ( + os.path.realpath(os.path.dirname(entry.path)) != cache_dir + or not _VIDEO_CACHE_FILE_PATTERN.fullmatch(entry.name) + or os.path.islink(entry.path) + ): + raise ValueError("cache file is outside the managed directory") + os.unlink(entry.path) + deleted_count += 1 + deleted_size += entry.size + except (OSError, ValueError) as exc: + failed_count += 1 + logger.warning( + f"failed to delete video cache file: file={entry.name}, error={exc}" + ) + + logger.info( + "finished cleaning video cache: " + f"candidates={candidate_count}, candidate_bytes={candidate_size}, " + f"deleted={deleted_count}, deleted_bytes={deleted_size}, failed={failed_count}" + ) + return VideoCacheCleanupResult( + deleted_count=deleted_count, + deleted_size=deleted_size, + failed_count=failed_count, + ) diff --git a/app/services/data/azure_voices.json b/app/services/data/azure_voices.json new file mode 100644 index 0000000..af27ec0 --- /dev/null +++ b/app/services/data/azure_voices.json @@ -0,0 +1,1326 @@ +[ + { + "name": "af-ZA-AdriNeural", + "gender": "Female" + }, + { + "name": "af-ZA-WillemNeural", + "gender": "Male" + }, + { + "name": "am-ET-AmehaNeural", + "gender": "Male" + }, + { + "name": "am-ET-MekdesNeural", + "gender": "Female" + }, + { + "name": "ar-AE-FatimaNeural", + "gender": "Female" + }, + { + "name": "ar-AE-HamdanNeural", + "gender": "Male" + }, + { + "name": "ar-BH-AliNeural", + "gender": "Male" + }, + { + "name": "ar-BH-LailaNeural", + "gender": "Female" + }, + { + "name": "ar-DZ-AminaNeural", + "gender": "Female" + }, + { + "name": "ar-DZ-IsmaelNeural", + "gender": "Male" + }, + { + "name": "ar-EG-SalmaNeural", + "gender": "Female" + }, + { + "name": "ar-EG-ShakirNeural", + "gender": "Male" + }, + { + "name": "ar-IQ-BasselNeural", + "gender": "Male" + }, + { + "name": "ar-IQ-RanaNeural", + "gender": "Female" + }, + { + "name": "ar-JO-SanaNeural", + "gender": "Female" + }, + { + "name": "ar-JO-TaimNeural", + "gender": "Male" + }, + { + "name": "ar-KW-FahedNeural", + "gender": "Male" + }, + { + "name": "ar-KW-NouraNeural", + "gender": "Female" + }, + { + "name": "ar-LB-LaylaNeural", + "gender": "Female" + }, + { + "name": "ar-LB-RamiNeural", + "gender": "Male" + }, + { + "name": "ar-LY-ImanNeural", + "gender": "Female" + }, + { + "name": "ar-LY-OmarNeural", + "gender": "Male" + }, + { + "name": "ar-MA-JamalNeural", + "gender": "Male" + }, + { + "name": "ar-MA-MounaNeural", + "gender": "Female" + }, + { + "name": "ar-OM-AbdullahNeural", + "gender": "Male" + }, + { + "name": "ar-OM-AyshaNeural", + "gender": "Female" + }, + { + "name": "ar-QA-AmalNeural", + "gender": "Female" + }, + { + "name": "ar-QA-MoazNeural", + "gender": "Male" + }, + { + "name": "ar-SA-HamedNeural", + "gender": "Male" + }, + { + "name": "ar-SA-ZariyahNeural", + "gender": "Female" + }, + { + "name": "ar-SY-AmanyNeural", + "gender": "Female" + }, + { + "name": "ar-SY-LaithNeural", + "gender": "Male" + }, + { + "name": "ar-TN-HediNeural", + "gender": "Male" + }, + { + "name": "ar-TN-ReemNeural", + "gender": "Female" + }, + { + "name": "ar-YE-MaryamNeural", + "gender": "Female" + }, + { + "name": "ar-YE-SalehNeural", + "gender": "Male" + }, + { + "name": "az-AZ-BabekNeural", + "gender": "Male" + }, + { + "name": "az-AZ-BanuNeural", + "gender": "Female" + }, + { + "name": "bg-BG-BorislavNeural", + "gender": "Male" + }, + { + "name": "bg-BG-KalinaNeural", + "gender": "Female" + }, + { + "name": "bn-BD-NabanitaNeural", + "gender": "Female" + }, + { + "name": "bn-BD-PradeepNeural", + "gender": "Male" + }, + { + "name": "bn-IN-BashkarNeural", + "gender": "Male" + }, + { + "name": "bn-IN-TanishaaNeural", + "gender": "Female" + }, + { + "name": "bs-BA-GoranNeural", + "gender": "Male" + }, + { + "name": "bs-BA-VesnaNeural", + "gender": "Female" + }, + { + "name": "ca-ES-EnricNeural", + "gender": "Male" + }, + { + "name": "ca-ES-JoanaNeural", + "gender": "Female" + }, + { + "name": "cs-CZ-AntoninNeural", + "gender": "Male" + }, + { + "name": "cs-CZ-VlastaNeural", + "gender": "Female" + }, + { + "name": "cy-GB-AledNeural", + "gender": "Male" + }, + { + "name": "cy-GB-NiaNeural", + "gender": "Female" + }, + { + "name": "da-DK-ChristelNeural", + "gender": "Female" + }, + { + "name": "da-DK-JeppeNeural", + "gender": "Male" + }, + { + "name": "de-AT-IngridNeural", + "gender": "Female" + }, + { + "name": "de-AT-JonasNeural", + "gender": "Male" + }, + { + "name": "de-CH-JanNeural", + "gender": "Male" + }, + { + "name": "de-CH-LeniNeural", + "gender": "Female" + }, + { + "name": "de-DE-AmalaNeural", + "gender": "Female" + }, + { + "name": "de-DE-ConradNeural", + "gender": "Male" + }, + { + "name": "de-DE-FlorianMultilingualNeural", + "gender": "Male" + }, + { + "name": "de-DE-KatjaNeural", + "gender": "Female" + }, + { + "name": "de-DE-KillianNeural", + "gender": "Male" + }, + { + "name": "de-DE-SeraphinaMultilingualNeural", + "gender": "Female" + }, + { + "name": "el-GR-AthinaNeural", + "gender": "Female" + }, + { + "name": "el-GR-NestorasNeural", + "gender": "Male" + }, + { + "name": "en-AU-NatashaNeural", + "gender": "Female" + }, + { + "name": "en-AU-WilliamNeural", + "gender": "Male" + }, + { + "name": "en-CA-ClaraNeural", + "gender": "Female" + }, + { + "name": "en-CA-LiamNeural", + "gender": "Male" + }, + { + "name": "en-GB-LibbyNeural", + "gender": "Female" + }, + { + "name": "en-GB-MaisieNeural", + "gender": "Female" + }, + { + "name": "en-GB-RyanNeural", + "gender": "Male" + }, + { + "name": "en-GB-SoniaNeural", + "gender": "Female" + }, + { + "name": "en-GB-ThomasNeural", + "gender": "Male" + }, + { + "name": "en-HK-SamNeural", + "gender": "Male" + }, + { + "name": "en-HK-YanNeural", + "gender": "Female" + }, + { + "name": "en-IE-ConnorNeural", + "gender": "Male" + }, + { + "name": "en-IE-EmilyNeural", + "gender": "Female" + }, + { + "name": "en-IN-NeerjaExpressiveNeural", + "gender": "Female" + }, + { + "name": "en-IN-NeerjaNeural", + "gender": "Female" + }, + { + "name": "en-IN-PrabhatNeural", + "gender": "Male" + }, + { + "name": "en-KE-AsiliaNeural", + "gender": "Female" + }, + { + "name": "en-KE-ChilembaNeural", + "gender": "Male" + }, + { + "name": "en-NG-AbeoNeural", + "gender": "Male" + }, + { + "name": "en-NG-EzinneNeural", + "gender": "Female" + }, + { + "name": "en-NZ-MitchellNeural", + "gender": "Male" + }, + { + "name": "en-NZ-MollyNeural", + "gender": "Female" + }, + { + "name": "en-PH-JamesNeural", + "gender": "Male" + }, + { + "name": "en-PH-RosaNeural", + "gender": "Female" + }, + { + "name": "en-SG-LunaNeural", + "gender": "Female" + }, + { + "name": "en-SG-WayneNeural", + "gender": "Male" + }, + { + "name": "en-TZ-ElimuNeural", + "gender": "Male" + }, + { + "name": "en-TZ-ImaniNeural", + "gender": "Female" + }, + { + "name": "en-US-AnaNeural", + "gender": "Female" + }, + { + "name": "en-US-AndrewMultilingualNeural", + "gender": "Male" + }, + { + "name": "en-US-AndrewNeural", + "gender": "Male" + }, + { + "name": "en-US-AriaNeural", + "gender": "Female" + }, + { + "name": "en-US-AvaMultilingualNeural", + "gender": "Female" + }, + { + "name": "en-US-AvaNeural", + "gender": "Female" + }, + { + "name": "en-US-BrianMultilingualNeural", + "gender": "Male" + }, + { + "name": "en-US-BrianNeural", + "gender": "Male" + }, + { + "name": "en-US-ChristopherNeural", + "gender": "Male" + }, + { + "name": "en-US-EmmaMultilingualNeural", + "gender": "Female" + }, + { + "name": "en-US-EmmaNeural", + "gender": "Female" + }, + { + "name": "en-US-EricNeural", + "gender": "Male" + }, + { + "name": "en-US-GuyNeural", + "gender": "Male" + }, + { + "name": "en-US-JennyNeural", + "gender": "Female" + }, + { + "name": "en-US-MichelleNeural", + "gender": "Female" + }, + { + "name": "en-US-RogerNeural", + "gender": "Male" + }, + { + "name": "en-US-SteffanNeural", + "gender": "Male" + }, + { + "name": "en-ZA-LeahNeural", + "gender": "Female" + }, + { + "name": "en-ZA-LukeNeural", + "gender": "Male" + }, + { + "name": "es-AR-ElenaNeural", + "gender": "Female" + }, + { + "name": "es-AR-TomasNeural", + "gender": "Male" + }, + { + "name": "es-BO-MarceloNeural", + "gender": "Male" + }, + { + "name": "es-BO-SofiaNeural", + "gender": "Female" + }, + { + "name": "es-CL-CatalinaNeural", + "gender": "Female" + }, + { + "name": "es-CL-LorenzoNeural", + "gender": "Male" + }, + { + "name": "es-CO-GonzaloNeural", + "gender": "Male" + }, + { + "name": "es-CO-SalomeNeural", + "gender": "Female" + }, + { + "name": "es-CR-JuanNeural", + "gender": "Male" + }, + { + "name": "es-CR-MariaNeural", + "gender": "Female" + }, + { + "name": "es-CU-BelkysNeural", + "gender": "Female" + }, + { + "name": "es-CU-ManuelNeural", + "gender": "Male" + }, + { + "name": "es-DO-EmilioNeural", + "gender": "Male" + }, + { + "name": "es-DO-RamonaNeural", + "gender": "Female" + }, + { + "name": "es-EC-AndreaNeural", + "gender": "Female" + }, + { + "name": "es-EC-LuisNeural", + "gender": "Male" + }, + { + "name": "es-ES-AlvaroNeural", + "gender": "Male" + }, + { + "name": "es-ES-ElviraNeural", + "gender": "Female" + }, + { + "name": "es-ES-XimenaNeural", + "gender": "Female" + }, + { + "name": "es-GQ-JavierNeural", + "gender": "Male" + }, + { + "name": "es-GQ-TeresaNeural", + "gender": "Female" + }, + { + "name": "es-GT-AndresNeural", + "gender": "Male" + }, + { + "name": "es-GT-MartaNeural", + "gender": "Female" + }, + { + "name": "es-HN-CarlosNeural", + "gender": "Male" + }, + { + "name": "es-HN-KarlaNeural", + "gender": "Female" + }, + { + "name": "es-MX-DaliaNeural", + "gender": "Female" + }, + { + "name": "es-MX-JorgeNeural", + "gender": "Male" + }, + { + "name": "es-NI-FedericoNeural", + "gender": "Male" + }, + { + "name": "es-NI-YolandaNeural", + "gender": "Female" + }, + { + "name": "es-PA-MargaritaNeural", + "gender": "Female" + }, + { + "name": "es-PA-RobertoNeural", + "gender": "Male" + }, + { + "name": "es-PE-AlexNeural", + "gender": "Male" + }, + { + "name": "es-PE-CamilaNeural", + "gender": "Female" + }, + { + "name": "es-PR-KarinaNeural", + "gender": "Female" + }, + { + "name": "es-PR-VictorNeural", + "gender": "Male" + }, + { + "name": "es-PY-MarioNeural", + "gender": "Male" + }, + { + "name": "es-PY-TaniaNeural", + "gender": "Female" + }, + { + "name": "es-SV-LorenaNeural", + "gender": "Female" + }, + { + "name": "es-SV-RodrigoNeural", + "gender": "Male" + }, + { + "name": "es-US-AlonsoNeural", + "gender": "Male" + }, + { + "name": "es-US-PalomaNeural", + "gender": "Female" + }, + { + "name": "es-UY-MateoNeural", + "gender": "Male" + }, + { + "name": "es-UY-ValentinaNeural", + "gender": "Female" + }, + { + "name": "es-VE-PaolaNeural", + "gender": "Female" + }, + { + "name": "es-VE-SebastianNeural", + "gender": "Male" + }, + { + "name": "et-EE-AnuNeural", + "gender": "Female" + }, + { + "name": "et-EE-KertNeural", + "gender": "Male" + }, + { + "name": "fa-IR-DilaraNeural", + "gender": "Female" + }, + { + "name": "fa-IR-FaridNeural", + "gender": "Male" + }, + { + "name": "fi-FI-HarriNeural", + "gender": "Male" + }, + { + "name": "fi-FI-NooraNeural", + "gender": "Female" + }, + { + "name": "fil-PH-AngeloNeural", + "gender": "Male" + }, + { + "name": "fil-PH-BlessicaNeural", + "gender": "Female" + }, + { + "name": "fr-BE-CharlineNeural", + "gender": "Female" + }, + { + "name": "fr-BE-GerardNeural", + "gender": "Male" + }, + { + "name": "fr-CA-AntoineNeural", + "gender": "Male" + }, + { + "name": "fr-CA-JeanNeural", + "gender": "Male" + }, + { + "name": "fr-CA-SylvieNeural", + "gender": "Female" + }, + { + "name": "fr-CA-ThierryNeural", + "gender": "Male" + }, + { + "name": "fr-CH-ArianeNeural", + "gender": "Female" + }, + { + "name": "fr-CH-FabriceNeural", + "gender": "Male" + }, + { + "name": "fr-FR-DeniseNeural", + "gender": "Female" + }, + { + "name": "fr-FR-EloiseNeural", + "gender": "Female" + }, + { + "name": "fr-FR-HenriNeural", + "gender": "Male" + }, + { + "name": "fr-FR-RemyMultilingualNeural", + "gender": "Male" + }, + { + "name": "fr-FR-VivienneMultilingualNeural", + "gender": "Female" + }, + { + "name": "ga-IE-ColmNeural", + "gender": "Male" + }, + { + "name": "ga-IE-OrlaNeural", + "gender": "Female" + }, + { + "name": "gl-ES-RoiNeural", + "gender": "Male" + }, + { + "name": "gl-ES-SabelaNeural", + "gender": "Female" + }, + { + "name": "gu-IN-DhwaniNeural", + "gender": "Female" + }, + { + "name": "gu-IN-NiranjanNeural", + "gender": "Male" + }, + { + "name": "he-IL-AvriNeural", + "gender": "Male" + }, + { + "name": "he-IL-HilaNeural", + "gender": "Female" + }, + { + "name": "hi-IN-MadhurNeural", + "gender": "Male" + }, + { + "name": "hi-IN-SwaraNeural", + "gender": "Female" + }, + { + "name": "hr-HR-GabrijelaNeural", + "gender": "Female" + }, + { + "name": "hr-HR-SreckoNeural", + "gender": "Male" + }, + { + "name": "hu-HU-NoemiNeural", + "gender": "Female" + }, + { + "name": "hu-HU-TamasNeural", + "gender": "Male" + }, + { + "name": "id-ID-ArdiNeural", + "gender": "Male" + }, + { + "name": "id-ID-GadisNeural", + "gender": "Female" + }, + { + "name": "is-IS-GudrunNeural", + "gender": "Female" + }, + { + "name": "is-IS-GunnarNeural", + "gender": "Male" + }, + { + "name": "it-IT-DiegoNeural", + "gender": "Male" + }, + { + "name": "it-IT-ElsaNeural", + "gender": "Female" + }, + { + "name": "it-IT-GiuseppeMultilingualNeural", + "gender": "Male" + }, + { + "name": "it-IT-IsabellaNeural", + "gender": "Female" + }, + { + "name": "iu-Cans-CA-SiqiniqNeural", + "gender": "Female" + }, + { + "name": "iu-Cans-CA-TaqqiqNeural", + "gender": "Male" + }, + { + "name": "iu-Latn-CA-SiqiniqNeural", + "gender": "Female" + }, + { + "name": "iu-Latn-CA-TaqqiqNeural", + "gender": "Male" + }, + { + "name": "ja-JP-KeitaNeural", + "gender": "Male" + }, + { + "name": "ja-JP-NanamiNeural", + "gender": "Female" + }, + { + "name": "jv-ID-DimasNeural", + "gender": "Male" + }, + { + "name": "jv-ID-SitiNeural", + "gender": "Female" + }, + { + "name": "ka-GE-EkaNeural", + "gender": "Female" + }, + { + "name": "ka-GE-GiorgiNeural", + "gender": "Male" + }, + { + "name": "kk-KZ-AigulNeural", + "gender": "Female" + }, + { + "name": "kk-KZ-DauletNeural", + "gender": "Male" + }, + { + "name": "km-KH-PisethNeural", + "gender": "Male" + }, + { + "name": "km-KH-SreymomNeural", + "gender": "Female" + }, + { + "name": "kn-IN-GaganNeural", + "gender": "Male" + }, + { + "name": "kn-IN-SapnaNeural", + "gender": "Female" + }, + { + "name": "ko-KR-HyunsuMultilingualNeural", + "gender": "Male" + }, + { + "name": "ko-KR-InJoonNeural", + "gender": "Male" + }, + { + "name": "ko-KR-SunHiNeural", + "gender": "Female" + }, + { + "name": "lo-LA-ChanthavongNeural", + "gender": "Male" + }, + { + "name": "lo-LA-KeomanyNeural", + "gender": "Female" + }, + { + "name": "lt-LT-LeonasNeural", + "gender": "Male" + }, + { + "name": "lt-LT-OnaNeural", + "gender": "Female" + }, + { + "name": "lv-LV-EveritaNeural", + "gender": "Female" + }, + { + "name": "lv-LV-NilsNeural", + "gender": "Male" + }, + { + "name": "mk-MK-AleksandarNeural", + "gender": "Male" + }, + { + "name": "mk-MK-MarijaNeural", + "gender": "Female" + }, + { + "name": "ml-IN-MidhunNeural", + "gender": "Male" + }, + { + "name": "ml-IN-SobhanaNeural", + "gender": "Female" + }, + { + "name": "mn-MN-BataaNeural", + "gender": "Male" + }, + { + "name": "mn-MN-YesuiNeural", + "gender": "Female" + }, + { + "name": "mr-IN-AarohiNeural", + "gender": "Female" + }, + { + "name": "mr-IN-ManoharNeural", + "gender": "Male" + }, + { + "name": "ms-MY-OsmanNeural", + "gender": "Male" + }, + { + "name": "ms-MY-YasminNeural", + "gender": "Female" + }, + { + "name": "mt-MT-GraceNeural", + "gender": "Female" + }, + { + "name": "mt-MT-JosephNeural", + "gender": "Male" + }, + { + "name": "my-MM-NilarNeural", + "gender": "Female" + }, + { + "name": "my-MM-ThihaNeural", + "gender": "Male" + }, + { + "name": "nb-NO-FinnNeural", + "gender": "Male" + }, + { + "name": "nb-NO-PernilleNeural", + "gender": "Female" + }, + { + "name": "ne-NP-HemkalaNeural", + "gender": "Female" + }, + { + "name": "ne-NP-SagarNeural", + "gender": "Male" + }, + { + "name": "nl-BE-ArnaudNeural", + "gender": "Male" + }, + { + "name": "nl-BE-DenaNeural", + "gender": "Female" + }, + { + "name": "nl-NL-ColetteNeural", + "gender": "Female" + }, + { + "name": "nl-NL-FennaNeural", + "gender": "Female" + }, + { + "name": "nl-NL-MaartenNeural", + "gender": "Male" + }, + { + "name": "pl-PL-MarekNeural", + "gender": "Male" + }, + { + "name": "pl-PL-ZofiaNeural", + "gender": "Female" + }, + { + "name": "ps-AF-GulNawazNeural", + "gender": "Male" + }, + { + "name": "ps-AF-LatifaNeural", + "gender": "Female" + }, + { + "name": "pt-BR-AntonioNeural", + "gender": "Male" + }, + { + "name": "pt-BR-FranciscaNeural", + "gender": "Female" + }, + { + "name": "pt-BR-ThalitaMultilingualNeural", + "gender": "Female" + }, + { + "name": "pt-PT-DuarteNeural", + "gender": "Male" + }, + { + "name": "pt-PT-RaquelNeural", + "gender": "Female" + }, + { + "name": "ro-RO-AlinaNeural", + "gender": "Female" + }, + { + "name": "ro-RO-EmilNeural", + "gender": "Male" + }, + { + "name": "ru-RU-DmitryNeural", + "gender": "Male" + }, + { + "name": "ru-RU-SvetlanaNeural", + "gender": "Female" + }, + { + "name": "si-LK-SameeraNeural", + "gender": "Male" + }, + { + "name": "si-LK-ThiliniNeural", + "gender": "Female" + }, + { + "name": "sk-SK-LukasNeural", + "gender": "Male" + }, + { + "name": "sk-SK-ViktoriaNeural", + "gender": "Female" + }, + { + "name": "sl-SI-PetraNeural", + "gender": "Female" + }, + { + "name": "sl-SI-RokNeural", + "gender": "Male" + }, + { + "name": "so-SO-MuuseNeural", + "gender": "Male" + }, + { + "name": "so-SO-UbaxNeural", + "gender": "Female" + }, + { + "name": "sq-AL-AnilaNeural", + "gender": "Female" + }, + { + "name": "sq-AL-IlirNeural", + "gender": "Male" + }, + { + "name": "sr-RS-NicholasNeural", + "gender": "Male" + }, + { + "name": "sr-RS-SophieNeural", + "gender": "Female" + }, + { + "name": "su-ID-JajangNeural", + "gender": "Male" + }, + { + "name": "su-ID-TutiNeural", + "gender": "Female" + }, + { + "name": "sv-SE-MattiasNeural", + "gender": "Male" + }, + { + "name": "sv-SE-SofieNeural", + "gender": "Female" + }, + { + "name": "sw-KE-RafikiNeural", + "gender": "Male" + }, + { + "name": "sw-KE-ZuriNeural", + "gender": "Female" + }, + { + "name": "sw-TZ-DaudiNeural", + "gender": "Male" + }, + { + "name": "sw-TZ-RehemaNeural", + "gender": "Female" + }, + { + "name": "ta-IN-PallaviNeural", + "gender": "Female" + }, + { + "name": "ta-IN-ValluvarNeural", + "gender": "Male" + }, + { + "name": "ta-LK-KumarNeural", + "gender": "Male" + }, + { + "name": "ta-LK-SaranyaNeural", + "gender": "Female" + }, + { + "name": "ta-MY-KaniNeural", + "gender": "Female" + }, + { + "name": "ta-MY-SuryaNeural", + "gender": "Male" + }, + { + "name": "ta-SG-AnbuNeural", + "gender": "Male" + }, + { + "name": "ta-SG-VenbaNeural", + "gender": "Female" + }, + { + "name": "te-IN-MohanNeural", + "gender": "Male" + }, + { + "name": "te-IN-ShrutiNeural", + "gender": "Female" + }, + { + "name": "th-TH-NiwatNeural", + "gender": "Male" + }, + { + "name": "th-TH-PremwadeeNeural", + "gender": "Female" + }, + { + "name": "tr-TR-AhmetNeural", + "gender": "Male" + }, + { + "name": "tr-TR-EmelNeural", + "gender": "Female" + }, + { + "name": "uk-UA-OstapNeural", + "gender": "Male" + }, + { + "name": "uk-UA-PolinaNeural", + "gender": "Female" + }, + { + "name": "ur-IN-GulNeural", + "gender": "Female" + }, + { + "name": "ur-IN-SalmanNeural", + "gender": "Male" + }, + { + "name": "ur-PK-AsadNeural", + "gender": "Male" + }, + { + "name": "ur-PK-UzmaNeural", + "gender": "Female" + }, + { + "name": "uz-UZ-MadinaNeural", + "gender": "Female" + }, + { + "name": "uz-UZ-SardorNeural", + "gender": "Male" + }, + { + "name": "vi-VN-HoaiMyNeural", + "gender": "Female" + }, + { + "name": "vi-VN-NamMinhNeural", + "gender": "Male" + }, + { + "name": "zh-CN-XiaoxiaoNeural", + "gender": "Female" + }, + { + "name": "zh-CN-XiaoyiNeural", + "gender": "Female" + }, + { + "name": "zh-CN-YunjianNeural", + "gender": "Male" + }, + { + "name": "zh-CN-YunxiNeural", + "gender": "Male" + }, + { + "name": "zh-CN-YunxiaNeural", + "gender": "Male" + }, + { + "name": "zh-CN-YunyangNeural", + "gender": "Male" + }, + { + "name": "zh-CN-liaoning-XiaobeiNeural", + "gender": "Female" + }, + { + "name": "zh-CN-shaanxi-XiaoniNeural", + "gender": "Female" + }, + { + "name": "zh-HK-HiuGaaiNeural", + "gender": "Female" + }, + { + "name": "zh-HK-HiuMaanNeural", + "gender": "Female" + }, + { + "name": "zh-HK-WanLungNeural", + "gender": "Male" + }, + { + "name": "zh-TW-HsiaoChenNeural", + "gender": "Female" + }, + { + "name": "zh-TW-HsiaoYuNeural", + "gender": "Female" + }, + { + "name": "zh-TW-YunJheNeural", + "gender": "Male" + }, + { + "name": "zu-ZA-ThandoNeural", + "gender": "Female" + }, + { + "name": "zu-ZA-ThembaNeural", + "gender": "Male" + }, + { + "name": "en-US-AvaMultilingualNeural-V2", + "gender": "Female" + }, + { + "name": "en-US-AndrewMultilingualNeural-V2", + "gender": "Male" + }, + { + "name": "en-US-EmmaMultilingualNeural-V2", + "gender": "Female" + }, + { + "name": "en-US-BrianMultilingualNeural-V2", + "gender": "Male" + }, + { + "name": "de-DE-FlorianMultilingualNeural-V2", + "gender": "Male" + }, + { + "name": "de-DE-SeraphinaMultilingualNeural-V2", + "gender": "Female" + }, + { + "name": "fr-FR-RemyMultilingualNeural-V2", + "gender": "Male" + }, + { + "name": "fr-FR-VivienneMultilingualNeural-V2", + "gender": "Female" + }, + { + "name": "zh-CN-XiaoxiaoMultilingualNeural-V2", + "gender": "Female" + } +] \ No newline at end of file diff --git a/app/services/llm.py b/app/services/llm.py new file mode 100644 index 0000000..4ef3f47 --- /dev/null +++ b/app/services/llm.py @@ -0,0 +1,958 @@ +import json +import logging +import re +from time import perf_counter +from typing import List + +from loguru import logger +from openai import AzureOpenAI, OpenAI +from openai.types.chat import ChatCompletion + +from app.config import config +from app.models.llm_provider import DEFAULT_LLM_PROVIDER_ID, get_llm_provider + +_max_retries = 5 +MIN_SCRIPT_PARAGRAPH_NUMBER = 1 +MAX_SCRIPT_PARAGRAPH_NUMBER = 10 +MAX_SCRIPT_PROMPT_LENGTH = 2000 +MAX_SCRIPT_SYSTEM_PROMPT_LENGTH = 8000 +_THINK_BLOCK_RE = re.compile(r"]*>.*?", re.IGNORECASE | re.DOTALL) +_UNCLOSED_THINK_BLOCK_RE = re.compile(r"]*>.*$", re.IGNORECASE | re.DOTALL) +_URL_USERINFO_RE = re.compile( + r"((?:https?|wss?)://)([^/\s?#@]*:[^/\s?#@]*@)", re.IGNORECASE +) +_SENSITIVE_QUERY_RE = re.compile( + r"([?&](?:api[_-]?key|access[_-]?token|token|key|secret|password)=)([^&#\s]+)", + re.IGNORECASE, +) + +DEFAULT_SCRIPT_SYSTEM_PROMPT = """ +# Role: Video Script Generator + +## Goals: +Generate a script for a video, depending on the subject of the video. + +## Constrains: +1. the script is to be returned as a string with the specified number of paragraphs. +2. do not under any circumstance reference this prompt in your response. +3. get straight to the point, don't start with unnecessary things like, "welcome to this video". +4. you must not include any type of markdown or formatting in the script, never use a title. +5. only return the raw content of the script. +6. do not include "voiceover", "narrator" or similar indicators of what should be spoken at the beginning of each paragraph or line. +7. you must not mention the prompt, or anything about the script itself. also, never talk about the amount of paragraphs or lines. just write the script. +8. respond in the same language as the video subject. +""".strip() + + +def _normalize_text_response(content, llm_provider: str) -> str: + # 不同 LLM SDK 在异常或被拦截场景下，可能返回 None、空字符串， + # 甚至返回非字符串对象。这里统一做兜底校验，避免后续直接调用 + # `.replace()` 时抛出 `NoneType` 之类的属性错误。 + if content is None: + raise ValueError(f"[{llm_provider}] returned empty text content") + + if not isinstance(content, str): + raise TypeError( + f"[{llm_provider}] returned non-text content: {type(content).__name__}" + ) + + # MiniMax M3、DeepSeek R1 这类 reasoning 模型可能会把内部推理包在 + # `...` 中返回。视频脚本和关键词只需要最终可朗读文本， + # 如果不在服务层统一清理，WebUI、字幕和配音都会把思考过程当正文处理。 + content = _THINK_BLOCK_RE.sub("", content) + content = _UNCLOSED_THINK_BLOCK_RE.sub("", content).strip() + if not content: + raise ValueError(f"[{llm_provider}] returned empty text content") + + return content.replace("\n", "") + + +def _sanitize_error_message(error: object) -> str: + """ + 清理返回给 WebUI/API 的错误信息，避免自定义 base_url 中的凭据泄露。 + + 一些 OpenAI-compatible SDK 会把请求 URL 原样拼进异常信息。如果用户为了 + 代理网关配置了 `https://user:pass@example.com/v1`，直接返回 `str(e)` + 就会把密码暴露给页面、API 调用方或后续日志。这里仅处理错误文案，不改变 + 实际请求地址，避免影响正常调用链路。 + """ + message = str(error) + message = _URL_USERINFO_RE.sub(r"\1***:***@", message) + message = _SENSITIVE_QUERY_RE.sub(r"\1***", message) + return message + + +def _extract_chat_completion_text(response, llm_provider: str) -> str: + # OpenAI 兼容接口在异常场景下，可能返回没有 choices、 + # 或者 choices/message/content 为空的响应对象。 + # 这里统一做结构校验，避免出现 `NoneType is not subscriptable` + # 这类底层属性访问错误。 + choices = getattr(response, "choices", None) + if not choices: + raise ValueError(f"[{llm_provider}] returned empty choices") + + first_choice = choices[0] + message = getattr(first_choice, "message", None) + if message is None: + raise ValueError(f"[{llm_provider}] returned empty message") + + content = getattr(message, "content", None) + return _normalize_text_response(content, llm_provider) + + +def _get_response_field(value, key: str): + """兼容 dict 和 SDK 响应对象的字段读取。""" + if isinstance(value, dict): + return value.get(key) + + try: + return value[key] + except (KeyError, TypeError, AttributeError): + return getattr(value, key, None) + + +def _extract_qwen_generation_text(response) -> str: + """ + 从 DashScope Generation 响应中提取文本。 + + Qwen 使用 `messages` 调用时返回的是 chat 结构： + `output.choices[0].message.content`；旧 completion 形态才会返回 + `output.text`。这里两个路径都兼容，避免 `output.text` 为 None 时 + 继续 `.replace()` 触发不可诊断的 AttributeError。 + """ + output = _get_response_field(response, "output") + choices = _get_response_field(output, "choices") if output else None + if choices is not None: + if not choices: + logger.warning("Qwen returned an empty choices list") + raise ValueError("[qwen] returned empty choices") + + first_choice = choices[0] + message = _get_response_field(first_choice, "message") + content = _get_response_field(message, "content") if message else None + if content is not None: + return _normalize_text_response(content, "qwen") + + text = _get_response_field(output, "text") if output else None + return _normalize_text_response(text, "qwen") + + +def _generate_response(prompt: str) -> str: + try: + llm_provider = str( + config.app.get("llm_provider", DEFAULT_LLM_PROVIDER_ID) + ).lower() + provider = get_llm_provider(llm_provider) + if provider is None: + raise ValueError(f"{llm_provider}: unsupported llm provider") + + logger.info(f"llm provider: {llm_provider}") + api_key = config.app.get(provider.config_key("api_key"), "") + configured_model = config.app.get(provider.config_key("model_name"), "") + model_name = provider.resolve_model_name(configured_model) + if configured_model and model_name != configured_model: + logger.warning( + f"{llm_provider} model '{configured_model}' is deprecated, " + f"fallback to '{model_name}'" + ) + configured_base_url = config.app.get(provider.config_key("base_url"), "") + base_url = provider.resolve_base_url(configured_base_url) + if configured_base_url and configured_base_url.strip().rstrip("/") in { + url.rstrip("/") for url in provider.deprecated_base_urls + }: + logger.warning( + f"{llm_provider} base URL '{configured_base_url}' is deprecated, " + f"fallback to '{base_url}'" + ) + adapter = provider.adapter + api_version = "" + + # Ollama 的默认地址依赖当前是否运行在容器中，无法作为静态 Registry + # 值保存；Registry 仍负责模型和必填规则，运行环境差异在这里解析。 + if llm_provider == "ollama": + api_key = "ollama" + if not base_url: + base_url = config.get_default_ollama_base_url() + + if adapter == "azure": + api_version = config.app.get( + provider.config_key("api_version"), "2024-02-15-preview" + ) + + extra_values = { + field.config_suffix: ( + config.app.get(provider.config_key(field.config_suffix), "") + or field.default_value + ) + for field in provider.extra_fields + } + + if provider.requires_api_key and not api_key: + raise ValueError( + f"{llm_provider}: api_key is not set, please set it in the config.toml file." + ) + if provider.requires_model_name and not model_name: + raise ValueError( + f"{llm_provider}: model_name is not set, please set it in the config.toml file." + ) + if provider.requires_base_url and not base_url: + raise ValueError( + f"{llm_provider}: base_url is not set, please set it in the config.toml file." + ) + + for field in provider.extra_fields: + if field.required and not extra_values[field.config_suffix]: + raise ValueError( + f"{llm_provider}: {field.config_suffix} is not set, " + "please set it in the config.toml file." + ) + + if adapter == "qwen": + import dashscope + from dashscope.api_entities.dashscope_response import GenerationResponse + + dashscope.api_key = api_key + response = dashscope.Generation.call( + model=model_name, messages=[{"role": "user", "content": prompt}] + ) + if response: + if isinstance(response, GenerationResponse): + status_code = response.status_code + if status_code != 200: + raise Exception( + f'[{llm_provider}] returned an error response: "{response}"' + ) + + return _extract_qwen_generation_text(response) + else: + raise Exception( + f'[{llm_provider}] returned an invalid response: "{response}"' + ) + else: + raise Exception(f"[{llm_provider}] returned an empty response") + + if adapter == "gemini": + from google import genai + from google.genai import types + + http_options = types.HttpOptions(base_url=base_url) if base_url else None + generation_config = types.GenerateContentConfig( + temperature=0.5, + top_p=1, + top_k=1, + max_output_tokens=2048, + safety_settings=[ + types.SafetySetting( + category="HARM_CATEGORY_HARASSMENT", + threshold="BLOCK_ONLY_HIGH", + ), + types.SafetySetting( + category="HARM_CATEGORY_HATE_SPEECH", + threshold="BLOCK_ONLY_HIGH", + ), + types.SafetySetting( + category="HARM_CATEGORY_SEXUALLY_EXPLICIT", + threshold="BLOCK_ONLY_HIGH", + ), + types.SafetySetting( + category="HARM_CATEGORY_DANGEROUS_CONTENT", + threshold="BLOCK_ONLY_HIGH", + ), + ], + ) + + try: + # 新版 google-genai 通过统一 Client 暴露模型服务。上下文管理器 + # 会在请求结束后关闭底层 HTTP 连接，避免频繁生成时积累连接资源。 + with genai.Client( + api_key=api_key, + http_options=http_options, + ) as client: + response = client.models.generate_content( + model=model_name, + contents=prompt, + config=generation_config, + ) + generated_text = response.text + except (AttributeError, IndexError, ValueError) as e: + logger.warning(f"gemini returned invalid response content: {str(e)}") + raise ValueError(f"[{llm_provider}] returned invalid response content") + + return _normalize_text_response(generated_text, llm_provider) + + if adapter == "cloudflare_ai_gateway": + account_id = extra_values["account_id"] + gateway_id = extra_values["gateway_id"] + # Cloudflare 当前推荐的 AI Gateway REST API 兼容 OpenAI SDK。 + # Account ID 用于构造统一端点，Gateway ID 通过请求头选择；这里 + # 不再调用 Workers AI 的 /ai/run/{model} 专用接口。 + client = OpenAI( + api_key=api_key, + base_url=( + f"https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1" + ), + default_headers={"cf-aig-gateway-id": gateway_id}, + ) + response = client.chat.completions.create( + model=model_name, + messages=[{"role": "user", "content": prompt}], + ) + return _extract_chat_completion_text(response, llm_provider) + + if adapter == "litellm": + import litellm + + if not model_name: + raise ValueError( + f"{llm_provider}: model_name is not set, please set it in the config.toml file." + ) + + response = litellm.completion( + model=model_name, + messages=[{"role": "user", "content": prompt}], + drop_params=True, + ) + + if not response: + raise ValueError(f"[{llm_provider}] returned empty response") + if not getattr(response, "choices", None): + raise ValueError(f"[{llm_provider}] returned empty response") + + return _extract_chat_completion_text(response, llm_provider) + + if adapter == "azure": + # Azure OpenAI SDK 使用 `azure_endpoint` 和 `api_version` 生成专用请求地址， + # 不能继续复用下面普通 OpenAI-compatible 的 `base_url` 初始化逻辑。 + # 这里在 Azure 分支内完成请求并立即返回，避免客户端被后续 fallback + # 覆盖，导致用户配置的 Azure 凭证通过校验但实际请求没有被使用。 + logger.info(f"requesting azure chat completion, model: {model_name}") + client = AzureOpenAI( + api_key=api_key, + api_version=api_version, + azure_endpoint=base_url, + ) + response = client.chat.completions.create( + model=model_name, messages=[{"role": "user", "content": prompt}] + ) + if response: + if isinstance(response, ChatCompletion): + return _extract_chat_completion_text(response, llm_provider) + else: + raise Exception( + f'[{llm_provider}] returned an invalid response: "{response}", please check your network ' + f"connection and try again." + ) + else: + raise Exception( + f"[{llm_provider}] returned an empty response, please check your network connection and try again." + ) + + if adapter == "modelscope": + content = "" + client = OpenAI( + api_key=api_key, + base_url=base_url, + ) + response = client.chat.completions.create( + model=model_name, + messages=[{"role": "user", "content": prompt}], + extra_body={"enable_thinking": False}, + stream=True, + ) + if response: + for chunk in response: + if not chunk.choices: + continue + delta = chunk.choices[0].delta + if delta and delta.content: + content += delta.content + + if not content.strip(): + raise ValueError("Empty content in stream response") + + return _normalize_text_response(content, llm_provider) + else: + raise Exception(f"[{llm_provider}] returned an empty response") + + client = OpenAI( + api_key=api_key, + base_url=base_url, + ) + + response = client.chat.completions.create( + model=model_name, messages=[{"role": "user", "content": prompt}] + ) + if response: + if isinstance(response, ChatCompletion): + return _extract_chat_completion_text(response, llm_provider) + else: + raise Exception( + f'[{llm_provider}] returned an invalid response: "{response}", please check your network ' + f"connection and try again." + ) + else: + raise Exception( + f"[{llm_provider}] returned an empty response, please check your network connection and try again." + ) + + except Exception as e: + return f"Error: {_sanitize_error_message(e)}" + + +def test_connection() -> tuple[bool, str, float]: + """ + 使用当前 Provider 配置发起一次最小请求，验证实际生成链路是否可用。 + + 连接测试直接复用 `_generate_response()`，因此会覆盖 API Key、Base URL、 + 模型名称和 Provider 专用字段，但不会进入脚本生成的重试逻辑，也不会发送 + 用户的视频主题或文案。返回值依次为成功状态、错误信息和请求耗时。 + """ + started_at = perf_counter() + response = _generate_response(prompt="Reply with exactly: OK") + elapsed = perf_counter() - started_at + + if not response: + error_message = "LLM returned an empty response" + logger.warning(f"llm connection test failed: {error_message}") + return False, error_message, elapsed + + if response.startswith("Error:"): + error_message = response.removeprefix("Error:").strip() + logger.warning(f"llm connection test failed: {error_message}") + return False, error_message, elapsed + + logger.info(f"llm connection test succeeded, elapsed: {elapsed:.2f}s") + return True, "", elapsed + + +def _limit_script_text(text: str | None, max_length: int, field_name: str) -> str: + value = (text or "").strip() + if len(value) <= max_length: + return value + + # API 层已经用 Pydantic 做长度校验；这里继续兜底，是为了保护 + # WebUI 或内部服务直接调用 generate_script 时不会把超长提示词发送给模型， + # 避免 token 成本异常和请求失败。 + logger.warning( + f"{field_name} is too long and will be truncated to {max_length} characters." + ) + return value[:max_length] + + +def _normalize_script_paragraph_number(paragraph_number: int | None) -> int: + try: + value = int(paragraph_number or MIN_SCRIPT_PARAGRAPH_NUMBER) + except (TypeError, ValueError): + value = MIN_SCRIPT_PARAGRAPH_NUMBER + + if value < MIN_SCRIPT_PARAGRAPH_NUMBER or value > MAX_SCRIPT_PARAGRAPH_NUMBER: + # WebUI 和 API 都会限制范围；这里兜底处理内部调用，避免异常参数直接扩大 + # LLM 生成成本或生成空结果。 + logger.warning( + f"script paragraph_number is out of range and will be clamped: {value}" + ) + return max(MIN_SCRIPT_PARAGRAPH_NUMBER, min(value, MAX_SCRIPT_PARAGRAPH_NUMBER)) + + return value + + +def build_script_prompt( + video_subject: str, + language: str = "", + paragraph_number: int = 1, + video_script_prompt: str = "", + custom_system_prompt: str = "", +) -> str: + paragraph_number = _normalize_script_paragraph_number(paragraph_number) + video_script_prompt = _limit_script_text( + video_script_prompt, MAX_SCRIPT_PROMPT_LENGTH, "video_script_prompt" + ) + custom_system_prompt = _limit_script_text( + custom_system_prompt, MAX_SCRIPT_SYSTEM_PROMPT_LENGTH, "custom_system_prompt" + ) + + # 将“脚本生成规则”和“运行时上下文”分开拼接。这样高级用户即使覆盖默认 + # system prompt，也不会漏掉视频主题、语言、段落数这些每次生成都必须带上的参数。 + prompt = custom_system_prompt or DEFAULT_SCRIPT_SYSTEM_PROMPT + prompt += f""" + +# Initialization: +- video subject: {video_subject} +- number of paragraphs: {paragraph_number} +""".rstrip() + if language: + prompt += f"\n- language: {language}" + if video_script_prompt: + prompt += f""" + +# Additional User Requirements: +{video_script_prompt} +""".rstrip() + + return prompt + + +def generate_script( + video_subject: str, + language: str = "", + paragraph_number: int = 1, + video_script_prompt: str = "", + custom_system_prompt: str = "", +) -> str: + paragraph_number = _normalize_script_paragraph_number(paragraph_number) + video_script_prompt = _limit_script_text( + video_script_prompt, MAX_SCRIPT_PROMPT_LENGTH, "video_script_prompt" + ) + custom_system_prompt = _limit_script_text( + custom_system_prompt, MAX_SCRIPT_SYSTEM_PROMPT_LENGTH, "custom_system_prompt" + ) + prompt = build_script_prompt( + video_subject=video_subject, + language=language, + paragraph_number=paragraph_number, + video_script_prompt=video_script_prompt, + custom_system_prompt=custom_system_prompt, + ) + final_script = "" + logger.info( + "generating video script: " + f"subject={video_subject}, paragraph_number={paragraph_number}, " + f"has_custom_prompt={bool(video_script_prompt.strip())}, " + f"has_custom_system_prompt={bool(custom_system_prompt.strip())}" + ) + + def format_response(response): + # Clean the script + # Remove asterisks, hashes + response = response.replace("*", "") + response = response.replace("#", "") + + # Remove markdown syntax + response = re.sub(r"\[.*\]", "", response) + response = re.sub(r"\(.*\)", "", response) + + # Split the script into paragraphs + paragraphs = response.split("\n\n") + + # Select the specified number of paragraphs + # selected_paragraphs = paragraphs[:paragraph_number] + + # Join the selected paragraphs into a single string + return "\n\n".join(paragraphs) + + for i in range(_max_retries): + try: + response = _generate_response(prompt=prompt) + if response: + final_script = format_response(response) + else: + logging.error("gpt returned an empty response") + + # Some upstream providers may return quota errors as plain text. + if final_script and "当日额度已消耗完" in final_script: + raise ValueError(final_script) + + if final_script: + break + except Exception as e: + logger.error(f"failed to generate script: {e}") + + if i < _max_retries: + logger.warning(f"failed to generate video script, trying again... {i + 1}") + if "Error: " in final_script: + logger.error(f"failed to generate video script: {final_script}") + else: + logger.success(f"completed: \n{final_script}") + return final_script.strip() + + +def _strip_code_fence(text: str) -> str: + """Strip a surrounding markdown code fence from an LLM response. + + Non-OpenAI providers (Claude, Gemini, …) frequently wrap JSON output in a + ```json … ``` fence even when asked to return raw JSON. Removing it lets the + first json.loads() succeed instead of falling through to the regex recovery + path (and spuriously logging a warning). Mirrors the DOTALL handling already + used in _parse_social_metadata(). + """ + t = (text or "").strip() + if t.startswith("```"): + t = re.sub(r"^```[a-zA-Z0-9]*\s*", "", t) + t = re.sub(r"\s*```$", "", t) + return t.strip() + + +def generate_terms( + video_subject: str, + video_script: str, + amount: int = 5, + match_script_order: bool = False, +) -> List[str]: + if match_script_order: + goal = ( + f"Generate {amount} chronological stock-video search terms that follow " + "the order of topics in the video script." + ) + ordering_rule = ( + "6. keep the terms in the same order as the script narration; " + "earlier terms must describe earlier visual moments." + ) + # 有序关键词模式下，示例数量要和 amount 保持一致，避免模型被固定 + # 的 4 个示例误导，导致长文案只返回少量关键词，影响素材覆盖度。 + example_terms = [ + "opening visual topic", + *[f"script visual topic {index}" for index in range(2, max(amount, 1))], + "final visual topic", + ] + output_example = json.dumps(example_terms[:amount], ensure_ascii=False) + else: + goal = ( + f"Generate {amount} search terms for stock videos, depending on the " + "subject of a video." + ) + ordering_rule = "" + output_example = ( + '["search term 1", "search term 2", "search term 3",' + '"search term 4", "search term 5"]' + ) + + prompt = f""" +# Role: Video Search Terms Generator + +## Goals: +{goal} + +## Constrains: +1. the search terms are to be returned as a json-array of strings. +2. each search term should consist of 1-3 words, always add the main subject of the video. +3. you must only return the json-array of strings. you must not return anything else. you must not return the script. +4. the search terms must be related to the subject of the video. +5. reply with english search terms only. +{ordering_rule} + +## Output Example: +{output_example} + +## Context: +### Video Subject +{video_subject} + +### Video Script +{video_script} + +Please note that you must use English for generating video search terms; Chinese is not accepted. +""".strip() + + logger.info(f"subject: {video_subject}, match_script_order: {match_script_order}") + + search_terms = [] + response = "" + for i in range(_max_retries): + try: + response = _generate_response(prompt) + if "Error: " in response: + logger.error(f"failed to generate video script: {response}") + return response + search_terms = json.loads(_strip_code_fence(response)) + if not isinstance(search_terms, list) or not all( + isinstance(term, str) for term in search_terms + ): + logger.error("response is not a list of strings.") + continue + + except Exception as e: + logger.warning(f"failed to generate video terms: {str(e)}") + if response: + match = re.search(r"\[.*]", response, re.DOTALL) + if match: + try: + search_terms = json.loads(match.group()) + except Exception as e: + # 这里保留重试流程，但必须记录 LLM 返回的非标准 JSON， + # 否则后续排查搜索词为空时无法定位 + # 是模型格式问题还是解析逻辑问题。 + logger.warning(f"failed to generate video terms: {str(e)}") + + if search_terms and len(search_terms) > 0: + break + if i < _max_retries: + logger.warning(f"failed to generate video terms, trying again... {i + 1}") + + logger.success(f"completed: \n{search_terms}") + return search_terms + + +# ============================================================================= +# Social publishing metadata +# +# 根据视频主题和脚本生成发布到短视频平台时常用的 title、caption 和 hashtags。 +# 这块能力只复用现有 LLM provider，不接入任何外部发布服务，也不影响视频生成主链路。 +# ============================================================================= + +# 不同平台的文案长度和 hashtag 数量偏好不同。这里使用保守上限，避免模型返回 +# 过长内容后调用方还需要二次裁剪。 +SOCIAL_PLATFORMS = { + "tiktok": {"title_max": 100, "caption_max": 2200, "hashtag_count": 5}, + "youtube_shorts": {"title_max": 100, "caption_max": 5000, "hashtag_count": 3}, + "instagram_reels": {"title_max": 125, "caption_max": 2200, "hashtag_count": 8}, + "facebook_reels": {"title_max": 125, "caption_max": 2200, "hashtag_count": 5}, +} +DEFAULT_SOCIAL_PLATFORM = "tiktok" +DEFAULT_SOCIAL_LANGUAGE = "auto" +MAX_SOCIAL_SUBJECT_LENGTH = 500 +MAX_SOCIAL_SCRIPT_LENGTH = 8000 +MAX_SOCIAL_LANGUAGE_LENGTH = 64 + +SOCIAL_PLATFORM_LABELS = { + "tiktok": "TikTok", + "youtube_shorts": "YouTube Shorts", + "instagram_reels": "Instagram Reels", + "facebook_reels": "Facebook Reels", +} + +# LLM 不可用时的通用兜底标签。这里故意不绑定某个国家或语种，保证 API +# 对中文、英文、越南语等不同场景都能返回可用结构。 +DEFAULT_SOCIAL_HASHTAGS = [ + "#shorts", + "#viral", + "#trending", + "#fyp", + "#video", + "#reels", + "#creator", + "#content", +] + + +def _resolve_social_platform(platform: str | None) -> str: + value = (platform or "").strip().lower() + return value if value in SOCIAL_PLATFORMS else DEFAULT_SOCIAL_PLATFORM + + +def _normalize_social_language(language: str | None) -> str: + value = (language or DEFAULT_SOCIAL_LANGUAGE).strip() + if len(value) > MAX_SOCIAL_LANGUAGE_LENGTH: + logger.warning( + "social metadata language is too long and will be truncated to " + f"{MAX_SOCIAL_LANGUAGE_LENGTH} characters." + ) + value = value[:MAX_SOCIAL_LANGUAGE_LENGTH] + return value or DEFAULT_SOCIAL_LANGUAGE + + +def _limit_social_text(text: str | None, max_length: int, field_name: str) -> str: + value = (text or "").strip() + if len(value) <= max_length: + return value + + # API 层会限制长度；这里继续兜底，是为了保护内部调用或未来 WebUI + # 直接调用时不会把超长内容发送给模型，避免 token 成本异常。 + logger.warning( + f"{field_name} is too long and will be truncated to {max_length} characters." + ) + return value[:max_length] + + +def _social_language_instruction(language: str | None) -> str: + language = _normalize_social_language(language) + if language.lower() == DEFAULT_SOCIAL_LANGUAGE: + return ( + "Use the same language as the video subject and script. If the subject " + "and script use different languages, prefer the script language." + ) + + return f'Write "title" and "caption" in this language: {language}.' + + +def _clamp_text(text, max_length: int) -> str: + value = ("" if text is None else str(text)).strip() + if max_length and len(value) > max_length: + return value[:max_length].rstrip() + return value + + +def _normalize_hashtags(raw, count: int) -> List[str]: + """ + 将 LLM 返回的 hashtag 统一整理成 `#tag` 格式。 + + LLM 可能返回字符串、数组、带空格的词组、重复标签或包含标点的内容。 + 这里集中清洗，可以让接口响应结构稳定，也避免平台发布时出现空标签、 + 重复标签或不符合常见格式的 hashtag。 + """ + if isinstance(raw, str): + candidates = re.split(r"[\s,]+", raw) + elif isinstance(raw, (list, tuple)): + # 数组里的每一项视为一个完整标签，因此 "du lich" 会变成 + # "#dulich"，而不是拆成两个标签。 + candidates = [str(entry) for entry in raw] + else: + candidates = [] + + seen = set() + result: List[str] = [] + for item in candidates: + tag = re.sub(r"[^\w]", "", item, flags=re.UNICODE) + if not tag: + continue + key = tag.lower() + if key in seen: + continue + seen.add(key) + result.append(f"#{tag}") + if count and len(result) >= count: + break + return result + + +def build_social_metadata_prompt( + video_subject: str, + video_script: str = "", + language: str = DEFAULT_SOCIAL_LANGUAGE, + platform: str = DEFAULT_SOCIAL_PLATFORM, +) -> str: + video_subject = _limit_social_text( + video_subject, MAX_SOCIAL_SUBJECT_LENGTH, "video_subject" + ) + video_script = _limit_social_text( + video_script, MAX_SOCIAL_SCRIPT_LENGTH, "video_script" + ) + platform = _resolve_social_platform(platform) + spec = SOCIAL_PLATFORMS[platform] + label = SOCIAL_PLATFORM_LABELS.get(platform, platform) + language_instruction = _social_language_instruction(language) + + prompt = f""" +# Role: Short-Video Social Media Copywriter + +## Goal +Write engaging publishing metadata for a short video that will be posted on {label}. + +## Constraints +1. Respond ONLY with a single valid minified JSON object. No markdown, no code fences, no commentary. +2. The JSON must contain exactly these keys: "title", "caption", "hashtags". +3. "title": a catchy hook, at most {spec["title_max"]} characters. +4. "caption": an engaging description that ends with a call to action, at most {spec["caption_max"]} characters. Do not put hashtags inside the caption. +5. "hashtags": a JSON array of exactly {spec["hashtag_count"]} strings. Each must start with "#", contain no spaces, and be relevant to the topic and to {label}. +6. {language_instruction} + +## Output Example +{{"title":"...","caption":"...","hashtags":["#example","#video"]}} + +## Context +### Video Subject +{video_subject} + +### Video Script +{video_script} +""".strip() + return prompt + + +def _parse_social_metadata(response: str, platform: str) -> dict: + spec = SOCIAL_PLATFORMS[_resolve_social_platform(platform)] + + data = None + try: + data = json.loads(_strip_code_fence(response)) + except Exception: + # 部分模型会在 JSON 外层包一段说明文字或 markdown fence。 + # API 调用方只需要稳定结构，所以这里尝试提取第一个 JSON object。 + match = re.search(r"\{.*\}", response or "", re.DOTALL) + if match: + data = json.loads(match.group()) + + if not isinstance(data, dict): + raise ValueError("social metadata response is not a JSON object") + + title = _clamp_text(data.get("title", ""), spec["title_max"]) + caption = _clamp_text(data.get("caption", ""), spec["caption_max"]) + hashtags = _normalize_hashtags(data.get("hashtags", []), spec["hashtag_count"]) + + if not title and not caption: + raise ValueError("social metadata response is missing both title and caption") + + return {"title": title, "caption": caption, "hashtags": hashtags} + + +def _fallback_social_metadata( + video_subject: str, video_script: str, platform: str +) -> dict: + spec = SOCIAL_PLATFORMS[_resolve_social_platform(platform)] + subject = (video_subject or "").strip() + script = (video_script or "").strip() + + title = subject + if not title and script: + # 没有主题时，用脚本第一句兜底生成 title，避免接口返回空标题。 + title = re.split(r"(?<=[.!?。！？])\s+", script)[0] + + return { + "title": _clamp_text(title, spec["title_max"]), + "caption": _clamp_text(script or subject, spec["caption_max"]), + "hashtags": _normalize_hashtags(DEFAULT_SOCIAL_HASHTAGS, spec["hashtag_count"]), + } + + +def generate_social_metadata( + video_subject: str, + video_script: str = "", + language: str = DEFAULT_SOCIAL_LANGUAGE, + platform: str = DEFAULT_SOCIAL_PLATFORM, +) -> dict: + """ + 生成短视频发布文案元数据。 + + 返回结构固定为 `{"title": str, "caption": str, "hashtags": List[str]}`。 + 如果 LLM 不可用或返回格式异常，会降级为通用启发式结果，保证 API + 调用方始终拿到可展示、可发布前编辑的数据结构。 + """ + platform = _resolve_social_platform(platform) + language = _normalize_social_language(language) + video_subject = _limit_social_text( + video_subject, MAX_SOCIAL_SUBJECT_LENGTH, "video_subject" + ) + video_script = _limit_social_text( + video_script, MAX_SOCIAL_SCRIPT_LENGTH, "video_script" + ) + prompt = build_social_metadata_prompt( + video_subject=video_subject, + video_script=video_script, + language=language, + platform=platform, + ) + logger.info(f"generating social metadata: platform={platform}, language={language}") + + response = "" + for i in range(_max_retries): + try: + response = _generate_response(prompt) + if isinstance(response, str) and "Error: " in response: + logger.error(f"failed to generate social metadata: {response}") + break + metadata = _parse_social_metadata(response, platform) + logger.success(f"completed: \n{metadata}") + return metadata + except Exception as e: + logger.warning(f"failed to parse social metadata: {str(e)}") + + if i < _max_retries - 1: + logger.warning( + f"failed to generate social metadata, trying again... {i + 1}" + ) + + logger.warning("falling back to heuristic social metadata") + return _fallback_social_metadata(video_subject, video_script, platform) + + +if __name__ == "__main__": + video_subject = "生命的意义是什么" + script = generate_script( + video_subject=video_subject, language="zh-CN", paragraph_number=1 + ) + print("######################") + print(script) + search_terms = generate_terms( + video_subject=video_subject, video_script=script, amount=5 + ) + print("######################") + print(search_terms) diff --git a/app/services/material.py b/app/services/material.py new file mode 100644 index 0000000..caf4a95 --- /dev/null +++ b/app/services/material.py @@ -0,0 +1,476 @@ +import os +import random +import threading +from typing import List +from urllib.parse import urlencode + +import requests +from loguru import logger +from moviepy.video.io.VideoFileClip import VideoFileClip + +from app.config import config +from app.models.schema import MaterialInfo, VideoAspect, VideoConcatMode +from app.utils import utils + +# Thread-safe counter for API key rotation +_api_key_counter = 0 +_api_key_lock = threading.Lock() + + +def _get_tls_verify() -> bool: + # 默认开启 TLS 证书校验，防止素材搜索和下载过程被中间人篡改。 + # 仅在企业代理、自签证书等明确需要的场景下，允许用户通过 + # `config.toml` 显式设置 `tls_verify = false` 临时关闭。 + tls_verify = config.app.get("tls_verify", True) + if isinstance(tls_verify, str): + tls_verify = tls_verify.strip().lower() not in ("0", "false", "no", "off") + + if not tls_verify: + logger.warning( + "TLS certificate verification is disabled by config.app.tls_verify=false. " + "Only use this in trusted proxy environments." + ) + + return bool(tls_verify) + + +def get_api_key(cfg_key: str): + api_keys = config.app.get(cfg_key) + if not api_keys: + raise ValueError( + f"\n\n##### {cfg_key} is not set #####\n\nPlease set it in the config.toml file: {config.config_file}\n\n" + f"{utils.to_json(config.app)}" + ) + + # if only one key is provided, return it + if isinstance(api_keys, str): + return api_keys + + global _api_key_counter + with _api_key_lock: + _api_key_counter += 1 + return api_keys[_api_key_counter % len(api_keys)] + + +def search_videos_pexels( + search_term: str, + minimum_duration: int, + video_aspect: VideoAspect = VideoAspect.portrait, +) -> List[MaterialInfo]: + aspect = VideoAspect(video_aspect) + video_orientation = aspect.name + video_width, video_height = aspect.to_resolution() + api_key = get_api_key("pexels_api_keys") + headers = { + "Authorization": api_key, + "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36", + } + # Build URL + params = {"query": search_term, "per_page": 20, "orientation": video_orientation} + query_url = f"https://api.pexels.com/videos/search?{urlencode(params)}" + logger.info(f"searching videos: {query_url}, with proxies: {config.proxy}") + + try: + r = requests.get( + query_url, + headers=headers, + proxies=config.proxy, + verify=_get_tls_verify(), + timeout=(30, 60), + ) + response = r.json() + video_items = [] + if "videos" not in response: + logger.error(f"search videos failed: {response}") + return video_items + videos = response["videos"] + # loop through each video in the result + for v in videos: + duration = v["duration"] + # check if video has desired minimum duration + if duration < minimum_duration: + continue + video_files = v["video_files"] + # loop through each url to determine the best quality + for video in video_files: + w = int(video["width"]) + h = int(video["height"]) + if w == video_width and h == video_height: + item = MaterialInfo() + item.provider = "pexels" + item.url = video["link"] + item.duration = duration + video_items.append(item) + break + return video_items + except Exception as e: + logger.error(f"search videos failed: {str(e)}") + + return [] + + +def search_videos_pixabay( + search_term: str, + minimum_duration: int, + video_aspect: VideoAspect = VideoAspect.portrait, +) -> List[MaterialInfo]: + aspect = VideoAspect(video_aspect) + + video_width, video_height = aspect.to_resolution() + + api_key = get_api_key("pixabay_api_keys") + # Build URL + params = { + "q": search_term, + "video_type": "all", # Accepted values: "all", "film", "animation" + "per_page": 50, + "key": api_key, + } + query_url = f"https://pixabay.com/api/videos/?{urlencode(params)}" + logger.info(f"searching videos: {query_url}, with proxies: {config.proxy}") + + try: + r = requests.get( + query_url, proxies=config.proxy, verify=_get_tls_verify(), timeout=(30, 60) + ) + response = r.json() + video_items = [] + if "hits" not in response: + logger.error(f"search videos failed: {response}") + return video_items + videos = response["hits"] + # loop through each video in the result + for v in videos: + duration = v["duration"] + # check if video has desired minimum duration + if duration < minimum_duration: + continue + video_files = v["videos"] + # loop through each url to determine the best quality + for video_type in video_files: + video = video_files[video_type] + w = int(video["width"]) + # h = int(video["height"]) + if w >= video_width: + item = MaterialInfo() + item.provider = "pixabay" + item.url = video["url"] + item.duration = duration + video_items.append(item) + break + return video_items + except Exception as e: + logger.error(f"search videos failed: {str(e)}") + + return [] + + +def search_videos_coverr( + search_term: str, + minimum_duration: int, + video_aspect: VideoAspect = VideoAspect.portrait, +) -> List[MaterialInfo]: + """ + Coverr (https://coverr.co) - free HD/4K stock videos, + subject to Coverr license terms (https://coverr.co/license). + + Coverr API notes (based on official docs at api.coverr.co/docs/): + - 鉴权: Authorization: Bearer + - 搜索端点: GET /videos?query=...,响应结构 {"hits": [...], ...} + - 加 ?urls=true 在搜索响应里直接返回 mp4 直链 + - URL 是 signed JWT(绑定 API key,无过期时间) + - Coverr 库以 16:9 横屏为主,9:16 portrait 占比极低(约 1%) + 因此本函数不做 aspect_ratio 过滤,由下游 video.py 的 + resize + letterbox 逻辑统一处理 + - duration 字段同时存在 number 和 string 两种形态,本函数都接受 + + 本函数使用 urls.mp4_download 字段作为下载地址 —— 按 Coverr 官方文档 + (https://api.coverr.co/docs/videos/#download-a-video) 的说法, + GET 这个 URL 本身就被 Coverr 当作一次合法的 download 事件计入统计, + 无需再调用 PATCH /videos/:id/stats/downloads。 + """ + api_key = get_api_key("coverr_api_keys") + headers = {"Authorization": f"Bearer {api_key}"} + params = { + "query": search_term, + "page_size": 20, + "urls": "true", + "sort": "popular", + } + query_url = f"https://api.coverr.co/videos?{urlencode(params)}" + logger.info(f"searching videos: {query_url}, with proxies: {config.proxy}") + + try: + r = requests.get( + query_url, + headers=headers, + proxies=config.proxy, + verify=_get_tls_verify(), + timeout=(30, 60), + ) + response = r.json() + video_items: List[MaterialInfo] = [] + + if not isinstance(response, dict) or "hits" not in response: + logger.error(f"search videos failed: {response}") + return video_items + + for v in response["hits"]: + # duration 在不同响应里可能是 number(11.625) 或 string("10.500000") + try: + duration = int(float(v.get("duration") or 0)) + except (TypeError, ValueError): + continue + if duration < minimum_duration: + continue + + video_id = v.get("id") + mp4_download_url = (v.get("urls") or {}).get("mp4_download") + if not video_id or not mp4_download_url: + continue + + item = MaterialInfo() + item.provider = "coverr" + item.url = mp4_download_url + item.duration = duration + video_items.append(item) + return video_items + except Exception as e: + logger.error(f"search videos failed: {str(e)}") + + return [] + + +def save_video(video_url: str, save_dir: str = "") -> str: + if not save_dir: + save_dir = utils.storage_dir("cache_videos") + + if not os.path.exists(save_dir): + os.makedirs(save_dir) + + url_without_query = video_url.split("?")[0] + url_hash = utils.md5(url_without_query) + video_id = f"vid-{url_hash}" + video_path = f"{save_dir}/{video_id}.mp4" + + # if video already exists, return the path + if os.path.exists(video_path) and os.path.getsize(video_path) > 0: + logger.info(f"video already exists: {video_path}") + return video_path + + headers = { + "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" + } + + # if video does not exist, download it + with open(video_path, "wb") as f: + f.write( + requests.get( + video_url, + headers=headers, + proxies=config.proxy, + verify=_get_tls_verify(), + timeout=(60, 240), + ).content + ) + + if os.path.exists(video_path) and os.path.getsize(video_path) > 0: + clip = None + try: + clip = VideoFileClip(video_path) + duration = clip.duration + fps = clip.fps + if duration > 0 and fps > 0: + return video_path + except Exception as e: + logger.warning(f"invalid video file: {video_path} => {str(e)}") + try: + os.remove(video_path) + except Exception as remove_error: + logger.warning( + f"failed to remove invalid video file: {video_path}, error: {str(remove_error)}" + ) + finally: + if clip is not None: + try: + clip.close() + except Exception as close_error: + logger.warning( + f"failed to close video clip: {video_path}, error: {str(close_error)}" + ) + return "" + + +def download_videos( + task_id: str, + search_terms: List[str], + source: str = "pexels", + video_aspect: VideoAspect = VideoAspect.portrait, + video_concat_mode: VideoConcatMode = VideoConcatMode.random, + audio_duration: float = 0.0, + max_clip_duration: int = 5, + match_script_order: bool = False, +) -> List[str]: + search_videos = search_videos_pexels + if source == "pixabay": + search_videos = search_videos_pixabay + elif source == "coverr": + search_videos = search_videos_coverr + + material_directory = config.app.get("material_directory", "").strip() + if material_directory == "task": + material_directory = utils.task_dir(task_id) + elif material_directory and not os.path.isdir(material_directory): + material_directory = "" + + if match_script_order: + return _download_videos_by_script_order( + task_id=task_id, + search_terms=search_terms, + search_videos=search_videos, + video_aspect=video_aspect, + audio_duration=audio_duration, + max_clip_duration=max_clip_duration, + material_directory=material_directory, + ) + + valid_video_items = [] + valid_video_urls = [] + found_duration = 0.0 + for search_term in search_terms: + video_items = search_videos( + search_term=search_term, + minimum_duration=max_clip_duration, + video_aspect=video_aspect, + ) + logger.info(f"found {len(video_items)} videos for '{search_term}'") + + for item in video_items: + if item.url not in valid_video_urls: + valid_video_items.append(item) + valid_video_urls.append(item.url) + found_duration += item.duration + + logger.info( + f"found total videos: {len(valid_video_items)}, required duration: {audio_duration} seconds, found duration: {found_duration} seconds" + ) + video_paths = [] + + concat_mode_value = getattr(video_concat_mode, "value", video_concat_mode) + if concat_mode_value == VideoConcatMode.random.value: + random.shuffle(valid_video_items) + + total_duration = 0.0 + for item in valid_video_items: + try: + logger.info(f"downloading video: {item.url}") + saved_video_path = save_video( + video_url=item.url, save_dir=material_directory + ) + if saved_video_path: + logger.info(f"video saved: {saved_video_path}") + video_paths.append(saved_video_path) + seconds = min(max_clip_duration, item.duration) + total_duration += seconds + if total_duration > audio_duration: + logger.info( + f"total duration of downloaded videos: {total_duration} seconds, skip downloading more" + ) + break + except Exception as e: + logger.error(f"failed to download video: {utils.to_json(item)} => {str(e)}") + logger.success(f"downloaded {len(video_paths)} videos") + return video_paths + + +def _download_videos_by_script_order( + task_id: str, + search_terms: List[str], + search_videos, + video_aspect: VideoAspect, + audio_duration: float, + max_clip_duration: int, + material_directory: str, +) -> List[str]: + """ + 按脚本文案顺序下载素材。 + + 默认下载逻辑会把所有关键词的候选素材合并成一个大列表；如果第一个 + 关键词返回很多结果，最终下载时可能一直消耗这个关键词的素材，后续 + 脚本主题就排不上时间线。这里按关键词分组后轮询下载： + 第 1 轮取每个关键词的第 1 个候选，第 2 轮取每个关键词的第 2 个候选。 + 这样在不重写视频合成引擎的前提下，尽量保证素材顺序贴近文案顺序。 + """ + logger.info("downloading videos with script-order material matching") + candidate_groups = [] + valid_video_urls = set() + found_duration = 0.0 + + for search_term in search_terms: + video_items = search_videos( + search_term=search_term, + minimum_duration=max_clip_duration, + video_aspect=video_aspect, + ) + logger.info(f"found {len(video_items)} videos for '{search_term}'") + + term_items = [] + for item in video_items: + if item.url in valid_video_urls: + continue + term_items.append(item) + valid_video_urls.add(item.url) + found_duration += item.duration + + if term_items: + candidate_groups.append((search_term, term_items)) + + logger.info( + f"found total ordered video candidates: {sum(len(items) for _, items in candidate_groups)}, " + f"required duration: {audio_duration} seconds, found duration: {found_duration} seconds" + ) + + video_paths = [] + total_duration = 0.0 + candidate_index = 0 + while candidate_groups and total_duration <= audio_duration: + has_candidate = False + for search_term, term_items in candidate_groups: + if candidate_index >= len(term_items): + continue + + has_candidate = True + item = term_items[candidate_index] + try: + logger.info( + f"downloading ordered video for '{search_term}': {item.url}" + ) + saved_video_path = save_video( + video_url=item.url, save_dir=material_directory + ) + if saved_video_path: + logger.info(f"video saved: {saved_video_path}") + video_paths.append(saved_video_path) + total_duration += min(max_clip_duration, item.duration) + if total_duration > audio_duration: + logger.info( + f"total duration of downloaded videos: {total_duration} seconds, skip downloading more" + ) + break + except Exception as e: + logger.error( + f"failed to download ordered video: {utils.to_json(item)} => {str(e)}" + ) + + if not has_candidate: + break + candidate_index += 1 + + logger.success(f"downloaded {len(video_paths)} ordered videos") + return video_paths + + +if __name__ == "__main__": + download_videos( + "test123", ["Money Exchange Medium"], audio_duration=100, source="pixabay" + ) diff --git a/app/services/state.py b/app/services/state.py new file mode 100644 index 0000000..c34e9f8 --- /dev/null +++ b/app/services/state.py @@ -0,0 +1,189 @@ +import ast +import copy +import threading +from abc import ABC, abstractmethod + +from app.config import config +from app.models import const + + +# Base class for state management +class BaseState(ABC): + @abstractmethod + def update_task(self, task_id: str, state: int, progress: int = 0, **kwargs): + pass + + @abstractmethod + def get_task(self, task_id: str): + pass + + @abstractmethod + def get_all_tasks(self, page: int, page_size: int): + pass + + +# Memory state management +class MemoryState(BaseState): + def __init__(self): + self._tasks = {} + self._lock = threading.RLock() + + def get_all_tasks(self, page: int, page_size: int): + start = (page - 1) * page_size + end = start + page_size + with self._lock: + tasks = [copy.deepcopy(task) for task in self._tasks.values()] + total = len(tasks) + return tasks[start:end], total + + def update_task( + self, + task_id: str, + state: int = const.TASK_STATE_PROCESSING, + progress: int = 0, + **kwargs, + ): + progress = int(progress) + if progress > 100: + progress = 100 + + with self._lock: + self._tasks[task_id] = { + "task_id": task_id, + "state": state, + "progress": progress, + **kwargs, + } + + def get_task(self, task_id: str): + with self._lock: + task = self._tasks.get(task_id, None) + return copy.deepcopy(task) if task is not None else None + + def delete_task(self, task_id: str): + with self._lock: + self._tasks.pop(task_id, None) + + +# Redis state management +class RedisState(BaseState): + """ + Redis-backed task state. + + Trust boundary: Redis is expected to be private to this application. Task + values are written by MoneyPrinterTurbo and converted back from strings for + compatibility with existing state records. Do not expose this Redis database + to untrusted writers without replacing deserialization with a stricter + schema-based format. + """ + + def __init__(self, host="localhost", port=6379, db=0, password=None): + import redis + + self._redis = redis.StrictRedis(host=host, port=port, db=db, password=password) + + def get_all_tasks(self, page: int, page_size: int): + start = (page - 1) * page_size + end = start + page_size + tasks = [] + cursor = 0 + total = 0 + while True: + cursor, keys = self._redis.scan(cursor, count=page_size) + batch_start = total + batch_size = len(keys) + total += batch_size + + # Redis SCAN 是分批返回 key。分页切片必须基于“当前批次起始索引” + # 计算，而不能用累积后的 total 反推，否则第一页会切到空数组， + # 第二页也可能只返回部分数据。 + if batch_start < end and total > start: + slice_start = max(0, start - batch_start) + slice_end = min(batch_size, end - batch_start) + for key in keys[slice_start:slice_end]: + task_data = self._redis.hgetall(key) + task = { + k.decode("utf-8"): self._convert_to_original_type(v) + for k, v in task_data.items() + } + tasks.append(task) + + # 即使当前页已经取满，也要继续 SCAN 到 cursor=0， + # 因为调用方需要准确 total 来渲染分页信息。 + if cursor == 0: + break + return tasks, total + + def update_task( + self, + task_id: str, + state: int = const.TASK_STATE_PROCESSING, + progress: int = 0, + **kwargs, + ): + progress = int(progress) + if progress > 100: + progress = 100 + + fields = { + "task_id": task_id, + "state": state, + "progress": progress, + **kwargs, + } + + for field, value in fields.items(): + self._redis.hset(task_id, field, str(value)) + + def get_task(self, task_id: str): + task_data = self._redis.hgetall(task_id) + if not task_data: + return None + + task = { + key.decode("utf-8"): self._convert_to_original_type(value) + for key, value in task_data.items() + } + return task + + def delete_task(self, task_id: str): + self._redis.delete(task_id) + + @staticmethod + def _convert_to_original_type(value): + """ + Convert values written by this application back to common Python types. + + This compatibility parser assumes Redis is inside the application's + trust boundary. If Redis can be written by untrusted clients, task state + should move to a strict JSON/schema parser instead of open-ended literal + conversion. + """ + value_str = value.decode("utf-8") + + try: + # try to convert byte string array to list + return ast.literal_eval(value_str) + except (ValueError, SyntaxError): + pass + + if value_str.isdigit(): + return int(value_str) + # Add more conversions here if needed + return value_str + + +# Global state +_enable_redis = config.app.get("enable_redis", False) +_redis_host = config.app.get("redis_host", "localhost") +_redis_port = config.app.get("redis_port", 6379) +_redis_db = config.app.get("redis_db", 0) +_redis_password = config.app.get("redis_password", None) + +state = ( + RedisState( + host=_redis_host, port=_redis_port, db=_redis_db, password=_redis_password + ) + if _enable_redis + else MemoryState() +) diff --git a/app/services/subtitle.py b/app/services/subtitle.py new file mode 100644 index 0000000..772a04d --- /dev/null +++ b/app/services/subtitle.py @@ -0,0 +1,313 @@ +import json +import os.path +import re +from timeit import default_timer as timer + +try: + from faster_whisper import WhisperModel +except ImportError: + WhisperModel = None +from loguru import logger + +from app.config import config +from app.utils import utils + +model_size = config.whisper.get("model_size", "large-v3") +device = config.whisper.get("device", "cpu") +compute_type = config.whisper.get("compute_type", "int8") +model = None + + +def create(audio_file, subtitle_file: str = ""): + global model + if WhisperModel is None: + logger.warning("faster_whisper not available, skipping whisper subtitle generation") + return "" + if not model: + model_path = f"{utils.root_dir()}/models/whisper-{model_size}" + model_bin_file = f"{model_path}/model.bin" + if not os.path.isdir(model_path) or not os.path.isfile(model_bin_file): + model_path = model_size + + logger.info( + f"loading model: {model_path}, device: {device}, compute_type: {compute_type}" + ) + try: + model = WhisperModel( + model_size_or_path=model_path, device=device, compute_type=compute_type + ) + except Exception as e: + logger.error( + f"failed to load model: {e} \n\n" + f"********************************************\n" + f"this may be caused by network issue. \n" + f"please download the model manually and put it in the 'models' folder. \n" + f"see [README.md FAQ](https://github.com/harry0703/MoneyPrinterTurbo) for more details.\n" + f"********************************************\n\n" + ) + return None + + logger.info(f"start, output file: {subtitle_file}") + if not subtitle_file: + subtitle_file = f"{audio_file}.srt" + + segments, info = model.transcribe( + audio_file, + beam_size=5, + word_timestamps=True, + vad_filter=True, + vad_parameters=dict(min_silence_duration_ms=500), + ) + + logger.info( + f"detected language: '{info.language}', probability: {info.language_probability:.2f}" + ) + + start = timer() + subtitles = [] + + def recognized(seg_text, seg_start, seg_end): + seg_text = seg_text.strip() + if not seg_text: + return + + msg = "[%.2fs -> %.2fs] %s" % (seg_start, seg_end, seg_text) + logger.debug(msg) + + subtitles.append( + {"msg": seg_text, "start_time": seg_start, "end_time": seg_end} + ) + + for segment in segments: + words_idx = 0 + words_len = len(segment.words) + + seg_start = 0 + seg_end = 0 + seg_text = "" + + if segment.words: + is_segmented = False + for word in segment.words: + if not is_segmented: + seg_start = word.start + is_segmented = True + + seg_end = word.end + # If it contains punctuation, then break the sentence. + seg_text += word.word + + if utils.str_contains_punctuation(word.word): + # remove last char + seg_text = seg_text[:-1] + if not seg_text: + continue + + recognized(seg_text, seg_start, seg_end) + + is_segmented = False + seg_text = "" + + if words_idx == 0 and segment.start < word.start: + seg_start = word.start + if words_idx == (words_len - 1) and segment.end > word.end: + seg_end = word.end + words_idx += 1 + + if not seg_text: + continue + + recognized(seg_text, seg_start, seg_end) + + end = timer() + + diff = end - start + logger.info(f"complete, elapsed: {diff:.2f} s") + + idx = 1 + lines = [] + for subtitle in subtitles: + text = subtitle.get("msg") + if text: + lines.append( + utils.text_to_srt( + idx, text, subtitle.get("start_time"), subtitle.get("end_time") + ) + ) + idx += 1 + + sub = "\n".join(lines) + "\n" + with open(subtitle_file, "w", encoding="utf-8") as f: + f.write(sub) + logger.info(f"subtitle file created: {subtitle_file}") + + +def file_to_subtitles(filename): + if not filename or not os.path.isfile(filename): + return [] + + times_texts = [] + current_times = None + current_text = "" + index = 0 + with open(filename, "r", encoding="utf-8") as f: + for line in f: + times = re.findall("([0-9]*:[0-9]*:[0-9]*,[0-9]*)", line) + if times: + current_times = line + elif line.strip() == "" and current_times: + index += 1 + times_texts.append((index, current_times.strip(), current_text.strip())) + current_times, current_text = None, "" + elif current_times: + current_text += line + + # Flush the final block. SRT files whose last subtitle is not followed by a + # trailing blank line never hit the blank-line branch above, so without this + # the last subtitle would be silently dropped. + if current_times: + index += 1 + times_texts.append((index, current_times.strip(), current_text.strip())) + return times_texts + + +def levenshtein_distance(s1, s2): + if len(s1) < len(s2): + return levenshtein_distance(s2, s1) + + if len(s2) == 0: + return len(s1) + + previous_row = range(len(s2) + 1) + for i, c1 in enumerate(s1): + current_row = [i + 1] + for j, c2 in enumerate(s2): + insertions = previous_row[j + 1] + 1 + deletions = current_row[j] + 1 + substitutions = previous_row[j] + (c1 != c2) + current_row.append(min(insertions, deletions, substitutions)) + previous_row = current_row + + return previous_row[-1] + + +def similarity(a, b): + distance = levenshtein_distance(a.lower(), b.lower()) + max_length = max(len(a), len(b)) + return 1 - (distance / max_length) + + +def correct(subtitle_file, video_script): + subtitle_items = file_to_subtitles(subtitle_file) + normalized_script = utils.normalize_script_for_subtitle_matching(video_script) + script_lines = utils.split_string_by_punctuations(normalized_script) + + corrected = False + new_subtitle_items = [] + script_index = 0 + subtitle_index = 0 + + while script_index < len(script_lines) and subtitle_index < len(subtitle_items): + script_line = script_lines[script_index].strip() + subtitle_line = subtitle_items[subtitle_index][2].strip() + + if script_line == subtitle_line: + new_subtitle_items.append(subtitle_items[subtitle_index]) + script_index += 1 + subtitle_index += 1 + else: + combined_subtitle = subtitle_line + start_time = subtitle_items[subtitle_index][1].split(" --> ")[0] + end_time = subtitle_items[subtitle_index][1].split(" --> ")[1] + next_subtitle_index = subtitle_index + 1 + + while next_subtitle_index < len(subtitle_items): + next_subtitle = subtitle_items[next_subtitle_index][2].strip() + if similarity( + script_line, combined_subtitle + " " + next_subtitle + ) > similarity(script_line, combined_subtitle): + combined_subtitle += " " + next_subtitle + end_time = subtitle_items[next_subtitle_index][1].split(" --> ")[1] + next_subtitle_index += 1 + else: + break + + if similarity(script_line, combined_subtitle) > 0.8: + logger.warning( + f"Merged/Corrected - Script: {script_line}, Subtitle: {combined_subtitle}" + ) + new_subtitle_items.append( + ( + len(new_subtitle_items) + 1, + f"{start_time} --> {end_time}", + script_line, + ) + ) + corrected = True + else: + logger.warning( + f"Mismatch - Script: {script_line}, Subtitle: {combined_subtitle}" + ) + new_subtitle_items.append( + ( + len(new_subtitle_items) + 1, + f"{start_time} --> {end_time}", + script_line, + ) + ) + corrected = True + + script_index += 1 + subtitle_index = next_subtitle_index + + # Process the remaining lines of the script. + while script_index < len(script_lines): + logger.warning(f"Extra script line: {script_lines[script_index]}") + if subtitle_index < len(subtitle_items): + new_subtitle_items.append( + ( + len(new_subtitle_items) + 1, + subtitle_items[subtitle_index][1], + script_lines[script_index], + ) + ) + subtitle_index += 1 + else: + new_subtitle_items.append( + ( + len(new_subtitle_items) + 1, + "00:00:00,000 --> 00:00:00,000", + script_lines[script_index], + ) + ) + script_index += 1 + corrected = True + + if corrected: + with open(subtitle_file, "w", encoding="utf-8") as fd: + for i, item in enumerate(new_subtitle_items): + fd.write(f"{i + 1}\n{item[1]}\n{item[2]}\n\n") + logger.info("Subtitle corrected") + else: + logger.success("Subtitle is correct") + + +if __name__ == "__main__": + task_id = "c12fd1e6-4b0a-4d65-a075-c87abe35a072" + task_dir = utils.task_dir(task_id) + subtitle_file = f"{task_dir}/subtitle.srt" + audio_file = f"{task_dir}/audio.mp3" + + subtitles = file_to_subtitles(subtitle_file) + print(subtitles) + + script_file = f"{task_dir}/script.json" + with open(script_file, "r") as f: + script_content = f.read() + s = json.loads(script_content) + script = s.get("script") + + correct(subtitle_file, script) + + subtitle_file = f"{task_dir}/subtitle-test.srt" + create(audio_file, subtitle_file) diff --git a/app/services/task.py b/app/services/task.py new file mode 100644 index 0000000..8a38e6a --- /dev/null +++ b/app/services/task.py @@ -0,0 +1,498 @@ +import math +import os.path +import re +from os import path + +from loguru import logger + +from app.config import config +from app.models import const +from app.models.schema import VideoConcatMode, VideoParams +from app.services import llm, material, subtitle, twelvelabs, video, voice, upload_post +from app.services import state as sm +from app.utils import file_security, utils + + +def generate_script(task_id, params): + logger.info("\n\n## generating video script") + video_script = params.video_script.strip() + if not video_script: + video_script = llm.generate_script( + video_subject=params.video_subject, + language=params.video_language, + paragraph_number=params.paragraph_number, + video_script_prompt=params.video_script_prompt, + custom_system_prompt=params.custom_system_prompt, + ) + else: + logger.debug(f"video script: \n{video_script}") + + if not video_script: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + logger.error("failed to generate video script.") + return None + + return video_script + + +def generate_terms(task_id, params, video_script): + logger.info("\n\n## generating video terms") + video_terms = params.video_terms + if not video_terms: + # 开启素材按文案顺序匹配后，关键词本身也必须按脚本叙事顺序生成； + # 否则后续即使顺序下载和顺序拼接，也只能复用一组全局主题词， + # 无法改善“后面内容的画面提前出现”的问题。 + video_terms = llm.generate_terms( + video_subject=params.video_subject, + video_script=video_script, + amount=8 if params.match_materials_to_script else 5, + match_script_order=params.match_materials_to_script, + ) + else: + if isinstance(video_terms, str): + video_terms = [term.strip() for term in re.split(r"[,，]", video_terms)] + elif isinstance(video_terms, list): + video_terms = [term.strip() for term in video_terms] + else: + raise ValueError("video_terms must be a string or a list of strings.") + + logger.debug(f"video terms: {utils.to_json(video_terms)}") + + if not video_terms: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + logger.error("failed to generate video terms.") + return None + + # 可选的 TwelveLabs Marengo 语义重排：未启用时返回原顺序，无任何副作用。 + # 顺序匹配模式下关键词顺序本身就是脚本叙事顺序，必须保持原样，故跳过。 + if not params.match_materials_to_script: + video_terms = twelvelabs.rerank_terms_by_subject( + video_subject=params.video_subject, + search_terms=video_terms, + ) + + return video_terms + + +def save_script_data(task_id, video_script, video_terms, params): + script_file = path.join(utils.task_dir(task_id), "script.json") + script_data = { + "script": video_script, + "search_terms": video_terms, + "params": params, + } + + with open(script_file, "w", encoding="utf-8") as f: + f.write(utils.to_json(script_data)) + + +def resolve_custom_audio_file(task_id: str, custom_audio_file: str | None) -> str: + requested_file = (custom_audio_file or "").strip() + if not requested_file: + return "" + + task_dir = utils.task_dir(task_id) + try: + return file_security.resolve_path_within_directory( + task_dir, + requested_file, + ) + except ValueError as exc: + task_dir_error = exc + + server_audio_file = path.realpath( + requested_file + if path.isabs(requested_file) + else path.join(utils.root_dir(), requested_file) + ) + if not path.isabs(requested_file): + project_root = path.realpath(utils.root_dir()) + try: + if path.commonpath([project_root, server_audio_file]) != project_root: + raise ValueError( + "relative custom audio paths must stay within the project directory" + ) + except ValueError as exc: + raise ValueError( + "custom audio file must be task-local or an existing server-side file" + ) from exc + + if not path.isfile(server_audio_file): + raise ValueError( + "custom audio file does not exist or is not a file" + ) from task_dir_error + + return server_audio_file + + +def generate_audio(task_id, params, video_script): + ''' + Generate audio for the video script. + If a custom audio file is provided, it will be used directly. + There will be no subtitle maker object returned in this case. + Otherwise, TTS will be used to generate the audio. + Returns: + - audio_file: path to the generated or provided audio file + - audio_duration: duration of the audio in seconds + - sub_maker: subtitle maker object if TTS is used, None otherwise + ''' + logger.info("\n\n## generating audio") + # /audio 和 /subtitle 请求模型不包含 custom_audio_file， + # 这里统一做兼容读取，避免直调接口时抛属性错误。 + requested_custom_audio_file = getattr(params, "custom_audio_file", None) + try: + custom_audio_file = resolve_custom_audio_file( + task_id, requested_custom_audio_file + ) + except ValueError as exc: + logger.error( + "custom audio file is invalid, " + f"task_id: {task_id}, path: {requested_custom_audio_file}, error: {str(exc)}" + ) + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + return None, None, None + + if not custom_audio_file: + logger.info("no custom audio file provided, using TTS to generate audio.") + audio_file = path.join(utils.task_dir(task_id), "audio.mp3") + sub_maker = voice.tts( + text=video_script, + voice_name=voice.parse_voice_name(params.voice_name), + voice_rate=params.voice_rate, + voice_file=audio_file, + ) + if sub_maker is None: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + logger.error( + """failed to generate audio: +1. check if the language of the voice matches the language of the video script. +2. check if the network is available. If you are in China, it is recommended to use a VPN and enable the global traffic mode. + """.strip() + ) + return None, None, None + audio_duration = math.ceil(voice.get_audio_duration(sub_maker)) + if audio_duration == 0: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + logger.error("failed to get audio duration.") + return None, None, None + return audio_file, audio_duration, sub_maker + else: + logger.info(f"using custom audio file: {custom_audio_file}") + audio_duration = voice.get_audio_duration(custom_audio_file) + if audio_duration == 0: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + logger.error("failed to get audio duration from custom audio file.") + return None, None, None + return custom_audio_file, audio_duration, None + +def generate_subtitle(task_id, params, video_script, sub_maker, audio_file): + ''' + Generate subtitle for the video script. + If subtitle generation is disabled or no subtitle maker is provided, it will return an empty string. + Otherwise, it will generate the subtitle using the specified provider. + Returns: + - subtitle_path: path to the generated subtitle file + ''' + logger.info("\n\n## generating subtitle") + if not params.subtitle_enabled: + return "" + + subtitle_path = path.join(utils.task_dir(task_id), "subtitle.srt") + subtitle_provider = config.app.get("subtitle_provider", "edge").strip().lower() + logger.info(f"\n\n## generating subtitle, provider: {subtitle_provider}") + + if sub_maker is None and subtitle_provider != "whisper": + # 自定义音频不会经过 TTS，因此没有 Edge/Azure 等 TTS 返回的 + # sub_maker 时间轴。只有 Whisper 可以直接从音频文件转写字幕； + # 其他字幕提供方继续保持原有行为，避免生成错误的空时间轴。 + logger.warning( + "subtitle maker is missing, skip subtitle generation for provider: " + f"{subtitle_provider}" + ) + return "" + + subtitle_fallback = False + if subtitle_provider == "edge": + voice.create_subtitle( + text=video_script, sub_maker=sub_maker, subtitle_file=subtitle_path + ) + if not os.path.exists(subtitle_path): + subtitle_fallback = True + logger.warning("subtitle file not found, fallback to whisper") + + if subtitle_provider == "whisper" or subtitle_fallback: + subtitle.create(audio_file=audio_file, subtitle_file=subtitle_path) + logger.info("\n\n## correcting subtitle") + subtitle.correct(subtitle_file=subtitle_path, video_script=video_script) + + subtitle_lines = subtitle.file_to_subtitles(subtitle_path) + if not subtitle_lines: + logger.warning(f"subtitle file is invalid: {subtitle_path}") + return "" + + return subtitle_path + + +def get_video_materials(task_id, params, video_terms, audio_duration): + if params.video_source == "local": + logger.info("\n\n## preprocess local materials") + materials = video.preprocess_video( + materials=params.video_materials, clip_duration=params.video_clip_duration + ) + if not materials: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + logger.error( + "no valid materials found, please check the materials and try again." + ) + return None + return [material_info.url for material_info in materials] + else: + logger.info(f"\n\n## downloading videos from {params.video_source}") + # 顺序匹配模式只在用户显式开启时生效。这里强制素材下载按关键词顺序 + # 轮询，避免某个早期关键词下载太多素材，把后续脚本主题挤出最终时间线。 + downloaded_videos = material.download_videos( + task_id=task_id, + search_terms=video_terms, + source=params.video_source, + video_aspect=params.video_aspect, + video_concat_mode=( + VideoConcatMode.sequential + if params.match_materials_to_script + else params.video_concat_mode + ), + audio_duration=audio_duration * params.video_count, + max_clip_duration=params.video_clip_duration, + match_script_order=params.match_materials_to_script, + ) + if not downloaded_videos: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + logger.error( + "failed to download videos, maybe the network is not available. if you are in China, please use a VPN." + ) + return None + return downloaded_videos + + +def generate_final_videos( + task_id, params, downloaded_videos, audio_file, subtitle_path +): + final_video_paths = [] + combined_video_paths = [] + # 多视频生成默认会打散素材以增加差异；但“按文案顺序匹配素材”追求的是 + # 时间线稳定性和可解释性，所以开启后所有输出都使用顺序拼接。 + if params.match_materials_to_script: + video_concat_mode = VideoConcatMode.sequential + elif params.video_count == 1: + video_concat_mode = params.video_concat_mode + else: + video_concat_mode = VideoConcatMode.random + video_transition_mode = params.video_transition_mode + + _progress = 50 + for i in range(params.video_count): + index = i + 1 + combined_video_path = path.join( + utils.task_dir(task_id), f"combined-{index}.mp4" + ) + logger.info(f"\n\n## combining video: {index} => {combined_video_path}") + video.combine_videos( + combined_video_path=combined_video_path, + video_paths=downloaded_videos, + audio_file=audio_file, + video_aspect=params.video_aspect, + video_concat_mode=video_concat_mode, + video_transition_mode=video_transition_mode, + max_clip_duration=params.video_clip_duration, + threads=params.n_threads, + ) + + _progress += 50 / params.video_count / 2 + sm.state.update_task(task_id, progress=_progress) + + final_video_path = path.join(utils.task_dir(task_id), f"final-{index}.mp4") + + logger.info(f"\n\n## generating video: {index} => {final_video_path}") + video.generate_video( + video_path=combined_video_path, + audio_path=audio_file, + subtitle_path=subtitle_path, + output_file=final_video_path, + params=params, + ) + + _progress += 50 / params.video_count / 2 + sm.state.update_task(task_id, progress=_progress) + + final_video_paths.append(final_video_path) + combined_video_paths.append(combined_video_path) + + return final_video_paths, combined_video_paths + + +def start(task_id, params: VideoParams, stop_at: str = "video"): + logger.info(f"start task: {task_id}, stop_at: {stop_at}") + sm.state.update_task(task_id, state=const.TASK_STATE_PROCESSING, progress=5) + + # 1. Generate script + video_script = generate_script(task_id, params) + if not video_script or "Error: " in video_script: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + return + + sm.state.update_task(task_id, state=const.TASK_STATE_PROCESSING, progress=10) + + if stop_at == "script": + sm.state.update_task( + task_id, state=const.TASK_STATE_COMPLETE, progress=100, script=video_script + ) + return {"script": video_script} + + # 2. Generate terms + video_terms = "" + if params.video_source != "local": + video_terms = generate_terms(task_id, params, video_script) + if not video_terms: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + return + + save_script_data(task_id, video_script, video_terms, params) + + if stop_at == "terms": + sm.state.update_task( + task_id, state=const.TASK_STATE_COMPLETE, progress=100, terms=video_terms + ) + return {"script": video_script, "terms": video_terms} + + sm.state.update_task(task_id, state=const.TASK_STATE_PROCESSING, progress=20) + + # 3. Generate audio + audio_file, audio_duration, sub_maker = generate_audio( + task_id, params, video_script + ) + if not audio_file: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + return + + sm.state.update_task(task_id, state=const.TASK_STATE_PROCESSING, progress=30) + + if stop_at == "audio": + sm.state.update_task( + task_id, + state=const.TASK_STATE_COMPLETE, + progress=100, + audio_file=audio_file, + ) + return {"audio_file": audio_file, "audio_duration": audio_duration} + + # 4. Generate subtitle + subtitle_path = generate_subtitle( + task_id, params, video_script, sub_maker, audio_file + ) + + if stop_at == "subtitle": + sm.state.update_task( + task_id, + state=const.TASK_STATE_COMPLETE, + progress=100, + subtitle_path=subtitle_path, + ) + return {"subtitle_path": subtitle_path} + + sm.state.update_task(task_id, state=const.TASK_STATE_PROCESSING, progress=40) + + # 5. Get video materials + downloaded_videos = get_video_materials( + task_id, params, video_terms, audio_duration + ) + if not downloaded_videos: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + return + + if stop_at == "materials": + sm.state.update_task( + task_id, + state=const.TASK_STATE_COMPLETE, + progress=100, + materials=downloaded_videos, + ) + return {"materials": downloaded_videos} + + sm.state.update_task(task_id, state=const.TASK_STATE_PROCESSING, progress=50) + + # 仅完整视频生成流程才需要处理视频拼接模式； + # 这样可以避免 /subtitle 和 /audio 这类请求访问不存在的字段。 + if type(params.video_concat_mode) is str: + params.video_concat_mode = VideoConcatMode(params.video_concat_mode) + + # 6. Generate final videos + final_video_paths, combined_video_paths = generate_final_videos( + task_id, params, downloaded_videos, audio_file, subtitle_path + ) + + if not final_video_paths: + sm.state.update_task(task_id, state=const.TASK_STATE_FAILED) + return + + logger.success( + f"task {task_id} finished, generated {len(final_video_paths)} videos." + ) + + # 7. Cross-post to social platforms (if enabled) + cross_post_results = [] + if upload_post.upload_post_service.is_configured() and upload_post.upload_post_service.auto_upload: + platforms = upload_post.upload_post_service.platforms + logger.info(f"\n\n## cross-posting videos to {', '.join(platforms)}") + + youtube_extra = None + if any(p.startswith("youtube") for p in platforms): + metadata = llm.generate_social_metadata( + video_subject=params.video_subject, + video_script=video_script, + language=params.video_language or "", + platform="youtube_shorts", + ) + youtube_extra = { + "youtube_title": metadata.get("title", params.video_subject), + "youtube_description": metadata.get("caption", ""), + "tags": metadata.get("hashtags", []), + "privacyStatus": upload_post.upload_post_service.youtube_privacy_status, + "containsSyntheticMedia": True, + } + + for video_path in final_video_paths: + result = upload_post.cross_post_video( + video_path=video_path, + title=params.video_subject or "Check out this video! #shorts #viral", + youtube_extra=youtube_extra, + ) + cross_post_results.append(result) + if result.get('success'): + logger.info(f"✅ Cross-posted: {video_path}") + else: + logger.warning(f"⚠️ Failed to cross-post: {video_path} - {result.get('error', 'Unknown error')}") + + kwargs = { + "videos": final_video_paths, + "combined_videos": combined_video_paths, + "script": video_script, + "terms": video_terms, + "audio_file": audio_file, + "audio_duration": audio_duration, + "subtitle_path": subtitle_path, + "materials": downloaded_videos, + "cross_post_results": cross_post_results if cross_post_results else None, + } + sm.state.update_task( + task_id, state=const.TASK_STATE_COMPLETE, progress=100, **kwargs + ) + return kwargs + + +if __name__ == "__main__": + task_id = "task_id" + params = VideoParams( + video_subject="金钱的作用", + voice_name="zh-CN-XiaoyiNeural-Female", + voice_rate=1.0, + ) + start(task_id, params, stop_at="video") diff --git a/app/services/twelvelabs.py b/app/services/twelvelabs.py new file mode 100644 index 0000000..6d4844d --- /dev/null +++ b/app/services/twelvelabs.py @@ -0,0 +1,166 @@ +""" +TwelveLabs (https://twelvelabs.io) integration — optional, opt-in helpers. + +This module wraps two TwelveLabs models so MoneyPrinterTurbo can make better +use of the stock/B-roll footage it downloads: + + * Marengo (multimodal embeddings, 512-dim) — used to *semantically reorder* + the LLM-generated search terms against the video subject, so that when the + timeline budget runs out the most on-topic footage is the footage that made + it in (instead of whatever the LLM happened to list first). + + * Pegasus (video understanding) — used to QA / describe a generated clip from + a public URL, e.g. to sanity-check that a downloaded clip actually matches + the script before it ships. + +The integration is fully opt-in and non-breaking: + * If `twelvelabs_api_keys` is not configured, every public function here is a + no-op that returns its input unchanged (or None), so default behavior is + identical to a build without TwelveLabs. + * The `twelvelabs` SDK is imported lazily, so the dependency is only required + when the feature is actually used. + +Config (config.toml, [app] section): + twelvelabs_api_keys = ["tlk_xxx"] # required to enable + twelvelabs_rerank_terms = true # opt-in: reorder search terms by relevance + twelvelabs_marengo_model = "marengo3.0" # optional override + twelvelabs_pegasus_model = "pegasus1.5" # optional override + +Configure a TwelveLabs API key from the TwelveLabs dashboard (https://twelvelabs.io) to enable this optional integration. +""" + +import math +from functools import lru_cache +from typing import List, Optional + +from loguru import logger + +from app.config import config +from app.services import material + +DEFAULT_MARENGO_MODEL = "marengo3.0" +DEFAULT_PEGASUS_MODEL = "pegasus1.5" +# Pegasus requires max_tokens in [512, 98304]; 512 is plenty for a one-line QA. +_PEGASUS_MIN_MAX_TOKENS = 512 + + +def is_enabled() -> bool: + """True only when at least one TwelveLabs API key is configured.""" + keys = config.app.get("twelvelabs_api_keys") + return bool(keys) + + +def _client(): + # Lazy import + rotated key reuse mirrors the other providers in + # material.py (get_api_key rotates across configured keys). + from twelvelabs import TwelveLabs + + api_key = material.get_api_key("twelvelabs_api_keys") + return TwelveLabs(api_key=api_key) + + +def _cosine(a: List[float], b: List[float]) -> float: + dot = sum(x * y for x, y in zip(a, b)) + na = math.sqrt(sum(x * x for x in a)) + nb = math.sqrt(sum(x * x for x in b)) + if na == 0 or nb == 0: + return 0.0 + return dot / (na * nb) + + +def embed_text(text: str, model: Optional[str] = None) -> Optional[List[float]]: + """ + Return a 512-dim Marengo text embedding, or None on failure / when disabled. + + Cached so repeated terms across a session don't re-hit the API. + """ + if not is_enabled() or not text or not text.strip(): + return None + model = model or config.app.get("twelvelabs_marengo_model", DEFAULT_MARENGO_MODEL) + try: + # lru_cache only memoizes successful returns; a raised exception is not + # cached, so a transient API error never poisons the cache. + return _embed_text_cached(text.strip(), model) + except Exception as e: # noqa: BLE001 - never break the pipeline on TL errors + logger.warning(f"TwelveLabs embed_text failed, skipping rerank: {e}") + return None + + +@lru_cache(maxsize=512) +def _embed_text_cached(text: str, model: str) -> List[float]: + client = _client() + resp = client.embed.create(model_name=model, text=text) + # SDK aliases the raw JSON 'float' vector key to `float_`. + return list(resp.text_embedding.segments[0].float_) + + +def rerank_terms_by_subject( + video_subject: str, + search_terms: List[str], + model: Optional[str] = None, +) -> List[str]: + """ + Reorder `search_terms` so the terms most semantically relevant to + `video_subject` come first (Marengo cosine similarity). + + Opt-in: only runs when TwelveLabs is enabled AND + `twelvelabs_rerank_terms` is truthy. Falls back to the original order on + any failure, so it can never make the pipeline worse. + """ + if not is_enabled() or not config.app.get("twelvelabs_rerank_terms"): + return search_terms + if not video_subject or len(search_terms) < 2: + return search_terms + + subject_vec = embed_text(video_subject, model) + if subject_vec is None: + return search_terms + + scored = [] + for term in search_terms: + vec = embed_text(term, model) + if vec is None: + # If any term can't be embedded, don't risk a partial reorder. + return search_terms + scored.append((term, _cosine(subject_vec, vec))) + + ranked = [term for term, _ in sorted(scored, key=lambda x: x[1], reverse=True)] + logger.info( + f"TwelveLabs Marengo reranked {len(ranked)} search terms by relevance " + f"to subject '{video_subject}': {ranked}" + ) + return ranked + + +def analyze_clip( + video_url: str, + prompt: str = "Describe what happens in this video in one sentence.", + model: Optional[str] = None, + max_tokens: int = _PEGASUS_MIN_MAX_TOKENS, +) -> Optional[str]: + """ + QA / describe a clip from a public URL with Pegasus, returning the model's + text answer (or None when disabled / on failure). + + Notes (TwelveLabs API constraints): + * Pegasus needs a publicly reachable URL (or an uploaded asset), not a + bare local path; the analyzed window must be >= 4s. + * max_tokens must be >= 512 for this model. + """ + if not is_enabled() or not video_url: + return None + model = model or config.app.get("twelvelabs_pegasus_model", DEFAULT_PEGASUS_MODEL) + try: + from twelvelabs.types import VideoContext_Url + + client = _client() + resp = client.analyze( + model_name=model, + video=VideoContext_Url(url=video_url), + prompt=prompt, + max_tokens=max(max_tokens, _PEGASUS_MIN_MAX_TOKENS), + ) + return resp.data + except Exception as e: # noqa: BLE001 + logger.warning(f"TwelveLabs analyze_clip failed: {e}") + return None diff --git a/app/services/upload_post.py b/app/services/upload_post.py new file mode 100644 index 0000000..bb58ae8 --- /dev/null +++ b/app/services/upload_post.py @@ -0,0 +1,136 @@ +""" +Upload-Post API integration for cross-posting videos to TikTok, Instagram and YouTube Shorts. + +Docs: https://docs.upload-post.com +""" +import os +from typing import Optional + +import requests +from loguru import logger +from app.config import config + + +class UploadPostService: + API_BASE = "https://api.upload-post.com" + + def __init__(self): + self.api_key = config.app.get("upload_post_api_key", "") + self.username = config.app.get("upload_post_username", "") + self.enabled = config.app.get("upload_post_enabled", False) + self.platforms = config.app.get("upload_post_platforms", ["tiktok", "instagram"]) + self.auto_upload = config.app.get("upload_post_auto_upload", False) + self.youtube_privacy_status = config.app.get("upload_post_youtube_privacy_status", "public") + + def is_configured(self) -> bool: + return bool(self.api_key and self.username and self.enabled) + + def upload_video( + self, + video_path: str, + title: str, + platforms: Optional[list] = None, + privacy_level: str = "PUBLIC_TO_EVERYONE", + youtube_extra: Optional[dict] = None, + ) -> dict: + if not self.is_configured(): + logger.warning("Upload-Post is not configured. Skipping cross-post.") + return {"success": False, "error": "Upload-Post not configured"} + + if platforms is None: + platforms = self.platforms + + if not os.path.exists(video_path): + logger.error(f"Video file not found: {video_path}") + return {"success": False, "error": f"Video file not found: {video_path}"} + + logger.info(f"Cross-posting video to {', '.join(platforms)} via Upload-Post...") + + try: + with open(video_path, 'rb') as video_file: + files = {'video': video_file} + + data = [ + ('user', self.username), + ('title', title[:2200]), + ('privacy_level', privacy_level), + ] + + for platform in platforms: + data.append(('platform[]', platform)) + + if youtube_extra and any(p.startswith("youtube") for p in platforms): + if "youtube_title" in youtube_extra: + data.append(('youtube_title', youtube_extra["youtube_title"][:100])) + if "youtube_description" in youtube_extra: + data.append(('youtube_description', youtube_extra["youtube_description"])) + for tag in youtube_extra.get("tags", []): + data.append(('tags[]', tag)) + data.append(('privacyStatus', youtube_extra.get("privacyStatus", "public"))) + data.append(('containsSyntheticMedia', "true")) + + headers = {'Authorization': f'Apikey {self.api_key}'} + + response = requests.post( + f"{self.API_BASE}/api/upload", + headers=headers, + data=data, + files=files, + timeout=300, + ) + + response.raise_for_status() + result = response.json() + + if result.get('success'): + logger.info(f"✅ Video cross-posted successfully! Request ID: {result.get('request_id')}") + else: + logger.warning(f"Cross-post failed: {result.get('message', 'Unknown error')}") + + return result + + except requests.exceptions.RequestException as e: + logger.error(f"Failed to cross-post video: {str(e)}") + return {"success": False, "error": str(e)} + + def check_status(self, request_id: str) -> dict: + """ + Check the status of an upload request. + + Args: + request_id (str): The request ID from upload + + Returns: + dict: Status information + """ + try: + headers = { + 'Authorization': f'Apikey {self.api_key}' + } + + response = requests.get( + f"{self.API_BASE}/api/uploadposts/status", + params={'request_id': request_id}, + headers=headers, + timeout=30 + ) + + response.raise_for_status() + return response.json() + + except requests.exceptions.RequestException as e: + logger.error(f"Failed to check status: {str(e)}") + return {"success": False, "error": str(e)} + + +# Singleton instance +upload_post_service = UploadPostService() + + +def cross_post_video( + video_path: str, + title: str, + platforms: Optional[list] = None, + youtube_extra: Optional[dict] = None, +) -> dict: + return upload_post_service.upload_video(video_path, title, platforms, youtube_extra=youtube_extra) diff --git a/app/services/utils/video_effects.py b/app/services/utils/video_effects.py new file mode 100644 index 0000000..72b57d7 --- /dev/null +++ b/app/services/utils/video_effects.py @@ -0,0 +1,145 @@ +import numpy as np +from moviepy import Clip, ColorClip, CompositeVideoClip, vfx +from PIL import Image + + +# FadeIn +def fadein_transition(clip: Clip, t: float) -> Clip: + return clip.with_effects([vfx.FadeIn(t)]) + + +# FadeOut +def fadeout_transition(clip: Clip, t: float) -> Clip: + return clip.with_effects([vfx.FadeOut(t)]) + + +# SlideIn +def slidein_transition(clip: Clip, t: float, side: str) -> Clip: + width, height = clip.size + + # MoviePy 内置 SlideIn 在当前这条处理链里对全屏素材不稳定， + # 会出现“逻辑上应用了转场，但画面几乎看不出变化”的情况。 + # 这里改成显式黑底 + 位移动画，保证转场效果可见且行为可控。 + def position(current_time: float): + progress = min(max(current_time / max(t, 0.001), 0), 1) + + if side == "left": + return (-width + width * progress, 0) + if side == "right": + return (width - width * progress, 0) + if side == "top": + return (0, -height + height * progress) + if side == "bottom": + return (0, height - height * progress) + return (0, 0) + + background = ColorClip(size=(width, height), color=(0, 0, 0)).with_duration( + clip.duration + ) + moving_clip = clip.with_position(position) + return CompositeVideoClip([background, moving_clip], size=(width, height)).with_duration( + clip.duration + ) + + +# SlideOut +def slideout_transition(clip: Clip, t: float, side: str) -> Clip: + width, height = clip.size + transition_start = max(clip.duration - t, 0) + + # SlideOut 同样改成显式位移，保证片段末尾能稳定滑出画面。 + def position(current_time: float): + if current_time <= transition_start: + return (0, 0) + + progress = min( + max((current_time - transition_start) / max(t, 0.001), 0), 1 + ) + + if side == "left": + return (-width * progress, 0) + if side == "right": + return (width * progress, 0) + if side == "top": + return (0, -height * progress) + if side == "bottom": + return (0, height * progress) + return (0, 0) + + background = ColorClip(size=(width, height), color=(0, 0, 0)).with_duration( + clip.duration + ) + moving_clip = clip.with_position(position) + return CompositeVideoClip([background, moving_clip], size=(width, height)).with_duration( + clip.duration + ) + + +# 保留原始设计的 20% 缩放幅度，让三秒左右的短片也有清晰可见的 Ken Burns 运动感。 +# 缩放稳定性由下方的亚像素中心采样保证，不通过削弱效果幅度来掩盖源视频编码闪烁。 +_ZOOM_MAX_SCALE = 1.2 + + +def _zoom_frame(frame: np.ndarray, scale_factor: float) -> np.ndarray: + """使用亚像素中心裁剪实现无黑边且稳定的缩放效果。 + + 不能先把裁剪宽高转换为整数：缩放比例连续变化时，整数边界会按不同步长跳动， + 并在奇偶尺寸切换时改变半像素采样相位，最终表现为画面抖动。Pillow 的 EXTENT + 变换可以直接接收浮点边界，在固定输出画布上完成亚像素采样；左右、上下边界 + 始终围绕同一个浮点中心对称，因此适用于整段视频持续缓慢缩放的场景。 + """ + if scale_factor <= 0: + raise ValueError("scale_factor must be greater than zero") + + # 1 倍缩放直接返回原帧，避免无意义的重采样造成首帧轻微模糊。 + if abs(scale_factor - 1.0) < 1e-9: + return frame + + height, width = frame.shape[:2] + crop_width = width / scale_factor + crop_height = height / scale_factor + left = (width - crop_width) / 2 + top = (height - crop_height) / 2 + right = left + crop_width + bottom = top + crop_height + + image = Image.fromarray(frame) + transformed = image.transform( + (width, height), + Image.Transform.EXTENT, + (left, top, right, bottom), + # 视频连续缩放更关注相邻帧的一致性。BICUBIC/LANCZOS 虽然单帧更锐利， + # 但高频纹理跨越采样网格时容易出现振铃和亮度闪烁；BILINEAR 更柔和， + # 能以少量锐度损失换取更稳定的动态观感。 + resample=Image.Resampling.BILINEAR, + ) + return np.asarray(transformed) + + +def zoomin_transition(clip: Clip, t: float) -> Clip: + """在整个片段内从原始画面平滑放大到 1.2 倍。""" + # t 暂时保留，用于与其它转场函数保持统一调用签名；缩放需要覆盖完整片段， + # 否则短暂缩放结束后画面会突然静止，不适合静态或低运动量素材。 + _ = t + duration = max(clip.duration, 0.001) + + def scale_effect(get_frame, current_time: float): + progress = min(max(current_time / duration, 0), 1) + scale_factor = 1 + (_ZOOM_MAX_SCALE - 1) * progress + return _zoom_frame(get_frame(current_time), scale_factor) + + return clip.transform(scale_effect) + + +def zoomout_transition(clip: Clip, t: float) -> Clip: + """在整个片段内从 1.2 倍平滑缩小到原始画面。""" + # 与 zoomin_transition 一致，t 仅用于兼容统一的转场调用接口。 + _ = t + duration = max(clip.duration, 0.001) + + def scale_effect(get_frame, current_time: float): + progress = min(max(current_time / duration, 0), 1) + scale_factor = _ZOOM_MAX_SCALE - (_ZOOM_MAX_SCALE - 1) * progress + return _zoom_frame(get_frame(current_time), scale_factor) + + return clip.transform(scale_effect) diff --git a/app/services/video.py b/app/services/video.py new file mode 100644 index 0000000..cf812d6 --- /dev/null +++ b/app/services/video.py @@ -0,0 +1,1322 @@ +import glob +import itertools +import io +import os +import random +import gc +import subprocess +import sys +import tempfile +import unicodedata +from contextlib import redirect_stdout +from functools import lru_cache +from typing import List +from loguru import logger +import numpy as np +from moviepy import ( + AudioFileClip, + ColorClip, + CompositeAudioClip, + CompositeVideoClip, + ImageClip, + TextClip, + VideoFileClip, + afx, +) +from moviepy.video.tools.subtitles import SubtitlesClip +from PIL import Image, ImageDraw, ImageFont + +from app.config import config +from app.models import const +from app.models.schema import ( + MaterialInfo, + VideoAspect, + VideoConcatMode, + VideoParams, + VideoTransitionMode, +) +from app.services.utils import video_effects +from app.utils import file_security, utils + +class SubClippedVideoClip: + def __init__( + self, + file_path, + start_time=None, + end_time=None, + width=None, + height=None, + duration=None, + source_file_path=None, + ): + self.file_path = file_path + self.start_time = start_time + self.end_time = end_time + self.width = width + self.height = height + self.source_file_path = source_file_path or file_path + if duration is None: + self.duration = end_time - start_time + else: + self.duration = duration + + def __str__(self): + return f"SubClippedVideoClip(file_path={self.file_path}, start_time={self.start_time}, end_time={self.end_time}, duration={self.duration}, width={self.width}, height={self.height})" + + +audio_codec = "aac" +# Docker 里的 ffmpeg/AAC 组合在默认配置下更容易出现音频质量波动， +# 这里显式抬高音频码率，避免成片阶段因为默认值过低而引入明显失真。 +audio_bitrate = "192k" +fps = 30 +# FFmpeg 按帧率拼接/转码时，最终时长可能比 MoviePy 读到的理论时长短几十毫秒。 +# 这里给视频素材多留一个很小的安全余量，避免音频末尾因为帧舍入出现黑屏、 +# 卡顿或最后一小段旁白没有画面的情况。 +_VIDEO_DURATION_SAFETY_MARGIN = 0.1 +_MIN_MATERIAL_DIMENSION = 480 +# 消息类应用和部分编码器会把画面尺寸向下取整，例如 WhatsApp 会把 9:16 的 +# 素材压成 478x850，比 480 少两个像素。直接按 480 硬卡会让这类素材全部被 +# 丢弃，最终以 "no valid materials found" 整体失败。这里留一个很小的容差， +# 既能放行仅仅因为取整而略低于阈值的素材，也仍然能挡住真正的低清素材。 +_MIN_DIMENSION_TOLERANCE = 10 +_BGM_EXTENSIONS = (".mp3",) +_DEFAULT_VIDEO_CODEC = "libx264" +_SUPPORTED_VIDEO_CODECS = ( + "libx264", + "h264_nvenc", + "h264_amf", + "h264_qsv", + "h264_mf", + "h264_videotoolbox", +) +_runtime_disabled_video_codecs = set() + + +def _get_required_video_duration(audio_duration: float) -> float: + """ + 返回视频素材拼接的目标时长。 + + 使用场景：合成视频时需要素材时长覆盖旁白音频。只做到“刚好等于” + 音频时长时，FFmpeg 可能因为帧率舍入让最终视频略短，因此统一加一个 + 轻量余量。函数独立出来，便于测试和后续按实际反馈调整余量大小。 + """ + return max(0.0, float(audio_duration) + _VIDEO_DURATION_SAFETY_MARGIN) + + +def is_material_resolution_acceptable(width: int, height: int) -> bool: + """ + 判断素材分辨率是否足够用于合成。 + + 标称最小值是 480x480，但允许比它低 `_MIN_DIMENSION_TOLERANCE` 个像素， + 以兼容编码器/消息应用向下取整导致的尺寸（例如 WhatsApp 的 478x850）。 + """ + min_dimension = _MIN_MATERIAL_DIMENSION - _MIN_DIMENSION_TOLERANCE + return width >= min_dimension and height >= min_dimension + + +def _prioritize_unique_source_clips( + subclipped_items: List[SubClippedVideoClip], + concat_mode: VideoConcatMode, +) -> List[SubClippedVideoClip]: + """ + 优先让每个源素材只出现一次，降低成片里同一素材反复出现的概率。 + + 线上素材经常会遇到“一个长视频被切成多个短片段”的情况。旧逻辑在 + random 模式下直接打乱所有短片段，导致同一个源视频的多个切片可能 + 分布在开头和中间，用户会感知为素材重复。本函数只调整片段顺序： + 先放每个源文件里最长的一个片段，剩余片段作为兜底；当素材总时长不足时， + 仍然允许后续片段补齐音频长度，避免破坏视频生成成功率。优先选择最长 + 片段是为了避免随机选中视频尾部的零碎短片段，导致明明有足够素材却过早复用。 + """ + if not subclipped_items: + return [] + + concat_mode_value = getattr(concat_mode, "value", concat_mode) + if concat_mode_value != VideoConcatMode.random.value: + return subclipped_items + + grouped_items: dict[str, list[SubClippedVideoClip]] = {} + for item in subclipped_items: + grouped_items.setdefault(item.source_file_path, []).append(item) + + primary_items = [] + overflow_items = [] + for items in grouped_items.values(): + primary_item = max(items, key=lambda item: item.duration) + primary_items.append(primary_item) + overflow_items.extend(item for item in items if item is not primary_item) + + random.shuffle(primary_items) + random.shuffle(overflow_items) + logger.info( + "prioritized unique video materials, " + f"sources: {len(grouped_items)}, " + f"primary clips: {len(primary_items)}, " + f"fallback clips: {len(overflow_items)}" + ) + return primary_items + overflow_items + + +def get_ffmpeg_binary(): + """ + 兼容历史上直接从 video 服务读取 FFmpeg 路径的调用方。 + + 真正的解析逻辑已经抽到 `app.utils.utils.get_ffmpeg_binary()`，视频、语音 + 和后续新增链路都应复用同一套优先级；这里保留薄包装，避免外部脚本或 + 旧测试直接导入 `app.services.video.get_ffmpeg_binary` 时出现 AttributeError。 + """ + return utils.get_ffmpeg_binary() + + +def _get_configured_video_codec() -> str: + """ + 读取用户配置的视频编码器。 + + 该配置面向高级用户，用于尝试启用 NVENC/AMF/QSV/VideoToolbox 等硬件 + 编码。这里刻意只允许固定白名单，避免开放任意 FFmpeg 参数后，用户填错 + 参数导致输出格式不可控，甚至让生成任务在后续阶段才失败。 + """ + configured_codec = str( + config.app.get("video_codec", _DEFAULT_VIDEO_CODEC) or _DEFAULT_VIDEO_CODEC + ).strip() + if configured_codec not in _SUPPORTED_VIDEO_CODECS: + logger.warning( + f"unsupported video codec configured: {configured_codec}, " + f"fallback to {_DEFAULT_VIDEO_CODEC}" + ) + return _DEFAULT_VIDEO_CODEC + return configured_codec + + +@lru_cache(maxsize=16) +def _ffmpeg_encoder_exists(ffmpeg_binary: str, codec: str) -> bool: + """ + 检查当前 FFmpeg 是否声明支持指定编码器。 + + 这只能证明 FFmpeg 编译时包含该 encoder，不能证明当前机器硬件和驱动 + 一定可用。因此实际编码失败时仍会再回退到 libx264。 + """ + try: + result = subprocess.run( + [ffmpeg_binary, "-hide_banner", "-encoders"], + capture_output=True, + text=True, + check=False, + timeout=10, + ) + except (OSError, subprocess.TimeoutExpired) as exc: + logger.warning( + "failed to inspect ffmpeg encoders, " + f"fallback to {_DEFAULT_VIDEO_CODEC}: {str(exc)}" + ) + return False + + if result.returncode != 0: + logger.warning( + "failed to inspect ffmpeg encoders, " + f"fallback to {_DEFAULT_VIDEO_CODEC}: {(result.stderr or result.stdout or '').strip()}" + ) + return False + return codec in result.stdout + + +def _get_effective_video_codec(preferred_codec: str | None = None) -> str: + """ + 返回本次实际使用的视频编码器。 + + 用户选择硬件编码器时，先做 FFmpeg encoder 列表检测；如果本进程里已经 + 实际编码失败过，也直接回退，避免一个任务里每个片段都重复失败。 + """ + selected_codec = preferred_codec or _get_configured_video_codec() + if selected_codec == _DEFAULT_VIDEO_CODEC: + return _DEFAULT_VIDEO_CODEC + + if selected_codec in _runtime_disabled_video_codecs: + logger.warning( + f"video codec {selected_codec} was disabled after a runtime failure, " + f"fallback to {_DEFAULT_VIDEO_CODEC}" + ) + return _DEFAULT_VIDEO_CODEC + + ffmpeg_binary = utils.get_ffmpeg_binary() + if not _ffmpeg_encoder_exists(ffmpeg_binary, selected_codec): + logger.warning( + f"ffmpeg encoder {selected_codec} is not available, " + f"fallback to {_DEFAULT_VIDEO_CODEC}" + ) + return _DEFAULT_VIDEO_CODEC + + return selected_codec + + +def _disable_runtime_video_codec(codec: str, reason: str): + if codec == _DEFAULT_VIDEO_CODEC: + return + _runtime_disabled_video_codecs.add(codec) + logger.warning( + f"video codec {codec} failed, fallback to {_DEFAULT_VIDEO_CODEC}. " + f"reason: {reason}" + ) + + +def _get_temp_audio_dir(output_dir: str) -> str: + """ + Return the directory to use for MoviePy's temporary audio file. + + On Windows, Windows Defender can lock files written to the task output + directory while scanning them, causing MoviePy to fail with a + PermissionError (WinError 32) on the TEMP_MPY_wvf_snd temp file and + leaving the final MP4 at 0 bytes. Using the system temp directory + sidesteps the scan without changing behaviour on other platforms. + + On Linux/macOS/Docker the output directory is returned unchanged so + existing behaviour is preserved. + """ + if sys.platform == "win32": + return tempfile.gettempdir() + return output_dir + + +def _fallback_write_videofile(clip, output_file: str, failed_codec: str, reason: str, **kwargs): + """ + 硬件编码失败后用 libx264 重试，只有重试成功才禁用该硬件编码器。 + + Windows 上 FFmpeg 失败原因比较复杂：可能是显卡/驱动不支持，也可能是输出 + 文件被占用、目录权限、杀软拦截等通用 IO 问题。只有 libx264 能成功写出时， + 才能判断原始失败大概率来自硬件编码器本身，避免误伤后续任务。 + """ + clip.write_videofile(output_file, codec=_DEFAULT_VIDEO_CODEC, **kwargs) + _disable_runtime_video_codec(failed_codec, reason) + return _DEFAULT_VIDEO_CODEC + + +def _write_videofile_with_codec_fallback(clip, output_file: str, codec: str, **kwargs): + """ + 使用指定编码器写出视频，失败时自动用 libx264 重试一次。 + + 硬件编码器是否可用不仅取决于 FFmpeg，还取决于显卡、驱动和当前运行环境。 + 生成任务不能因为高级编码器不可用而整体失败，所以这里把回退集中处理。 + """ + effective_codec = _get_effective_video_codec(codec) + try: + clip.write_videofile(output_file, codec=effective_codec, **kwargs) + return effective_codec + except Exception as exc: + if effective_codec == _DEFAULT_VIDEO_CODEC: + raise + return _fallback_write_videofile( + clip, + output_file, + failed_codec=effective_codec, + reason=str(exc), + **kwargs, + ) + + +def _escape_ffmpeg_concat_path(file_path: str) -> str: + # concat demuxer 使用单引号包裹路径，路径中的单引号需要先转义。 + return file_path.replace("'", "'\\''") + + +def _format_ffmpeg_concat_path(file_path: str) -> str: + """ + 生成 concat demuxer 文件列表中的路径。 + + FFmpeg 官方文档要求 concat list 中的特殊字符和空格需要转义；Windows + 绝对路径里的反斜杠也容易被解析成转义字符。这里统一转成正斜杠形式， + 让 `C:\\Users\\...` 变成 `C:/Users/...`，再处理单引号，兼容 macOS/Linux。 + """ + absolute_path = os.path.abspath(file_path) + return _escape_ffmpeg_concat_path(absolute_path.replace("\\", "/")) + + +def concat_video_clips_with_ffmpeg( + clip_files: List[str], + output_file: str, + threads: int, + output_dir: str, + max_duration: float | None = None, +): + concat_list_file = os.path.join(output_dir, "ffmpeg-concat-list.txt") + with open(concat_list_file, "w", encoding="utf-8") as fp: + for clip_file in clip_files: + fp.write(f"file '{_format_ffmpeg_concat_path(clip_file)}'\n") + + def build_command(codec: str) -> list[str]: + command = [ + utils.get_ffmpeg_binary(), + "-y", + "-f", + "concat", + "-safe", + "0", + "-i", + concat_list_file, + "-c:v", + codec, + "-threads", + str(threads or 2), + "-pix_fmt", + "yuv420p", + ] + if max_duration is not None and max_duration > 0: + command.extend(["-t", f"{max_duration:.3f}"]) + command.append(output_file) + return command + + def run_concat(codec: str): + command = build_command(codec) + # 使用 ffmpeg 只做一次串联与编码，避免 MoviePy 逐段合并时反复重编码， + # 从而降低画质劣化与颜色偏移风险。 + result = subprocess.run( + command, + capture_output=True, + text=True, + check=False, + ) + if result.returncode != 0: + error_message = (result.stderr or result.stdout or "").strip() + raise RuntimeError(error_message or "ffmpeg concat failed") + return codec + + try: + effective_codec = _get_effective_video_codec() + try: + return run_concat(effective_codec) + except Exception as exc: + if effective_codec == _DEFAULT_VIDEO_CODEC: + raise + result_codec = run_concat(_DEFAULT_VIDEO_CODEC) + _disable_runtime_video_codec(effective_codec, str(exc)) + return result_codec + finally: + delete_files(concat_list_file) + + +def _sanitize_image_file(image_path: str) -> str: + # 某些本地图片虽然能被 Pillow 打开，但会因为损坏的 EXIF/eXIf 元数据导致 + # ImageClip 在解析阶段直接抛异常。这里重新导出一份“干净图片”，把坏元数据剥离掉。 + image_root, _ = os.path.splitext(image_path) + sanitized_path = f"{image_root}.sanitized.png" + + with Image.open(image_path) as image: + image.load() + # 统一导出为 PNG，避免 JPEG/PNG 不同元数据路径继续把坏块带过去。 + cleaned_image = Image.new(image.mode, image.size) + cleaned_image.putdata(list(image.getdata())) + cleaned_image.save(sanitized_path) + + return sanitized_path + + +def _open_image_clip_with_fallback(image_path: str): + # 优先直接打开原始图片；如果因为损坏元数据失败，再尝试生成无元数据副本。 + try: + return ImageClip(image_path), image_path + except Exception as exc: + logger.warning( + f"failed to open image directly, trying sanitized copy: {image_path}, error: {str(exc)}" + ) + sanitized_path = _sanitize_image_file(image_path) + return ImageClip(sanitized_path), sanitized_path + + +def _open_video_clip_quietly(video_path: str, audio: bool = False) -> VideoFileClip: + """ + 安静地打开视频文件，避免 MoviePy 2.1.x 把 ffmpeg 探测信息直接打印到 stdout。 + + 背景： + 当前依赖版本的 `FFMPEG_VideoReader` 内部存在 `print(self.infos)` 和 + `print(ffmpeg command)`，读取无音轨的中间视频时会输出 + `audio_found: False`。这只是输入素材 metadata，不代表最终成片没有音频， + 但会误导 WebUI/终端用户以为生成失败。 + + 实现： + 1. 只在打开 VideoFileClip 的短窗口内重定向 stdout； + 2. 默认 `audio=False`，因为项目视频素材阶段不需要保留素材原声， + 最终音频会在 `generate_video()` 阶段统一挂载； + 3. 如果依赖库确实输出了内容，降级为 debug 日志，便于必要时排查。 + """ + captured_stdout = io.StringIO() + with redirect_stdout(captured_stdout): + clip = VideoFileClip(video_path, audio=audio) + + moviepy_stdout = captured_stdout.getvalue().strip() + if moviepy_stdout: + logger.debug( + "suppressed MoviePy video reader stdout for " + f"{video_path}, chars: {len(moviepy_stdout)}" + ) + + return clip + + +def close_clip(clip): + if clip is None: + return + + try: + # close main resources + if hasattr(clip, 'reader') and clip.reader is not None: + clip.reader.close() + + # close audio resources + if hasattr(clip, 'audio') and clip.audio is not None: + if hasattr(clip.audio, 'reader') and clip.audio.reader is not None: + clip.audio.reader.close() + del clip.audio + + # close mask resources + if hasattr(clip, 'mask') and clip.mask is not None: + if hasattr(clip.mask, 'reader') and clip.mask.reader is not None: + clip.mask.reader.close() + del clip.mask + + # handle child clips in composite clips + if hasattr(clip, 'clips') and clip.clips: + for child_clip in clip.clips: + if child_clip is not clip: # avoid possible circular references + close_clip(child_clip) + + # clear clip list + if hasattr(clip, 'clips'): + clip.clips = [] + + except Exception as e: + logger.error(f"failed to close clip: {str(e)}") + + del clip + gc.collect() + +def delete_files(files: List[str] | str): + if isinstance(files, str): + files = [files] + + for file in files: + try: + os.remove(file) + except Exception as e: + logger.debug(f"failed to delete file {file}: {str(e)}") + + +def _resolve_bgm_file_path(song_dir: str, bgm_file: str) -> str: + # 背景音乐只允许读取 resource/songs 目录内的文件，避免用户输入任意路径后 + # 被 MoviePy 打开。这里兼容两种常见输入： + # 1. output000.mp3：来自 BGM 列表或用户只填写文件名 + # 2. ./resource/songs/output000.mp3：用户按项目目录结构填写的相对路径 + # 两种写法最终都会再次通过 resource/songs 白名单校验，不能绕过目录限制。 + try: + return file_security.resolve_path_within_directory(song_dir, bgm_file) + except ValueError as song_dir_exc: + if os.path.isabs(bgm_file): + raise song_dir_exc + + project_relative_file = os.path.join(utils.root_dir(), bgm_file) + try: + return file_security.resolve_path_within_directory( + song_dir, project_relative_file + ) + except ValueError as root_dir_exc: + raise ValueError(str(root_dir_exc)) from song_dir_exc + + +def get_bgm_file(bgm_type: str = "random", bgm_file: str = ""): + if not bgm_type: + return "" + + if bgm_file: + song_dir = utils.song_dir() + try: + resolved_bgm_file = _resolve_bgm_file_path(song_dir, bgm_file) + except ValueError as exc: + # API 请求里的 bgm_file 来自用户输入，不能直接把任意绝对路径交给 + # MoviePy 打开。这里强制限制到 resource/songs 目录，阻止读取 + # /etc/passwd、配置文件、密钥等非背景音乐文件。 + logger.warning( + f"reject unsafe bgm file: {bgm_file}, song_dir: {song_dir}, error: {str(exc)}" + ) + return "" + + if not resolved_bgm_file.lower().endswith(_BGM_EXTENSIONS): + logger.warning(f"reject unsupported bgm file extension: {resolved_bgm_file}") + return "" + + return resolved_bgm_file + + if bgm_type == "random": + suffix = "*.mp3" + song_dir = utils.song_dir() + files = glob.glob(os.path.join(song_dir, suffix)) + # 当背景音乐目录为空时，直接回退为“不使用 BGM”，避免 random.choice([]) 抛异常。 + if not files: + logger.warning(f"no bgm files found in song directory: {song_dir}") + return "" + return random.choice(files) + + return "" + + +def combine_videos( + combined_video_path: str, + video_paths: List[str], + audio_file: str, + video_aspect: VideoAspect = VideoAspect.portrait, + video_concat_mode: VideoConcatMode = VideoConcatMode.random, + video_transition_mode: VideoTransitionMode = None, + max_clip_duration: int = 5, + threads: int = 2, +) -> str: + audio_clip = AudioFileClip(audio_file) + try: + # 这里只需要读取旁白音频时长来决定素材视频拼接长度；后续不会再使用 + # audio_clip。读取完成后立即关闭，避免早退或异常路径泄漏文件句柄。 + audio_duration = audio_clip.duration + finally: + close_clip(audio_clip) + logger.info(f"audio duration: {audio_duration} seconds") + logger.info(f"maximum clip duration: {max_clip_duration} seconds") + required_video_duration = _get_required_video_duration(audio_duration) + logger.info( + f"required video duration: {required_video_duration:.2f} seconds " + f"(audio duration + {_VIDEO_DURATION_SAFETY_MARGIN:.2f}s safety margin)" + ) + + # 兼容 API 直接调用时未传转场模式的情况，避免后续访问 .value 时崩溃。 + transition_value = getattr(video_transition_mode, "value", video_transition_mode) + output_dir = os.path.dirname(combined_video_path) + + aspect = VideoAspect(video_aspect) + video_width, video_height = aspect.to_resolution() + + processed_clips = [] + subclipped_items = [] + video_duration = 0 + for video_path in video_paths: + clip = _open_video_clip_quietly(video_path) + clip_duration = clip.duration + clip_w, clip_h = clip.size + close_clip(clip) + + start_time = 0 + + while start_time < clip_duration: + end_time = min(start_time + max_clip_duration, clip_duration) + + # 保留所有有效分段。 + # 这样既不会丢掉“整段视频本身就短于 max_clip_duration”的素材， + # 也不会吞掉长视频最后剩下的一小段尾部内容。 + if end_time > start_time: + subclipped_items.append( + SubClippedVideoClip( + file_path=video_path, + start_time=start_time, + end_time=end_time, + width=clip_w, + height=clip_h, + source_file_path=video_path, + ) + ) + + start_time = end_time + if video_concat_mode.value == VideoConcatMode.sequential.value: + break + + subclipped_items = _prioritize_unique_source_clips( + subclipped_items=subclipped_items, + concat_mode=video_concat_mode, + ) + + logger.debug(f"total subclipped items: {len(subclipped_items)}") + + # Add downloaded clips over and over until the duration of the audio (max_duration) has been reached + for i, subclipped_item in enumerate(subclipped_items): + if video_duration >= required_video_duration: + break + + logger.debug( + f"processing clip {i+1}: {subclipped_item.width}x{subclipped_item.height}, " + f"source: {os.path.basename(subclipped_item.source_file_path)}, " + f"current duration: {video_duration:.2f}s, " + f"remaining: {required_video_duration - video_duration:.2f}s" + ) + + try: + clip = _open_video_clip_quietly(subclipped_item.file_path).subclipped( + subclipped_item.start_time, subclipped_item.end_time + ) + clip_duration = clip.duration + # Not all videos are same size, so we need to resize them + clip_w, clip_h = clip.size + if clip_w != video_width or clip_h != video_height: + clip_ratio = clip.w / clip.h + video_ratio = video_width / video_height + logger.debug(f"resizing clip, source: {clip_w}x{clip_h}, ratio: {clip_ratio:.2f}, target: {video_width}x{video_height}, ratio: {video_ratio:.2f}") + + if clip_ratio == video_ratio: + clip = clip.resized(new_size=(video_width, video_height)) + else: + if clip_ratio > video_ratio: + scale_factor = video_width / clip_w + else: + scale_factor = video_height / clip_h + + new_width = int(clip_w * scale_factor) + new_height = int(clip_h * scale_factor) + + background = ColorClip(size=(video_width, video_height), color=(0, 0, 0)).with_duration(clip_duration) + clip_resized = clip.resized(new_size=(new_width, new_height)).with_position("center") + clip = CompositeVideoClip([background, clip_resized]) + + shuffle_side = random.choice(["left", "right", "top", "bottom"]) + if transition_value in (None, VideoTransitionMode.none.value): + clip = clip + elif transition_value == VideoTransitionMode.fade_in.value: + clip = video_effects.fadein_transition(clip, 1) + elif transition_value == VideoTransitionMode.fade_out.value: + clip = video_effects.fadeout_transition(clip, 1) + elif transition_value == VideoTransitionMode.slide_in.value: + clip = video_effects.slidein_transition(clip, 1, shuffle_side) + elif transition_value == VideoTransitionMode.slide_out.value: + clip = video_effects.slideout_transition(clip, 1, shuffle_side) + elif transition_value == VideoTransitionMode.zoom_in.value: + clip = video_effects.zoomin_transition(clip, 1) + elif transition_value == VideoTransitionMode.zoom_out.value: + clip = video_effects.zoomout_transition(clip, 1) + elif transition_value == VideoTransitionMode.shuffle.value: + transition_funcs = [ + lambda c: video_effects.fadein_transition(c, 1), + lambda c: video_effects.fadeout_transition(c, 1), + lambda c: video_effects.slidein_transition(c, 1, shuffle_side), + lambda c: video_effects.slideout_transition(c, 1, shuffle_side), + lambda c: video_effects.zoomin_transition(c, 1), + lambda c: video_effects.zoomout_transition(c, 1), + ] + shuffle_transition = random.choice(transition_funcs) + clip = shuffle_transition(clip) + + if clip.duration > max_clip_duration: + clip = clip.subclipped(0, max_clip_duration) + + # wirte clip to temp file + clip_file = f"{output_dir}/temp-clip-{i+1}.mp4" + _write_videofile_with_codec_fallback( + clip, + clip_file, + codec=_get_configured_video_codec(), + logger=None, + fps=fps, + ) + + # Store clip duration before closing + clip_duration_saved = clip.duration + close_clip(clip) + + processed_clips.append( + SubClippedVideoClip( + file_path=clip_file, + duration=clip_duration_saved, + width=clip_w, + height=clip_h, + source_file_path=subclipped_item.source_file_path, + ) + ) + video_duration += clip_duration_saved + + except Exception as e: + logger.error(f"failed to process clip: {str(e)}") + + # loop processed clips until the video duration covers the audio duration and the small safety margin. + if video_duration < required_video_duration: + logger.warning( + f"video duration ({video_duration:.2f}s) is shorter than required duration " + f"({required_video_duration:.2f}s), looping clips to match audio length." + ) + base_clips = processed_clips.copy() + for clip in itertools.cycle(base_clips): + if video_duration >= required_video_duration: + break + processed_clips.append(clip) + video_duration += clip.duration + logger.info( + f"video duration: {video_duration:.2f}s, audio duration: {audio_duration:.2f}s, " + f"required duration: {required_video_duration:.2f}s, " + f"looped {len(processed_clips)-len(base_clips)} clips" + ) + + # merge video clips progressively, avoid loading all videos at once to avoid memory overflow + logger.info("starting clip merging process") + if not processed_clips: + logger.warning("no clips available for merging") + return combined_video_path + + clip_files = [clip.file_path for clip in processed_clips] + logger.info(f"concatenating {len(clip_files)} clips with ffmpeg") + concat_video_clips_with_ffmpeg( + clip_files=clip_files, + output_file=combined_video_path, + threads=threads, + output_dir=output_dir, + max_duration=audio_duration, + ) + + # clean temp files + delete_files(clip_files) + + logger.info("video combining completed") + return combined_video_path + + +def wrap_text(text, max_width, font="Arial", fontsize=60): + # 字幕换行必须在真正创建 TextClip 前完成，否则 MoviePy 只会按原始文本 + # 计算渲染区域。这里用 PIL 按当前字体和字号测量宽度，确保每一行都尽量 + # 控制在视频可用宽度内，避免大字号或中文长句直接溢出画面。 + font = ImageFont.truetype(font, fontsize) + max_width = int(max_width) + + def get_text_size(inner_text): + inner_text = inner_text.strip() + if not inner_text: + return 0, fontsize + left, top, right, bottom = font.getbbox(inner_text) + return right - left, bottom - top + + width, height = get_text_size(text) + if width <= max_width: + return text, height + + def split_long_token(token): + # 当一个 token 本身就超宽时（常见于中文无空格长句，或英文超长单词）， + # 退化为字符级拆分。关键点是：检测到 candidate 超宽时，先提交上一个 + # 仍然合法的 current，再把当前字符放入下一行，不能把超宽字符塞回上一行。 + lines = [] + current = "" + for char in token: + candidate = f"{current}{char}" + candidate_width, _ = get_text_size(candidate) + if candidate_width <= max_width or not current: + current = candidate + continue + lines.append(current) + current = char + if current: + lines.append(current) + return lines + + lines = [] + current = "" + words = text.split(" ") + for word in words: + candidate = f"{current} {word}".strip() if current else word + candidate_width, _ = get_text_size(candidate) + if candidate_width <= max_width: + current = candidate + continue + + if current: + lines.append(current) + + word_width, _ = get_text_size(word) + if word_width <= max_width: + current = word + else: + lines.extend(split_long_token(word)) + current = "" + + if current: + lines.append(current) + + line_start_punctuation = "，。！？；：、,.!?;:)]}）】》」』”’" + for index in range(1, len(lines)): + # 中文长句按字符拆分时，最后一个句号、逗号等闭合标点可能被单独 + # 放到下一行，导致字幕背景被异常撑高，视觉上像一个小点掉在正文 + # 下方。这里在不重新设计换行算法的前提下，把上一行最后一个字 + # 移到标点行前面，让标点跟随文字显示，兼容中英文常见闭合标点。 + if not lines[index] or lines[index][0] not in line_start_punctuation: + continue + if len(lines[index - 1]) <= 1: + continue + + candidate = f"{lines[index - 1][-1]}{lines[index]}" + candidate_width, _ = get_text_size(candidate) + if candidate_width <= max_width: + lines[index] = candidate + lines[index - 1] = lines[index - 1][:-1] + + result = "\n".join(line.strip() for line in lines if line.strip()).strip() + height = len(lines) * height + return result, height + + +def _hex_to_rgb(color: str) -> tuple[int, int, int]: + # 字幕背景色来自 API/WebUI 参数，可能为空或格式不规范。这里统一只接受 + # #RRGGBB 形式，非法值回退为黑色，避免 PIL 渲染阶段抛出异常中断任务。 + if isinstance(color, str) and color.startswith("#") and len(color) == 7: + try: + return (int(color[1:3], 16), int(color[3:5], 16), int(color[5:7], 16)) + except ValueError: + pass + return (0, 0, 0) + + +def _rounded_subtitle_background_clip( + width: int, + height: int, + color: str, + alpha: int = 140, + radius: int = 16, +) -> ImageClip: + # 新字幕背景仅在用户显式开启时使用：通过 RGBA 图片绘制圆角半透明底板， + # 再交给 MoviePy 作为透明 ImageClip 参与合成。这样默认路径完全不变， + # 同时可以低成本试验更柔和的字幕视觉效果。 + rgb = _hex_to_rgb(color) + safe_alpha = max(0, min(255, int(alpha))) + img = Image.new("RGBA", (width, height), (0, 0, 0, 0)) + draw = ImageDraw.Draw(img) + draw.rounded_rectangle( + [0, 0, max(0, width - 1), max(0, height - 1)], + radius=max(0, int(radius)), + fill=(rgb[0], rgb[1], rgb[2], safe_alpha), + ) + return ImageClip(np.array(img), transparent=True) + + +def _get_visible_center_position( + text_clip: TextClip, + container_width: int, + container_height: int, +) -> tuple[int, int]: + """ + 按文字真实可见像素把 TextClip 放到背景容器中心。 + + MoviePy 的 TextClip 会按字体行高和 baseline 创建透明画布。很多字体的 + 可见字形并不在这个画布的几何中心，直接 `with_position("center")` + 会把整块透明画布居中，导致字幕看起来偏上或偏下。这里读取 TextClip + 的透明 mask，只根据实际有像素的 bbox 计算偏移，让用户看到的文字 + 在字幕背景里视觉居中。 + """ + x = int(round((container_width - text_clip.w) / 2)) + y = int(round((container_height - text_clip.h) / 2)) + + try: + if text_clip.mask is None: + return x, y + + mask_frame = text_clip.mask.get_frame(0) + ys, _ = np.where(mask_frame > 0.01) + if len(ys) == 0: + return x, y + + visible_top = int(ys.min()) + visible_bottom = int(ys.max()) + visible_height = visible_bottom - visible_top + 1 + y = int(round((container_height - visible_height) / 2 - visible_top)) + except Exception as exc: + logger.debug(f"failed to center subtitle text by visible mask: {str(exc)}") + + return x, y + + +def subtitle_colors_are_indistinguishable(params: VideoParams) -> bool: + """判断字幕文字和背景是否同色，提醒用户可能无法看清字幕。""" + if not params.subtitle_enabled or not params.text_background_color: + return False + + def normalize_color(value): + if isinstance(value, bool): + return "#000000" if value else "" + return str(value or "").strip().lower() + + text_color = normalize_color(params.text_fore_color) + background_color = normalize_color(params.text_background_color) + return bool(text_color and text_color == background_color) + + +@lru_cache(maxsize=64) +def _subtitle_font_supports_sample(font_path: str, sample: str) -> bool: + """检查字体是否包含样本文字需要的字形，并缓存重复检查结果。""" + try: + font = ImageFont.truetype(font_path, 30) + missing_mask = font.getmask("\U0010ffff") + missing_signature = ( + missing_mask.size, + missing_mask.getbbox(), + bytes(missing_mask), + ) + for char in sample: + char_mask = font.getmask(char) + char_signature = ( + char_mask.size, + char_mask.getbbox(), + bytes(char_mask), + ) + if char_mask.getbbox() is None or char_signature == missing_signature: + return False + return True + except Exception as e: + # 字体探测失败不应阻止用户生成；保留日志供环境兼容问题排查。 + logger.warning(f"failed to inspect subtitle font glyphs: {font_path}, {e}") + return True + + +def subtitle_font_supports_text(font_path: str, text: str) -> bool: + """检查字体能否绘制文本中的字母和数字，忽略空白及标点符号。""" + sample = "".join( + dict.fromkeys( + char + for char in str(text or "") + if unicodedata.category(char)[0] in {"L", "N"} + ) + )[:64] + if not sample: + return True + return _subtitle_font_supports_sample(font_path, sample) + + +def generate_video( + video_path: str, + audio_path: str, + subtitle_path: str, + output_file: str, + params: VideoParams, +): + aspect = VideoAspect(params.video_aspect) + video_width, video_height = aspect.to_resolution() + + logger.info(f"generating video: {video_width} x {video_height}") + logger.info(f" ① video: {video_path}") + logger.info(f" ② audio: {audio_path}") + logger.info(f" ③ subtitle: {subtitle_path}") + logger.info(f" ④ output: {output_file}") + + # https://github.com/harry0703/MoneyPrinterTurbo/issues/217 + # PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'final-1.mp4.tempTEMP_MPY_wvf_snd.mp3' + # write into the same directory as the output file + output_dir = os.path.dirname(output_file) + + font_path = "" + if params.subtitle_enabled: + if not params.font_name: + params.font_name = "STHeitiMedium.ttc" + font_path = os.path.join(utils.font_dir(), params.font_name) + if os.name == "nt": + font_path = font_path.replace("\\", "/") + + logger.info(f" ⑤ font: {font_path}") + + def resolve_subtitle_background_color(): + # 兼容历史参数：API 里 `text_background_color` 既可能是布尔值， + # 也可能是实际颜色字符串。统一在这里归一化，避免把 True/False + # 直接传给 TextClip 后出现不可预期的渲染结果。 + if isinstance(params.text_background_color, bool): + return "#000000" if params.text_background_color else None + return params.text_background_color + + def create_text_clip(subtitle_item): + params.font_size = int(params.font_size) + params.stroke_width = int(params.stroke_width) + phrase = subtitle_item[1] + max_width = video_width * 0.9 + bg_color = resolve_subtitle_background_color() + rounded_bg_enabled = bool( + getattr(params, "rounded_subtitle_background", False) and bg_color + ) + has_subtitle_background = bool(bg_color) + # 圆角背景按文字真实宽度生成，左右留白应更克制；旧矩形背景仍保留 + # 较大的安全边距，避免历史配置中的长字幕贴边或被裁切。 + padding_ratio = 0.4 if rounded_bg_enabled else 0.6 + pad_x = int(params.font_size * padding_ratio) if has_subtitle_background else 0 + # 字幕背景需要给文字左右留出明确内边距。先从可用宽度中扣除 + # padding 再换行，避免长英文或大字号刚好撑满 90% 视频宽度后， + # 文字贴到背景框边缘，看起来像被裁切。普通矩形背景和圆角背景 + # 都走这条逻辑；无背景字幕则保持原有最大宽度。 + text_max_width = max(1, int(max_width) - 2 * pad_x) + wrapped_txt, txt_height = wrap_text( + phrase, + max_width=text_max_width, + font=font_path, + fontsize=params.font_size, + ) + interline = int(params.font_size * 0.25) + line_count = wrapped_txt.count("\n") + 1 + vertical_padding = int(params.font_size * 0.35) + text_clip_margin_y = max( + int(params.font_size * 0.3), int(params.stroke_width * 2) + ) + # MoviePy 在 `method=label` 下会自动收缩文本框高度，遇到多行字幕、 + # 描边或背景色时，容易把最后一行的下半部分裁掉。这里显式传入 + # 一个更保守的高度，把行间距和额外上下留白一并算进去，保证字幕 + # 背景框与文字本身都能完整渲染出来。 + clip_h = int(txt_height + vertical_padding + (interline * line_count)) + + if rounded_bg_enabled: + # 圆角背景需要贴合文字宽度，而不是沿用 90% 视频宽度。这里先用 + # PIL 测量最长一行文字，再加水平内边距，避免短字幕出现过宽底板。 + try: + font = ImageFont.truetype(font_path, params.font_size) + text_w = max( + int(font.getbbox(line)[2] - font.getbbox(line)[0]) + for line in wrapped_txt.split("\n") + ) + except Exception as exc: + logger.warning( + f"failed to measure subtitle text width, fallback to max width: {str(exc)}" + ) + text_w = int(max_width) + + box_w = max(1, min(int(max_width), text_w + 2 * pad_x)) + radius = max(8, int(params.font_size * 0.4)) + text_clip = TextClip( + text=wrapped_txt, + font=font_path, + font_size=params.font_size, + color=params.text_fore_color, + bg_color=None, + stroke_color=params.stroke_color, + stroke_width=params.stroke_width, + interline=interline, + size=(box_w, None), + text_align="center", + margin=(0, text_clip_margin_y), + ) + clip_h = max(clip_h, text_clip.h) + bg_clip = _rounded_subtitle_background_clip( + width=box_w, + height=clip_h, + color=bg_color, + alpha=140, + radius=radius, + ) + text_position = _get_visible_center_position(text_clip, box_w, clip_h) + _clip = CompositeVideoClip( + [bg_clip, text_clip.with_position(text_position)], + size=(box_w, clip_h), + ) + elif bg_color: + size = ( + int(max_width), + clip_h, + ) + text_clip = TextClip( + text=wrapped_txt, + font=font_path, + font_size=params.font_size, + color=params.text_fore_color, + bg_color=None, + stroke_color=params.stroke_color, + stroke_width=params.stroke_width, + interline=interline, + size=(int(max_width), None), + text_align="center", + margin=(0, text_clip_margin_y), + ) + size = (size[0], max(size[1], text_clip.h)) + bg_clip = _rounded_subtitle_background_clip( + width=size[0], + height=size[1], + color=bg_color, + alpha=255, + radius=0, + ) + text_position = _get_visible_center_position(text_clip, size[0], size[1]) + _clip = CompositeVideoClip( + [bg_clip, text_clip.with_position(text_position)], + size=size, + ) + else: + size = ( + int(max_width), + clip_h, + ) + _clip = TextClip( + text=wrapped_txt, + font=font_path, + font_size=params.font_size, + color=params.text_fore_color, + bg_color=None, + stroke_color=params.stroke_color, + stroke_width=params.stroke_width, + interline=interline, + size=size, + text_align="center", + ) + duration = subtitle_item[0][1] - subtitle_item[0][0] + _clip = _clip.with_start(subtitle_item[0][0]) + _clip = _clip.with_end(subtitle_item[0][1]) + _clip = _clip.with_duration(duration) + if params.subtitle_position == "bottom": + _clip = _clip.with_position(("center", video_height * 0.95 - _clip.h)) + elif params.subtitle_position == "top": + _clip = _clip.with_position(("center", video_height * 0.05)) + elif params.subtitle_position == "custom": + # Ensure the subtitle is fully within the screen bounds + margin = 10 # Additional margin, in pixels + max_y = video_height - _clip.h - margin + min_y = margin + custom_y = (video_height - _clip.h) * (params.custom_position / 100) + custom_y = max( + min_y, min(custom_y, max_y) + ) # Constrain the y value within the valid range + _clip = _clip.with_position(("center", custom_y)) + else: # center + _clip = _clip.with_position(("center", "center")) + return _clip + + video_clip = _open_video_clip_quietly(video_path) + audio_clip = AudioFileClip(audio_path).with_effects( + [afx.MultiplyVolume(params.voice_volume)] + ) + + def make_textclip(text): + return TextClip( + text=text, + font=font_path, + font_size=params.font_size, + ) + + if subtitle_path and os.path.exists(subtitle_path): + sub = SubtitlesClip( + subtitles=subtitle_path, encoding="utf-8", make_textclip=make_textclip + ) + text_clips = [] + for item in sub.subtitles: + clip = create_text_clip(subtitle_item=item) + text_clips.append(clip) + video_clip = CompositeVideoClip([video_clip, *text_clips]) + + bgm_file = get_bgm_file(bgm_type=params.bgm_type, bgm_file=params.bgm_file) + if bgm_file: + try: + bgm_clip = AudioFileClip(bgm_file).with_effects( + [ + afx.MultiplyVolume(params.bgm_volume), + afx.AudioFadeOut(3), + afx.AudioLoop(duration=video_clip.duration), + ] + ) + audio_clip = CompositeAudioClip([audio_clip, bgm_clip]) + except Exception as e: + logger.error(f"failed to add bgm: {str(e)}") + + video_clip = video_clip.with_audio(audio_clip) + # 显式沿用输入音频的采样率；如果取不到，再回退到 MoviePy 默认的 44100Hz。 + # 这样可以减少不同运行环境，尤其是 Docker 环境中再次重采样带来的音质波动。 + output_audio_fps = int(getattr(audio_clip, "fps", 0) or 44100) + _write_videofile_with_codec_fallback( + video_clip, + output_file=output_file, + codec=_get_configured_video_codec(), + audio_codec=audio_codec, + audio_fps=output_audio_fps, + audio_bitrate=audio_bitrate, + temp_audiofile_path=_get_temp_audio_dir(output_dir), + threads=params.n_threads or 2, + logger=None, + fps=fps, + ) + video_clip.close() + del video_clip + + +def preprocess_video(materials: List[MaterialInfo], clip_duration=4): + # WebUI 在某些二次生成场景下可能传入空素材列表，这里直接返回空结果，避免抛出 NoneType 异常。 + if not materials: + return [] + + # 仅返回通过预处理校验的素材，避免低分辨率图片继续进入后续的视频合成流程。 + valid_materials = [] + local_videos_dir = utils.storage_dir("local_videos", create=True) + + for material in materials: + if not material.url: + continue + + try: + material_source_path = file_security.resolve_path_within_directory( + local_videos_dir, material.url + ) + except ValueError as exc: + # local video_source 的素材路径来自 API 参数，必须限制在专用素材目录。 + # 允许用户传文件名，也兼容历史返回的绝对路径，但不允许逃逸到系统 + # 其他目录，避免任意文件读取或通过 MoviePy 探测本地敏感文件。 + logger.warning( + f"skip unsafe local material: {material.url}, " + f"local_videos_dir: {local_videos_dir}, error: {str(exc)}" + ) + continue + + ext = utils.parse_extension(material_source_path) + try: + # 图片素材直接按图片方式读取，避免先走 VideoFileClip 误判后触发不稳定的回退分支。 + if ext in const.FILE_TYPE_IMAGES: + clip, material_source_path = _open_image_clip_with_fallback( + material_source_path + ) + else: + clip = _open_video_clip_quietly(material_source_path) + except Exception: + # 非标准扩展名或探测失败时再回退到图片模式，兼容历史上直接传本地图片路径的情况。 + try: + clip, material_source_path = _open_image_clip_with_fallback( + material_source_path + ) + except Exception as exc: + logger.warning( + f"skip unreadable local material: {material.url}, error: {str(exc)}" + ) + continue + try: + width = clip.size[0] + height = clip.size[1] + if not is_material_resolution_acceptable(width, height): + logger.warning( + f"low resolution material: {width}x{height}, minimum " + f"{_MIN_MATERIAL_DIMENSION}x{_MIN_MATERIAL_DIMENSION} required " + f"(tolerance {_MIN_DIMENSION_TOLERANCE}px)" + ) + # 探测到低分辨率素材后立即关闭资源，并且不要把该素材返回给后续流程。 + close_clip(clip) + continue + + if ext in const.FILE_TYPE_IMAGES: + logger.info(f"processing image: {material_source_path}") + # 探测尺寸时已经打开过一次素材，这里先释放探测句柄，再重新创建用于导出的图片 clip。 + close_clip(clip) + # Create an image clip and set its duration to 3 seconds + clip = ( + ImageClip(material_source_path) + .with_duration(clip_duration) + .with_position("center") + ) + # Apply a zoom effect using the resize method. + # A lambda function is used to make the zoom effect dynamic over time. + # The zoom effect starts from the original size and gradually scales up to 120%. + # t represents the current time, and clip.duration is the total duration of the clip (3 seconds). + # Note: 1 represents 100% size, so 1.2 represents 120% size. + zoom_clip = clip.resized( + lambda t: 1 + (clip_duration * 0.03) * (t / clip.duration) + ) + + # Optionally, create a composite video clip containing the zoomed clip. + # This is useful when you want to add other elements to the video. + final_clip = CompositeVideoClip([zoom_clip]) + + # Output the video to a file. + video_file = f"{material_source_path}.mp4" + final_clip.write_videofile(video_file, fps=30, logger=None) + close_clip(clip) + close_clip(final_clip) + material.url = video_file + logger.success(f"image processed: {video_file}") + else: + # 普通视频素材只需要读取尺寸做校验，校验完成后立即释放句柄即可。 + close_clip(clip) + # Update url to the resolved absolute path so that downstream + # stages (combine_videos) can open the file without re-resolving. + material.url = material_source_path + except Exception: + close_clip(clip) + raise + + valid_materials.append(material) + + return valid_materials diff --git a/app/services/voice.py b/app/services/voice.py new file mode 100644 index 0000000..2c7d983 --- /dev/null +++ b/app/services/voice.py @@ -0,0 +1,1812 @@ +import asyncio +import base64 +import io +import inspect +import json +import math +import os +import queue +import re +import subprocess +import threading +import time +import unicodedata +from datetime import datetime +from typing import Union +from xml.sax.saxutils import escape, unescape + +import edge_tts +import requests +from edge_tts import SubMaker +from loguru import logger +from moviepy.video.tools import subtitles +from moviepy.audio.io.AudioFileClip import AudioFileClip +from openai import OpenAI + +from app.config import config +from app.utils import utils + +_DEFAULT_EDGE_TTS_TIMEOUT_SECONDS = 30.0 +_MIMO_DEFAULT_BASE_URL = "https://api.xiaomimimo.com/v1" +_MIMO_DEFAULT_TTS_MODEL = "mimo-v2.5-tts" +NO_VOICE_NAME = "no-voice" +# `none` 是 PR #981 里曾使用过的无配音标识。这里短期兼容这个值，避免 +# 已经手动调用过该分支的 API 用户升级后立即失效；WebUI 和新代码统一使用 +# 更明确的 `no-voice`。 +_NO_VOICE_ALIASES = {NO_VOICE_NAME, "none"} + + +def _configure_pydub_ffmpeg(audio_segment_cls): + configured_ffmpeg = utils.get_ffmpeg_binary() + if configured_ffmpeg: + audio_segment_cls.converter = configured_ffmpeg + + +def mktimestamp(time_unit: float) -> str: + """ + 将 edge_tts 使用的 100 纳秒时间单位转换为字幕时间戳。 + + edge_tts 7.x 不再导出旧版本里的 `mktimestamp`，但项目里旧字幕链路 + 还需要这个格式化函数来兼容 Azure v2、Gemini、SiliconFlow 这些 + 手工构造的字幕时间轴，因此这里内置一个等价实现。 + """ + hour = math.floor(time_unit / 10**7 / 3600) + minute = math.floor((time_unit / 10**7 / 60) % 60) + seconds = (time_unit / 10**7) % 60 + return f"{hour:02d}:{minute:02d}:{seconds:06.3f}" + + +def get_siliconflow_voices() -> list[str]: + """ + 获取硅基流动的声音列表 + + Returns: + 声音列表，格式为 ["siliconflow:FunAudioLLM/CosyVoice2-0.5B:alex", ...] + """ + # 硅基流动的声音列表和对应的性别（用于显示） + voices_with_gender = [ + ("FunAudioLLM/CosyVoice2-0.5B", "alex", "Male"), + ("FunAudioLLM/CosyVoice2-0.5B", "anna", "Female"), + ("FunAudioLLM/CosyVoice2-0.5B", "bella", "Female"), + ("FunAudioLLM/CosyVoice2-0.5B", "benjamin", "Male"), + ("FunAudioLLM/CosyVoice2-0.5B", "charles", "Male"), + ("FunAudioLLM/CosyVoice2-0.5B", "claire", "Female"), + ("FunAudioLLM/CosyVoice2-0.5B", "david", "Male"), + ("FunAudioLLM/CosyVoice2-0.5B", "diana", "Female"), + ] + + # 添加siliconflow:前缀，并格式化为显示名称 + return [ + f"siliconflow:{model}:{voice}-{gender}" + for model, voice, gender in voices_with_gender + ] + + +def get_gemini_voices() -> list[str]: + """ + 获取Gemini TTS的声音列表 + + Returns: + 声音列表，格式为 ["gemini:Zephyr-Female", "gemini:Puck-Male", ...] + """ + # Gemini TTS支持的语音列表 + voices_with_gender = [ + ("Zephyr", "Female"), + ("Puck", "Male"), + ("Charon", "Male"), + ("Kore", "Female"), + ("Fenrir", "Male"), + ("Aoede", "Female"), + ("Thalia", "Female"), + ("Sage", "Male"), + ("Echo", "Female"), + ("Harmony", "Female"), + ("Lux", "Female"), + ("Nova", "Female"), + ("Vale", "Male"), + ("Orion", "Male"), + ("Atlas", "Male"), + ] + + # 添加gemini:前缀，并格式化为显示名称 + return [ + f"gemini:{voice}-{gender}" + for voice, gender in voices_with_gender + ] + + +def get_mimo_voices() -> list[str]: + """ + 获取 Xiaomi MiMo V2.5 TTS 的预置音色列表。 + + 当前只接入官方文档里的 `mimo-v2.5-tts` 预置音色模式。音色设计 + `mimo-v2.5-tts-voicedesign` 和音色复刻 `mimo-v2.5-tts-voiceclone` + 需要额外的输入表单和素材上传流程，先不混入普通 TTS 下拉框，避免 + 用户误以为选择一个 voice id 就能完成所有高级能力。 + """ + voices_with_gender = [ + ("mimo_default", "Female"), + ("冰糖", "Female"), + ("茉莉", "Female"), + ("苏打", "Male"), + ("白桦", "Male"), + ("Mia", "Female"), + ("Chloe", "Female"), + ("Milo", "Male"), + ("Dean", "Male"), + ] + + return [f"mimo:{voice}-{gender}" for voice, gender in voices_with_gender] + + +def get_elevenlabs_voices(api_key: str) -> list[str]: + if not api_key: + return [] + try: + url = "https://api.elevenlabs.io/v2/voices" + params = {"is_favorite": "true", "page_size": 100} + headers = {"xi-api-key": api_key} + response = requests.get(url, params=params, headers=headers, timeout=10) + if response.status_code != 200: + logger.warning( + f"ElevenLabs voices fetch failed with status {response.status_code}: {response.text}" + ) + return [] + data = response.json() + voices = data.get("voices", []) + return [ + f"elevenlabs:{v['voice_id']}:{v['name']}" + for v in voices + if v.get("voice_id") and v.get("name") and v.get("status") != "disabled" + ] + except Exception as e: + logger.warning(f"ElevenLabs voices fetch failed: {str(e)}") + return [] + + +def get_chatterbox_voices() -> list[str]: + """Return the configured Chatterbox voices. + + Chatterbox is self-hosted, so there is no global voice catalog. Operators + list the voice names exposed by their server via ``[chatterbox] voices`` + (a TOML array, or a comma-separated string). Each entry is normalised to + the ``chatterbox:`` format used by the TTS dispatcher. + """ + voices = config.chatterbox.get("voices", []) or [] + if isinstance(voices, str): + voices = [v.strip() for v in voices.split(",") if v.strip()] + result = [] + for v in voices: + v = str(v).strip() + if not v: + continue + result.append(v if v.startswith("chatterbox:") else f"chatterbox:{v}") + if not result: + # keep the dropdown usable even before any voice is configured + result = ["chatterbox:default-Female"] + return result + + +_AZURE_VOICES_DATA_FILE = os.path.join( + os.path.dirname(__file__), "data", "azure_voices.json" +) +_azure_voices_cache = None + + +def _load_azure_voices() -> list[dict]: + global _azure_voices_cache + if _azure_voices_cache is None: + with open(_AZURE_VOICES_DATA_FILE, "r", encoding="utf-8") as f: + _azure_voices_cache = json.load(f) + return _azure_voices_cache + + +def get_all_azure_voices(filter_locals=None) -> list[str]: + voices = [] + for item in _load_azure_voices(): + name = item["name"] + gender = item["gender"] + # 应用过滤条件 + if filter_locals and any( + name.lower().startswith(fl.lower()) for fl in filter_locals + ): + voices.append(f"{name}-{gender}") + elif not filter_locals: + voices.append(f"{name}-{gender}") + + voices.sort() + return voices + + +def parse_voice_name(name: str): + # zh-CN-XiaoyiNeural-Female + # zh-CN-YunxiNeural-Male + # zh-CN-XiaoxiaoMultilingualNeural-V2-Female + name = name.replace("-Female", "").replace("-Male", "").strip() + return name + + +def is_azure_v2_voice(voice_name: str): + voice_name = parse_voice_name(voice_name) + if voice_name.endswith("-V2"): + return voice_name.replace("-V2", "").strip() + return "" + + +def is_siliconflow_voice(voice_name: str): + """检查是否是硅基流动的声音""" + return voice_name.startswith("siliconflow:") + + +def is_gemini_voice(voice_name: str): + """检查是否是Gemini TTS的声音""" + return voice_name.startswith("gemini:") + + +def is_mimo_voice(voice_name: str): + """检查是否是 Xiaomi MiMo TTS 的声音""" + return voice_name.startswith("mimo:") + + +def is_elevenlabs_voice(voice_name: str) -> bool: + return (voice_name or "").startswith("elevenlabs:") + + +def is_chatterbox_voice(voice_name: str) -> bool: + return (voice_name or "").startswith("chatterbox:") + + +def is_no_voice(voice_name: str | None) -> bool: + """ + 判断用户是否明确选择了“无配音”模式。 + + 这里刻意不把空字符串当成无配音：空 voice 更可能是配置损坏、旧版本 + WebUI 状态丢失或接口参数缺失。只有明确的 sentinel 才进入静音分支， + 这样可以避免把真实错误伪装成正常生成。 + """ + return str(voice_name or "").strip().lower() in _NO_VOICE_ALIASES + + +def estimate_no_voice_duration(text: str) -> float: + """ + 为无配音模式估算一个稳定的视频时间轴长度。 + + 无配音仍需要一个音频占位来驱动现有素材裁剪、字幕时间轴和最终合成。 + 估算策略尽量简单： + 1. 中文等 CJK 字符按约 4.2 字/秒估算； + 2. 英文/数字按约 2.7 词/秒估算； + 3. 其他语种文字按约 4.0 字符/秒兜底估算，覆盖俄语、阿拉伯语、 + 日文假名、韩文等非 ASCII 文本； + 4. 每个断句补一点停顿，让字幕切换不至于过于紧凑； + 5. 最少 3 秒，避免极短脚本生成 0 秒音频。 + """ + normalized_text = (text or "").strip() + if not normalized_text: + return 3.0 + + cjk_chars = len(re.findall(r"[\u4e00-\u9fff]", normalized_text)) + words = len(re.findall(r"[A-Za-z0-9]+", normalized_text)) + ascii_word_chars = sum(len(word) for word in re.findall(r"[A-Za-z0-9]+", normalized_text)) + other_text_chars = 0 + for char in normalized_text: + # Unicode category 以 L 开头表示各语种字母，N 表示数字。前面已经单独 + # 统计了 CJK 和 ASCII 单词，这里只统计剩余文字，避免英文被重复计时。 + category = unicodedata.category(char) + if category.startswith(("L", "N")): + other_text_chars += 1 + other_text_chars = max(other_text_chars - cjk_chars - ascii_word_chars, 0) + sentence_count = max(len(utils.split_string_by_punctuations(normalized_text)), 1) + + cjk_duration = cjk_chars / 4.2 + word_duration = words / 2.7 + other_text_duration = other_text_chars / 4.0 + pause_duration = max(sentence_count - 1, 0) * 0.35 + return max(3.0, cjk_duration + word_duration + other_text_duration + pause_duration) + + +def generate_silent_audio(duration_seconds: float, output_file: str) -> bool: + """ + 生成 MP3 静音音频，作为“无配音”模式的时间轴占位。 + + 使用 FFmpeg 的 anullsrc 直接生成静音，比先构造临时 WAV 再转码更少中间 + 文件。失败时返回 False，让上层按普通 TTS 失败路径处理并记录日志。 + """ + ensure_file_path_exists(output_file) + duration_seconds = max(float(duration_seconds or 0), 0.1) + ffmpeg_binary = utils.get_ffmpeg_binary() + command = [ + ffmpeg_binary, + "-y", + "-f", + "lavfi", + "-i", + "anullsrc=r=44100:cl=mono", + "-t", + f"{duration_seconds:.3f}", + "-codec:a", + "libmp3lame", + "-q:a", + "4", + output_file, + ] + + logger.info( + f"generating silent audio for no-voice mode, duration: {duration_seconds:.2f}s" + ) + result = subprocess.run( + command, + capture_output=True, + text=True, + check=False, + ) + if result.returncode != 0: + logger.error( + "failed to generate silent audio: " + f"{(result.stderr or result.stdout or '').strip()}" + ) + return False + if not os.path.exists(output_file) or os.path.getsize(output_file) <= 0: + logger.error( + "silent audio output file is missing or empty, " + f"file: {output_file}, duration: {duration_seconds:.2f}s" + ) + return False + return True + + +def tts( + text: str, + voice_name: str, + voice_rate: float, + voice_file: str, + voice_volume: float = 1.0, +) -> Union[SubMaker, None]: + if is_no_voice(voice_name): + duration_seconds = estimate_no_voice_duration(text) + if not generate_silent_audio(duration_seconds, voice_file): + return None + + sub_maker = ensure_legacy_submaker_fields(SubMaker()) + return populate_legacy_submaker_with_full_text( + sub_maker=sub_maker, + text=text, + audio_duration_seconds=duration_seconds, + ) + + if is_azure_v2_voice(voice_name): + return azure_tts_v2( + text, + voice_name, + voice_file, + voice_rate=voice_rate, + ) + elif is_siliconflow_voice(voice_name): + # 从voice_name中提取模型和声音 + # 格式: siliconflow:model:voice-Gender + parts = voice_name.split(":") + if len(parts) >= 3: + model = parts[1] + # 移除性别后缀，例如 "alex-Male" -> "alex" + voice_with_gender = parts[2] + voice = voice_with_gender.split("-")[0] + # 构建完整的voice参数，格式为 "model:voice" + full_voice = f"{model}:{voice}" + return siliconflow_tts( + text, model, full_voice, voice_rate, voice_file, voice_volume + ) + else: + logger.error(f"Invalid siliconflow voice name format: {voice_name}") + return None + elif is_gemini_voice(voice_name): + # 从voice_name中提取声音名称 + # 格式: gemini:voice-Gender + parts = voice_name.split(":") + if len(parts) >= 2: + # 移除性别后缀，例如 "Zephyr-Female" -> "Zephyr" + voice_with_gender = parts[1] + voice = voice_with_gender.split("-")[0] + return gemini_tts(text, voice, voice_rate, voice_file, voice_volume) + else: + logger.error(f"Invalid gemini voice name format: {voice_name}") + return None + elif is_mimo_voice(voice_name): + # 从voice_name中提取声音名称 + # 格式: mimo:voice-Gender；如果调用方已执行 parse_voice_name， + # 则可能是 mimo:voice。两种格式都兼容。 + parts = voice_name.split(":") + if len(parts) >= 2: + voice_with_gender = parts[1] + voice = voice_with_gender.split("-")[0] + return mimo_tts(text, voice, voice_rate, voice_file, voice_volume) + else: + logger.error(f"Invalid mimo voice name format: {voice_name}") + return None + elif is_elevenlabs_voice(voice_name): + # 格式: elevenlabs:{voice_id}:{name} + parts = voice_name.split(":") + if len(parts) >= 2: + voice_id = parts[1] + return elevenlabs_tts(text, voice_id, voice_file, voice_rate, voice_volume) + else: + logger.error(f"Invalid elevenlabs voice name format: {voice_name}") + return None + elif is_chatterbox_voice(voice_name): + # 格式: chatterbox:，voice 可带显示用的 -Female/-Male 后缀 + parts = voice_name.split(":", 1) + if len(parts) >= 2 and parts[1].strip(): + chatterbox_voice = parts[1].strip() + if chatterbox_voice.endswith(("-Female", "-Male")): + chatterbox_voice = chatterbox_voice.rsplit("-", 1)[0] + return chatterbox_tts( + text, chatterbox_voice, voice_file, voice_rate, voice_volume + ) + else: + logger.error(f"Invalid chatterbox voice name format: {voice_name}") + return None + return azure_tts_v1(text, voice_name, voice_rate, voice_file) + + +def convert_rate_to_percent(rate: float) -> str: + # edge-tts requires a sign-prefixed percentage (e.g. "+0%", "-20%"). + # Rounding can yield 0 for rates near but not equal to 1.0 (e.g. 1.004, + # 0.997); those must still be returned as "+0%", not the unsigned "0%" + # which edge-tts rejects with ValueError: Invalid rate '0%'. + # API 或批处理调用可能传入 0、0.0、None 或无法转换的空值；这些值不代表 + # 合法语速，直接计算会变成 -100% 或抛异常。这里统一回退到正常语速， + # 避免生成极慢音频或让 TTS 流程在边界输入下失败。 + try: + rate = float(rate) + except (TypeError, ValueError): + rate = 1.0 + if rate <= 0: + rate = 1.0 + percent = round((rate - 1.0) * 100) + if percent >= 0: + return f"+{percent}%" + return f"{percent}%" + + +def ensure_file_path_exists(file_path: str) -> None: + """ + 确保输出文件所在目录一定存在。 + + 这里单独做一层兜底，是因为 edge_tts 7.x 在真正发起网络请求之前， + 就会先打开目标音频文件；如果目录不存在，会直接因为本地文件路径报错， + 从而掩盖真正的 TTS 行为结果。 + """ + dir_path = os.path.dirname(file_path) + if dir_path: + os.makedirs(dir_path, exist_ok=True) + + +def ensure_legacy_submaker_fields(sub_maker: SubMaker) -> SubMaker: + """ + 为项目里仍然沿用旧字幕结构的调用方补齐兼容字段。 + + edge_tts 7.x 的 `SubMaker` 主要暴露 `cues/get_srt()`，但项目里 Azure v2、 + Gemini、SiliconFlow 这些路径仍然会直接读写 `subs/offset`。这里统一补齐， + 避免升级 edge_tts 后这些非 edge 路径被连带破坏。 + """ + if not hasattr(sub_maker, "subs"): + sub_maker.subs = [] + if not hasattr(sub_maker, "offset"): + sub_maker.offset = [] + return sub_maker + + +def populate_legacy_submaker_with_full_text( + sub_maker: SubMaker, text: str, audio_duration_seconds: float +) -> SubMaker: + """ + 用整段文本填充项目历史沿用的 `subs/offset` 字幕结构。 + + 背景： + 1. edge_tts 7.x 的 `SubMaker` 不再提供旧版本里的 `create_sub()`； + 2. 项目里 Gemini、SiliconFlow 等非 edge 路径依然需要返回一个 + 带 `subs/offset` 的对象，供后续统一计算音频时长和生成字幕； + 3. 对于拿不到逐词边界的 TTS 服务，需要至少按脚本断句切成多个片段， + 这样后续 `subtitle_provider=edge` 的聚合逻辑才能继续工作，而不是 + 因为整段文本无法和脚本断句逐行匹配而回退 Whisper。 + + Args: + sub_maker: 需要写入兼容字段的字幕对象 + text: 原始脚本文本 + audio_duration_seconds: 音频总时长，单位秒 + + Returns: + 已填充兼容字幕数据的 SubMaker 对象 + """ + sub_maker = ensure_legacy_submaker_fields(sub_maker) + + # 清空旧值，避免调用方重复复用对象时出现脏数据叠加。 + sub_maker.subs = [] + sub_maker.offset = [] + + normalized_text = (text or "").strip() + if not normalized_text: + return sub_maker + + audio_duration_100ns = max(int(audio_duration_seconds * 10000000), 1) + + # Gemini / SiliconFlow 这类路径拿不到逐词边界时，仍然尽量沿用项目 + # 原来的“按标点断句 + 按字符数比例分配时长”的策略。这样既能让 + # create_subtitle() 匹配脚本断句，也能避免再次回退 Whisper。 + sentences = utils.split_string_by_punctuations(normalized_text) + if not sentences: + sentences = [normalized_text] + + total_chars = sum(len(sentence) for sentence in sentences) + if total_chars <= 0: + sub_maker.subs.append(normalized_text) + sub_maker.offset.append((0, audio_duration_100ns)) + return sub_maker + + current_offset = 0 + for index, sentence in enumerate(sentences): + cleaned_sentence = sentence.strip() + if not cleaned_sentence: + continue + + # 前面的句子按字符数比例分配时长，最后一句兜底吃掉剩余时长， + # 避免整数取整导致总时长丢失或字幕结束时间短于音频。 + if index == len(sentences) - 1: + sentence_end = audio_duration_100ns + else: + sentence_chars = len(cleaned_sentence) + sentence_duration = max( + int(audio_duration_100ns * (sentence_chars / total_chars)), + 1, + ) + sentence_end = min(current_offset + sentence_duration, audio_duration_100ns) + + sub_maker.subs.append(cleaned_sentence) + sub_maker.offset.append((current_offset, sentence_end)) + current_offset = sentence_end + + return sub_maker + + +def create_edge_tts_communicate( + text: str, voice_name: str, rate_str: str +) -> edge_tts.Communicate: + """ + 按当前已安装的 edge_tts 版本构造 Communicate 对象。 + + 背景： + 1. 主线代码已经升级到 edge_tts 7.x，并使用 `boundary` 参数拿到更细的边界事件； + 2. 但 Windows 便携包如果更新失败，现场环境可能仍然停留在旧版 edge_tts； + 3. 旧版 `Communicate.__init__()` 不接受 `boundary`，会直接抛出 + `unexpected keyword argument 'boundary'`，导致整个 TTS 链路失败。 + + 因此这里先根据构造函数签名探测当前版本支持的参数，再决定是否传入 + `boundary`，让同一份代码同时兼容旧版和新版依赖。 + """ + communicate_kwargs = {"rate": rate_str} + communicate_signature = inspect.signature(edge_tts.Communicate) + + if "boundary" in communicate_signature.parameters: + communicate_kwargs["boundary"] = "WordBoundary" + + return edge_tts.Communicate(text, voice_name, **communicate_kwargs) + + +def get_edge_tts_timeout_seconds() -> Union[float, None]: + """ + 获取 Azure TTS V1 单次流式请求的超时时间。 + + 背景： + Edge consumer TTS 在网络不通、服务端限流、voice 与文本语言不匹配等场景下， + 可能长时间卡在 `stream_sync()` 内部，日志只停留在 `start`。这里提供一个 + 默认超时，避免 WebUI 任务长期无反馈。 + + 使用方式： + - 默认 30 秒，覆盖常见短视频脚本的首包等待时间； + - 如用户处于慢网络或代理环境，可在 `config.toml` 里设置 + `edge_tts_timeout = 60`； + - 设置为 0 或负数表示显式禁用超时，保留完全向后兼容。 + """ + raw_timeout = config.app.get( + "edge_tts_timeout", _DEFAULT_EDGE_TTS_TIMEOUT_SECONDS + ) + try: + timeout_seconds = float(raw_timeout) + except (TypeError, ValueError): + logger.warning( + "invalid edge_tts_timeout: " + f"{raw_timeout}, fallback to {_DEFAULT_EDGE_TTS_TIMEOUT_SECONDS}s" + ) + timeout_seconds = _DEFAULT_EDGE_TTS_TIMEOUT_SECONDS + + if timeout_seconds <= 0: + return None + + return timeout_seconds + + +def _stream_edge_tts_sync_with_timeout( + communicate, on_chunk, timeout_seconds: float +) -> None: + """ + 带总超时地消费 edge_tts 7.x 的同步流。 + + 实现原因： + `stream_sync()` 本身是阻塞迭代器，网络层卡住时主线程无法及时恢复。 + 这里把阻塞迭代放到 daemon 线程中，主线程通过 Queue 获取 chunk， + 到达超时时间后直接抛出 TimeoutError，让外层重试和错误日志继续工作。 + + 注意： + daemon 线程只作为兜底保护使用，最多随 Azure TTS V1 的 3 次重试产生 + 少量残留线程；进程退出时会自动回收。相比 WebUI 任务永久卡住，这是 + 更可控的失败模式。 + """ + stream_queue = queue.Queue() + done_marker = object() + + def _produce_chunks(): + try: + for chunk in communicate.stream_sync(): + stream_queue.put(("chunk", chunk)) + stream_queue.put(("done", done_marker)) + except Exception as e: + stream_queue.put(("error", e)) + + thread = threading.Thread(target=_produce_chunks, daemon=True) + thread.start() + + deadline = time.monotonic() + timeout_seconds + while True: + remaining_seconds = deadline - time.monotonic() + if remaining_seconds <= 0: + raise TimeoutError( + f"edge_tts stream timed out after {timeout_seconds:g}s" + ) + + try: + item_type, payload = stream_queue.get( + timeout=min(0.5, remaining_seconds) + ) + except queue.Empty: + continue + + if item_type == "chunk": + on_chunk(payload) + elif item_type == "error": + raise payload + elif item_type == "done": + return + + +def stream_edge_tts_chunks( + communicate, on_chunk, timeout_seconds: Union[float, None] = None +) -> None: + """ + 统一消费 edge_tts 的同步流和旧版异步流。 + + edge_tts 7.x 提供 `stream_sync()`，可以在同步函数里直接迭代； + 更早的版本通常只有异步 `stream()`。为了让 `azure_tts_v1()` 在 + 旧依赖残留场景下仍能继续工作，这里统一做一层流式兼容。 + + Args: + communicate: edge_tts.Communicate 实例 + on_chunk: 每拿到一个事件块时执行的回调 + timeout_seconds: 单次流式请求总超时；为 None 时不启用超时。 + """ + if hasattr(communicate, "stream_sync"): + if timeout_seconds: + _stream_edge_tts_sync_with_timeout( + communicate, on_chunk, timeout_seconds + ) + return + + for chunk in communicate.stream_sync(): + on_chunk(chunk) + return + + if not hasattr(communicate, "stream"): + raise AttributeError("edge_tts communicate object has no stream method") + + async def _consume_async_stream(): + async for chunk in communicate.stream(): + on_chunk(chunk) + + # 这里显式创建独立事件循环，而不是复用外部上下文，目的是避免 + # 在同步调用栈里遇到“当前线程没有事件循环”或跨线程复用循环的问题。 + loop = asyncio.new_event_loop() + try: + if timeout_seconds: + loop.run_until_complete( + asyncio.wait_for(_consume_async_stream(), timeout=timeout_seconds) + ) + else: + loop.run_until_complete(_consume_async_stream()) + finally: + loop.close() + + +def azure_tts_v1( + text: str, voice_name: str, voice_rate: float, voice_file: str +) -> Union[SubMaker, None]: + voice_name = parse_voice_name(voice_name) + text = text.strip() + rate_str = convert_rate_to_percent(voice_rate) + for i in range(3): + try: + logger.info(f"start, voice name: {voice_name}, try: {i + 1}") + + # 这里同时兼容 edge_tts 7.x 和旧版便携包里可能残留的老依赖： + # 1. 新版支持 `boundary` + `stream_sync()` + # 2. 旧版不支持 `boundary`，且通常只暴露异步 `stream()` + ensure_file_path_exists(voice_file) + communicate = create_edge_tts_communicate(text, voice_name, rate_str) + sub_maker = edge_tts.SubMaker() + timeout_seconds = get_edge_tts_timeout_seconds() + + with open(voice_file, "wb") as file: + def _handle_chunk(chunk): + chunk_type = chunk["type"] + if chunk_type == "audio": + file.write(chunk["data"]) + elif chunk_type in ["WordBoundary", "SentenceBoundary"]: + # 无论来自 7.x 的同步流，还是旧版异步流，只要事件结构 + # 里仍有边界信息，就统一喂给 SubMaker，保证后续字幕链路 + # 仍然走项目现有逻辑。 + sub_maker.feed(chunk) + + stream_edge_tts_chunks( + communicate, _handle_chunk, timeout_seconds=timeout_seconds + ) + + if not sub_maker.get_srt(): + logger.warning("failed, sub_maker.get_srt() is empty") + continue + + logger.info(f"completed, output file: {voice_file}") + return sub_maker + except Exception as e: + logger.error(f"failed, error: {str(e)}") + # TTS 流式写入如果在首包前超时或网络异常，会留下 0 字节音频文件。 + # 这种文件既不可播放，也可能误导后续排查，因此失败后只清理空文件； + # 如果已经写入了部分数据，则保留现场文件，便于分析服务端返回内容。 + if os.path.exists(voice_file) and os.path.getsize(voice_file) == 0: + try: + os.remove(voice_file) + except Exception as remove_error: + logger.warning( + "failed to remove empty tts file: " + f"{voice_file}, error: {str(remove_error)}" + ) + return None + + +def siliconflow_tts( + text: str, + model: str, + voice: str, + voice_rate: float, + voice_file: str, + voice_volume: float = 1.0, +) -> Union[SubMaker, None]: + """ + 使用硅基流动的API生成语音 + + Args: + text: 要转换为语音的文本 + model: 模型名称，如 "FunAudioLLM/CosyVoice2-0.5B" + voice: 声音名称，如 "FunAudioLLM/CosyVoice2-0.5B:alex" + voice_rate: 语音速度，范围[0.25, 4.0] + voice_file: 输出的音频文件路径 + voice_volume: 语音音量，范围[0.6, 5.0]，需要转换为硅基流动的增益范围[-10, 10] + + Returns: + SubMaker对象或None + """ + text = text.strip() + api_key = config.siliconflow.get("api_key", "") + + if not api_key: + logger.error("SiliconFlow API key is not set") + return None + + # 将voice_volume转换为硅基流动的增益范围 + # 默认voice_volume为1.0，对应gain为0 + gain = voice_volume - 1.0 + # 确保gain在[-10, 10]范围内 + gain = max(-10, min(10, gain)) + + url = "https://api.siliconflow.cn/v1/audio/speech" + + payload = { + "model": model, + "input": text, + "voice": voice, + "response_format": "mp3", + "sample_rate": 32000, + "stream": False, + "speed": voice_rate, + "gain": gain, + } + + headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"} + + for i in range(3): # 尝试3次 + try: + logger.info( + f"start siliconflow tts, model: {model}, voice: {voice}, try: {i + 1}" + ) + + response = requests.post(url, json=payload, headers=headers) + + if response.status_code == 200: + # 保存音频文件 + with open(voice_file, "wb") as f: + f.write(response.content) + + # 这里仍然沿用项目原有的字幕结构，因此需要补齐旧字段。 + sub_maker = ensure_legacy_submaker_fields(SubMaker()) + + # 获取音频文件的实际长度 + try: + # 尝试使用moviepy获取音频长度 + from moviepy import AudioFileClip + + audio_clip = AudioFileClip(voice_file) + audio_duration = audio_clip.duration + audio_clip.close() + + # 将音频长度转换为100纳秒单位（与edge_tts兼容） + audio_duration_100ns = int(audio_duration * 10000000) + + # 使用文本分割来创建更准确的字幕 + # 将文本按标点符号分割成句子 + sentences = utils.split_string_by_punctuations(text) + + if sentences: + # 计算每个句子的大致时长（按字符数比例分配） + total_chars = sum(len(s) for s in sentences) + char_duration = ( + audio_duration_100ns / total_chars if total_chars > 0 else 0 + ) + + current_offset = 0 + for sentence in sentences: + if not sentence.strip(): + continue + + # 计算当前句子的时长 + sentence_chars = len(sentence) + sentence_duration = int(sentence_chars * char_duration) + + # 添加到SubMaker + sub_maker.subs.append(sentence) + sub_maker.offset.append( + (current_offset, current_offset + sentence_duration) + ) + + # 更新偏移量 + current_offset += sentence_duration + else: + # 如果无法分割，则使用整个文本作为一个字幕 + sub_maker.subs = [text] + sub_maker.offset = [(0, audio_duration_100ns)] + + except Exception as e: + logger.warning(f"Failed to create accurate subtitles: {str(e)}") + # 回退到简单的字幕 + sub_maker.subs = [text] + # 使用音频文件的实际长度，如果无法获取，则假设为10秒 + sub_maker.offset = [ + ( + 0, + audio_duration_100ns + if "audio_duration_100ns" in locals() + else 10000000, + ) + ] + + logger.success(f"siliconflow tts succeeded: {voice_file}") + logger.debug( + "siliconflow subtitle timeline generated, " + f"subs: {len(sub_maker.subs)}, offsets: {len(sub_maker.offset)}" + ) + return sub_maker + else: + logger.error( + f"siliconflow tts failed with status code {response.status_code}: {response.text}" + ) + except Exception as e: + logger.error(f"siliconflow tts failed: {str(e)}") + + return None + + +def _build_azure_v2_ssml(text: str, voice_name: str, voice_rate: float) -> str: + """构造 Azure Speech V2 使用的 SSML，并安全规范化语速参数。""" + try: + normalized_rate = float(voice_rate) + except (TypeError, ValueError): + normalized_rate = 1.0 + normalized_rate = max(0.25, min(4.0, normalized_rate)) + + voice_locale_parts = voice_name.split("-", 2) + voice_locale = ( + "-".join(voice_locale_parts[:2]) + if len(voice_locale_parts) >= 2 + else "en-US" + ) + escaped_text = escape(text) + escaped_voice_name = escape(voice_name, {'"': """}) + return ( + '' + f'' + f'{escaped_text}' + "" + ) + + +def azure_tts_v2( + text: str, + voice_name: str, + voice_file: str, + voice_rate: float = 1.0, +) -> Union[SubMaker, None]: + voice_name = is_azure_v2_voice(voice_name) + if not voice_name: + logger.error(f"invalid voice name: {voice_name}") + raise ValueError(f"invalid voice name: {voice_name}") + text = text.strip() + ssml = _build_azure_v2_ssml(text, voice_name, voice_rate) + + def _format_duration_to_offset(duration) -> int: + if isinstance(duration, str): + time_obj = datetime.strptime(duration, "%H:%M:%S.%f") + milliseconds = ( + (time_obj.hour * 3600000) + + (time_obj.minute * 60000) + + (time_obj.second * 1000) + + (time_obj.microsecond // 1000) + ) + return milliseconds * 10000 + + if isinstance(duration, int): + return duration + + return 0 + + for i in range(3): + try: + logger.info( + f"start, voice name: {voice_name}, rate: {voice_rate}, try: {i + 1}" + ) + + import azure.cognitiveservices.speech as speechsdk + + sub_maker = ensure_legacy_submaker_fields(SubMaker()) + + def speech_synthesizer_word_boundary_cb(evt: speechsdk.SessionEventArgs): + # print('WordBoundary event:') + # print('\tBoundaryType: {}'.format(evt.boundary_type)) + # print('\tAudioOffset: {}ms'.format((evt.audio_offset + 5000))) + # print('\tDuration: {}'.format(evt.duration)) + # print('\tText: {}'.format(evt.text)) + # print('\tTextOffset: {}'.format(evt.text_offset)) + # print('\tWordLength: {}'.format(evt.word_length)) + + duration = _format_duration_to_offset(str(evt.duration)) + offset = _format_duration_to_offset(evt.audio_offset) + sub_maker.subs.append(evt.text) + sub_maker.offset.append((offset, offset + duration)) + + # Creates an instance of a speech config with specified subscription key and service region. + speech_key = config.azure.get("speech_key", "") + service_region = config.azure.get("speech_region", "") + if not speech_key or not service_region: + logger.error("Azure speech key or region is not set") + return None + + audio_config = speechsdk.audio.AudioOutputConfig( + filename=voice_file, use_default_speaker=True + ) + speech_config = speechsdk.SpeechConfig( + subscription=speech_key, region=service_region + ) + speech_config.speech_synthesis_voice_name = voice_name + # speech_config.set_property(property_id=speechsdk.PropertyId.SpeechServiceResponse_RequestSentenceBoundary, + # value='true') + speech_config.set_property( + property_id=speechsdk.PropertyId.SpeechServiceResponse_RequestWordBoundary, + value="true", + ) + + speech_config.set_speech_synthesis_output_format( + speechsdk.SpeechSynthesisOutputFormat.Audio48Khz192KBitRateMonoMp3 + ) + speech_synthesizer = speechsdk.SpeechSynthesizer( + audio_config=audio_config, speech_config=speech_config + ) + speech_synthesizer.synthesis_word_boundary.connect( + speech_synthesizer_word_boundary_cb + ) + + # speak_text_async() 不支持语速参数。使用 SSML prosody 后，试听和 + # 正式生成都会按 WebUI/API 传入的 voice_rate 调整语速。 + result = speech_synthesizer.speak_ssml_async(ssml).get() + if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted: + logger.success(f"azure v2 speech synthesis succeeded: {voice_file}") + return sub_maker + elif result.reason == speechsdk.ResultReason.Canceled: + cancellation_details = result.cancellation_details + logger.error( + f"azure v2 speech synthesis canceled: {cancellation_details.reason}" + ) + if cancellation_details.reason == speechsdk.CancellationReason.Error: + logger.error( + f"azure v2 speech synthesis error: {cancellation_details.error_details}" + ) + logger.info(f"completed, output file: {voice_file}") + except Exception as e: + logger.error(f"failed, error: {str(e)}") + return None + + +def gemini_tts( + text: str, + voice_name: str, + voice_rate: float, + voice_file: str, + voice_volume: float = 1.0, +) -> Union[SubMaker, None]: + """ + 使用Google Gemini TTS生成语音 + + Args: + text: 要转换的文本 + voice_name: 语音名称，如 "Zephyr", "Puck" 等 + voice_rate: 语音速率（当前未使用） + voice_file: 输出音频文件路径 + voice_volume: 音频音量（当前未使用） + + Returns: + SubMaker对象或None + """ + import base64 + import io + from pydub import AudioSegment + from google import genai + from google.genai import types + _configure_pydub_ffmpeg(AudioSegment) + + try: + api_key = config.app.get("gemini_api_key", "") + if not api_key: + logger.error("Gemini API key is not set") + return None + + logger.info(f"start, voice name: {voice_name}, try: 1") + + generation_config = types.GenerateContentConfig( + response_modalities=["AUDIO"], + speech_config=types.SpeechConfig( + voice_config=types.VoiceConfig( + prebuilt_voice_config=types.PrebuiltVoiceConfig( + voice_name=voice_name + ) + ) + ), + ) + + # google-genai 使用统一 Client 调用文本和 TTS 模型。上下文管理器确保 + # 请求结束后释放 HTTP 连接，同时保留原有 PCM 转码和字幕时间轴逻辑。 + with genai.Client(api_key=api_key) as client: + response = client.models.generate_content( + model="gemini-2.5-flash-preview-tts", + contents=text, + config=generation_config, + ) + + # 检查响应 + if not response.candidates or not response.candidates[0].content: + logger.error("No audio content received from Gemini TTS") + return None + + # 获取音频数据 + audio_data = None + for part in response.candidates[0].content.parts: + if hasattr(part, 'inline_data') and part.inline_data: + audio_data = part.inline_data.data + break + + if not audio_data: + logger.error("No audio data found in response") + return None + + # 音频数据已经是原始字节，不需要base64解码 + if isinstance(audio_data, str): + # 如果是字符串，则需要base64解码 + audio_bytes = base64.b64decode(audio_data) + else: + # 如果已经是字节，直接使用 + audio_bytes = audio_data + + # 尝试不同的音频格式 - Gemini可能返回不同的格式 + audio_segment = None + + # Gemini返回Linear PCM格式，按照文档参数解析 + try: + audio_segment = AudioSegment.from_file( + io.BytesIO(audio_bytes), + format="raw", + frame_rate=24000, # Gemini TTS默认采样率 + channels=1, # 单声道 + sample_width=2 # 16-bit + ) + except Exception as e: + logger.error(f"Failed to load PCM audio: {e}") + return None + + # pydub 会返回打开的输出文件对象。批量生成时若不主动关闭，文件描述符 + # 会持续累积，并在 Windows 上增加后续覆盖或删除音频文件失败的概率。 + exported_audio = audio_segment.export(voice_file, format="mp3") + exported_audio.close() + + logger.info(f"completed, output file: {voice_file}") + + # Gemini 拿不到 edge_tts 那种逐词边界事件，因此这里退回到 + # 项目原有的 `subs/offset` 兼容结构，至少保证后续字幕与时长 + # 计算链路可继续工作。 + sub_maker = ensure_legacy_submaker_fields(SubMaker()) + audio_duration = len(audio_segment) / 1000.0 # 转换为秒 + return populate_legacy_submaker_with_full_text( + sub_maker=sub_maker, + text=text, + audio_duration_seconds=audio_duration, + ) + + except ImportError as e: + logger.error(f"Missing required package for Gemini TTS: {str(e)}. Please install: pip install pydub") + return None + except Exception as e: + logger.error(f"Gemini TTS failed, error: {str(e)}") + return None + + +def mimo_tts( + text: str, + voice_name: str, + voice_rate: float, + voice_file: str, + voice_volume: float = 1.0, +) -> Union[SubMaker, None]: + """ + 使用 Xiaomi MiMo V2.5 TTS 生成语音。 + + 官方接口兼容 OpenAI Chat Completions，但 TTS 有两个关键差异： + 1. 待合成文本必须放在 `assistant` 消息里； + 2. 音频以 `message.audio.data` 的 base64 字符串返回。 + + MiMo 当前没有返回逐词时间轴，因此这里复用项目已有的 legacy + SubMaker 兜底方案：根据最终音频时长和脚本文本断句生成字幕时间轴。 + """ + from pydub import AudioSegment + + text = (text or "").strip() + if not text: + logger.error("MiMo TTS text is empty") + return None + + api_key = config.app.get("mimo_api_key", "") + if not api_key: + logger.error("MiMo API key is not set") + return None + + base_url = config.app.get("mimo_base_url", "") or _MIMO_DEFAULT_BASE_URL + model_name = config.app.get("mimo_tts_model_name", "") or _MIMO_DEFAULT_TTS_MODEL + style_prompt = config.app.get( + "mimo_tts_style_prompt", + "请用自然、清晰、适合短视频旁白的语气朗读。", + ) + + _configure_pydub_ffmpeg(AudioSegment) + + for i in range(3): + try: + logger.info( + f"start mimo tts, model: {model_name}, voice: {voice_name}, try: {i + 1}" + ) + ensure_file_path_exists(voice_file) + + client = OpenAI(api_key=api_key, base_url=base_url) + completion = client.chat.completions.create( + model=model_name, + messages=[ + {"role": "user", "content": style_prompt}, + {"role": "assistant", "content": text}, + ], + audio={ + "format": "wav", + "voice": voice_name, + }, + ) + + if not completion or not getattr(completion, "choices", None): + raise ValueError("MiMo TTS returned empty response") + + message = completion.choices[0].message + audio = getattr(message, "audio", None) + audio_data = None + if isinstance(audio, dict): + audio_data = audio.get("data") + elif audio is not None: + audio_data = getattr(audio, "data", None) + + if not audio_data: + raise ValueError("MiMo TTS returned empty audio data") + + audio_bytes = base64.b64decode(audio_data) + audio_segment = AudioSegment.from_file(io.BytesIO(audio_bytes), format="wav") + + output_format = utils.parse_extension(voice_file) or "mp3" + if output_format == "wav": + with open(voice_file, "wb") as f: + f.write(audio_bytes) + else: + audio_segment.export(voice_file, format=output_format) + + audio_duration = len(audio_segment) / 1000.0 + sub_maker = ensure_legacy_submaker_fields(SubMaker()) + logger.success(f"mimo tts succeeded: {voice_file}") + logger.debug( + "mimo subtitle timeline generated, " + f"duration: {audio_duration:.3f}s, output_format: {output_format}" + ) + return populate_legacy_submaker_with_full_text( + sub_maker=sub_maker, + text=text, + audio_duration_seconds=audio_duration, + ) + except Exception as e: + logger.error(f"mimo tts failed: {str(e)}") + + return None + + +def elevenlabs_tts( + text: str, + voice_id: str, + voice_file: str, + voice_rate: float = 1.0, + voice_volume: float = 1.0, + model_id: str = "", +) -> Union[SubMaker, None]: + text = (text or "").strip() + if not text: + logger.error("ElevenLabs TTS text is empty") + return None + + api_key = config.elevenlabs.get("api_key", "") + if not api_key: + logger.error("ElevenLabs API key is not set") + return None + + if not model_id: + model_id = config.elevenlabs.get("model_id", "eleven_multilingual_v2") + + url = f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}" + headers = { + "xi-api-key": api_key, + "Content-Type": "application/json", + } + payload = { + "text": text, + "model_id": model_id, + "voice_settings": { + "stability": 0.5, + "similarity_boost": 0.75, + "style": 0.0, + "use_speaker_boost": True, + }, + } + + # Errors where retrying will never help (auth/access/validation failures). + _NON_RETRYABLE_CODES = {401, 403, 422} + _NON_RETRYABLE_STATUSES = {"voice_disabled", "voice_access_denied", "unauthorized"} + + for i in range(3): + try: + logger.info(f"start elevenlabs tts, voice_id: {voice_id}, try: {i + 1}") + ensure_file_path_exists(voice_file) + + response = requests.post(url, json=payload, headers=headers, timeout=60) + if response.status_code != 200: + error_status = "" + try: + detail = response.json().get("detail", {}) + if isinstance(detail, dict): + error_status = detail.get("status", "") + except Exception: + pass + + if response.status_code in _NON_RETRYABLE_CODES or error_status in _NON_RETRYABLE_STATUSES: + logger.error( + f"ElevenLabs TTS failed (non-retryable) — voice_id: {voice_id}, " + f"status: {response.status_code}, error: {error_status or response.text[:200]}. " + "Please select a different ElevenLabs voice." + ) + return None + + logger.error( + f"elevenlabs tts failed with status {response.status_code}: {response.text[:200]}" + ) + continue + + with open(voice_file, "wb") as f: + f.write(response.content) + + audio_clip = AudioFileClip(voice_file) + audio_duration = audio_clip.duration + audio_clip.close() + + sub_maker = ensure_legacy_submaker_fields(SubMaker()) + logger.success(f"elevenlabs tts succeeded: {voice_file}") + return populate_legacy_submaker_with_full_text( + sub_maker=sub_maker, + text=text, + audio_duration_seconds=audio_duration, + ) + except Exception as e: + logger.error(f"elevenlabs tts failed: {str(e)}") + + return None + + +def chatterbox_tts( + text: str, + voice: str, + voice_file: str, + voice_rate: float = 1.0, + voice_volume: float = 1.0, + model_id: str = "", +) -> Union[SubMaker, None]: + """Generate speech with a self-hosted Chatterbox TTS server. + + Chatterbox (Resemble AI, MIT) is an open-source, locally hosted TTS model + with zero-shot voice cloning — a self-hostable alternative to ElevenLabs. + This talks to an OpenAI-compatible ``/audio/speech`` endpoint, so it works + with the common community servers (e.g. devnen/Chatterbox-TTS-Server, + travisvn/chatterbox-tts-api). Configure ``[chatterbox] base_url`` (and an + optional ``api_key``). + + Like ElevenLabs, Chatterbox does not return word-level timestamps, so the + subtitle path falls back to the full-text SubMaker. For tighter subtitle + sync set ``subtitle_provider = "whisper"``. + """ + text = (text or "").strip() + if not text: + logger.error("Chatterbox TTS text is empty") + return None + + base_url = (config.chatterbox.get("base_url", "") or "").strip().rstrip("/") + if not base_url: + logger.error( + "Chatterbox base_url is not set, please configure [chatterbox] base_url in config.toml" + ) + return None + + api_key = config.chatterbox.get("api_key", "") + if not model_id: + model_id = config.chatterbox.get("model_id", "chatterbox") or "chatterbox" + + url = f"{base_url}/audio/speech" + headers = {"Content-Type": "application/json"} + if api_key: + headers["Authorization"] = f"Bearer {api_key}" + payload = { + "model": model_id, + "input": text, + "voice": voice, + "response_format": "mp3", + # OpenAI speech API accepts speed 0.25-4.0; MoneyPrinterTurbo's rate is a + # 1.0-centred multiplier, so it maps directly (clamped to the valid range). + "speed": max(0.25, min(4.0, float(voice_rate or 1.0))), + } + # voice_volume is accepted for parity with the other TTS providers but is + # intentionally not sent: the OpenAI /audio/speech contract has no volume + # field, so Chatterbox servers ignore it. Adjust loudness via voice_rate + # (speed) or in post-processing instead. + + for i in range(3): + try: + logger.info(f"start chatterbox tts, voice: {voice}, try: {i + 1}") + ensure_file_path_exists(voice_file) + + response = requests.post(url, json=payload, headers=headers, timeout=120) + if response.status_code != 200: + logger.error( + f"chatterbox tts failed with status {response.status_code}: {response.text[:200]}" + ) + continue + + with open(voice_file, "wb") as f: + f.write(response.content) + + audio_clip = AudioFileClip(voice_file) + audio_duration = audio_clip.duration + audio_clip.close() + + sub_maker = ensure_legacy_submaker_fields(SubMaker()) + logger.success(f"chatterbox tts succeeded: {voice_file}") + return populate_legacy_submaker_with_full_text( + sub_maker=sub_maker, + text=text, + audio_duration_seconds=audio_duration, + ) + except Exception as e: + logger.error(f"chatterbox tts failed: {str(e)}") + + return None + + +def _format_text(text: str) -> str: + """ + 清理字幕对齐前的脚本文本。 + + 这里不能只在 LLM 生成阶段处理，因为用户也可能手动粘贴脚本，或通过 + API 直接传入包含 Markdown 标记的文本。TTS 通常不会朗读 `---`、 + `___`、`***` 这类分隔符行，也不会朗读 `_` 这种强调标记；如果字幕 + 对齐仍保留这些字符，`create_subtitle()` 会一直等待不存在的 cue， + 最终导致字幕文件缺失并在 Whisper fallback 校正时补出全 0 时间轴。 + """ + text = text.replace("[", " ") + text = text.replace("]", " ") + text = text.replace("(", " ") + text = text.replace(")", " ") + text = text.replace("{", " ") + text = text.replace("}", " ") + return utils.normalize_script_for_subtitle_matching(text) + + +def _build_subtitle_formatter(): + """ + 返回统一的 SRT 行格式化函数。 + + 这里单独拆成一个小工具，是为了让 edge_tts 7.x 的 cues 路径 + 和项目原有的 legacy `subs/offset` 路径共用同一套字幕落盘格式， + 避免两套逻辑各自产生细微格式差异。 + """ + + def formatter(idx: int, start_time: float, end_time: float, sub_text: str) -> str: + start_t = mktimestamp(start_time).replace(".", ",") + end_t = mktimestamp(end_time).replace(".", ",") + return f"{idx}\n{start_t} --> {end_t}\n{sub_text}\n" + + return formatter + + +# 阿拉伯语变音符号和 Tatweel 拉长符在 edge-tts 返回文本中可能出现， +# 这些字符不影响语义，但会导致脚本文本和字幕 cue 字符串精确匹配失败。 +_ARABIC_DIACRITICS = re.compile("[\u0610-\u061A\u064B-\u065F\u0670\u0640\u06D6-\u06ED]") + + +def _normalize_arabic(text: str) -> str: + """统一阿拉伯语常见字母变体，提升字幕 cue 与脚本行的匹配容错率。 + + edge-tts 对阿拉伯语可能返回与原脚本不同的字母形态，例如把 أ/إ/آ + 归一成 ا，或者携带变音符号。这里仅在最后一层匹配兜底中使用， + 不改变原始字幕文本，避免影响最终展示内容。 + """ + text = _ARABIC_DIACRITICS.sub("", text) + for src, dst in ( + ("أإآٱ", "ا"), + ("ىئ", "ي"), + ("ة", "ه"), + ("ؤ", "و"), + ): + for ch in src: + text = text.replace(ch, dst) + return text + + +def _match_script_line(script_lines: list[str], current_text: str, sub_index: int) -> str: + """ + 尝试把当前累计的字幕文本，与脚本中的某一条标准断句匹配起来。 + + 这里复用了项目原有的“按标点拆脚本，再逐段比对”的思路： + 1. 优先精确匹配； + 2. 再做一次去标点和 Markdown `_` 格式符后的匹配； + 3. 最后做一次阿拉伯语字符形态归一化匹配。 + + 这样可以兼容： + - TTS 返回里可能缺失或单独拆分的标点； + - 中文场景下词边界和脚本文本不完全一一对应的情况。 + """ + if len(script_lines) <= sub_index: + return "" + + target_line = script_lines[sub_index] + if current_text == target_line: + return target_line.strip() + + current_text_normalized = re.sub(r"[_\W]+", "", current_text) + target_line_normalized = re.sub(r"[_\W]+", "", target_line) + if current_text_normalized == target_line_normalized: + return target_line.strip() + + # 最后一层阿拉伯语容错：edge-tts 返回的字母形态、变音符号或 Tatweel + # 可能和脚本不同。只在常规匹配失败后归一化比较，非阿拉伯语文本不会受影响。 + current_ar = re.sub(r"[_\W]+", "", _normalize_arabic(current_text)) + target_ar = re.sub(r"[_\W]+", "", _normalize_arabic(target_line)) + if current_ar and current_ar == target_ar: + return target_line.strip() + + return "" + + +def _write_subtitle_items(sub_items: list[str], subtitle_file: str) -> bool: + """ + 将已经聚合好的字幕段写入到 SRT 文件，并做一次基本可读性验证。 + + 返回值： + - `True`：字幕文件成功落盘且可被 moviepy 解析； + - `False`：字幕文件写入或解析失败。 + """ + try: + ensure_file_path_exists(subtitle_file) + with open(subtitle_file, "w", encoding="utf-8") as file: + file.write("\n".join(sub_items) + "\n") + + sbs = subtitles.file_to_subtitles(subtitle_file, encoding="utf-8") + duration = max([tb for ((ta, tb), txt) in sbs]) if sbs else 0 + logger.info( + f"completed, subtitle file created: {subtitle_file}, duration: {duration}" + ) + return True + except Exception as e: + logger.error(f"failed, error: {str(e)}") + if os.path.exists(subtitle_file): + os.remove(subtitle_file) + return False + + +def _build_subtitle_items_from_edge_cues( + sub_maker: SubMaker, script_lines: list[str] +) -> list[str]: + """ + 将 edge_tts 7.x 的细粒度 `cues` 聚合为按脚本断句的 SRT 片段。 + + 背景： + edge_tts 7.x 的 `SubMaker.get_srt()` 更偏向逐词/逐短语的时间轴。 + 对英文做逐词高亮尚可，但中文短视频字幕如果直接照搬，会出现 + “金钱 / 是 / 一种 / 社会 / 工具” 这种阅读体验很差的效果。 + + 实现策略： + 1. 逐个消费 cues 中的 `content`； + 2. 累积成一段候选文本； + 3. 当候选文本与脚本里当前目标断句匹配时，收敛为一个完整字幕段； + 4. 使用第一条 cue 的开始时间和最后一条 cue 的结束时间，保证时间轴连续。 + """ + formatter = _build_subtitle_formatter() + sub_items = [] + sub_index = 0 + current_text = "" + current_start_time = None + + for cue in sub_maker.cues: + cue_text = unescape(cue.content) + if current_start_time is None: + current_start_time = int(cue.start.total_seconds() * 10000000) + + current_end_time = int(cue.end.total_seconds() * 10000000) + current_text += cue_text + + matched_text = _match_script_line(script_lines, current_text, sub_index) + if not matched_text: + continue + + sub_index += 1 + sub_items.append( + formatter( + idx=sub_index, + start_time=current_start_time, + end_time=current_end_time, + sub_text=matched_text, + ) + ) + current_text = "" + current_start_time = None + + if current_text.strip(): + logger.warning( + f"edge cues still have unmatched text after aggregation: {current_text}" + ) + + return sub_items + + +def _build_subtitle_items_from_legacy_submaker( + sub_maker: SubMaker, script_lines: list[str] +) -> list[str]: + """ + 将项目原有 `subs/offset` 结构聚合为按脚本断句的 SRT 片段。 + + 这部分保留了原来的核心思路，只是拆成独立函数，便于与 edge_tts 7.x + 的 cues 聚合逻辑共享同一套断句匹配与落盘流程。 + """ + formatter = _build_subtitle_formatter() + start_time = -1.0 + sub_items = [] + sub_index = 0 + sub_line = "" + + legacy_offsets = getattr(sub_maker, "offset", []) + legacy_subs = getattr(sub_maker, "subs", []) + for _, (offset, sub) in enumerate(zip(legacy_offsets, legacy_subs)): + current_start_time, current_end_time = offset + if start_time < 0: + start_time = current_start_time + + sub_line += unescape(sub) + matched_text = _match_script_line(script_lines, sub_line, sub_index) + if not matched_text: + continue + + sub_index += 1 + sub_items.append( + formatter( + idx=sub_index, + start_time=start_time, + end_time=current_end_time, + sub_text=matched_text, + ) + ) + start_time = -1.0 + sub_line = "" + + if sub_line.strip(): + logger.warning( + f"legacy subtitle items still have unmatched text after aggregation: {sub_line}" + ) + + return sub_items + + +def create_subtitle(sub_maker: SubMaker, text: str, subtitle_file: str): + """ + 优化字幕文件 + 1. 将字幕文件按照标点符号分割成多行 + 2. 逐行匹配字幕文件中的文本 + 3. 生成新的字幕文件 + """ + text = _format_text(text) + script_lines = utils.split_string_by_punctuations(text) + try: + if hasattr(sub_maker, "cues") and sub_maker.cues: + sub_items = _build_subtitle_items_from_edge_cues(sub_maker, script_lines) + else: + sub_items = _build_subtitle_items_from_legacy_submaker( + sub_maker, script_lines + ) + + if len(sub_items) != len(script_lines): + logger.warning( + f"failed, sub_items len: {len(sub_items)}, script_lines len: {len(script_lines)}" + ) + return + + _write_subtitle_items(sub_items, subtitle_file) + except Exception as e: + logger.error(f"failed, error: {str(e)}") + + +def _get_audio_duration_from_submaker(sub_maker: SubMaker): + """ + 获取音频时长 + """ + # 优先兼容 edge_tts 7.x 的 cues 结构； + # 如果是项目里其他 TTS 手工填充的旧结构，则继续读取 offset。 + if hasattr(sub_maker, "cues") and sub_maker.cues: + return sub_maker.cues[-1].end.total_seconds() + + legacy_offsets = getattr(sub_maker, "offset", []) + if not legacy_offsets: + return 0.0 + return legacy_offsets[-1][1] / 10000000 + +def _get_audio_duration_from_file(audio_file: str) -> float: + """ + 获取音频文件时长（支持 mp3/m4a/wav/aac 等 ffmpeg 可解码的格式） + """ + if not os.path.exists(audio_file): + logger.error(f"audio file does not exist: {audio_file}") + return 0.0 + + try: + # Use moviepy (ffmpeg) to read the duration of any supported audio format + with AudioFileClip(audio_file) as audio: + return audio.duration # Duration in seconds + except Exception as e: + logger.error(f"Failed to get audio duration from file: {str(e)}") + return 0.0 + +def get_audio_duration(target: Union[str, SubMaker]) -> float: + """ + 获取音频时长 + 如果是SubMaker对象，则从SubMaker中获取时长 + 如果是音频文件路径，则从音频文件中获取时长（支持 mp3/m4a/wav 等格式） + """ + if isinstance(target, SubMaker): + return _get_audio_duration_from_submaker(target) + elif isinstance(target, str): + return _get_audio_duration_from_file(target) + else: + logger.error(f"Invalid target type: {type(target)}") + return 0.0 + +if __name__ == "__main__": + voice_name = "zh-CN-XiaoxiaoMultilingualNeural-V2-Female" + voice_name = parse_voice_name(voice_name) + voice_name = is_azure_v2_voice(voice_name) + print(voice_name) + + voices = get_all_azure_voices() + print(len(voices)) + + async def _do(): + temp_dir = utils.storage_dir("temp") + + voice_names = [ + "zh-CN-XiaoxiaoMultilingualNeural", + # 女性 + "zh-CN-XiaoxiaoNeural", + "zh-CN-XiaoyiNeural", + # 男性 + "zh-CN-YunyangNeural", + "zh-CN-YunxiNeural", + ] + text = """ + 静夜思是唐代诗人李白创作的一首五言古诗。这首诗描绘了诗人在寂静的夜晚，看到窗前的明月，不禁想起远方的家乡和亲人，表达了他对家乡和亲人的深深思念之情。全诗内容是：“床前明月光，疑是地上霜。举头望明月，低头思故乡。”在这短短的四句诗中，诗人通过“明月”和“思故乡”的意象，巧妙地表达了离乡背井人的孤独与哀愁。首句“床前明月光”设景立意，通过明亮的月光引出诗人的遐想；“疑是地上霜”增添了夜晚的寒冷感，加深了诗人的孤寂之情；“举头望明月”和“低头思故乡”则是情感的升华，展现了诗人内心深处的乡愁和对家的渴望。这首诗简洁明快，情感真挚，是中国古典诗歌中非常著名的一首，也深受后人喜爱和推崇。 + """ + + text = """ + What is the meaning of life? This question has puzzled philosophers, scientists, and thinkers of all kinds for centuries. Throughout history, various cultures and individuals have come up with their interpretations and beliefs around the purpose of life. Some say it's to seek happiness and self-fulfillment, while others believe it's about contributing to the welfare of others and making a positive impact in the world. Despite the myriad of perspectives, one thing remains clear: the meaning of life is a deeply personal concept that varies from one person to another. It's an existential inquiry that encourages us to reflect on our values, desires, and the essence of our existence. + """ + + text = """ + 预计未来3天深圳冷空气活动频繁，未来两天持续阴天有小雨，出门带好雨具； + 10-11日持续阴天有小雨，日温差小，气温在13-17℃之间，体感阴凉； + 12日天气短暂好转，早晚清凉； + """ + + text = "[Opening scene: A sunny day in a suburban neighborhood. A young boy named Alex, around 8 years old, is playing in his front yard with his loyal dog, Buddy.]\n\n[Camera zooms in on Alex as he throws a ball for Buddy to fetch. Buddy excitedly runs after it and brings it back to Alex.]\n\nAlex: Good boy, Buddy! You're the best dog ever!\n\n[Buddy barks happily and wags his tail.]\n\n[As Alex and Buddy continue playing, a series of potential dangers loom nearby, such as a stray dog approaching, a ball rolling towards the street, and a suspicious-looking stranger walking by.]\n\nAlex: Uh oh, Buddy, look out!\n\n[Buddy senses the danger and immediately springs into action. He barks loudly at the stray dog, scaring it away. Then, he rushes to retrieve the ball before it reaches the street and gently nudges it back towards Alex. Finally, he stands protectively between Alex and the stranger, growling softly to warn them away.]\n\nAlex: Wow, Buddy, you're like my superhero!\n\n[Just as Alex and Buddy are about to head inside, they hear a loud crash from a nearby construction site. They rush over to investigate and find a pile of rubble blocking the path of a kitten trapped underneath.]\n\nAlex: Oh no, Buddy, we have to help!\n\n[Buddy barks in agreement and together they work to carefully move the rubble aside, allowing the kitten to escape unharmed. The kitten gratefully nuzzles against Buddy, who responds with a friendly lick.]\n\nAlex: We did it, Buddy! We saved the day again!\n\n[As Alex and Buddy walk home together, the sun begins to set, casting a warm glow over the neighborhood.]\n\nAlex: Thanks for always being there to watch over me, Buddy. You're not just my dog, you're my best friend.\n\n[Buddy barks happily and nuzzles against Alex as they disappear into the sunset, ready to face whatever adventures tomorrow may bring.]\n\n[End scene.]" + + text = "大家好，我是乔哥，一个想帮你把信用卡全部还清的家伙！\n今天我们要聊的是信用卡的取现功能。\n你是不是也曾经因为一时的资金紧张，而拿着信用卡到ATM机取现？如果是，那你得好好看看这个视频了。\n现在都2024年了，我以为现在不会再有人用信用卡取现功能了。前几天一个粉丝发来一张图片，取现1万。\n信用卡取现有三个弊端。\n一，信用卡取现功能代价可不小。会先收取一个取现手续费，比如这个粉丝，取现1万，按2.5%收取手续费，收取了250元。\n二，信用卡正常消费有最长56天的免息期，但取现不享受免息期。从取现那一天开始，每天按照万5收取利息，这个粉丝用了11天，收取了55元利息。\n三，频繁的取现行为，银行会认为你资金紧张，会被标记为高风险用户，影响你的综合评分和额度。\n那么，如果你资金紧张了，该怎么办呢？\n乔哥给你支一招，用破思机摩擦信用卡，只需要少量的手续费，而且还可以享受最长56天的免息期。\n最后，如果你对玩卡感兴趣，可以找乔哥领取一本《卡神秘籍》，用卡过程中遇到任何疑惑，也欢迎找乔哥交流。\n别忘了，关注乔哥，回复用卡技巧，免费领取《2024用卡技巧》，让我们一起成为用卡高手！" + + text = """ + 2023全年业绩速览 +公司全年累计实现营业收入1476.94亿元，同比增长19.01%，归母净利润747.34亿元，同比增长19.16%。EPS达到59.49元。第四季度单季，营业收入444.25亿元，同比增长20.26%，环比增长31.86%；归母净利润218.58亿元，同比增长19.33%，环比增长29.37%。这一阶段 +的业绩表现不仅突显了公司的增长动力和盈利能力，也反映出公司在竞争激烈的市场环境中保持了良好的发展势头。 +2023年Q4业绩速览 +第四季度，营业收入贡献主要增长点；销售费用高增致盈利能力承压；税金同比上升27%，扰动净利率表现。 +业绩解读 +利润方面，2023全年贵州茅台，>归母净利润增速为19%，其中营业收入正贡献18%，营业成本正贡献百分之一，管理费用正贡献百分之一点四。(注：归母净利润增速值=营业收入增速+各科目贡献，展示贡献/拖累的前四名科目，且要求贡献值/净利润增速>15%) +""" + text = "静夜思是唐代诗人李白创作的一首五言古诗。这首诗描绘了诗人在寂静的夜晚，看到窗前的明月，不禁想起远方的家乡和亲人" + + text = _format_text(text) + lines = utils.split_string_by_punctuations(text) + print(lines) + + for voice_name in voice_names: + voice_file = f"{temp_dir}/tts-{voice_name}.mp3" + subtitle_file = f"{temp_dir}/tts.mp3.srt" + sub_maker = azure_tts_v2( + text=text, voice_name=voice_name, voice_file=voice_file + ) + create_subtitle(sub_maker=sub_maker, text=text, subtitle_file=subtitle_file) + audio_duration = get_audio_duration(sub_maker) + print(f"voice: {voice_name}, audio duration: {audio_duration}s") + + loop = asyncio.get_event_loop_policy().get_event_loop() + try: + loop.run_until_complete(_do()) + finally: + loop.close() diff --git a/app/utils/file_security.py b/app/utils/file_security.py new file mode 100644 index 0000000..00366fc --- /dev/null +++ b/app/utils/file_security.py @@ -0,0 +1,35 @@ +import os + + +def resolve_path_within_directory( + base_dir: str, + unsafe_path: str, + *, + require_file: bool = True, +) -> str: + # 用户传入的路径可能是文件名、相对路径、绝对路径，也可能夹带 `../`。 + # 这里统一解析成真实路径，并用 commonpath 判断它是否仍在允许目录内。 + # 这样比简单判断字符串前缀可靠，可以覆盖符号链接、重复分隔符、相对路径 + # 等场景，适用于上传目录、素材目录、任务产物目录这类白名单目录。 + if not unsafe_path: + raise ValueError("empty path is not allowed") + + base_dir_real = os.path.realpath(base_dir) + candidate_path = unsafe_path + if not os.path.isabs(candidate_path): + candidate_path = os.path.join(base_dir_real, candidate_path) + + resolved_path = os.path.realpath(candidate_path) + try: + common_path = os.path.commonpath([base_dir_real, resolved_path]) + except ValueError as exc: + # Windows 下不同盘符会触发 ValueError，这类路径一定不属于允许目录。 + raise ValueError("path is outside the allowed directory") from exc + + if common_path != base_dir_real: + raise ValueError("path is outside the allowed directory") + + if require_file and not os.path.isfile(resolved_path): + raise ValueError("file does not exist") + + return resolved_path diff --git a/app/utils/utils.py b/app/utils/utils.py new file mode 100644 index 0000000..196bb96 --- /dev/null +++ b/app/utils/utils.py @@ -0,0 +1,343 @@ +import json +import os +import re +import shutil +from functools import lru_cache +from pathlib import Path +import threading +from typing import Any, Iterable +from uuid import uuid4 + +from loguru import logger + +from app.models import const + + +def get_response(status: int, data: Any = None, message: str = ""): + obj = { + "status": status, + } + if data: + obj["data"] = data + if message: + obj["message"] = message + return obj + + +def to_json(obj): + try: + # Define a helper function to handle different types of objects + def serialize(o): + # If the object is a serializable type, return it directly + if isinstance(o, (int, float, bool, str)) or o is None: + return o + # If the object is binary data, convert it to a base64-encoded string + elif isinstance(o, bytes): + return "*** binary data ***" + # If the object is a dictionary, recursively process each key-value pair + elif isinstance(o, dict): + return {k: serialize(v) for k, v in o.items()} + # If the object is a list or tuple, recursively process each element + elif isinstance(o, (list, tuple)): + return [serialize(item) for item in o] + # If the object is a custom type, attempt to return its __dict__ attribute + elif hasattr(o, "__dict__"): + return serialize(o.__dict__) + # Return None for other cases (or choose to raise an exception) + else: + return None + + # Use the serialize function to process the input object + serialized_obj = serialize(obj) + + # Serialize the processed object into a JSON string + return json.dumps(serialized_obj, ensure_ascii=False, indent=4) + except Exception as e: + logger.error(f"failed to serialize object to json: {str(e)}") + return None + + +def get_uuid(remove_hyphen: bool = False): + u = str(uuid4()) + if remove_hyphen: + u = u.replace("-", "") + return u + + +def root_dir(): + return os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))) + + +def storage_dir(sub_dir: str = "", create: bool = False): + d = os.path.join(root_dir(), "storage") + if sub_dir: + d = os.path.join(d, sub_dir) + if create and not os.path.exists(d): + os.makedirs(d) + + return d + + +def resource_dir(sub_dir: str = ""): + d = os.path.join(root_dir(), "resource") + if sub_dir: + d = os.path.join(d, sub_dir) + return d + + +def task_dir(sub_dir: str = ""): + d = os.path.join(storage_dir(), "tasks") + if sub_dir: + d = os.path.join(d, sub_dir) + if not os.path.exists(d): + os.makedirs(d) + return d + + +def font_dir(sub_dir: str = ""): + d = resource_dir("fonts") + if sub_dir: + d = os.path.join(d, sub_dir) + if not os.path.exists(d): + os.makedirs(d) + return d + + +def song_dir(sub_dir: str = ""): + d = resource_dir("songs") + if sub_dir: + d = os.path.join(d, sub_dir) + if not os.path.exists(d): + os.makedirs(d) + return d + + +def public_dir(sub_dir: str = ""): + d = resource_dir("public") + if sub_dir: + d = os.path.join(d, sub_dir) + if not os.path.exists(d): + os.makedirs(d) + return d + + +def get_ffmpeg_binary() -> str: + """ + 解析当前进程应该使用的 FFmpeg 可执行文件。 + + 增加原因： + 1. 视频编码、静音音频生成、pydub 音频转码都依赖 FFmpeg； + 2. Windows 便携包、Docker 和用户自定义安装目录经常出现 PATH 不一致； + 3. 集中解析可以让所有调用方使用同一套优先级，减少某条链路能跑、 + 另一条链路找不到 FFmpeg 的现场问题。 + + 优先级： + 1. IMAGEIO_FFMPEG_EXE：MoviePy/imageio 约定的显式配置； + 2. 系统 PATH 中的 ffmpeg； + 3. imageio-ffmpeg 依赖提供的内置二进制； + 4. 字符串 "ffmpeg" 兜底，交给 subprocess 在运行时暴露更具体错误。 + """ + configured_ffmpeg = os.environ.get("IMAGEIO_FFMPEG_EXE") + if configured_ffmpeg: + return configured_ffmpeg + + system_ffmpeg = shutil.which("ffmpeg") + if system_ffmpeg: + return system_ffmpeg + + try: + import imageio_ffmpeg + + bundled_ffmpeg = imageio_ffmpeg.get_ffmpeg_exe() + if bundled_ffmpeg: + return bundled_ffmpeg + except Exception as exc: + logger.warning(f"failed to resolve bundled ffmpeg binary: {str(exc)}") + + return "ffmpeg" + + +def run_in_background(func, *args, **kwargs): + def run(): + try: + func(*args, **kwargs) + except Exception as e: + logger.error(f"run_in_background error: {e}", exc_info=True) + + thread = threading.Thread(target=run, daemon=False) + thread.start() + return thread + + +def time_convert_seconds_to_hmsm(seconds) -> str: + hours = int(seconds // 3600) + seconds = seconds % 3600 + minutes = int(seconds // 60) + milliseconds = int(seconds * 1000) % 1000 + seconds = int(seconds % 60) + return "{:02d}:{:02d}:{:02d},{:03d}".format(hours, minutes, seconds, milliseconds) + + +def text_to_srt(idx: int, msg: str, start_time: float, end_time: float) -> str: + start_time = time_convert_seconds_to_hmsm(start_time) + end_time = time_convert_seconds_to_hmsm(end_time) + srt = """%d +%s --> %s +%s + """ % ( + idx, + start_time, + end_time, + msg, + ) + return srt + + +def str_contains_punctuation(word): + for p in const.PUNCTUATIONS: + if p in word: + return True + return False + + +def split_string_by_punctuations(s): + result = [] + txt = "" + + previous_char = "" + next_char = "" + for i in range(len(s)): + char = s[i] + if char == "\n": + result.append(txt.strip()) + txt = "" + continue + + if i > 0: + previous_char = s[i - 1] + if i < len(s) - 1: + next_char = s[i + 1] + + if char == "." and previous_char.isdigit() and next_char.isdigit(): + # # In the case of "withdraw 10,000, charged at 2.5% fee", the dot in "2.5" should not be treated as a line break marker + txt += char + continue + + if char == "," and previous_char.isdigit() and next_char.isdigit(): + # 英文数字里的千分位逗号不是断句符，例如 "1,000 years"。 + # Edge TTS 的 word boundary 通常会把这种数字整体作为连续内容返回； + # 如果这里拆成 "1" 和 "000 years"，后续字幕聚合会无法匹配脚本原文， + # 进而错误回退到 Whisper。 + txt += char + continue + + if char not in const.PUNCTUATIONS: + txt += char + else: + result.append(txt.strip()) + txt = "" + result.append(txt.strip()) + # filter empty string + result = list(filter(None, result)) + return result + + +def normalize_script_for_subtitle_matching(video_script: str) -> str: + """ + 清理字幕匹配前的脚本文本。 + + 用户可能手动输入 Markdown 分隔符、标题强调或 `_` 这类格式符号。 + 这些字符通常不会出现在 TTS/Whisper 的识别结果里；如果继续参与 + 字幕逐行匹配，脚本行数量会大于真实字幕行数量，最终可能补出 + `00:00:00,000 --> 00:00:00,000`，导致剪辑软件无法导入 SRT。 + """ + video_script = video_script or "" + underscore_count = video_script.count("_") + video_script = video_script.replace("_", "") + cleaned_lines = [] + removed_separator_lines = 0 + for line in video_script.splitlines(): + line = line.strip() + # Markdown 分隔符或强调符号单独成行时不会被 TTS 朗读，必须从 + # 脚本行里移除，避免字幕聚合卡在这类“不可发声”的目标行上。 + if re.fullmatch(r"[-*_]{3,}", line): + removed_separator_lines += 1 + continue + cleaned_lines.append(line) + + normalized_script = "\n".join(cleaned_lines).strip() + if underscore_count or removed_separator_lines: + logger.debug( + "normalized script for subtitle matching, " + f"removed underscores: {underscore_count}, " + f"removed markdown separator lines: {removed_separator_lines}" + ) + return normalized_script + + +def md5(text): + import hashlib + + return hashlib.md5(text.encode("utf-8")).hexdigest() + + +def resolve_ui_language( + saved_language: str | None, + browser_locale: str | None, + supported_languages: Iterable[str], + default_language: str = "en", +) -> str: + """ + 按“已保存设置、浏览器语言、默认语言”的优先级选择界面语言。 + + 浏览器通常返回带地区的 locale，例如 ``zh-CN``、``pt-BR``。语言文件使用 + ``zh``、``pt`` 这类基础代码，因此先尝试完整匹配，再回退到连字符前的语言 + 代码。函数保持纯逻辑，避免把浏览器上下文和配置写入耦合到工具层，便于测试。 + """ + supported = [str(language).strip() for language in supported_languages] + supported_by_lower = { + language.lower(): language for language in supported if language + } + + def match_language(value: str | None) -> str | None: + normalized = str(value or "").strip().replace("_", "-").lower() + if not normalized: + return None + if normalized in supported_by_lower: + return supported_by_lower[normalized] + base_language = normalized.split("-", 1)[0] + return supported_by_lower.get(base_language) + + saved_match = match_language(saved_language) + if saved_match: + return saved_match + + browser_match = match_language(browser_locale) + if browser_match: + return browser_match + + default_match = match_language(default_language) + if default_match: + return default_match + + # 正常项目始终包含英文；保留空语言集合兜底，避免损坏的语言目录让页面 + # 初始化直接抛异常，后续翻译函数会继续显示原始 key 以便诊断。 + return supported[0] if supported else default_language + + +@lru_cache(maxsize=8) +def load_locales(i18n_dir): + # WebUI 每次交互都会触发 Streamlit 重新执行脚本，语言文件运行期不会变化， + # 因此缓存解析结果，避免反复读取和解析所有 i18n JSON 文件。 + _locales = {} + for root, dirs, files in os.walk(i18n_dir): + for file in files: + if file.endswith(".json"): + lang = file.split(".")[0] + with open(os.path.join(root, file), "r", encoding="utf-8") as f: + _locales[lang] = json.loads(f.read()) + return _locales + + +def parse_extension(filename): + return Path(filename).suffix.lower().lstrip('.') diff --git a/cli.py b/cli.py new file mode 100644 index 0000000..781339c --- /dev/null +++ b/cli.py @@ -0,0 +1,757 @@ +from __future__ import annotations + +import argparse +import json +import math +import os +import re +import shutil +from typing import TYPE_CHECKING, Sequence +from uuid import UUID, uuid4 + +from loguru import logger + +if TYPE_CHECKING: + from app.models.schema import VideoParams + + +DEFAULT_VOICE_NAME = "zh-CN-XiaoxiaoNeural-Female" +_PIPELINE_STAGES = ("script", "terms", "audio", "subtitle", "materials", "video") +_CUSTOM_AUDIO_EXTENSIONS = {".mp3", ".wav", ".m4a", ".aac", ".flac", ".ogg"} + + +class _CliHelpFormatter( + argparse.ArgumentDefaultsHelpFormatter, + argparse.RawDescriptionHelpFormatter, +): + """在保留多行示例排版的同时，自动展示有意义的默认值。""" + + def _get_help_string(self, action): + help_text = action.help or "" + if ( + "%(default)" not in help_text + and action.default not in (None, "", argparse.SUPPRESS) + and action.option_strings + and "default:" not in help_text.lower() + ): + help_text += " (default: %(default)s)" + return help_text + + +def _positive_int(value: str) -> int: + parsed = int(value) + if parsed < 1: + raise argparse.ArgumentTypeError(f"value must be >= 1, got {parsed}") + return parsed + + +def _paragraph_count(value: str) -> int: + parsed = int(value) + if parsed < 1 or parsed > 10: + raise argparse.ArgumentTypeError( + f"paragraph-number must be between 1 and 10, got {parsed}" + ) + return parsed + + +def _non_negative_float(value: str) -> float: + parsed = float(value) + if not math.isfinite(parsed) or parsed < 0: + raise argparse.ArgumentTypeError(f"value must be a finite number >= 0, got {value!r}") + return parsed + + +def _positive_float(value: str) -> float: + parsed = float(value) + if not math.isfinite(parsed) or parsed <= 0: + raise argparse.ArgumentTypeError(f"value must be a finite number > 0, got {value!r}") + return parsed + + +def _percent_position(value: str) -> float: + parsed = float(value) + if not math.isfinite(parsed) or parsed < 0 or parsed > 100: + raise argparse.ArgumentTypeError( + f"custom-position must be a finite number between 0 and 100, got {value!r}" + ) + return parsed + + +def _hex_color(value: str) -> str: + if not re.fullmatch(r"#[0-9a-fA-F]{6}", value): + raise argparse.ArgumentTypeError( + f"color must use #RRGGBB format, got {value!r}" + ) + return value + + +def _task_id(value: str) -> str: + """CLI 自定义任务标识只接受 UUID，避免该值被解释为文件系统路径。""" + try: + return str(UUID(value.strip())) + except (AttributeError, ValueError) as exc: + raise argparse.ArgumentTypeError( + f"task-id must be a valid UUID, got {value!r}" + ) from exc + + +_TRANSITION_MODE_VALUES = { + "none": None, + "shuffle": "Shuffle", + "fade-in": "FadeIn", + "fade-out": "FadeOut", + "slide-in": "SlideIn", + "slide-out": "SlideOut", +} + + +def _transition_mode(value: str) -> str | None: + normalized = value.strip().lower() + if normalized not in _TRANSITION_MODE_VALUES: + allowed = ", ".join(_TRANSITION_MODE_VALUES) + raise argparse.ArgumentTypeError( + f"video-transition-mode must be one of: {allowed}" + ) + return _TRANSITION_MODE_VALUES[normalized] + + +def _bgm_type(value: str) -> str: + normalized = value.strip().lower() + if normalized == "none": + return "" + if normalized in {"", "random", "custom"}: + return normalized + raise argparse.ArgumentTypeError("bgm-type must be one of: none, random, custom") + + +def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace: + parser = argparse.ArgumentParser( + description=( + "Generate MoneyPrinterTurbo videos without the WebUI.\n\n" + "Provider settings and credentials are read from config.toml.\n" + "Default full-video generation requires a configured LLM and Pexels API key.\n" + "The default Edge TTS voice requires no API key." + ), + epilog=""" +Examples: + Generate a complete video with the default Edge TTS voice: + uv run python cli.py --video-subject "How AI is changing everyday life" + + Generate from local files. Relative paths use the current working directory; + absolute paths are also accepted: + uv run python cli.py --video-subject "How AI is changing everyday life" \\ + --video-source local --video-materials "./1.mp4,./2.mp4" + + Generate with a prepared script and no voiceover: + uv run python cli.py --video-script "Your complete script" \\ + --voice-name no-voice --stop-at video + + Stop after script generation: + uv run python cli.py --video-subject "How AI is changing everyday life" --stop-at script + +Pipeline stages: + script Generate or return the script. + terms Generate material search terms; unavailable with local materials. + audio Generate TTS, silent audio, or use --custom-audio-file. + subtitle Generate subtitles when enabled. + materials Download online materials or preprocess local files. + video Generate the final video and run configured cross-posting. + The command stops immediately after the selected stage and prints that stage's result. + +Output and exit status: + Task files are written to storage/tasks//. A successful command prints one + JSON object to stdout and exits with 0. Task failures exit with 1; argument errors + exit with 2. Runtime logs are written to stderr. +""", + formatter_class=_CliHelpFormatter, + ) + + content_group = parser.add_argument_group("script and content") + content_group.add_argument( + "--video-subject", + default="", + help="video topic; required unless --video-script is provided", + ) + content_group.add_argument( + "--video-script", + default="", + help="complete script; skips LLM script generation when provided", + ) + content_group.add_argument( + "--video-terms", + default=None, + help="comma-separated material search terms; generated automatically when omitted", + ) + content_group.add_argument( + "--video-language", + default=None, + help=( + "script language code, such as zh-CN or en-US (default: auto-detect)" + ), + ) + content_group.add_argument( + "--paragraph-number", + type=_paragraph_count, + default=None, + help="number of generated script paragraphs, from 1 to 10 (default: 1)", + ) + content_group.add_argument( + "--video-script-prompt", + default=None, + help="additional requirements for LLM script generation", + ) + content_group.add_argument( + "--custom-system-prompt", + default=None, + help="replace the default LLM system prompt for script generation", + ) + + material_group = parser.add_argument_group("materials and pipeline") + material_group.add_argument( + "--video-source", + default="pexels", + choices=["pexels", "pixabay", "coverr", "local"], + help="video material provider; online providers require matching API keys in config.toml", + ) + material_group.add_argument( + "--video-materials", + default="", + metavar="PATH[,PATH...]", + help=( + "comma-separated local image/video paths for --video-source local; relative " + "paths use the current working directory, then storage/local_videos as a " + "compatibility fallback; absolute paths are accepted" + ), + ) + material_group.add_argument( + "--stop-at", + default="video", + choices=_PIPELINE_STAGES, + help="stop after this pipeline stage; see the stage order below", + ) + + video_group = parser.add_argument_group("video output") + video_group.add_argument( + "--video-count", + type=_positive_int, + default=1, + help="number of output videos, at least 1", + ) + video_group.add_argument( + "--video-aspect", + choices=["9:16", "16:9", "1:1"], + default="9:16", + help="output aspect ratio: portrait, landscape, or square", + ) + video_group.add_argument( + "--video-concat-mode", + choices=["random", "sequential"], + default=None, + help="source clip concatenation order (default: random)", + ) + video_group.add_argument( + "--video-transition-mode", + type=_transition_mode, + default=None, + metavar="{none,shuffle,fade-in,fade-out,slide-in,slide-out}", + help="transition applied between source clips (default: none)", + ) + video_group.add_argument( + "--video-clip-duration", + type=_positive_int, + default=None, + help=( + "maximum duration of each source clip in seconds, at least 1 (default: 5)" + ), + ) + video_group.add_argument( + "--match-materials-to-script", + default=None, + action=argparse.BooleanOptionalAction, + help=( + "preserve script keyword order while selecting and concatenating materials " + "(default: disabled)" + ), + ) + video_group.add_argument( + "--n-threads", + type=_positive_int, + default=None, + help="FFmpeg worker thread count, at least 1 (default: 2)", + ) + + audio_group = parser.add_argument_group("voiceover and background music") + audio_group.add_argument( + "--voice-name", + default=DEFAULT_VOICE_NAME, + help=( + "TTS voice identifier; use 'no-voice' for silent output. Provider-specific " + "identifiers use prefixes such as gemini:, mimo:, elevenlabs:, and chatterbox:" + ), + ) + audio_group.add_argument( + "--voice-volume", + type=_non_negative_float, + default=None, + help=( + "final voiceover volume multiplier, a finite number >= 0 (default: 1.0)" + ), + ) + audio_group.add_argument( + "--voice-rate", + type=_positive_float, + default=None, + help=( + "speech rate multiplier, a finite number > 0 (default: 1.0)" + ), + ) + audio_group.add_argument( + "--custom-audio-file", + default=None, + metavar="PATH", + help=( + "existing MP3/WAV/M4A/AAC/FLAC/OGG voiceover; relative paths use the " + "current working directory. This skips TTS; set subtitle_provider=whisper " + "to transcribe it" + ), + ) + audio_group.add_argument( + "--bgm-type", + type=_bgm_type, + default=None, + metavar="{none,random,custom}", + help=( + "background music mode; --bgm-file implies custom when omitted " + "(default: random)" + ), + ) + audio_group.add_argument( + "--bgm-file", + default=None, + metavar="PATH", + help=( + "custom MP3 inside resource/songs; accepts a filename or a path " + "relative to the project root" + ), + ) + audio_group.add_argument( + "--bgm-volume", + type=_non_negative_float, + default=None, + help=( + "background music volume multiplier, a finite number >= 0 (default: 0.2)" + ), + ) + + subtitle_group = parser.add_argument_group("subtitles") + subtitle_group.add_argument( + "--subtitle-enabled", + default=True, + action=argparse.BooleanOptionalAction, + help=( + "enable subtitles; use --no-subtitle-enabled to disable " + "(default: enabled)" + ), + ) + subtitle_group.add_argument( + "--font-name", + default=None, + help=( + "subtitle font filename inside resource/fonts " + "(default: STHeitiMedium.ttc)" + ), + ) + subtitle_group.add_argument( + "--subtitle-position", + choices=["top", "center", "bottom", "custom"], + default=None, + help=( + "subtitle vertical position (default: [ui].subtitle_position from " + "config.toml; bottom when unset)" + ), + ) + subtitle_group.add_argument( + "--custom-position", + type=_percent_position, + default=None, + help=( + "custom position as percent from top, 0-100; requires " + "--subtitle-position custom (default: [ui].custom_position from " + "config.toml; 70 when unset)" + ), + ) + subtitle_group.add_argument( + "--text-fore-color", + type=_hex_color, + default=None, + help=( + "subtitle text color in #RRGGBB format; quote the value in shells " + "that treat # as a comment (default: #FFFFFF)" + ), + ) + subtitle_group.add_argument( + "--font-size", + type=_positive_int, + default=None, + help="subtitle font size (default: 60)", + ) + subtitle_group.add_argument( + "--stroke-color", + type=_hex_color, + default=None, + help="subtitle outline color in #RRGGBB format (default: #000000)", + ) + subtitle_group.add_argument( + "--stroke-width", + type=_non_negative_float, + default=None, + help=( + "subtitle outline width, a finite number >= 0 (default: 1.5)" + ), + ) + subtitle_group.add_argument( + "--subtitle-background-enabled", + default=None, + action=argparse.BooleanOptionalAction, + help=( + "enable subtitle background; use --no-subtitle-background-enabled to " + "disable (default: enabled)" + ), + ) + subtitle_group.add_argument( + "--subtitle-background-color", + type=_hex_color, + default=None, + help="subtitle background color in #RRGGBB format (default: #000000)", + ) + subtitle_group.add_argument( + "--rounded-subtitle-background", + default=None, + action=argparse.BooleanOptionalAction, + help="use a rounded subtitle background (default: disabled)", + ) + + execution_group = parser.add_argument_group("execution") + execution_group.add_argument( + "--task-id", + type=_task_id, + default=None, + help="custom UUID used for storage/tasks/; generated automatically when omitted", + ) + args = parser.parse_args(argv) + + if not args.video_subject.strip() and not args.video_script.strip(): + parser.error("one of --video-subject or --video-script is required") + + if args.video_source == "local" and args.stop_at == "terms": + parser.error( + "--stop-at terms has no effect with --video-source local " + "(search terms are not generated for local sources)" + ) + + stage_requires_materials = args.stop_at in {"materials", "video"} + has_video_materials = bool((args.video_materials or "").strip()) + if args.video_source == "local" and stage_requires_materials and not has_video_materials: + parser.error( + "--video-materials is required with --video-source local when " + "--stop-at is materials or video" + ) + if args.video_source != "local" and has_video_materials: + parser.error("--video-materials can only be used with --video-source local") + + if args.bgm_file: + if args.bgm_type in (None, "custom"): + args.bgm_type = "custom" + else: + parser.error("--bgm-file cannot be combined with --bgm-type none or random") + elif args.bgm_type == "custom": + parser.error("--bgm-file is required when --bgm-type is custom") + + if args.custom_position is not None and args.subtitle_position != "custom": + parser.error("--custom-position requires --subtitle-position custom") + if args.stop_at == "subtitle" and not args.subtitle_enabled: + parser.error("--stop-at subtitle cannot be combined with --no-subtitle-enabled") + if args.subtitle_background_enabled is False and ( + args.subtitle_background_color is not None + or args.rounded_subtitle_background is True + ): + parser.error( + "subtitle background color or rounding cannot be enabled together with " + "--no-subtitle-background-enabled" + ) + + return args + + +def build_video_params(args: argparse.Namespace) -> VideoParams: + # 参数帮助和校验不需要加载应用配置。仅在真正构建任务参数时导入模型， + # 避免执行 ``cli.py -h`` 时产生配置初始化日志。 + from app.models.schema import MaterialInfo, VideoParams + + video_terms = args.video_terms + if video_terms: + video_terms = [ + term.strip() for term in re.split(r"[,，]", video_terms) if term.strip() + ] + + video_materials = None + materials_arg = args.video_materials or "" + if materials_arg.strip(): + video_materials = [ + # Actual duration will be detected during video processing; use 0 as placeholder. + MaterialInfo(provider="local", url=item.strip(), duration=0) + for item in materials_arg.split(",") + if item.strip() + ] + + params_kwargs = { + "video_subject": args.video_subject.strip(), + "video_script": args.video_script, + "video_terms": video_terms, + "video_source": args.video_source, + "video_materials": video_materials, + "video_count": args.video_count, + "video_aspect": args.video_aspect, + "voice_name": args.voice_name, + "subtitle_enabled": args.subtitle_enabled, + } + + optional_arg_names = [ + "video_language", + "paragraph_number", + "video_script_prompt", + "custom_system_prompt", + "video_concat_mode", + "video_transition_mode", + "video_clip_duration", + "match_materials_to_script", + "n_threads", + "voice_volume", + "voice_rate", + "custom_audio_file", + "bgm_type", + "bgm_file", + "bgm_volume", + "font_name", + "subtitle_position", + "custom_position", + "text_fore_color", + "font_size", + "stroke_color", + "stroke_width", + "rounded_subtitle_background", + ] + for name in optional_arg_names: + value = getattr(args, name) + if value is not None: + params_kwargs[name] = value + + if args.subtitle_background_enabled is False: + params_kwargs["text_background_color"] = False + params_kwargs["rounded_subtitle_background"] = False + elif args.subtitle_background_color is not None: + params_kwargs["text_background_color"] = args.subtitle_background_color + elif args.subtitle_background_enabled is True: + params_kwargs["text_background_color"] = True + + return VideoParams(**params_kwargs) + + +def _resolve_cli_file( + raw_path: str, + *, + description: str, + fallback_dir: str | None = None, +) -> str: + """ + 将 CLI 文件参数按当前工作目录解析为绝对路径， + 并在任务开始前确认存在。 + + 本地素材旧版本始终相对 ``storage/local_videos`` 解析。为兼容已有脚本， + 当前目录找不到相对路径时允许回退该目录；绝对路径始终按用户输入 + 直接解析。 + """ + expanded_path = os.path.expanduser(raw_path.strip()) + if not expanded_path: + raise ValueError(f"{description} path cannot be empty") + + candidate = ( + expanded_path + if os.path.isabs(expanded_path) + else os.path.join(os.getcwd(), expanded_path) + ) + resolved_path = os.path.realpath(candidate) + if not os.path.isfile(resolved_path) and fallback_dir and not os.path.isabs(expanded_path): + resolved_path = os.path.realpath(os.path.join(fallback_dir, expanded_path)) + + if not os.path.isfile(resolved_path): + raise ValueError(f"{description} file does not exist: {raw_path}") + return resolved_path + + +def _path_is_within_directory(file_path: str, directory: str) -> bool: + try: + return os.path.commonpath( + [os.path.realpath(directory), os.path.realpath(file_path)] + ) == os.path.realpath(directory) + except ValueError: + # Windows 不同盘符无法计算 commonpath，此时文件显然不在目标目录内。 + return False + + +def _resolve_managed_resource_file( + raw_path: str, + *, + resource_dir: str, + description: str, +) -> str: + """解析项目资源文件，并确保绝对路径仍位于对应资源目录内。""" + from app.utils import utils + + expanded_path = os.path.expanduser(raw_path.strip()) + candidates = ( + [expanded_path] + if os.path.isabs(expanded_path) + else [ + os.path.join(resource_dir, expanded_path), + os.path.join(utils.root_dir(), expanded_path), + ] + ) + for candidate in candidates: + resolved_path = os.path.realpath(candidate) + if os.path.isfile(resolved_path) and _path_is_within_directory( + resolved_path, resource_dir + ): + return resolved_path + raise ValueError( + f"{description} file must exist inside {resource_dir}: {raw_path}" + ) + + +def prepare_cli_files(params: VideoParams, stop_at: str) -> None: + """ + 在调用 LLM/TTS 前准备 CLI 文件，避免长流程运行到后期才报告路径错误。 + + 服务层为了保护 API 请求，只允许读取 ``storage/local_videos`` 内的素材。 + CLI 是本地入口，接受当前目录相对路径和绝对路径。目录外素材会 + 复制到受控目录，再把参数替换为服务层可安全使用的绝对路径。 + """ + from app.models import const + from app.utils import utils + + local_material_extensions = { + *(f".{extension}" for extension in const.FILE_TYPE_VIDEOS), + *(f".{extension}" for extension in const.FILE_TYPE_IMAGES), + ".avi", + ".flv", + } + + if params.custom_audio_file: + params.custom_audio_file = _resolve_cli_file( + params.custom_audio_file, + description="custom audio", + ) + audio_extension = os.path.splitext(params.custom_audio_file)[1].lower() + if audio_extension not in _CUSTOM_AUDIO_EXTENSIONS: + allowed = ", ".join(sorted(_CUSTOM_AUDIO_EXTENSIONS)) + raise ValueError( + f"unsupported custom audio type {audio_extension or ''}; " + f"allowed extensions: {allowed}" + ) + + if params.bgm_type == "custom" and params.bgm_file: + params.bgm_file = _resolve_managed_resource_file( + params.bgm_file, + resource_dir=utils.song_dir(), + description="background music", + ) + if not params.bgm_file.lower().endswith(".mp3"): + raise ValueError("background music must use the .mp3 extension") + + if params.subtitle_enabled and params.font_name and stop_at == "video": + font_path = _resolve_managed_resource_file( + params.font_name, + resource_dir=utils.font_dir(), + description="subtitle font", + ) + if not font_path.lower().endswith((".ttf", ".ttc")): + raise ValueError("subtitle font must use the .ttf or .ttc extension") + # 下游根据 resource/fonts 内的文件名拼接路径，因此仍保留纯文件名。 + params.font_name = os.path.basename(font_path) + + if params.video_source != "local" or stop_at not in {"materials", "video"}: + return + + local_videos_dir = utils.storage_dir("local_videos", create=True) + resolved_materials: list[tuple[MaterialInfo, str, str]] = [] + for material in params.video_materials or []: + source_path = _resolve_cli_file( + material.url, + description="local material", + fallback_dir=local_videos_dir, + ) + extension = os.path.splitext(source_path)[1].lower() + if extension not in local_material_extensions: + allowed = ", ".join(sorted(local_material_extensions)) + raise ValueError( + f"unsupported local material type {extension or ''}: " + f"{material.url}; allowed extensions: {allowed}" + ) + resolved_materials.append((material, source_path, extension)) + + # 所有输入检查通过后再复制，避免第二个文件无效时留下第一个文件的 + # 孤儿副本。 + prepared_paths: dict[str, str] = {} + for material, source_path, extension in resolved_materials: + prepared_path = prepared_paths.get(source_path) + if prepared_path is None: + if _path_is_within_directory(source_path, local_videos_dir): + prepared_path = source_path + else: + prepared_path = os.path.join( + local_videos_dir, + f"cli-material-{uuid4().hex}{extension}", + ) + shutil.copy2(source_path, prepared_path) + logger.info( + "copied CLI local material into managed storage: " + f"source={source_path}, target={prepared_path}" + ) + prepared_paths[source_path] = prepared_path + + material.url = prepared_path + + +def run_cli(argv: Sequence[str] | None = None) -> int: + args = parse_args(argv) + try: + params = build_video_params(args) + prepare_cli_files(params, stop_at=args.stop_at) + except (ValueError, OSError) as exc: + logger.error(f"invalid CLI input: {exc}") + return 2 + + # 帮助参数会在 parse_args 中直接退出。把业务服务延迟到这里导入， + # 保证 -h/--help 输出干净，同时不改变实际任务的初始化流程。 + from app.services import task as tm + from app.utils import utils + + task_id = args.task_id or utils.get_uuid() + logger.info(f"start CLI task: task_id={task_id}, stop_at={args.stop_at}") + try: + result = tm.start(task_id=task_id, params=params, stop_at=args.stop_at) + except Exception as exc: + logger.exception( + f"CLI task failed with an unexpected error: task_id={task_id}, error={exc}" + ) + return 1 + if not result: + logger.error(f"CLI task failed: task_id={task_id}, stop_at={args.stop_at}") + return 1 + + print(json.dumps({"task_id": task_id, "result": result}, ensure_ascii=False)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(run_cli()) diff --git a/config.example.toml b/config.example.toml new file mode 100644 index 0000000..4edb6cc --- /dev/null +++ b/config.example.toml @@ -0,0 +1,334 @@ +# MoneyPrinterTurbo configuration example. +# The app copies this file to config.toml on first run. Keep API keys and other +# credentials in config.toml; do not commit your local config.toml. + +# ============================================================================= +# API Service / API 服务 +# ============================================================================= +# Log level: DEBUG, INFO, WARNING, or ERROR. +log_level = "DEBUG" +# Use 127.0.0.1 for local-only access. 0.0.0.0 listens on all network interfaces. +listen_host = "0.0.0.0" +listen_port = 8080 + +[app] + +# ----------------------------------------------------------------------------- +# General / 通用设置 +# ----------------------------------------------------------------------------- + +# Hide the basic configuration panel in the WebUI. +# 是否隐藏 WebUI 基础配置面板。 +hide_config = false + +# Timeout in seconds for a single Edge TTS streaming request. +# 单次 Edge TTS 流式请求超时时间。慢网络可适当调大，设置为 0 表示禁用超时。 +edge_tts_timeout = 30 + +# Verify TLS certificates for external APIs and material downloads. +# 默认应保持开启；仅在可信代理或自签名证书环境中临时关闭。 +tls_verify = true + +# ----------------------------------------------------------------------------- +# Video Materials / 视频素材 +# ----------------------------------------------------------------------------- + +# Available values: "pexels", "pixabay", "coverr", "local". +video_source = "pexels" + +# API key lists support key rotation. Use straight ASCII double quotes and +# separate multiple keys with commas, for example: ["key-1", "key-2"]. +# Key 必须使用英文半角双引号，多个 Key 使用英文逗号分隔。 + +# Register at https://www.pexels.com/api/ +pexels_api_keys = [] + +# Register at https://pixabay.com/api/docs/ +pixabay_api_keys = [] + +# Register at https://coverr.co/developers?ctx=header_navigation +coverr_api_keys = [] + +# Optional TwelveLabs integration for semantic material ranking and video QA. +# Install the optional dependency with: uv sync --extra twelvelabs +# Create an API key at https://playground.twelvelabs.io/ +twelvelabs_api_keys = [] +# Reorder search terms using Marengo before downloading materials. This option +# is ignored when match_materials_to_script is enabled. +twelvelabs_rerank_terms = false +# Optional model overrides. Uncomment only when a different model is required. +# twelvelabs_marengo_model = "marengo3.0" +# twelvelabs_pegasus_model = "pegasus1.5" + +# Match search terms and downloaded materials to the script narrative order. +# 默认关闭，开启后会减少随机性，使素材顺序更贴近文案结构。 +match_materials_to_script = false + +# ----------------------------------------------------------------------------- +# LLM Providers / 大模型提供商 +# ----------------------------------------------------------------------------- + +# Provider definitions, default models, and default Base URLs are maintained in +# app/models/llm_provider.py. Leave model_name and base_url empty to follow the +# Registry defaults; only set them when you need an explicit override. +llm_provider = "moonshot" + +# Kimi / Moonshot AI +# API key: https://platform.kimi.com/console/api-keys +moonshot_api_key = "" +moonshot_base_url = "" +moonshot_model_name = "" + +# OpenAI or another OpenAI Chat Completions-compatible provider. +# API key: https://platform.openai.com/api-keys +openai_api_key = "" +openai_base_url = "" +openai_model_name = "" + +# Google Gemini +# API key: https://aistudio.google.com/app/apikey +gemini_api_key = "" +gemini_model_name = "" + +# DeepSeek +# API key: https://platform.deepseek.com/api_keys +deepseek_api_key = "" +deepseek_base_url = "" +deepseek_model_name = "" + +# Alibaba Cloud Qwen +# API key: https://dashscope.console.aliyun.com/apiKey +qwen_api_key = "" +qwen_model_name = "" + +# Microsoft Azure OpenAI. model_name is the deployment name. +# Documentation: https://learn.microsoft.com/azure/ai-services/openai/reference +azure_api_key = "" +azure_base_url = "" +azure_model_name = "" +azure_api_version = "2024-02-15-preview" + +# ByteDance VolcEngine Ark +# Console: https://console.volcengine.com/ark +volcengine_api_key = "" +volcengine_base_url = "" +volcengine_model_name = "" + +# xAI Grok +# API key: https://console.x.ai/ +grok_api_key = "" +grok_base_url = "" +grok_model_name = "" + +# MiniMax. The API key and Base URL must use the same platform. +# China: https://platform.minimaxi.com/ | https://api.minimaxi.com/v1 +# International: https://platform.minimax.io/ | https://api.minimax.io/v1 +minimax_api_key = "" +minimax_base_url = "" +minimax_model_name = "" + +# Xiaomi MiMo +# Documentation: https://platform.xiaomimimo.com/docs/zh-CN/quick-start/first-api-call +mimo_api_key = "" +mimo_base_url = "" +mimo_model_name = "" + +# Cloudflare AI Gateway. The token requires AI Gateway Read/Edit and Workers AI +# Read permissions. Leave gateway_id empty to use the Registry default. +# Dashboard: https://dash.cloudflare.com/ +cloudflare_api_key = "" +cloudflare_account_id = "" +cloudflare_gateway_id = "" +cloudflare_model_name = "" + +# Alibaba ModelScope. Bind an Alibaba Cloud account before using API inference. +# Documentation: https://modelscope.cn/docs/model-service/API-Inference/intro +modelscope_api_key = "" +modelscope_base_url = "" +modelscope_model_name = "" + +# AIHubMix, an OpenAI-compatible multi-model gateway. +# API key: https://aihubmix.com/ +aihubmix_api_key = "" +aihubmix_base_url = "" +aihubmix_model_name = "" + +# AIML API, an OpenAI-compatible multi-model API. +# API key: https://aimlapi.com/app/keys +aimlapi_api_key = "" +aimlapi_base_url = "" +aimlapi_model_name = "" + +# EvoLink, an OpenAI-compatible multi-model gateway. +# API key: https://evolink.ai/dashboard/keys +evolink_api_key = "" +evolink_base_url = "" +evolink_model_name = "" + +# Ollama. Leave base_url empty to use the environment-aware local default. +# Set model_name to a model installed by `ollama pull` or shown by `ollama list`. +ollama_base_url = "" +ollama_model_name = "" + +# OneAPI. Use the token, Base URL, and model ID from your OneAPI deployment. +# Project: https://github.com/songquanpeng/one-api +oneapi_api_key = "" +oneapi_base_url = "" +oneapi_model_name = "" + +# LiteLLM uses provider credentials from environment variables, such as +# OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, or AWS_ACCESS_KEY_ID. +# Model formats: openai/gpt-4o, anthropic/claude-sonnet-4, ollama/llama3. +# Providers: https://docs.litellm.ai/docs/providers +litellm_model_name = "" + +# Groq +# API key: https://console.groq.com/keys +groq_api_key = "" +groq_base_url = "" +groq_model_name = "" + +# Pollinations AI: https://enter.pollinations.ai/ +pollinations_api_key = "" +pollinations_base_url = "" +pollinations_model_name = "" + +# ----------------------------------------------------------------------------- +# Speech, Subtitles, and Video / 语音、字幕与视频 +# ----------------------------------------------------------------------------- + +# Xiaomi MiMo TTS shares mimo_api_key and mimo_base_url with the MiMo LLM. +mimo_tts_model_name = "mimo-v2.5-tts" +mimo_tts_style_prompt = "请用自然、清晰、适合短视频旁白的语气朗读。" + +# Subtitle provider: "edge" or "whisper". Leave empty to skip generation. +subtitle_provider = "edge" + +# FFmpeg is normally downloaded and detected automatically. If detection fails, +# download it from https://www.gyan.dev/ffmpeg/builds/ and set the executable. +# Windows example: ffmpeg_path = "C:\\path\\to\\ffmpeg.exe" +# ffmpeg_path = "" + +# Leave unset to follow the app default (libx264). Hardware encoders are optional; +# unsupported encoders automatically fall back to libx264. +# Available values: "libx264", "h264_nvenc", "h264_amf", "h264_qsv", +# "h264_mf", "h264_videotoolbox". +# video_codec = "libx264" + +# ----------------------------------------------------------------------------- +# Storage and Task Runtime / 存储与任务运行 +# ----------------------------------------------------------------------------- + +# Public base URL used to build generated video download links. Leave empty to +# use the current API service address, or set an external reverse-proxy URL. +endpoint = "" + +# Material storage: "" uses ./storage/cache_videos, "task" stores materials in +# each task directory, and an absolute path stores materials in that directory. +# Example: material_directory = "/path/to/videos" +material_directory = "" + +# Optional Redis-backed task state. Keep Redis private to this application; +# untrusted writers must not be able to modify serialized task records. +enable_redis = false +redis_host = "localhost" +redis_port = 6379 +redis_db = 0 +redis_password = "" + +# Maximum number of concurrent and queued API video-generation tasks. +max_concurrent_tasks = 5 +max_queued_tasks = 100 + +# ----------------------------------------------------------------------------- +# Cross-platform Publishing / 跨平台发布 +# ----------------------------------------------------------------------------- + +# Upload-Post can publish generated videos to TikTok, Instagram, and YouTube. +# Create an account and API key at https://upload-post.com/ +# API documentation: https://docs.upload-post.com/ +upload_post_enabled = false +upload_post_api_key = "" +upload_post_username = "" +# Available values: "tiktok", "instagram", "youtube". +upload_post_platforms = ["tiktok", "instagram"] +# Automatically publish successful video outputs after generation. +upload_post_auto_upload = false +# YouTube privacy: "public", "unlisted", or "private". +upload_post_youtube_privacy_status = "public" + +# ============================================================================= +# Whisper Subtitles / Whisper 字幕 +# ============================================================================= + +[whisper] +# The model is downloaded automatically on first use unless a matching local +# directory exists at ./models/whisper-{model_size}. +model_size = "large-v3" +# device: "cpu" or "cuda". Common compute types include "int8" for CPU and +# "float16" or "int8_float16" for CUDA. +device = "cpu" +compute_type = "int8" + +# ============================================================================= +# Material Request Proxy / 素材请求代理 +# ============================================================================= + +[proxy] +# Optional proxy used by Pexels, Pixabay, Coverr, and material downloads. +# Format: "http://:@:" +# Documentation: https://requests.readthedocs.io/en/latest/user/advanced/#proxies +# http = "http://127.0.0.1:3128" +# https = "http://127.0.0.1:1080" + +# ============================================================================= +# TTS Providers / 语音合成服务 +# ============================================================================= + +[azure] +# Azure Speech TTS credentials. These are separate from Azure OpenAI settings. +# API key: https://portal.azure.com/#view/Microsoft_Azure_ProjectOxford/CognitiveServicesHub/~/SpeechServices +speech_key = "" +speech_region = "" + +[siliconflow] +# SiliconFlow TTS API key: https://siliconflow.cn/ +api_key = "" + +[elevenlabs] +# API key: https://elevenlabs.io/app/settings/api-keys +# Favorite voices in the ElevenLabs voice library are shown in the WebUI. +api_key = "" +# Available models include eleven_multilingual_v2, eleven_flash_v2_5, and eleven_v3. +model_id = "eleven_multilingual_v2" + +[chatterbox] +# OpenAI-compatible Chatterbox TTS server. The defaults target +# travisvn/chatterbox-tts-api running locally on port 4123. +base_url = "http://127.0.0.1:4123/v1" +api_key = "" +model_id = "chatterbox" +# Voice names exposed by the configured Chatterbox server. +voices = ["default-Female"] + +# ============================================================================= +# WebUI Preferences / WebUI 偏好 +# ============================================================================= + +[ui] +# Hide generation logs in the WebUI. +hide_log = false + +# The WebUI writes the remaining preferences automatically. Uncomment values +# below only when you want to define initial defaults manually. +# language = "zh" +# tts_server = "azure-tts-v1" +# voice_name = "" +# font_name = "MicrosoftYaHeiBold.ttc" +# font_size = 60 +# text_fore_color = "#FFFFFF" +# subtitle_position = "bottom" # "top", "center", "bottom", or "custom" +# custom_position = 70.0 # Percentage from the top when using "custom" +# subtitle_background_enabled = false +# subtitle_background_color = "#000000" +# rounded_subtitle_background = false diff --git a/docker-compose.gpu.yml b/docker-compose.gpu.yml new file mode 100644 index 0000000..cddfc64 --- /dev/null +++ b/docker-compose.gpu.yml @@ -0,0 +1,35 @@ +# GPU override file +# Usage: docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d +# +# Prerequisites: +# 1. NVIDIA GPU driver installed on host +# 2. NVIDIA Container Toolkit installed +# 3. Verify with: docker info | grep nvidia +# +# This file overrides the default docker-compose.yml to: +# - Build with Dockerfile.gpu (NVIDIA CUDA base image) +# - Attach GPU devices to both services because video generation can run in +# either the WebUI process or the API process +services: + webui: + build: + context: . + dockerfile: Dockerfile.gpu + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: 1 + capabilities: [gpu] + api: + build: + context: . + dockerfile: Dockerfile.gpu + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: 1 + capabilities: [gpu] diff --git a/docker-compose.release.yml b/docker-compose.release.yml new file mode 100644 index 0000000..c461cbe --- /dev/null +++ b/docker-compose.release.yml @@ -0,0 +1,21 @@ +x-common-volumes: &common-volumes + - ./config.toml:/MoneyPrinterTurbo/config.toml + - ./storage:/MoneyPrinterTurbo/storage + +services: + webui: + image: ghcr.io/harry0703/moneyprinterturbo:latest + container_name: "moneyprinterturbo-webui" + ports: + - "127.0.0.1:8501:8501" + command: [ "streamlit", "run", "./webui/Main.py", "--server.address=0.0.0.0", "--server.port=8501", "--browser.serverAddress=127.0.0.1", "--server.enableCORS=True", "--browser.gatherUsageStats=False", "--client.toolbarMode=minimal", "--logger.hideWelcomeMessage=True", "--server.showEmailPrompt=False" ] + volumes: *common-volumes + restart: always + api: + image: ghcr.io/harry0703/moneyprinterturbo:latest + container_name: "moneyprinterturbo-api" + ports: + - "127.0.0.1:8080:8080" + command: [ "python3", "main.py" ] + volumes: *common-volumes + restart: always diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..97f501e --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,24 @@ +x-common-volumes: &common-volumes + - ./:/MoneyPrinterTurbo + +services: + webui: + build: + context: . + dockerfile: Dockerfile + container_name: "moneyprinterturbo-webui" + ports: + - "127.0.0.1:8501:8501" + command: [ "streamlit", "run", "./webui/Main.py", "--server.address=0.0.0.0", "--server.port=8501", "--browser.serverAddress=127.0.0.1", "--server.enableCORS=True", "--browser.gatherUsageStats=False", "--client.toolbarMode=minimal", "--logger.hideWelcomeMessage=True", "--server.showEmailPrompt=False" ] + volumes: *common-volumes + restart: always + api: + build: + context: . + dockerfile: Dockerfile + container_name: "moneyprinterturbo-api" + ports: + - "127.0.0.1:8080:8080" + command: [ "python3", "main.py" ] + volumes: *common-volumes + restart: always diff --git a/docs/MoneyPrinterTurbo.ipynb b/docs/MoneyPrinterTurbo.ipynb new file mode 100644 index 0000000..e643d2c --- /dev/null +++ b/docs/MoneyPrinterTurbo.ipynb @@ -0,0 +1,187 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Run MoneyPrinterTurbo on Google Colab\n", + "\n", + "This notebook installs [MoneyPrinterTurbo](https://github.com/harry0703/MoneyPrinterTurbo) in an isolated Python 3.11 environment and launches its WebUI. Colab runtimes are temporary, so generated files are removed when the runtime is reset." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Install MoneyPrinterTurbo\n", + "\n", + "The setup is safe to run again: it clones the repository on the first run and updates it on later runs. Dependencies are installed in the project's virtual environment to avoid conflicts with Colab's preinstalled packages." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "S8Eu-aQarY_B" + }, + "outputs": [], + "source": [ + "import os\n", + "import subprocess\n", + "from pathlib import Path\n", + "\n", + "REPO_DIR = Path(\"/content/MoneyPrinterTurbo\")\n", + "REPO_URL = \"https://github.com/harry0703/MoneyPrinterTurbo.git\"\n", + "\n", + "# Update an existing checkout so rerunning this cell does not fail during clone.\n", + "if (REPO_DIR / \".git\").is_dir():\n", + " subprocess.run(\n", + " [\"git\", \"-C\", str(REPO_DIR), \"pull\", \"--ff-only\"], check=True\n", + " )\n", + "elif REPO_DIR.exists():\n", + " raise RuntimeError(f\"{REPO_DIR} exists but is not a Git repository\")\n", + "else:\n", + " subprocess.run(\n", + " [\"git\", \"clone\", \"--depth\", \"1\", REPO_URL, str(REPO_DIR)], check=True\n", + " )\n", + "\n", + "os.chdir(REPO_DIR)\n", + "subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"-q\", \"uv\", \"pyngrok\"], check=True)\n", + "subprocess.run([\"uv\", \"python\", \"install\", \"3.11\"], check=True)\n", + "# Use the lockfile in an isolated environment without changing Colab's preinstalled packages.\n", + "subprocess.run([\"uv\", \"sync\", \"--frozen\", \"--python\", \"3.11\"], check=True)\n", + "print(f\"MoneyPrinterTurbo is ready in {REPO_DIR}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Configure the Colab tunnel\n", + "\n", + "MoneyPrinterTurbo itself does not require ngrok for normal local use. This notebook uses ngrok only because the Streamlit server runs inside a remote Colab runtime.\n", + "\n", + "Create an account and copy your authentication token from the [ngrok dashboard](https://dashboard.ngrok.com/get-started/your-authtoken). The next cell reads it without displaying or storing it in the notebook." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from getpass import getpass\n", + "\n", + "from pyngrok import ngrok\n", + "\n", + "# Close tunnels from earlier runs before configuring a new one.\n", + "ngrok.kill()\n", + "\n", + "ngrok_token = getpass(\"Enter your ngrok authentication token: \").strip()\n", + "if not ngrok_token:\n", + " raise ValueError(\"An ngrok authentication token is required\")\n", + "ngrok.set_auth_token(ngrok_token)\n", + "del ngrok_token\n", + "print(\"ngrok authentication configured\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. Launch the WebUI\n", + "\n", + "The cell waits for Streamlit to become healthy before creating the public URL. If startup fails, it includes the recent server log in the error. After opening the URL, use **Settings** to configure the required model and media API keys." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "collapsed": true, + "id": "oahsIOXmwjl9", + "outputId": "ee23a96c-af21-4207-deb7-9fab69e0c05e" + }, + "outputs": [], + "source": [ + "import time\n", + "from urllib.error import URLError\n", + "from urllib.request import urlopen\n", + "\n", + "PORT = 8501\n", + "LOG_PATH = Path(\"/content/moneyprinterturbo-webui.log\")\n", + "\n", + "# Make this cell safe to rerun by closing the previous process, log handle, and tunnel.\n", + "if \"streamlit_proc\" in globals() and streamlit_proc.poll() is None:\n", + " streamlit_proc.terminate()\n", + " streamlit_proc.wait(timeout=10)\n", + "if \"streamlit_log\" in globals() and not streamlit_log.closed:\n", + " streamlit_log.close()\n", + "ngrok.kill()\n", + "\n", + "streamlit_log = LOG_PATH.open(\"w\", encoding=\"utf-8\")\n", + "streamlit_proc = subprocess.Popen(\n", + " [\n", + " \"uv\",\n", + " \"run\",\n", + " \"streamlit\",\n", + " \"run\",\n", + " \"webui/Main.py\",\n", + " f\"--server.port={PORT}\",\n", + " \"--server.address=0.0.0.0\",\n", + " \"--browser.gatherUsageStats=False\",\n", + " \"--client.toolbarMode=minimal\",\n", + " \"--server.showEmailPrompt=False\",\n", + " ],\n", + " cwd=REPO_DIR,\n", + " stdout=streamlit_log,\n", + " stderr=subprocess.STDOUT,\n", + " text=True,\n", + ")\n", + "\n", + "# Poll the health endpoint because startup time varies across Colab runtimes.\n", + "deadline = time.time() + 90\n", + "server_ready = False\n", + "while time.time() < deadline:\n", + " if streamlit_proc.poll() is not None:\n", + " break\n", + " try:\n", + " with urlopen(f\"http://127.0.0.1:{PORT}/_stcore/health\", timeout=2) as response:\n", + " server_ready = response.status == 200\n", + " except (URLError, TimeoutError):\n", + " pass\n", + " if server_ready:\n", + " break\n", + " time.sleep(2)\n", + "\n", + "if not server_ready:\n", + " streamlit_log.flush()\n", + " recent_log = LOG_PATH.read_text(encoding=\"utf-8\", errors=\"replace\")[-4000:]\n", + " raise RuntimeError(f\"Streamlit failed to start. Recent log:\\n{recent_log}\")\n", + "\n", + "# Use an explicit IPv4 URL because pyngrok may otherwise route localhost through ::1.\n", + "public_tunnel = ngrok.connect(addr=f\"http://127.0.0.1:{PORT}\", proto=\"http\", bind_tls=True)\n", + "print(\"MoneyPrinterTurbo is ready:\")\n", + "print(public_tunnel.public_url)\n", + "print(f\"Server log: {LOG_PATH}\")" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/docs/api.jpg b/docs/api.jpg new file mode 100644 index 0000000..8af1670 Binary files /dev/null and b/docs/api.jpg differ diff --git a/docs/skill/SKILL.md b/docs/skill/SKILL.md new file mode 100644 index 0000000..a94218f --- /dev/null +++ b/docs/skill/SKILL.md @@ -0,0 +1,132 @@ +--- +name: moneyprinterturbo-video +description: Use this skill whenever the user wants to create a finished video from a topic, title, idea, prompt, or script with MoneyPrinterTurbo. This includes short-form, voice-over, educational, marketing, social-media, and stock-footage videos. Also use it when the user mentions MoneyPrinterTurbo, provides this Skill URL, asks an AI agent to install or configure MoneyPrinterTurbo, needs missing API keys identified, wants a failed generation repaired, or wants a generated MP4 located and delivered. Use this skill when the expected outcome is a final video file, not setup instructions. +compatibility: Requires an AI agent with terminal, network, filesystem, and long-running command support. Supports macOS and Windows and uses uv exclusively. +metadata: + author: "harry0703@hotmail.com" + version: "1.3.2" + upstream: "https://github.com/harry0703/MoneyPrinterTurbo" +--- + +# MoneyPrinterTurbo Video Generation + +The user only needs to provide a video topic or script. Complete installation, configuration reuse, generation, waiting, and final MP4 delivery automatically. Do not stop after giving instructions or commands. + +## Required Behavior + +1. Ask the user only for required API credentials that are missing, rejected, or unusable. Combine all required credentials into one request. +2. Do not ask for confirmation before installing, generating, waiting, using defaults, or returning the result. +3. Do not create or repeatedly update a detailed plan for a standard generation request. Send one short progress update and execute. +4. Run the helper as one foreground command with a timeout of at least 20 minutes. +5. Never poll with `sleep`, `echo`, `ps`, repeated `ls`, or repeated `tail`. If the terminal returns a resumable session ID, continue waiting on that same session. +6. Do not read the full log after success. Read only the short reported error or the relevant log tail after failure. +7. Never print API keys, tokens, the full `config.toml`, or credential-bearing configuration fragments. + +## Defaults + +Unless the user requests otherwise, generate one Chinese `9:16` portrait video with Pexels footage, the default Chinese Edge TTS voice, subtitles, and background music. Install MoneyPrinterTurbo under the user's home directory. + +## Execution + +### 1. Locate the helper + +Resolve `SKILL_DIR` from this `SKILL.md` file. The helper is the adjacent `mpt_agent.py`. Set the terminal tool's working directory to `SKILL_DIR` and invoke the helper by its relative filename. Do not put the absolute helper path in the command, and do not run an extra `ls` or `dir` check. + +This is required on Windows because some agent terminal validators remove backslashes from absolute paths embedded in commands. Using `mpt_agent.py` with `workdir=SKILL_DIR` avoids that failure and works on both macOS and Windows. + +If the client loaded only the remote `SKILL.md`, download the helper from the official repository to a temporary directory, then use that temporary directory as the command working directory: + +```text +https://raw.githubusercontent.com/harry0703/MoneyPrinterTurbo/main/docs/skill/mpt_agent.py +``` + +### 2. Run the helper + +Do not run a separate `uv --version` preflight. Run the helper directly. If the shell explicitly reports that uv is missing, install uv and retry the same helper command once. + +macOS uv installation: + +```bash +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +Windows PowerShell uv installation: + +```powershell +powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" +``` + +Use this foreground command with `workdir=SKILL_DIR` and a timeout of at least 20 minutes: + +```bash +uv run --no-project --python 3.11 python mpt_agent.py --subject "