docs: make Chinese README the default
This commit is contained in:
wehub-resource-sync
@@ -1,6 +1,12 @@
|
||||
<!-- WEHUB_ZH_README -->
|
||||
> [!NOTE]
|
||||
> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。
|
||||
> [English](./README.en.md) · [原始项目](https://github.com/microsoft/VibeVoice) · [上游 README](https://github.com/microsoft/VibeVoice/blob/HEAD/README.md)
|
||||
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
|
||||
|
||||
<div align="center">
|
||||
|
||||
## 🎙️ VibeVoice: Open-Source Frontier Voice AI
|
||||
## 🎙️ VibeVoice：开源前沿语音 AI
|
||||
[](https://microsoft.github.io/VibeVoice)
|
||||
[](https://huggingface.co/collections/microsoft/vibevoice-68a2ef24a875c44be47b034f)
|
||||
[](https://openreview.net/pdf?id=FihSkzyxdv)
|
||||
@@ -22,43 +28,43 @@
|
||||
|
||||
<div align="left">
|
||||
|
||||
<h3>📰 News</h3>
|
||||
<h3>📰 新闻</h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<strong>2026-03-06: 🚀 VibeVoice ASR is now part of a <a href="https://huggingface.co/microsoft/VibeVoice-ASR-HF">Transformers release</a>! You can now use our speech recognition model directly through the Hugging Face Transformers library for seamless integration into your projects.</strong>
|
||||
<strong>2026-03-06：🚀 VibeVoice ASR 现已纳入 <a href="https://huggingface.co/microsoft/VibeVoice-ASR-HF">Transformers 发布版本</a>！你现在可以通过 Hugging Face Transformers 库直接使用我们的语音识别模型，无缝集成到你的项目中。</strong>
|
||||
|
||||
<strong>2026-01-21:</strong> 📣 We open-sourced <a href="docs/vibevoice-asr.md"><strong>VibeVoice-ASR</strong></a>, a unified speech-to-text model designed to handle 60-minute long-form audio in a single pass, generating structured transcriptions containing Who (Speaker), When (Timestamps), and What (Content), with support for User-Customized Context. Try it in [Playground](https://aka.ms/vibevoice-asr).
|
||||
- ⭐️ VibeVoice-ASR is natively multilingual, supporting over 50 languages — check the [supported languages](docs/vibevoice-asr.md#language-distribution) for details.
|
||||
- 🔥 The VibeVoice-ASR [finetuning code](finetuning-asr/README.md) is now available!
|
||||
- ⚡️ **vLLM inference** is now supported for faster inference; see [vllm-asr](docs/vibevoice-vllm-asr.md) for more details.
|
||||
- 📑 [VibeVoice-ASR Technique Report](https://arxiv.org/pdf/2601.18184) is available.
|
||||
<strong>2026-01-21：</strong> 📣 我们开源了 <a href="docs/vibevoice-asr.md"><strong>VibeVoice-ASR</strong></a>，这是一款统一的语音转文本（Speech-to-Text）模型，旨在单次处理长达 60 分钟的长音频，生成包含 Who（说话人）、When（时间戳）和 What（内容）的结构化转录结果，并支持用户自定义上下文（User-Customized Context）。可在 [Playground](https://aka.ms/vibevoice-asr). 试用。
|
||||
- ⭐️ VibeVoice-ASR 原生支持多语言，覆盖 50 多种语言 — 详见[支持的语言](docs/vibevoice-asr.md#language-distribution)。
|
||||
- 🔥 VibeVoice-ASR [微调代码](finetuning-asr/README.md) 现已发布！
|
||||
- ⚡️ 现已支持 **vLLM 推理**，可实现更快推理；详见 [vllm-asr](docs/vibevoice-vllm-asr.md)。
|
||||
- 📑 [VibeVoice-ASR 技术报告](https://arxiv.org/pdf/2601.18184) 现已发布。
|
||||
|
||||
2025-12-16: 📣 We added experimental speakers to <a href="docs/vibevoice-realtime-0.5b.md"><strong>VibeVoice‑Realtime‑0.5B</strong></a> for exploration, including multilingual voices in nine languages (DE, FR, IT, JP, KR, NL, PL, PT, ES) and 11 distinct English style voices. [Try it](docs/vibevoice-realtime-0.5b.md#optional-more-experimental-voices). More speaker types will be added over time.
|
||||
2025-12-16：📣 我们为 <a href="docs/vibevoice-realtime-0.5b.md"><strong>VibeVoice‑Realtime‑0.5B</strong></a> 新增了实验性说话人供探索，包括九种语言（DE、FR、IT、JP、KR、NL、PL、PT、ES）的多语言音色，以及 11 种不同的英语风格音色。[立即试用](docs/vibevoice-realtime-0.5b.md#optional-more-experimental-voices)。后续将陆续增加更多说话人类型。
|
||||
|
||||
2025-12-03: 📣 We open-sourced <a href="docs/vibevoice-realtime-0.5b.md"><strong>VibeVoice‑Realtime‑0.5B</strong></a>, a real‑time text‑to‑speech model that supports streaming text input and robust long-form speech generation. Try it on [Colab](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/vibevoice_realtime_colab.ipynb).
|
||||
2025-12-03：📣 我们开源了 <a href="docs/vibevoice-realtime-0.5b.md"><strong>VibeVoice‑Realtime‑0.5B</strong></a>，这是一款支持流式文本输入、可稳健生成长篇语音的实时文本转语音（Text-to-Speech）模型。可在 [Colab](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/vibevoice_realtime_colab.ipynb). 试用。
|
||||
|
||||
|
||||
2025-09-05: VibeVoice is an open-source research framework intended to advance collaboration in the speech synthesis community. After release, we discovered instances where the tool was used in ways inconsistent with the stated intent. Since responsible use of AI is one of Microsoft’s guiding principles, we have removed the VibeVoice-TTS code from this repository.
|
||||
2025-09-05：VibeVoice 是一个旨在推动语音合成社区协作的开源研究框架。发布之后，我们发现该工具存在与既定用途不一致的使用情况。由于负责任地使用 AI 是 Microsoft 的指导原则之一，我们已从本仓库中移除 VibeVoice-TTS 代码。
|
||||
|
||||
|
||||
2025-08-25: 📣 We open-sourced <a href="docs/vibevoice-tts.md"><strong>VibeVoice-TTS</strong></a>, a long-form multi-speaker text-to-speech model that can synthesize speech up to 90 minutes long with up to 4 distinct speakers. — accepted as an [Oral](https://openreview.net/forum?id=FihSkzyxdv) at ICLR 2026! 🔥
|
||||
2025-08-25：📣 我们开源了 <a href="docs/vibevoice-tts.md"><strong>VibeVoice-TTS</strong></a>，这是一款长篇幅多说话人文本转语音模型，可合成最长 90 分钟、最多 4 位不同说话人的语音。—— 已被 ICLR 2026 接收为 [Oral](https://openreview.net/forum?id=FihSkzyxdv)！🔥
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
## 概述
|
||||
|
||||
VibeVoice is a **family of open-source frontier voice AI models** that includes both Text-to-Speech (TTS) and Automatic Speech Recognition (ASR) models.
|
||||
VibeVoice 是一个**开源前沿语音 AI 模型家族**，涵盖文本转语音（Text-to-Speech，TTS）与自动语音识别（Automatic Speech Recognition，ASR）模型。
|
||||
|
||||
A core innovation of VibeVoice is its use of continuous speech tokenizers (Acoustic and Semantic) operating at an ultra-low frame rate of **7.5 Hz**. These tokenizers efficiently preserve audio fidelity while significantly boosting computational efficiency for processing long sequences. VibeVoice employs a [next-token diffusion](https://arxiv.org/abs/2412.08635) framework, leveraging a Large Language Model (LLM) to understand textual context and dialogue flow, and a diffusion head to generate high-fidelity acoustic details.
|
||||
VibeVoice 的一项核心创新是采用以 **7.5 Hz** 超低帧率运行的连续语音分词器（Acoustic 与 Semantic）。这些分词器在高效保留音频保真度的同时，显著提升了处理长序列的计算效率。VibeVoice 采用 [next-token diffusion](https://arxiv.org/abs/2412.08635) 框架，借助大语言模型（Large Language Model，LLM）理解文本上下文与对话流程，并通过扩散头（diffusion head）生成高保真声学细节。
|
||||
|
||||
For more information, demos, and examples, please visit our [Project Page](https://microsoft.github.io/VibeVoice).
|
||||
更多信息、演示与示例，请访问我们的[项目主页](https://microsoft.github.io/VibeVoice).
|
||||
|
||||
|
||||
<div align="center">
|
||||
|
||||
| Model | Weight | Quick Try |
|
||||
| 模型 | 权重 | 快速试用 |
|
||||
|-------|--------------|---------|
|
||||
| VibeVoice-ASR-7B | [HF Link](https://huggingface.co/microsoft/VibeVoice-ASR) | [Playground](https://aka.ms/vibevoice-asr) |
|
||||
| VibeVoice-TTS-1.5B | [HF Link](https://huggingface.co/microsoft/VibeVoice-1.5B) | Disabled |
|
||||
@@ -66,23 +72,23 @@ For more information, demos, and examples, please visit our [Project Page](https
|
||||
|
||||
</div>
|
||||
|
||||
## Models
|
||||
## 模型
|
||||
|
||||
|
||||
### 1. 📖 [VibeVoice-ASR](docs/vibevoice-asr.md) - Long-form Speech Recognition
|
||||
### 1. 📖 [VibeVoice-ASR](docs/vibevoice-asr.md) - 长音频语音识别
|
||||
|
||||
**VibeVoice-ASR** is a unified speech-to-text model designed to handle **60-minute long-form audio** in a single pass, generating structured transcriptions containing **Who (Speaker), When (Timestamps), and What (Content)**, with support for **Customized Hotwords**.
|
||||
**VibeVoice-ASR** 是一款统一的语音转文本模型，旨在单次处理**长达 60 分钟的长音频**，生成包含 **Who（说话人）、When（时间戳）和 What（内容）** 的结构化转录结果，并支持**自定义热词（Customized Hotwords）**。
|
||||
|
||||
- **🕒 60-minute Single-Pass Processing**:
|
||||
Unlike conventional ASR models that slice audio into short chunks (often losing global context), VibeVoice ASR accepts up to **60 minutes** of continuous audio input within 64K token length. This ensures consistent speaker tracking and semantic coherence across the entire hour.
|
||||
- **🕒 60 分钟单次处理**：
|
||||
与将音频切分为短片段（往往会丢失全局上下文）的传统 ASR 模型不同，VibeVoice ASR 可在 64K token 长度内接受最长 **60 分钟**的连续音频输入。这确保在整个一小时内保持一致的说话人跟踪与语义连贯性。
|
||||
|
||||
- **👤 Customized Hotwords**:
|
||||
Users can provide customized hotwords (e.g., specific names, technical terms, or background info) to guide the recognition process, significantly improving accuracy on domain-specific content.
|
||||
- **👤 自定义热词**：
|
||||
用户可提供自定义热词（例如特定姓名、技术术语或背景信息）以引导识别过程，显著提升领域特定内容的识别准确率。
|
||||
|
||||
- **📝 Rich Transcription (Who, When, What)**:
|
||||
The model jointly performs ASR, diarization, and timestamping, producing a structured output that indicates *who* said *what* and *when*.
|
||||
- **📝 丰富转录（Who、When、What）**：
|
||||
该模型联合执行 ASR、说话人分离（diarization）与时间戳标注，输出结构化结果，标明*谁在何时说了什么*。
|
||||
|
||||
[📖 Documentation](docs/vibevoice-asr.md) | [🤗 Hugging Face](https://huggingface.co/microsoft/VibeVoice-ASR) | [🎮 Playground](https://aka.ms/vibevoice-asr) | [🛠️ Finetuning](finetuning-asr/README.md) | [📊 Paper](docs/VibeVoice-ASR-Report.pdf)
|
||||
[📖 文档](docs/vibevoice-asr.md) | [🤗 Hugging Face](https://huggingface.co/microsoft/VibeVoice-ASR) | [🎮 Playground](https://aka.ms/vibevoice-asr) | [🛠️ 微调](finetuning-asr/README.md) | [📊 论文](docs/VibeVoice-ASR-Report.pdf)
|
||||
|
||||
|
||||
<p align="center">
|
||||
@@ -99,24 +105,24 @@ https://github.com/user-attachments/assets/acde5602-dc17-4314-9e3b-c630bc84aefa
|
||||
</div>
|
||||
<br>
|
||||
|
||||
### 2. 🎙️ [VibeVoice-TTS](docs/vibevoice-tts.md) - Long-form Multi-speaker TTS
|
||||
### 2. 🎙️ [VibeVoice-TTS](docs/vibevoice-tts.md) - 长篇幅多说话人 TTS
|
||||
|
||||
**Best for**: Long-form conversational audio, podcasts, multi-speaker dialogues
|
||||
**最适合**：长篇幅对话音频、播客、多说话人对话
|
||||
|
||||
- **⏱️ 90-minute Long-form Generation**:
|
||||
Synthesizes conversational/single-speaker speech up to **90 minutes** in a single pass, maintaining speaker consistency and semantic coherence throughout.
|
||||
- **⏱️ 90 分钟长篇幅生成**：
|
||||
单次可合成最长 **90 分钟**的对话式/单说话人语音，全程保持说话人一致性与语义连贯性。
|
||||
|
||||
- **👥 Multi-speaker Support**:
|
||||
Supports up to **4 distinct speakers** in a single conversation, with natural turn-taking and speaker consistency across long dialogues.
|
||||
- **👥 多说话人支持**：
|
||||
单次对话最多支持 **4 位不同说话人**，在长对话中实现自然的轮替与一致的说话人表现。
|
||||
|
||||
- **🎭 Expressive Speech**:
|
||||
Generates expressive, natural-sounding speech that captures conversational dynamics and emotional nuances.
|
||||
- **🎭 富有表现力的语音**：
|
||||
生成富有表现力、听起来自然的语音，捕捉对话动态与情感细微差别。
|
||||
|
||||
- **🌐 Multi-lingual Support**:
|
||||
Supports English, Chinese and other languages.
|
||||
- **🌐 多语言支持**：
|
||||
支持英语、中文及其他语言。
|
||||
|
||||
|
||||
[📖 Documentation](docs/vibevoice-tts.md) | [🤗 Hugging Face](https://huggingface.co/microsoft/VibeVoice-1.5B) | [📊 Paper](https://arxiv.org/pdf/2508.19205)
|
||||
[📖 文档](docs/vibevoice-tts.md) | [🤗 Hugging Face](https://huggingface.co/microsoft/VibeVoice-1.5B) | [📊 论文](https://arxiv.org/pdf/2508.19205)
|
||||
|
||||
|
||||
<div align="center">
|
||||
@@ -132,21 +138,21 @@ https://github.com/user-attachments/assets/0967027c-141e-4909-bec8-091558b1b784
|
||||
</div>
|
||||
|
||||
|
||||
**Chinese**
|
||||
**中文**
|
||||
<div align="center">
|
||||
|
||||
https://github.com/user-attachments/assets/322280b7-3093-4c67-86e3-10be4746c88f
|
||||
|
||||
</div>
|
||||
|
||||
**Cross-Lingual**
|
||||
**跨语言**
|
||||
<div align="center">
|
||||
|
||||
https://github.com/user-attachments/assets/838d8ad9-a201-4dde-bb45-8cd3f59ce722
|
||||
|
||||
</div>
|
||||
|
||||
**Spontaneous Singing**
|
||||
**即兴演唱**
|
||||
<div align="center">
|
||||
|
||||
https://github.com/user-attachments/assets/6f27a8a5-0c60-4f57-87f3-7dea2e11c730
|
||||
@@ -154,7 +160,7 @@ https://github.com/user-attachments/assets/6f27a8a5-0c60-4f57-87f3-7dea2e11c730
|
||||
</div>
|
||||
|
||||
|
||||
**Long Conversation with 4 people**
|
||||
**四人长对话**
|
||||
<div align="center">
|
||||
|
||||
https://github.com/user-attachments/assets/a357c4b6-9768-495c-a576-1618f6275727
|
||||
@@ -167,16 +173,16 @@ https://github.com/user-attachments/assets/a357c4b6-9768-495c-a576-1618f6275727
|
||||
|
||||
<br>
|
||||
|
||||
### 3. ⚡ [VibeVoice-Streaming](docs/vibevoice-realtime-0.5b.md) - Real-time Streaming TTS
|
||||
### 3. ⚡ [VibeVoice-Streaming](docs/vibevoice-realtime-0.5b.md) - 实时流式 TTS
|
||||
|
||||
VibeVoice-Realtime is a **lightweight real‑time** text-to-speech model supporting **streaming text input** and **robust long-form speech generation**.
|
||||
VibeVoice-Realtime 是一款**轻量级实时**文本转语音（text-to-speech，TTS）模型，支持**流式文本输入**和**稳健的长篇语音生成**。
|
||||
|
||||
- Parameter size: 0.5B (deployment-friendly)
|
||||
- Real-time TTS (~300 milliseconds first audible latency)
|
||||
- Streaming text input
|
||||
- Robust long-form speech generation (~10 minutes)
|
||||
- 参数量：0.5B（便于部署）
|
||||
- 实时 TTS（首次可听延迟约 300 毫秒）
|
||||
- 流式文本输入
|
||||
- 稳健的长篇语音生成（约 10 分钟）
|
||||
|
||||
[📖 Documentation](docs/vibevoice-realtime-0.5b.md) | [🤗 Hugging Face](https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B) | [🚀 Colab](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/vibevoice_realtime_colab.ipynb)
|
||||
[📖 文档](docs/vibevoice-realtime-0.5b.md) | [🤗 Hugging Face](https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B) | [🚀 Colab](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/vibevoice_realtime_colab.ipynb)
|
||||
|
||||
|
||||
<div align="center" id="generated-example-audio-vibevoice-realtime">
|
||||
@@ -187,21 +193,21 @@ https://github.com/user-attachments/assets/0901d274-f6ae-46ef-a0fd-3c4fba4f76dc
|
||||
|
||||
<br>
|
||||
|
||||
## Contributing
|
||||
## 贡献
|
||||
|
||||
Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed contribution guidelines.
|
||||
请参阅 [CONTRIBUTING.md](CONTRIBUTING.md) 了解详细的贡献指南。
|
||||
|
||||
|
||||
|
||||
## ⚠️ Risks and Limitations
|
||||
## ⚠️ 风险与限制
|
||||
|
||||
|
||||
While efforts have been made to optimize it through various techniques, it may still produce outputs that are unexpected, biased, or inaccurate. VibeVoice inherits any biases, errors, or omissions produced by its base model (specifically, Qwen2.5 1.5b in this release).
|
||||
Potential for Deepfakes and Disinformation: High-quality synthetic speech can be misused to create convincing fake audio content for impersonation, fraud, or spreading disinformation. Users must ensure transcripts are reliable, check content accuracy, and avoid using generated content in misleading ways. Users are expected to use the generated content and to deploy the models in a lawful manner, in full compliance with all applicable laws and regulations in the relevant jurisdictions. It is best practice to disclose the use of AI when sharing AI-generated content.
|
||||
尽管我们已通过多种技术对其进行优化，模型仍可能产生出人意料、带有偏见或不准确的输出。VibeVoice 会继承其基座模型（本版本具体为 Qwen2.5 1.5b）所产生的任何偏见、错误或遗漏。
|
||||
深度伪造与虚假信息风险：高质量合成语音可能被滥用于制作逼真的虚假音频内容，以进行冒充、欺诈或传播虚假信息。用户必须确保转录文本可靠、核查内容准确性，并避免以误导性方式使用生成内容。用户应以合法方式使用生成内容并部署模型，全面遵守相关司法管辖区适用的全部法律法规。分享 AI 生成内容时，最佳做法是披露 AI 的使用情况。
|
||||
|
||||
|
||||
We do not recommend using VibeVoice in commercial or real-world applications without further testing and development. This model is intended for research and development purposes only. Please use responsibly.
|
||||
未经进一步测试与开发，我们不建议将 VibeVoice 用于商业或真实场景应用。本模型仅供研究与开发用途。请负责任地使用。
|
||||
|
||||
## Star History
|
||||
## Star 历史
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user