Compare commits

..

3 Commits

Author SHA1 Message Date
wehub-resource-sync e50ac961fb docs: make Chinese README the default 2026-07-13 10:35:07 +00:00
wehub-resource-sync 2ea16835a1 docs: preserve upstream English README 2026-07-13 10:34:31 +00:00
wehub-resource-sync 8492c4cc85 chore: import upstream snapshot with attribution 2026-07-13 13:23:40 +08:00
12 changed files with 502 additions and 140 deletions
+6 -6
View File
@@ -17,10 +17,10 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.10
- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
python-version: '3.10'
python-version: 3.9
- name: Install module
run: |
@@ -47,10 +47,10 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.10
- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
python-version: '3.10'
python-version: 3.9
- name: Install module
run: |
@@ -69,10 +69,10 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.10
- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
python-version: '3.10'
python-version: 3.9
- name: Install dependencies
run: |
+297
View File
@@ -0,0 +1,297 @@
[![CI](https://github.com/SYSTRAN/faster-whisper/workflows/CI/badge.svg)](https://github.com/SYSTRAN/faster-whisper/actions?query=workflow%3ACI) [![PyPI version](https://badge.fury.io/py/faster-whisper.svg)](https://badge.fury.io/py/faster-whisper)
# Faster Whisper transcription with CTranslate2
**faster-whisper** is a reimplementation of OpenAI's Whisper model using [CTranslate2](https://github.com/OpenNMT/CTranslate2/), which is a fast inference engine for Transformer models.
This implementation is up to 4 times faster than [openai/whisper](https://github.com/openai/whisper) for the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU.
## Benchmark
### Whisper
For reference, here's the time and memory usage that are required to transcribe [**13 minutes**](https://www.youtube.com/watch?v=0u7tTptBo9I) of audio using different implementations:
* [openai/whisper](https://github.com/openai/whisper)@[v20240930](https://github.com/openai/whisper/tree/v20240930)
* [whisper.cpp](https://github.com/ggerganov/whisper.cpp)@[v1.7.2](https://github.com/ggerganov/whisper.cpp/tree/v1.7.2)
* [transformers](https://github.com/huggingface/transformers)@[v4.46.3](https://github.com/huggingface/transformers/tree/v4.46.3)
* [faster-whisper](https://github.com/SYSTRAN/faster-whisper)@[v1.1.0](https://github.com/SYSTRAN/faster-whisper/tree/v1.1.0)
### Large-v2 model on GPU
| Implementation | Precision | Beam size | Time | VRAM Usage |
| --- | --- | --- | --- | --- |
| openai/whisper | fp16 | 5 | 2m23s | 4708MB |
| whisper.cpp (Flash Attention) | fp16 | 5 | 1m05s | 4127MB |
| transformers (SDPA)[^1] | fp16 | 5 | 1m52s | 4960MB |
| faster-whisper | fp16 | 5 | 1m03s | 4525MB |
| faster-whisper (`batch_size=8`) | fp16 | 5 | 17s | 6090MB |
| faster-whisper | int8 | 5 | 59s | 2926MB |
| faster-whisper (`batch_size=8`) | int8 | 5 | 16s | 4500MB |
### distil-whisper-large-v3 model on GPU
| Implementation | Precision | Beam size | Time | YT Commons WER |
| --- | --- | --- | --- | --- |
| transformers (SDPA) (`batch_size=16`) | fp16 | 5 | 46m12s | 14.801 |
| faster-whisper (`batch_size=16`) | fp16 | 5 | 25m50s | 13.527 |
*GPU Benchmarks are Executed with CUDA 12.4 on a NVIDIA RTX 3070 Ti 8GB.*
[^1]: transformers OOM for any batch size > 1
### Small model on CPU
| Implementation | Precision | Beam size | Time | RAM Usage |
| --- | --- | --- | --- | --- |
| openai/whisper | fp32 | 5 | 6m58s | 2335MB |
| whisper.cpp | fp32 | 5 | 2m05s | 1049MB |
| whisper.cpp (OpenVINO) | fp32 | 5 | 1m45s | 1642MB |
| faster-whisper | fp32 | 5 | 2m37s | 2257MB |
| faster-whisper (`batch_size=8`) | fp32 | 5 | 1m06s | 4230MB |
| faster-whisper | int8 | 5 | 1m42s | 1477MB |
| faster-whisper (`batch_size=8`) | int8 | 5 | 51s | 3608MB |
*Executed with 8 threads on an Intel Core i7-12700K.*
## Requirements
* Python 3.9 or greater
Unlike openai-whisper, FFmpeg does **not** need to be installed on the system. The audio is decoded with the Python library [PyAV](https://github.com/PyAV-Org/PyAV) which bundles the FFmpeg libraries in its package.
### GPU
GPU execution requires the following NVIDIA libraries to be installed:
* [cuBLAS for CUDA 12](https://developer.nvidia.com/cublas)
* [cuDNN 9 for CUDA 12](https://developer.nvidia.com/cudnn)
**Note**: The latest versions of `ctranslate2` only support CUDA 12 and cuDNN 9. For CUDA 11 and cuDNN 8, the current workaround is downgrading to the `3.24.0` version of `ctranslate2`, for CUDA 12 and cuDNN 8, downgrade to the `4.4.0` version of `ctranslate2`, (This can be done with `pip install --force-reinstall ctranslate2==4.4.0` or specifying the version in a `requirements.txt`).
There are multiple ways to install the NVIDIA libraries mentioned above. The recommended way is described in the official NVIDIA documentation, but we also suggest other installation methods below.
<details>
<summary>Other installation methods (click to expand)</summary>
**Note:** For all these methods below, keep in mind the above note regarding CUDA versions. Depending on your setup, you may need to install the _CUDA 11_ versions of libraries that correspond to the CUDA 12 libraries listed in the instructions below.
#### Use Docker
The libraries (cuBLAS, cuDNN) are installed in this official NVIDIA CUDA Docker images: `nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04`.
#### Install with `pip` (Linux only)
On Linux these libraries can be installed with `pip`. Note that `LD_LIBRARY_PATH` must be set before launching Python.
```bash
pip install nvidia-cublas-cu12 nvidia-cudnn-cu12==9.*
export LD_LIBRARY_PATH=`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`
```
#### Download the libraries from Purfview's repository (Windows & Linux)
Purfview's [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) provides the required NVIDIA libraries for Windows & Linux in a [single archive](https://github.com/Purfview/whisper-standalone-win/releases/tag/libs). Decompress the archive and place the libraries in a directory included in the `PATH`.
</details>
## Installation
The module can be installed from [PyPI](https://pypi.org/project/faster-whisper/):
```bash
pip install faster-whisper
```
<details>
<summary>Other installation methods (click to expand)</summary>
### Install the master branch
```bash
pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faster-whisper/archive/refs/heads/master.tar.gz"
```
### Install a specific commit
```bash
pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faster-whisper/archive/a4f1cc8f11433e454c3934442b5e1a4ed5e865c3.tar.gz"
```
</details>
## Usage
### Faster-whisper
```python
from faster_whisper import WhisperModel
model_size = "large-v3"
# Run on GPU with FP16
model = WhisperModel(model_size, device="cuda", compute_type="float16")
# or run on GPU with INT8
# model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")
# or run on CPU with INT8
# model = WhisperModel(model_size, device="cpu", compute_type="int8")
segments, info = model.transcribe("audio.mp3", beam_size=5)
print("Detected language '%s' with probability %f" % (info.language, info.language_probability))
for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
```
**Warning:** `segments` is a *generator* so the transcription only starts when you iterate over it. The transcription can be run to completion by gathering the segments in a list or a `for` loop:
```python
segments, _ = model.transcribe("audio.mp3")
segments = list(segments) # The transcription will actually run here.
```
### Batched Transcription
The following code snippet illustrates how to run batched transcription on an example audio file. `BatchedInferencePipeline.transcribe` is a drop-in replacement for `WhisperModel.transcribe`
```python
from faster_whisper import WhisperModel, BatchedInferencePipeline
model = WhisperModel("turbo", device="cuda", compute_type="float16")
batched_model = BatchedInferencePipeline(model=model)
segments, info = batched_model.transcribe("audio.mp3", batch_size=16)
for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
```
### Faster Distil-Whisper
The Distil-Whisper checkpoints are compatible with the Faster-Whisper package. In particular, the latest [distil-large-v3](https://huggingface.co/distil-whisper/distil-large-v3)
checkpoint is intrinsically designed to work with the Faster-Whisper transcription algorithm. The following code snippet
demonstrates how to run inference with distil-large-v3 on a specified audio file:
```python
from faster_whisper import WhisperModel
model_size = "distil-large-v3"
model = WhisperModel(model_size, device="cuda", compute_type="float16")
segments, info = model.transcribe("audio.mp3", beam_size=5, language="en", condition_on_previous_text=False)
for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
```
For more information about the distil-large-v3 model, refer to the original [model card](https://huggingface.co/distil-whisper/distil-large-v3).
### Word-level timestamps
```python
segments, _ = model.transcribe("audio.mp3", word_timestamps=True)
for segment in segments:
for word in segment.words:
print("[%.2fs -> %.2fs] %s" % (word.start, word.end, word.word))
```
### VAD filter
The library integrates the [Silero VAD](https://github.com/snakers4/silero-vad) model to filter out parts of the audio without speech:
```python
segments, _ = model.transcribe("audio.mp3", vad_filter=True)
```
The default behavior is conservative and only removes silence longer than 2 seconds. See the available VAD parameters and default values in the [source code](https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/vad.py). They can be customized with the dictionary argument `vad_parameters`:
```python
segments, _ = model.transcribe(
"audio.mp3",
vad_filter=True,
vad_parameters=dict(min_silence_duration_ms=500),
)
```
Vad filter is enabled by default for batched transcription.
### Logging
The library logging level can be configured like this:
```python
import logging
logging.basicConfig()
logging.getLogger("faster_whisper").setLevel(logging.DEBUG)
```
### Going further
See more model and transcription options in the [`WhisperModel`](https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/transcribe.py) class implementation.
## Community integrations
Here is a non exhaustive list of open-source projects using faster-whisper. Feel free to add your project to the list!
* [speaches](https://github.com/speaches-ai/speaches) is an OpenAI compatible server using `faster-whisper`. It's easily deployable with Docker, works with OpenAI SDKs/CLI, supports streaming, and live transcription.
* [WhisperX](https://github.com/m-bain/whisperX) is an award-winning Python library that offers speaker diarization and accurate word-level timestamps using wav2vec2 alignment
* [whisper-ctranslate2](https://github.com/Softcatala/whisper-ctranslate2) is a command line client based on faster-whisper and compatible with the original client from openai/whisper.
* [whisper-diarize](https://github.com/MahmoudAshraf97/whisper-diarization) is a speaker diarization tool that is based on faster-whisper and NVIDIA NeMo.
* [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) Standalone CLI executables of faster-whisper for Windows, Linux & macOS.
* [asr-sd-pipeline](https://github.com/hedrergudene/asr-sd-pipeline) provides a scalable, modular, end to end multi-speaker speech to text solution implemented using AzureML pipelines.
* [Open-Lyrics](https://github.com/zh-plus/Open-Lyrics) is a Python library that transcribes voice files using faster-whisper, and translates/polishes the resulting text into `.lrc` files in the desired language using OpenAI-GPT.
* [wscribe](https://github.com/geekodour/wscribe) is a flexible transcript generation tool supporting faster-whisper, it can export word level transcript and the exported transcript then can be edited with [wscribe-editor](https://github.com/geekodour/wscribe-editor)
* [aTrain](https://github.com/BANDAS-Center/aTrain) is a graphical user interface implementation of faster-whisper developed at the BANDAS-Center at the University of Graz for transcription and diarization in Windows ([Windows Store App](https://apps.microsoft.com/detail/atrain/9N15Q44SZNS2)) and Linux.
* [Whisper-Streaming](https://github.com/ufal/whisper_streaming) implements real-time mode for offline Whisper-like speech-to-text models with faster-whisper as the most recommended back-end. It implements a streaming policy with self-adaptive latency based on the actual source complexity, and demonstrates the state of the art.
* [WhisperLive](https://github.com/collabora/WhisperLive) is a nearly-live implementation of OpenAI's Whisper which uses faster-whisper as the backend to transcribe audio in real-time.
* [Faster-Whisper-Transcriber](https://github.com/BBC-Esq/ctranslate2-faster-whisper-transcriber) is a simple but reliable voice transcriber that provides a user-friendly interface.
* [Open-dubbing](https://github.com/softcatala/open-dubbing) is open dubbing is an AI dubbing system which uses machine learning models to automatically translate and synchronize audio dialogue into different languages.
* [Whisper-FastAPI](https://github.com/heimoshuiyu/whisper-fastapi) whisper-fastapi is a very simple script that provides an API backend compatible with OpenAI, HomeAssistant, and Konele (Android voice typing) formats.
## Model conversion
When loading a model from its size such as `WhisperModel("large-v3")`, the corresponding CTranslate2 model is automatically downloaded from the [Hugging Face Hub](https://huggingface.co/Systran).
We also provide a script to convert any Whisper models compatible with the Transformers library. They could be the original OpenAI models or user fine-tuned models.
For example the command below converts the [original "large-v3" Whisper model](https://huggingface.co/openai/whisper-large-v3) and saves the weights in FP16:
```bash
pip install transformers[torch]>=4.23
ct2-transformers-converter --model openai/whisper-large-v3 --output_dir whisper-large-v3-ct2
--copy_files tokenizer.json preprocessor_config.json --quantization float16
```
* The option `--model` accepts a model name on the Hub or a path to a model directory.
* If the option `--copy_files tokenizer.json` is not used, the tokenizer configuration is automatically downloaded when the model is loaded later.
Models can also be converted from the code. See the [conversion API](https://opennmt.net/CTranslate2/python/ctranslate2.converters.TransformersConverter.html).
### Load a converted model
1. Directly load the model from a local directory:
```python
model = faster_whisper.WhisperModel("whisper-large-v3-ct2")
```
2. [Upload your model to the Hugging Face Hub](https://huggingface.co/docs/transformers/model_sharing#upload-with-the-web-interface) and load it from its name:
```python
model = faster_whisper.WhisperModel("username/whisper-large-v3-ct2")
```
## Comparing performance against other implementations
If you are comparing the performance against other Whisper implementations, you should make sure to run the comparison with similar settings. In particular:
* Verify that the same transcription options are used, especially the same beam size. For example in openai/whisper, `model.transcribe` uses a default beam size of 1 but here we use a default beam size of 5.
* Transcription speed is closely affected by the number of words in the transcript, so ensure that other implementations have a similar WER (Word Error Rate) to this one.
* When running on CPU, make sure to set the same number of threads. Many frameworks will read the environment variable `OMP_NUM_THREADS`, which can be set when running your script:
```bash
OMP_NUM_THREADS=4 python3 my_script.py
```
+87 -81
View File
@@ -1,25 +1,31 @@
<!-- WEHUB_ZH_README -->
> [!NOTE]
> 本文档由 WeHub 基于上游 README 翻译整理,属于社区翻译,非官方中文文档。
> [English](./README.en.md) · [原始项目](https://github.com/SYSTRAN/faster-whisper) · [上游 README](https://github.com/SYSTRAN/faster-whisper/blob/HEAD/README.md)
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
[![CI](https://github.com/SYSTRAN/faster-whisper/workflows/CI/badge.svg)](https://github.com/SYSTRAN/faster-whisper/actions?query=workflow%3ACI) [![PyPI version](https://badge.fury.io/py/faster-whisper.svg)](https://badge.fury.io/py/faster-whisper)
# Faster Whisper transcription with CTranslate2
# 使用 CTranslate2 的 Faster Whisper 转录
**faster-whisper** is a reimplementation of OpenAI's Whisper model using [CTranslate2](https://github.com/OpenNMT/CTranslate2/), which is a fast inference engine for Transformer models.
**faster-whisper** 是使用 [CTranslate2](https://github.com/OpenNMT/CTranslate2/), 对 OpenAI Whisper 模型的重新实现,CTranslate2 是一个用于 Transformer 模型的快速推理引擎。
This implementation is up to 4 times faster than [openai/whisper](https://github.com/openai/whisper) for the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU.
在相同精度下,该实现的运行速度最高可达 [openai/whisper](https://github.com/openai/whisper) 的 4 倍,同时占用更少内存。通过在 CPU GPU 上使用 8 位量化,效率还可以进一步提升。
## Benchmark
## 基准测试
### Whisper
For reference, here's the time and memory usage that are required to transcribe [**13 minutes**](https://www.youtube.com/watch?v=0u7tTptBo9I) of audio using different implementations:
作为参考,以下是使用不同实现转录 [**13 分钟**](https://www.youtube.com/watch?v=0u7tTptBo9I) 音频所需的时间和内存占用:
* [openai/whisper](https://github.com/openai/whisper)@[v20240930](https://github.com/openai/whisper/tree/v20240930)
* [whisper.cpp](https://github.com/ggerganov/whisper.cpp)@[v1.7.2](https://github.com/ggerganov/whisper.cpp/tree/v1.7.2)
* [transformers](https://github.com/huggingface/transformers)@[v4.46.3](https://github.com/huggingface/transformers/tree/v4.46.3)
* [faster-whisper](https://github.com/SYSTRAN/faster-whisper)@[v1.1.0](https://github.com/SYSTRAN/faster-whisper/tree/v1.1.0)
### Large-v2 model on GPU
### GPU 上的 Large-v2 模型
| Implementation | Precision | Beam size | Time | VRAM Usage |
| 实现 | 精度 | Beam size | 时间 | VRAM 占用 |
| --- | --- | --- | --- | --- |
| openai/whisper | fp16 | 5 | 2m23s | 4708MB |
| whisper.cpp (Flash Attention) | fp16 | 5 | 1m05s | 4127MB |
@@ -29,19 +35,19 @@ For reference, here's the time and memory usage that are required to transcribe
| faster-whisper | int8 | 5 | 59s | 2926MB |
| faster-whisper (`batch_size=8`) | int8 | 5 | 16s | 4500MB |
### distil-whisper-large-v3 model on GPU
### GPU 上的 distil-whisper-large-v3 模型
| Implementation | Precision | Beam size | Time | YT Commons WER |
| 实现 | 精度 | Beam size | 时间 | YT Commons WER |
| --- | --- | --- | --- | --- |
| transformers (SDPA) (`batch_size=16`) | fp16 | 5 | 46m12s | 14.801 |
| faster-whisper (`batch_size=16`) | fp16 | 5 | 25m50s | 13.527 |
*GPU Benchmarks are Executed with CUDA 12.4 on a NVIDIA RTX 3070 Ti 8GB.*
[^1]: transformers OOM for any batch size > 1
*GPU 基准测试在配备 8GB 显存的 NVIDIA RTX 3070 Ti 上,使用 CUDA 12.4 执行。*
[^1]: 对于 batch size > 1 的任何情况,transformers 会出现 OOM
### Small model on CPU
### CPU 上的 Small 模型
| Implementation | Precision | Beam size | Time | RAM Usage |
| 实现 | 精度 | Beam size | 时间 | RAM 占用 |
| --- | --- | --- | --- | --- |
| openai/whisper | fp32 | 5 | 6m58s | 2335MB |
| whisper.cpp | fp32 | 5 | 2m05s | 1049MB |
@@ -51,39 +57,39 @@ For reference, here's the time and memory usage that are required to transcribe
| faster-whisper | int8 | 5 | 1m42s | 1477MB |
| faster-whisper (`batch_size=8`) | int8 | 5 | 51s | 3608MB |
*Executed with 8 threads on an Intel Core i7-12700K.*
* Intel Core i7-12700K 上使用 8 个线程执行。*
## Requirements
## 环境要求
* Python 3.10 or greater
* Python 3.9 或更高版本
Unlike openai-whisper, FFmpeg does **not** need to be installed on the system. The audio is decoded with the Python library [PyAV](https://github.com/PyAV-Org/PyAV) which bundles the FFmpeg libraries in its package.
openai-whisper 不同,系统**不**需要安装 FFmpeg。音频解码通过 Python [PyAV](https://github.com/PyAV-Org/PyAV) 完成,该库在其软件包中捆绑了 FFmpeg 库。
### GPU
GPU execution requires the following NVIDIA libraries to be installed:
GPU 执行需要安装以下 NVIDIA 库:
* [cuBLAS for CUDA 12](https://developer.nvidia.com/cublas)
* [cuDNN 9 for CUDA 12](https://developer.nvidia.com/cudnn)
**Note**: The latest versions of `ctranslate2` only support CUDA 12 and cuDNN 9. For CUDA 11 and cuDNN 8, the current workaround is downgrading to the `3.24.0` version of `ctranslate2`, for CUDA 12 and cuDNN 8, downgrade to the `4.4.0` version of `ctranslate2`, (This can be done with `pip install --force-reinstall ctranslate2==4.4.0` or specifying the version in a `requirements.txt`).
**注意**:最新版本的 `ctranslate2` 仅支持 CUDA 12 cuDNN 9。对于 CUDA 11 cuDNN 8,目前的变通方法是降级到 `ctranslate2``3.24.0` 版本;对于 CUDA 12 cuDNN 8,则降级到 `ctranslate2``4.4.0` 版本(可通过 `pip install --force-reinstall ctranslate2==4.4.0` 完成,或在 `requirements.txt` 中指定版本)。
There are multiple ways to install the NVIDIA libraries mentioned above. The recommended way is described in the official NVIDIA documentation, but we also suggest other installation methods below.
安装上述 NVIDIA 库有多种方式。推荐方式见官方 NVIDIA 文档,但我们也建议在下方采用其他安装方法。
<details>
<summary>Other installation methods (click to expand)</summary>
<summary>其他安装方法(点击展开)</summary>
**Note:** For all these methods below, keep in mind the above note regarding CUDA versions. Depending on your setup, you may need to install the _CUDA 11_ versions of libraries that correspond to the CUDA 12 libraries listed in the instructions below.
**注意:** 对于以下所有方法,请记住上述关于 CUDA 版本的说明。根据你的环境,你可能需要安装与下方说明中列出的 CUDA 12 库相对应的 _CUDA 11_ 版本库。
#### Use Docker
#### 使用 Docker
The libraries (cuBLAS, cuDNN) are installed in this official NVIDIA CUDA Docker images: `nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04`.
这些库(cuBLAScuDNN)已安装在官方 NVIDIA CUDA Docker 镜像中:`nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04`
#### Install with `pip` (Linux only)
#### 使用 `pip` 安装(仅 Linux
On Linux these libraries can be installed with `pip`. Note that `LD_LIBRARY_PATH` must be set before launching Python.
Linux 上,可以使用 `pip` 安装这些库。请注意,在启动 Python 之前必须设置 `LD_LIBRARY_PATH`
```bash
pip install nvidia-cublas-cu12 nvidia-cudnn-cu12==9.*
@@ -91,30 +97,30 @@ pip install nvidia-cublas-cu12 nvidia-cudnn-cu12==9.*
export LD_LIBRARY_PATH=`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`
```
#### Download the libraries from Purfview's repository (Windows & Linux)
#### 从 Purfview 的仓库下载库(Windows Linux
Purfview's [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) provides the required NVIDIA libraries for Windows & Linux in a [single archive](https://github.com/Purfview/whisper-standalone-win/releases/tag/libs). Decompress the archive and place the libraries in a directory included in the `PATH`.
Purfview [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) Windows Linux 提供了所需的 NVIDIA 库,打包在[单个压缩包](https://github.com/Purfview/whisper-standalone-win/releases/tag/libs). 中。解压后将库放置在包含在 `PATH` 中的目录里。
</details>
## Installation
## 安装
The module can be installed from [PyPI](https://pypi.org/project/faster-whisper/):
该模块可从 [PyPI](https://pypi.org/project/faster-whisper/): 安装:
```bash
pip install faster-whisper
```
<details>
<summary>Other installation methods (click to expand)</summary>
<summary>其他安装方法(点击展开)</summary>
### Install the master branch
### 安装 master 分支
```bash
pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faster-whisper/archive/refs/heads/master.tar.gz"
```
### Install a specific commit
### 安装特定提交
```bash
pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faster-whisper/archive/a4f1cc8f11433e454c3934442b5e1a4ed5e865c3.tar.gz"
@@ -122,7 +128,7 @@ pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faste
</details>
## Usage
## 用法
### Faster-whisper
@@ -147,15 +153,16 @@ for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
```
**Warning:** `segments` is a *generator* so the transcription only starts when you iterate over it. The transcription can be run to completion by gathering the segments in a list or a `for` loop:
**警告:** `segments` 是一个*生成器*(generator),因此只有在你对其进行迭代时,转录才会开始。可以通过将片段收集到列表中,或使用 `for` 循环,来运行转录直至完成:
```python
segments, _ = model.transcribe("audio.mp3")
segments = list(segments) # The transcription will actually run here.
```
### Batched Transcription
The following code snippet illustrates how to run batched transcription on an example audio file. `BatchedInferencePipeline.transcribe` is a drop-in replacement for `WhisperModel.transcribe`
### 批量转录
以下代码片段演示了如何对示例音频文件运行批量转录。`BatchedInferencePipeline.transcribe` 可直接替代 `WhisperModel.transcribe`
```python
from faster_whisper import WhisperModel, BatchedInferencePipeline
@@ -170,9 +177,8 @@ for segment in segments:
### Faster Distil-Whisper
The Distil-Whisper checkpoints are compatible with the Faster-Whisper package. In particular, the latest [distil-large-v3](https://huggingface.co/distil-whisper/distil-large-v3)
checkpoint is intrinsically designed to work with the Faster-Whisper transcription algorithm. The following code snippet
demonstrates how to run inference with distil-large-v3 on a specified audio file:
Distil-Whisper 检查点与 Faster-Whisper 软件包兼容。特别是,最新的 [distil-large-v3](https://huggingface.co/distil-whisper/distil-large-v3)
检查点本质上就是为配合 Faster-Whisper 转录算法而设计的。以下代码片段演示了如何在指定音频文件上使用 distil-large-v3 运行推理:
```python
from faster_whisper import WhisperModel
@@ -186,9 +192,9 @@ for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
```
For more information about the distil-large-v3 model, refer to the original [model card](https://huggingface.co/distil-whisper/distil-large-v3).
有关 distil-large-v3 模型的更多信息,请参阅原始 [model card](https://huggingface.co/distil-whisper/distil-large-v3).
### Word-level timestamps
### 词级时间戳(Word-level timestamps
```python
segments, _ = model.transcribe("audio.mp3", word_timestamps=True)
@@ -198,15 +204,15 @@ for segment in segments:
print("[%.2fs -> %.2fs] %s" % (word.start, word.end, word.word))
```
### VAD filter
### VAD 滤波器(VAD filter
The library integrates the [Silero VAD](https://github.com/snakers4/silero-vad) model to filter out parts of the audio without speech:
该库集成了 [Silero VAD](https://github.com/snakers4/silero-vad) 模型,用于滤除音频中没有语音的部分:
```python
segments, _ = model.transcribe("audio.mp3", vad_filter=True)
```
The default behavior is conservative and only removes silence longer than 2 seconds. See the available VAD parameters and default values in the [source code](https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/vad.py). They can be customized with the dictionary argument `vad_parameters`:
默认行为较为保守,仅移除长于 2 秒的静音。可在 [source code](https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/vad.py). 中查看可用的 VAD 参数及其默认值。可通过字典参数 `vad_parameters` 进行自定义:
```python
segments, _ = model.transcribe(
@@ -215,11 +221,11 @@ segments, _ = model.transcribe(
vad_parameters=dict(min_silence_duration_ms=500),
)
```
Vad filter is enabled by default for batched transcription.
批量转写默认启用 Vad 滤波器。
### Logging
### 日志(Logging
The library logging level can be configured like this:
可按如下方式配置库的日志级别:
```python
import logging
@@ -228,37 +234,37 @@ logging.basicConfig()
logging.getLogger("faster_whisper").setLevel(logging.DEBUG)
```
### Going further
### 进一步探索(Going further
See more model and transcription options in the [`WhisperModel`](https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/transcribe.py) class implementation.
[`WhisperModel`](https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/transcribe.py) 类实现中查看更多模型与转写选项。
## Community integrations
## 社区集成(Community integrations
Here is a non exhaustive list of open-source projects using faster-whisper. Feel free to add your project to the list!
以下是使用 faster-whisper 的开源项目列表(非穷尽)。欢迎将你的项目添加到列表中!
* [speaches](https://github.com/speaches-ai/speaches) is an OpenAI compatible server using `faster-whisper`. It's easily deployable with Docker, works with OpenAI SDKs/CLI, supports streaming, and live transcription.
* [WhisperX](https://github.com/m-bain/whisperX) is an award-winning Python library that offers speaker diarization and accurate word-level timestamps using wav2vec2 alignment
* [whisper-ctranslate2](https://github.com/Softcatala/whisper-ctranslate2) is a command line client based on faster-whisper and compatible with the original client from openai/whisper.
* [whisper-diarize](https://github.com/MahmoudAshraf97/whisper-diarization) is a speaker diarization tool that is based on faster-whisper and NVIDIA NeMo.
* [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) Standalone CLI executables of faster-whisper for Windows, Linux & macOS.
* [asr-sd-pipeline](https://github.com/hedrergudene/asr-sd-pipeline) provides a scalable, modular, end to end multi-speaker speech to text solution implemented using AzureML pipelines.
* [Open-Lyrics](https://github.com/zh-plus/Open-Lyrics) is a Python library that transcribes voice files using faster-whisper, and translates/polishes the resulting text into `.lrc` files in the desired language using OpenAI-GPT.
* [wscribe](https://github.com/geekodour/wscribe) is a flexible transcript generation tool supporting faster-whisper, it can export word level transcript and the exported transcript then can be edited with [wscribe-editor](https://github.com/geekodour/wscribe-editor)
* [aTrain](https://github.com/BANDAS-Center/aTrain) is a graphical user interface implementation of faster-whisper developed at the BANDAS-Center at the University of Graz for transcription and diarization in Windows ([Windows Store App](https://apps.microsoft.com/detail/atrain/9N15Q44SZNS2)) and Linux.
* [Whisper-Streaming](https://github.com/ufal/whisper_streaming) implements real-time mode for offline Whisper-like speech-to-text models with faster-whisper as the most recommended back-end. It implements a streaming policy with self-adaptive latency based on the actual source complexity, and demonstrates the state of the art.
* [WhisperLive](https://github.com/collabora/WhisperLive) is a nearly-live implementation of OpenAI's Whisper which uses faster-whisper as the backend to transcribe audio in real-time.
* [Faster-Whisper-Transcriber](https://github.com/BBC-Esq/ctranslate2-faster-whisper-transcriber) is a simple but reliable voice transcriber that provides a user-friendly interface.
* [Open-dubbing](https://github.com/softcatala/open-dubbing) is open dubbing is an AI dubbing system which uses machine learning models to automatically translate and synchronize audio dialogue into different languages.
* [Whisper-FastAPI](https://github.com/heimoshuiyu/whisper-fastapi) whisper-fastapi is a very simple script that provides an API backend compatible with OpenAI, HomeAssistant, and Konele (Android voice typing) formats.
* [speaches](https://github.com/speaches-ai/speaches) 是基于 `faster-whisper` 的 OpenAI 兼容服务器。它易于通过 Docker 部署,可与 OpenAI SDKs/CLI 配合使用,并支持流式传输与实时转写。
* [WhisperX](https://github.com/m-bain/whisperX) 是一款屡获殊荣的 Python 库,通过 wav2vec2 对齐提供说话人分离(speaker diarization)和精确的词级时间戳
* [whisper-ctranslate2](https://github.com/Softcatala/whisper-ctranslate2) 是基于 faster-whisper 的命令行客户端,并与 openai/whisper 的原始客户端兼容。
* [whisper-diarize](https://github.com/MahmoudAshraf97/whisper-diarization) 是一款基于 faster-whisper NVIDIA NeMo 的说话人分离工具。
* [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) faster-whisper WindowsLinux macOS 上的独立 CLI 可执行文件。
* [asr-sd-pipeline](https://github.com/hedrergudene/asr-sd-pipeline) 提供基于 AzureML pipelines 实现的可扩展、模块化、端到端的多说话人语音转文本解决方案。
* [Open-Lyrics](https://github.com/zh-plus/Open-Lyrics) 是一款 Python 库,使用 faster-whisper 转写语音文件,并利用 OpenAI-GPT 将结果文本翻译/润色为所需语言的 `.lrc` 文件。
* [wscribe](https://github.com/geekodour/wscribe) 是一款灵活的转录生成工具,支持 faster-whisper,可导出词级转录,导出的转录可使用 [wscribe-editor](https://github.com/geekodour/wscribe-editor) 进行编辑
* [aTrain](https://github.com/BANDAS-Center/aTrain) 是格拉茨大学 BANDAS-Center 开发的 faster-whisper 图形用户界面实现,用于在 Windows[Windows Store App](https://apps.microsoft.com/detail/atrain/9N15Q44SZNS2)))和 Linux 上进行转写与说话人分离。
* [Whisper-Streaming](https://github.com/ufal/whisper_streaming) 为离线类 Whisper 语音转文本模型实现实时模式,并以 faster-whisper 作为最推荐的后端。它实现了基于实际源复杂度的自适应延迟流式策略,并展示了当前最先进水平。
* [WhisperLive](https://github.com/collabora/WhisperLive) OpenAI Whisper 的近实时实现,使用 faster-whisper 作为后端实时转写音频。
* [Faster-Whisper-Transcriber](https://github.com/BBC-Esq/ctranslate2-faster-whisper-transcriber) 是一款简单但可靠的语音转写器,提供用户友好的界面。
* [Open-dubbing](https://github.com/softcatala/open-dubbing) open dubbing 是一款 AI 配音系统,使用机器学习模型自动将音频对话翻译并同步到不同语言。
* [Whisper-FastAPI](https://github.com/heimoshuiyu/whisper-fastapi) whisper-fastapi 是一个非常简单的脚本,提供兼容 OpenAIHomeAssistant KoneleAndroid 语音输入)格式的 API 后端。
## Model conversion
## 模型转换(Model conversion
When loading a model from its size such as `WhisperModel("large-v3")`, the corresponding CTranslate2 model is automatically downloaded from the [Hugging Face Hub](https://huggingface.co/Systran).
当按尺寸加载模型(例如 `WhisperModel("large-v3")`)时,相应的 CTranslate2 模型会自动从 [Hugging Face Hub](https://huggingface.co/Systran). 下载。
We also provide a script to convert any Whisper models compatible with the Transformers library. They could be the original OpenAI models or user fine-tuned models.
我们还提供了一个脚本,用于转换任何与 Transformers 库兼容的 Whisper 模型。这些模型可以是原始 OpenAI 模型,也可以是用户微调后的模型。
For example the command below converts the [original "large-v3" Whisper model](https://huggingface.co/openai/whisper-large-v3) and saves the weights in FP16:
例如,下面的命令会转换 [original "large-v3" Whisper model](https://huggingface.co/openai/whisper-large-v3) 并以 FP16 格式保存权重:
```bash
pip install transformers[torch]>=4.23
@@ -267,30 +273,30 @@ ct2-transformers-converter --model openai/whisper-large-v3 --output_dir whisper-
--copy_files tokenizer.json preprocessor_config.json --quantization float16
```
* The option `--model` accepts a model name on the Hub or a path to a model directory.
* If the option `--copy_files tokenizer.json` is not used, the tokenizer configuration is automatically downloaded when the model is loaded later.
* 选项 `--model` 接受 Hub 上的模型名称或模型目录路径。
* 如果未使用选项 `--copy_files tokenizer.json`,则在后续加载模型时会自动下载 tokenizer 配置。
Models can also be converted from the code. See the [conversion API](https://opennmt.net/CTranslate2/python/ctranslate2.converters.TransformersConverter.html).
也可以从代码中进行模型转换。请参阅 [conversion API](https://opennmt.net/CTranslate2/python/ctranslate2.converters.TransformersConverter.html).
### Load a converted model
### 加载已转换的模型
1. Directly load the model from a local directory:
1. 直接从本地目录加载模型:
```python
model = faster_whisper.WhisperModel("whisper-large-v3-ct2")
```
2. [Upload your model to the Hugging Face Hub](https://huggingface.co/docs/transformers/model_sharing#upload-with-the-web-interface) and load it from its name:
2. [Upload your model to the Hugging Face Hub](https://huggingface.co/docs/transformers/model_sharing#upload-with-the-web-interface) 并按名称加载:
```python
model = faster_whisper.WhisperModel("username/whisper-large-v3-ct2")
```
## Comparing performance against other implementations
## 与其他实现的性能对比
If you are comparing the performance against other Whisper implementations, you should make sure to run the comparison with similar settings. In particular:
如果你正在将性能与其他 Whisper 实现进行对比,应确保在相似设置下运行对比。特别要注意:
* Verify that the same transcription options are used, especially the same beam size. For example in openai/whisper, `model.transcribe` uses a default beam size of 1 but here we use a default beam size of 5.
* Transcription speed is closely affected by the number of words in the transcript, so ensure that other implementations have a similar WER (Word Error Rate) to this one.
* When running on CPU, make sure to set the same number of threads. Many frameworks will read the environment variable `OMP_NUM_THREADS`, which can be set when running your script:
* 确认使用相同的转写选项,尤其是相同的 beam size。例如在 openai/whisper 中,`model.transcribe` 默认 beam size 为 1,而此处默认 beam size 5
* 转写速度与转录文本中的词数密切相关,因此请确保其他实现与本实现的 WERWord Error Rate,词错误率)相近。
* 在 CPU 上运行时,请确保设置相同的线程数。许多框架会读取环境变量 `OMP_NUM_THREADS`,可在运行脚本时设置:
```bash
OMP_NUM_THREADS=4 python3 my_script.py
+7
View File
@@ -0,0 +1,7 @@
# WeHub 来源说明
- 原始项目:`SYSTRAN/faster-whisper`
- 原始仓库:https://github.com/SYSTRAN/faster-whisper
- 导入方式:上游默认分支的最新快照
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
- 本文件仅用于记录来源,不代表 WeHub 是原项目作者
Binary file not shown.
+17 -3
View File
@@ -418,23 +418,34 @@ class BatchedInferencePipeline:
"Set 'vad_filter' to True or provide 'clip_timestamps'."
)
clip_timestamps_provided = False
audio_chunks, chunks_metadata = collect_chunks(
audio, clip_timestamps, max_duration=chunk_length
)
else:
clip_timestamps_provided = True
clip_timestamps = [
{k: int(v * sampling_rate) for k, v in segment.items()}
for segment in clip_timestamps
]
audio_chunks, chunks_metadata = [], []
for clip in clip_timestamps:
for i, clip in enumerate(clip_timestamps):
audio_chunks.append(audio[clip["start"] : clip["end"]])
clip_duration = (clip["end"] - clip["start"]) / sampling_rate
if clip_duration > 30:
self.model.logger.warning(
"Segment %d is longer than 30 seconds, "
"only the first 30 seconds will be transcribed",
i,
)
chunks_metadata.append(
{
"offset": clip["start"] / sampling_rate,
"duration": (clip["end"] - clip["start"]) / sampling_rate,
"duration": clip_duration,
"segments": [clip],
}
)
@@ -559,7 +570,10 @@ class BatchedInferencePipeline:
options,
log_progress,
)
segments = restore_speech_timestamps(segments, clip_timestamps, sampling_rate)
if not clip_timestamps_provided:
segments = restore_speech_timestamps(
segments, clip_timestamps, sampling_rate
)
return segments, info
+1 -20
View File
@@ -5,7 +5,6 @@ import re
from typing import List, Optional, Union
import huggingface_hub
import requests
from tqdm.auto import tqdm
@@ -106,7 +105,6 @@ def download_model(
if output_dir is not None:
kwargs["local_dir"] = output_dir
kwargs["local_dir_use_symlinks"] = False
if cache_dir is not None:
kwargs["cache_dir"] = cache_dir
@@ -114,24 +112,7 @@ def download_model(
if use_auth_token is not None:
kwargs["token"] = use_auth_token
try:
return huggingface_hub.snapshot_download(repo_id, **kwargs)
except (
huggingface_hub.utils.HfHubHTTPError,
requests.exceptions.ConnectionError,
) as exception:
logger = get_logger()
logger.warning(
"An error occured while synchronizing the model %s from the Hugging Face Hub:\n%s",
repo_id,
exception,
)
logger.warning(
"Trying to load the model directly from the local cache, if it exists."
)
kwargs["local_files_only"] = True
return huggingface_hub.snapshot_download(repo_id, **kwargs)
return huggingface_hub.snapshot_download(repo_id, **kwargs)
def format_timestamp(
+58 -24
View File
@@ -27,11 +27,15 @@ class VadOptions:
min_speech_duration_ms: Final speech chunks shorter min_speech_duration_ms are thrown out.
max_speech_duration_s: Maximum duration of speech chunks in seconds. Chunks longer
than max_speech_duration_s will be split at the timestamp of the last silence that
lasts more than 100ms (if any), to prevent aggressive cutting. Otherwise, they will be
split aggressively just before max_speech_duration_s.
lasts more than min_silence_at_max_speech (if any), to prevent aggressive cutting.
Otherwise, they will be split aggressively just before max_speech_duration_s.
min_silence_duration_ms: In the end of each speech chunk wait for min_silence_duration_ms
before separating it
speech_pad_ms: Final speech chunks are padded by speech_pad_ms each side
min_silence_at_max_speech: Minimum silence duration in ms which is used to avoid abrupt cuts
when max_speech_duration_s is reached.
use_max_poss_sil_at_max_speech: Whether to use the maximum possible silence at
max_speech_duration_s or not. If not, the last silence is used.
"""
threshold: float = 0.5
@@ -40,6 +44,8 @@ class VadOptions:
max_speech_duration_s: float = float("inf")
min_silence_duration_ms: int = 2000
speech_pad_ms: int = 400
min_silence_at_max_speech: int = 98
use_max_poss_sil_at_max_speech: bool = True
def get_speech_timestamps(
@@ -69,6 +75,9 @@ def get_speech_timestamps(
min_silence_duration_ms = vad_options.min_silence_duration_ms
window_size_samples = 512
speech_pad_ms = vad_options.speech_pad_ms
min_silence_at_max_speech = vad_options.min_silence_at_max_speech
use_max_poss_sil_at_max_speech = vad_options.use_max_poss_sil_at_max_speech
min_speech_samples = sampling_rate * min_speech_duration_ms / 1000
speech_pad_samples = sampling_rate * speech_pad_ms / 1000
max_speech_samples = (
@@ -77,7 +86,7 @@ def get_speech_timestamps(
- 2 * speech_pad_samples
)
min_silence_samples = sampling_rate * min_silence_duration_ms / 1000
min_silence_samples_at_max_speech = sampling_rate * 98 / 1000
min_silence_samples_at_max_speech = sampling_rate * min_silence_at_max_speech / 1000
audio_length_samples = len(audio)
@@ -91,6 +100,8 @@ def get_speech_timestamps(
triggered = False
speeches = []
current_speech = {}
possible_ends = []
if neg_threshold is None:
neg_threshold = max(threshold - 0.15, 0.01)
@@ -100,45 +111,67 @@ def get_speech_timestamps(
prev_end = next_start = 0
for i, speech_prob in enumerate(speech_probs):
cur_sample = window_size_samples * i
if (speech_prob >= threshold) and temp_end:
sil_dur = cur_sample - temp_end
if sil_dur > min_silence_samples_at_max_speech:
possible_ends.append((temp_end, sil_dur))
temp_end = 0
if next_start < prev_end:
next_start = window_size_samples * i
next_start = cur_sample
if (speech_prob >= threshold) and not triggered:
triggered = True
current_speech["start"] = window_size_samples * i
current_speech["start"] = cur_sample
continue
if (
triggered
and (window_size_samples * i) - current_speech["start"] > max_speech_samples
):
if prev_end:
if triggered and (cur_sample - current_speech["start"] > max_speech_samples):
if use_max_poss_sil_at_max_speech and possible_ends:
prev_end, dur = max(possible_ends, key=lambda x: x[1])
current_speech["end"] = prev_end
speeches.append(current_speech)
current_speech = {}
# previously reached silence (< neg_thres) and is still not speech (< thres)
if next_start < prev_end:
triggered = False
else:
next_start = prev_end + dur
if next_start < prev_end + cur_sample:
current_speech["start"] = next_start
else:
triggered = False
prev_end = next_start = temp_end = 0
possible_ends = []
else:
current_speech["end"] = window_size_samples * i
speeches.append(current_speech)
current_speech = {}
prev_end = next_start = temp_end = 0
triggered = False
continue
if prev_end:
current_speech["end"] = prev_end
speeches.append(current_speech)
current_speech = {}
if next_start < prev_end:
triggered = False
else:
current_speech["start"] = next_start
prev_end = next_start = temp_end = 0
possible_ends = []
else:
current_speech["end"] = cur_sample
speeches.append(current_speech)
current_speech = {}
prev_end = next_start = temp_end = 0
triggered = False
possible_ends = []
continue
if (speech_prob < neg_threshold) and triggered:
if not temp_end:
temp_end = window_size_samples * i
# condition to avoid cutting in very short silence
if (window_size_samples * i) - temp_end > min_silence_samples_at_max_speech:
temp_end = cur_sample
sil_dur_now = cur_sample - temp_end
if (
not use_max_poss_sil_at_max_speech
and sil_dur_now > min_silence_samples_at_max_speech
):
prev_end = temp_end
if (window_size_samples * i) - temp_end < min_silence_samples:
if sil_dur_now < min_silence_samples:
continue
else:
current_speech["end"] = temp_end
@@ -149,6 +182,7 @@ def get_speech_timestamps(
current_speech = {}
prev_end = next_start = temp_end = 0
triggered = False
possible_ends = []
continue
if (
+1 -1
View File
@@ -1,3 +1,3 @@
"""Version information."""
__version__ = "1.2.0"
__version__ = "1.2.1"
+2 -2
View File
@@ -1,6 +1,6 @@
ctranslate2>=4.0,<5
huggingface_hub>=0.13
huggingface_hub>=0.23
tokenizers>=0.13,<1
onnxruntime>=1.14,<2
av>=11
tqdm
tqdm
+2 -2
View File
@@ -45,13 +45,13 @@ setup(
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
],
keywords="openai whisper speech ctranslate2 inference quantization transformer",
python_requires=">=3.10",
python_requires=">=3.9",
install_requires=install_requires,
extras_require={
"conversion": conversion_requires,
+24 -1
View File
@@ -245,7 +245,7 @@ def test_transcribe_signature():
def test_monotonic_timestamps(physcisworks_path):
model = WhisperModel("tiny")
model = WhisperModel("base")
pipeline = BatchedInferencePipeline(model=model)
segments, info = model.transcribe(physcisworks_path, word_timestamps=True)
@@ -290,3 +290,26 @@ def test_cliptimestamps_segments(jfk_path):
" And so my fellow Americans ask not what your country can do for you, "
"ask what you can do for your country."
)
def test_cliptimestamps_timings(physcisworks_path):
model = WhisperModel("tiny")
pipeline = BatchedInferencePipeline(model=model)
audio = decode_audio(physcisworks_path)
clip_timestamps = [{"start": 0.0, "end": 5.0}, {"start": 6.0, "end": 15.0}]
transcripts = [
" Now I want to return to the conservation of mechanical energy.",
(
" I have here a pendulum. I have an object that weighs 15 kilograms"
" and I can lift it up one meter, which I have done now."
),
]
segments, info = pipeline.transcribe(audio, clip_timestamps=clip_timestamps)
segments = list(segments)
assert len(segments) == 2
for segment, clip, transcript in zip(segments, clip_timestamps, transcripts):
assert clip["start"] == segment.start
assert clip["end"] == segment.end
assert segment.text == transcript