chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:22:28 +08:00
commit c56bef871b
9296 changed files with 1854228 additions and 0 deletions
@@ -0,0 +1,15 @@
---
title: "External Integrations"
id: external-integrations-audio
slug: "/external-integrations-audio"
description: "External integrations that enable working with audio in Haystack by transcribing files or converting text to audio."
---
# External Integrations
External integrations that enable working with audio in Haystack by transcribing files or converting text to audio.
| Name | Description |
| --- | --- |
| [AssemblyAI](https://haystack.deepset.ai/integrations/assemblyai) | Perform speech recognition, speaker diarization and summarization. |
| [Elevenlabs](https://haystack.deepset.ai/integrations/elevenlabs) | Convert text to speech using ElevenLabs API. |
@@ -0,0 +1,90 @@
---
title: "LocalWhisperTranscriber"
id: localwhispertranscriber
slug: "/localwhispertranscriber"
description: "Use `LocalWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model using your local installation of Whisper."
---
# LocalWhisperTranscriber
Use `LocalWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model using your local installation of Whisper.
<div className="key-value-table">
| | |
| --- | --- |
| **Most common position in a pipeline** | As the first component in an indexing pipeline |
| **Mandatory run variables** | `sources`: A list of paths or binary streams that you want to transcribe |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Audio](/reference/audio-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/audio/whisper_local.py |
| **Package name** | `haystack-ai` |
</div>
## Overview
The component also needs to know which Whisper model to work with. Specify this in the `model` parameter when initializing the component. All transcription is completed on the executing machine, and the audio is never sent to a third-party provider.
See other optional parameters you can specify in our [API documentation](/reference/audio-api).
See the [Whisper API documentation](https://platform.openai.com/docs/guides/speech-to-text) and the official Whisper [GitHub repo](https://github.com/openai/whisper) for the supported audio formats and languages.
To work with the `LocalWhisperTranscriber`, install torch and [Whisper](https://github.com/openai/whisper) first with the following commands:
```python
pip install 'transformers[torch]'
pip install -U openai-whisper
```
## Usage
### On its own
Heres an example of how to use `LocalWhisperTranscriber` on its own:
```python
import requests
from haystack.components.audio import LocalWhisperTranscriber
response = requests.get(
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
)
with open("kennedy_speech.mp3", "wb") as file:
file.write(response.content)
transcriber = LocalWhisperTranscriber(model="tiny")
transcription = transcriber.run(sources=["./kennedy_speech.mp3"])
print(transcription["documents"][0].content)
```
### In a pipeline
The pipeline below fetches an audio file from a specified URL and transcribes it. It first retrieves the audio file using `LinkContentFetcher`, then transcribes the audio into text with `LocalWhisperTranscriber`, and finally outputs the transcription text.
```python
from haystack.components.audio import LocalWhisperTranscriber
from haystack.components.fetchers import LinkContentFetcher
from haystack import Pipeline
pipe = Pipeline()
pipe.add_component("fetcher", LinkContentFetcher())
pipe.add_component("transcriber", LocalWhisperTranscriber(model="tiny"))
pipe.connect("fetcher", "transcriber")
result = pipe.run(
data={
"fetcher": {
"urls": [
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
],
},
},
)
print(result["transcriber"]["documents"][0].content)
```
## Additional References
🧑‍🍳 Cookbook: [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)
@@ -0,0 +1,98 @@
---
title: "RemoteWhisperTranscriber"
id: remotewhispertranscriber
slug: "/remotewhispertranscriber"
description: "Use `RemoteWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model."
---
# RemoteWhisperTranscriber
Use `RemoteWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model.
<div className="key-value-table">
| | |
| --- | --- |
| **Most common position in a pipeline** | As the first component in an indexing pipeline |
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with an environment variable `OPENAI_API_KEY`. |
| **Mandatory run variables** | `sources`: A list of paths or binary streams that you want to transcribe |
| **Output variables** | `documents`: A list of documents |
| **API reference** | [Audio](/reference/audio-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/audio/whisper_remote.py |
| **Package name** | `haystack-ai` |
</div>
## Overview
`RemoteWhisperTranscriber` works with OpenAI-compatible clients and isn't limited to just OpenAI as a provider. For example, [Groq](https://console.groq.com/docs/speech-text) offers a drop-in replacement that can be used as well. You can set the API key in one of two ways:
1. Through the `api_key` initialization parameter, where the key is resolved using [Secret API](../../concepts/secret-management.mdx).
2. By setting it in the `OPENAI_API_KEY` environment variable, which the system will use to access the key.
```python
from haystack.components.audio import RemoteWhisperTranscriber
transcriber = RemoteWhisperTranscriber()
```
Additionally, the component requires the following parameters to work:
- `model` specifies the Whisper model.
- `api_base_url` specifies the OpenAI base URL and defaults to `"https://api.openai.com/v1"`. If you are using Whisper provider other than OpenAI set this parameter according to provider's documentation.
See other optional parameters in our [API documentation](/reference/audio-api).
See the [Whisper API documentation](https://platform.openai.com/docs/guides/speech-to-text) and the official Whisper [GitHub repo](https://github.com/openai/whisper) for the supported audio formats and languages.
## Usage
### On its own
Heres an example of how to use `RemoteWhisperTranscriber` to transcribe a local file:
```python
import requests
from haystack.components.audio import RemoteWhisperTranscriber
response = requests.get(
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
)
with open("kennedy_speech.mp3", "wb") as file:
file.write(response.content)
transcriber = RemoteWhisperTranscriber()
transcription = transcriber.run(sources=["./kennedy_speech.mp3"])
print(transcription["documents"][0].content)
```
### In a pipeline
The pipeline below fetches an audio file from a specified URL and transcribes it. It first retrieves the audio file using `LinkContentFetcher`, then transcribes the audio into text with `RemoteWhisperTranscriber`, and finally outputs the transcription text.
```python
from haystack.components.audio import RemoteWhisperTranscriber
from haystack.components.fetchers import LinkContentFetcher
from haystack import Pipeline
pipe = Pipeline()
pipe.add_component("fetcher", LinkContentFetcher())
pipe.add_component("transcriber", RemoteWhisperTranscriber())
pipe.connect("fetcher", "transcriber")
result = pipe.run(
data={
"fetcher": {
"urls": [
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
],
},
},
)
print(result["transcriber"]["documents"][0].content)
```
## Additional References
🧑‍🍳 Cookbook: [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)