chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,262 @@
|
||||
---
|
||||
title: "Agent"
|
||||
id: agent
|
||||
slug: "/agent"
|
||||
description: "The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`."
|
||||
---
|
||||
|
||||
# Agent
|
||||
|
||||
The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or user input |
|
||||
| **Mandatory init variables** | `chat_generator`: An instance of a Chat Generator that supports tools |
|
||||
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)s |
|
||||
| **Output variables** | `messages`: Chat history with tool and model responses |
|
||||
| **API reference** | [Agents](/reference/agents-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/agents/agent.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `Agent` component is a loop-based system that uses a chat-based large language model (LLM) and external tools to solve complex user queries. It works iteratively—calling tools, updating state, and generating prompts—until one of the configurable `exit_conditions` is met.
|
||||
|
||||
It can:
|
||||
|
||||
- Dynamically select tools based on user input,
|
||||
- Maintain and validate runtime state using a schema,
|
||||
- Stream token-level outputs from the LLM.
|
||||
|
||||
The `Agent` returns a dictionary containing:
|
||||
|
||||
- `messages`: the full conversation history,
|
||||
- Additional dynamic keys based on `state_schema`.
|
||||
|
||||
### Parameters
|
||||
|
||||
To initialize the `Agent` component, you need to provide it with an instance of a Chat Generator that supports tools. You can pass a list of [tools](../../tools/tool.mdx) or [`ComponentTool`](../../tools/componenttool.mdx) instances, or wrap them in a [`Toolset`](../../tools/toolset.mdx) to manage them as a group.
|
||||
|
||||
You can additionally configure:
|
||||
|
||||
- A `system_prompt` for your Agent,
|
||||
- A list of `exit_conditions` strings that will cause the agent to return. Can be either:
|
||||
- “text”, which means that the Agent will exit as soon as the LLM replies only with a text response,
|
||||
- or specific tool names.
|
||||
- A `state_schema` for one agent invocation run. It defines extra information – such as documents or context – that tools can read from or write to during execution. You can use this schema to pass parameters that tools can both produce and consume.
|
||||
- `streaming_callback` to stream the tokens from the LLM directly in output.
|
||||
|
||||
:::info
|
||||
For a complete list of available parameters, refer to the [Agents API Documentation](/reference/agents-api).
|
||||
:::
|
||||
|
||||
### Agents as Tools
|
||||
|
||||
You can wrap an `Agent` using [`ComponentTool`](../../tools/componenttool.mdx) to create multi-agent systems where specialized agents act as tools for a coordinator agent.
|
||||
|
||||
When wrapping an `Agent` as a `ComponentTool`, use the `outputs_to_string` parameter with `{"source": "last_message"}` to extract only the agent's final response text, rather than the execution trace with tool calls to keep the coordinator agent's context clean and focused.
|
||||
|
||||
```python
|
||||
## Wrap the agent as a ComponentTool with outputs_to_string
|
||||
research_tool = ComponentTool(
|
||||
component=research_agent, # another agent component
|
||||
name="research_specialist",
|
||||
description="A specialist that can research topics from the knowledge base",
|
||||
outputs_to_string={"source": "last_message"}, ## Extract only the final response
|
||||
)
|
||||
|
||||
## Create a coordinator agent that uses the specialist
|
||||
coordinator_agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
|
||||
tools=[research_tool],
|
||||
system_prompt="You are a coordinator that delegates research tasks to a specialist.",
|
||||
exit_conditions=["text"],
|
||||
)
|
||||
|
||||
## Warm up and run
|
||||
research_agent.warm_up()
|
||||
coordinator_agent.warm_up()
|
||||
|
||||
result = coordinator_agent.run(
|
||||
messages=[ChatMessage.from_user("Tell me about Haystack")],
|
||||
)
|
||||
|
||||
print(result["last_message"].text)
|
||||
```
|
||||
|
||||
### Streaming
|
||||
|
||||
You can stream output as it’s generated. Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
|
||||
|
||||
```python
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
|
||||
## Configure any `Generator` or `ChatGenerator` with a streaming callback
|
||||
component = SomeGeneratorOrChatGenerator(streaming_callback=print_streaming_chunk)
|
||||
|
||||
## If this is a `ChatGenerator`, pass a list of messages:
|
||||
## from haystack.dataclasses import ChatMessage
|
||||
## component.run([ChatMessage.from_user("Your question here")])
|
||||
|
||||
## If this is a (non-chat) `Generator`, pass a prompt:
|
||||
## component.run({"prompt": "Your prompt here"})
|
||||
```
|
||||
|
||||
:::info
|
||||
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
|
||||
:::
|
||||
|
||||
See our [Streaming Support](../generators/guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
|
||||
|
||||
Give preference to `print_streaming_chunk` by default. Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.tools.tool import Tool
|
||||
from haystack.components.agents import Agent
|
||||
from typing import List
|
||||
|
||||
|
||||
## Tool Function
|
||||
def calculate(expression: str) -> dict:
|
||||
try:
|
||||
result = eval(expression, {"__builtins__": {}})
|
||||
return {"result": result}
|
||||
except Exception as e:
|
||||
return {"error": str(e)}
|
||||
|
||||
|
||||
## Tool Definition
|
||||
calculator_tool = Tool(
|
||||
name="calculator",
|
||||
description="Evaluate basic math expressions.",
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"expression": {
|
||||
"type": "string",
|
||||
"description": "Math expression to evaluate",
|
||||
},
|
||||
},
|
||||
"required": ["expression"],
|
||||
},
|
||||
function=calculate,
|
||||
outputs_to_state={"calc_result": {"source": "result"}},
|
||||
)
|
||||
|
||||
## Agent Setup
|
||||
agent = Agent(
|
||||
chat_generator=OpenAIChatGenerator(),
|
||||
tools=[calculator_tool],
|
||||
exit_conditions=["calculator"],
|
||||
state_schema={
|
||||
"calc_result": {"type": int},
|
||||
},
|
||||
)
|
||||
|
||||
## Run the Agent
|
||||
agent.warm_up()
|
||||
response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")])
|
||||
|
||||
## Output
|
||||
print(response["messages"])
|
||||
print("Calc Result:", response.get("calc_result"))
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The example pipeline below creates a database assistant using `OpenAIChatGenerator`, `LinkContentFetcher`, and custom database tool. It reads the given URL and processes the page content, then builds a prompt for the AI. The assistant uses this information to write people's names and titles from the given page to the database.
|
||||
|
||||
```python
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
|
||||
from haystack.components.converters.html import HTMLToDocument
|
||||
from haystack.components.fetchers.link_content import LinkContentFetcher
|
||||
from haystack.core.pipeline import Pipeline
|
||||
from haystack.tools import tool
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from typing import Optional
|
||||
from haystack.dataclasses import ChatMessage, Document
|
||||
|
||||
document_store = InMemoryDocumentStore() # create a document store or an SQL database
|
||||
|
||||
|
||||
@tool
|
||||
def add_database_tool(
|
||||
name: str,
|
||||
surname: str,
|
||||
job_title: Optional[str],
|
||||
other: Optional[str],
|
||||
):
|
||||
"""Use this tool to add names to the database with information about them"""
|
||||
document_store.write_documents(
|
||||
[
|
||||
Document(
|
||||
content=name + " " + surname + " " + (job_title or ""),
|
||||
meta={"other": other},
|
||||
),
|
||||
],
|
||||
)
|
||||
return
|
||||
|
||||
|
||||
database_asistant = Agent(
|
||||
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
|
||||
tools=[add_database_tool],
|
||||
system_prompt="""
|
||||
You are a database assistant.
|
||||
Your task is to extract the names of people mentioned in the given context and add them to a knowledge base, along with additional relevant information about them that can be extracted from the context.
|
||||
Do not use you own knowledge, stay grounded to the given context.
|
||||
Do not ask the user for confirmation. Instead, automatically update the knowledge base and return a brief summary of the people added, including the information stored for each.
|
||||
""",
|
||||
exit_conditions=["text"],
|
||||
max_agent_steps=100,
|
||||
raise_on_tool_invocation_failure=False,
|
||||
)
|
||||
|
||||
extraction_agent = Pipeline()
|
||||
extraction_agent.add_component("fetcher", LinkContentFetcher())
|
||||
extraction_agent.add_component("converter", HTMLToDocument())
|
||||
extraction_agent.add_component(
|
||||
"builder",
|
||||
ChatPromptBuilder(
|
||||
template=[
|
||||
ChatMessage.from_user("""
|
||||
{% for doc in docs %}
|
||||
{{ doc.content|default|truncate(25000) }}
|
||||
{% endfor %}
|
||||
"""),
|
||||
],
|
||||
required_variables=["docs"],
|
||||
),
|
||||
)
|
||||
|
||||
extraction_agent.add_component("database_agent", database_asistant)
|
||||
extraction_agent.connect("fetcher.streams", "converter.sources")
|
||||
extraction_agent.connect("converter.documents", "builder.docs")
|
||||
extraction_agent.connect("builder", "database_agent")
|
||||
|
||||
agent_output = extraction_agent.run(
|
||||
{"fetcher": {"urls": ["https://en.wikipedia.org/wiki/Deepset"]}},
|
||||
)
|
||||
|
||||
print(agent_output["database_agent"]["messages"][-1].text)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Build a GitHub Issue Resolver Agent](https://haystack.deepset.ai/cookbook/github_issue_resolver_agent)
|
||||
|
||||
📓 Tutorials:
|
||||
- [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)
|
||||
- [Creating a Multi-Agent System](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system)
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
title: "Audio"
|
||||
id: audio
|
||||
slug: "/audio"
|
||||
description: "Use these components to work with audio in Haystack by transcribing files or converting text to audio."
|
||||
---
|
||||
|
||||
# Audio
|
||||
|
||||
Use these components to work with audio in Haystack by transcribing files or converting text to audio.
|
||||
|
||||
| Name | Description |
|
||||
| --- | --- |
|
||||
| [LocalWhisperTranscriber](audio/localwhispertranscriber.mdx) | Transcribe audio files using OpenAI's Whisper model using your local installation of Whisper. |
|
||||
| [RemoteWhisperTranscriber](audio/remotewhispertranscriber.mdx) | Transcribe audio files using OpenAI's Whisper model. |
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
---
|
||||
title: "External Integrations"
|
||||
id: external-integrations-audio
|
||||
slug: "/external-integrations-audio"
|
||||
description: "External integrations that enable working with audio in Haystack by transcribing files or converting text to audio."
|
||||
---
|
||||
|
||||
# External Integrations
|
||||
|
||||
External integrations that enable working with audio in Haystack by transcribing files or converting text to audio.
|
||||
|
||||
| Name | Description |
|
||||
| --- | --- |
|
||||
| [AssemblyAI](https://haystack.deepset.ai/integrations/assemblyai) | Perform speech recognition, speaker diarization and summarization. |
|
||||
| [Elevenlabs](https://haystack.deepset.ai/integrations/elevenlabs) | Convert text to speech using ElevenLabs’ API. |
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
---
|
||||
title: "LocalWhisperTranscriber"
|
||||
id: localwhispertranscriber
|
||||
slug: "/localwhispertranscriber"
|
||||
description: "Use `LocalWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model using your local installation of Whisper."
|
||||
---
|
||||
|
||||
# LocalWhisperTranscriber
|
||||
|
||||
Use `LocalWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model using your local installation of Whisper.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | As the first component in an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: A list of paths or binary streams that you want to transcribe |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Audio](/reference/audio-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/audio/whisper_local.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The component also needs to know which Whisper model to work with. Specify this in the `model` parameter when initializing the component. All transcription is completed on the executing machine, and the audio is never sent to a third-party provider.
|
||||
|
||||
See other optional parameters you can specify in our [API documentation](/reference/audio-api).
|
||||
|
||||
See the [Whisper API documentation](https://platform.openai.com/docs/guides/speech-to-text) and the official Whisper [GitHub repo](https://github.com/openai/whisper) for the supported audio formats and languages.
|
||||
|
||||
To work with the `LocalWhisperTranscriber`, install torch and [Whisper](https://github.com/openai/whisper) first with the following commands:
|
||||
|
||||
```python
|
||||
pip install 'transformers[torch]'
|
||||
pip install -U openai-whisper
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here’s an example of how to use `LocalWhisperTranscriber` on its own:
|
||||
|
||||
```python
|
||||
import requests
|
||||
from haystack.components.audio import LocalWhisperTranscriber
|
||||
|
||||
response = requests.get(
|
||||
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
|
||||
)
|
||||
with open("kennedy_speech.mp3", "wb") as file:
|
||||
file.write(response.content)
|
||||
|
||||
transcriber = LocalWhisperTranscriber(model="tiny")
|
||||
transcriber.warm_up()
|
||||
|
||||
transcription = transcriber.run(sources=["./kennedy_speech.mp3"])
|
||||
print(transcription["documents"][0].content)
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The pipeline below fetches an audio file from a specified URL and transcribes it. It first retrieves the audio file using `LinkContentFetcher`, then transcribes the audio into text with `LocalWhisperTranscriber`, and finally outputs the transcription text.
|
||||
|
||||
```python
|
||||
from haystack.components.audio import LocalWhisperTranscriber
|
||||
from haystack.components.fetchers import LinkContentFetcher
|
||||
from haystack import Pipeline
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("fetcher", LinkContentFetcher())
|
||||
pipe.add_component("transcriber", LocalWhisperTranscriber(model="tiny"))
|
||||
|
||||
pipe.connect("fetcher", "transcriber")
|
||||
result = pipe.run(
|
||||
data={
|
||||
"fetcher": {
|
||||
"urls": [
|
||||
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
|
||||
],
|
||||
},
|
||||
},
|
||||
)
|
||||
print(result["transcriber"]["documents"][0].content)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)
|
||||
+97
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: "RemoteWhisperTranscriber"
|
||||
id: remotewhispertranscriber
|
||||
slug: "/remotewhispertranscriber"
|
||||
description: "Use `RemoteWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model."
|
||||
---
|
||||
|
||||
# RemoteWhisperTranscriber
|
||||
|
||||
Use `RemoteWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | As the first component in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with an environment variable `OPENAI_API_KEY`. |
|
||||
| **Mandatory run variables** | `sources`: A list of paths or binary streams that you want to transcribe |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Audio](/reference/audio-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/audio/whisper_remote.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`RemoteWhisperTranscriber` works with OpenAI-compatible clients and isn't limited to just OpenAI as a provider. For example, [Groq](https://console.groq.com/docs/speech-text) offers a drop-in replacement that can be used as well. You can set the API key in one of two ways:
|
||||
|
||||
1. Through the `api_key` initialization parameter, where the key is resolved using [Secret API](../../concepts/secret-management.mdx).
|
||||
2. By setting it in the `OPENAI_API_KEY` environment variable, which the system will use to access the key.
|
||||
|
||||
```python
|
||||
from haystack.components.audio import RemoteWhisperTranscriber
|
||||
|
||||
transcriber = RemoteWhisperTranscriber()
|
||||
```
|
||||
|
||||
Additionally, the component requires the following parameters to work:
|
||||
|
||||
- `model` specifies the Whisper model.
|
||||
- `api_base_url` specifies the OpenAI base URL and defaults to `"https://api.openai.com/v1"`. If you are using Whisper provider other than OpenAI set this parameter according to provider's documentation.
|
||||
|
||||
See other optional parameters in our [API documentation](/reference/audio-api).
|
||||
|
||||
See the [Whisper API documentation](https://platform.openai.com/docs/guides/speech-to-text) and the official Whisper [GitHub repo](https://github.com/openai/whisper) for the supported audio formats and languages.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here’s an example of how to use `RemoteWhisperTranscriber` to transcribe a local file:
|
||||
|
||||
```python
|
||||
import requests
|
||||
from haystack.components.audio import RemoteWhisperTranscriber
|
||||
|
||||
response = requests.get(
|
||||
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
|
||||
)
|
||||
with open("kennedy_speech.mp3", "wb") as file:
|
||||
file.write(response.content)
|
||||
|
||||
transcriber = RemoteWhisperTranscriber()
|
||||
transcription = transcriber.run(sources=["./kennedy_speech.mp3"])
|
||||
|
||||
print(transcription["documents"][0].content)
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The pipeline below fetches an audio file from a specified URL and transcribes it. It first retrieves the audio file using `LinkContentFetcher`, then transcribes the audio into text with `RemoteWhisperTranscriber`, and finally outputs the transcription text.
|
||||
|
||||
```python
|
||||
from haystack.components.audio import RemoteWhisperTranscriber
|
||||
from haystack.components.fetchers import LinkContentFetcher
|
||||
from haystack import Pipeline
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("fetcher", LinkContentFetcher())
|
||||
pipe.add_component("transcriber", RemoteWhisperTranscriber())
|
||||
|
||||
pipe.connect("fetcher", "transcriber")
|
||||
result = pipe.run(
|
||||
data={
|
||||
"fetcher": {
|
||||
"urls": [
|
||||
"https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
|
||||
],
|
||||
},
|
||||
},
|
||||
)
|
||||
print(result["transcriber"]["documents"][0].content)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)
|
||||
@@ -0,0 +1,13 @@
|
||||
---
|
||||
title: "Builders"
|
||||
id: builders
|
||||
slug: "/builders"
|
||||
---
|
||||
|
||||
# Builders
|
||||
|
||||
| Component | Description |
|
||||
| --- | --- |
|
||||
| [AnswerBuilder](builders/answerbuilder.mdx) | Creates `GeneratedAnswer` objects from the query and the answer. |
|
||||
| [PromptBuilder](builders/promptbuilder.mdx) | Renders prompt templates with given parameters. |
|
||||
| [ChatPromptBuilder](builders/chatpromptbuilder.mdx) | PromptBuilder for chat messages. |
|
||||
+113
@@ -0,0 +1,113 @@
|
||||
---
|
||||
title: "AnswerBuilder"
|
||||
id: answerbuilder
|
||||
slug: "/answerbuilder"
|
||||
description: "Use this component in pipelines that contain a Generator to parse its replies."
|
||||
---
|
||||
|
||||
# AnswerBuilder
|
||||
|
||||
Use this component in pipelines that contain a Generator to parse its replies.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Use in pipelines (such as a RAG pipeline) after a [Generator](../generators.mdx) component to create [`GeneratedAnswer`](../../concepts/data-classes.mdx#generatedanswer) objects from its replies. |
|
||||
| **Mandatory run variables** | `query`: A query string <br /> <br />`replies`: A list of strings, or a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects that are replies from a Generator |
|
||||
| **Output variables** | `answers`: A list of `GeneratedAnswer` objects |
|
||||
| **API reference** | [Builders](/reference/builders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/answer_builder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`AnswerBuilder` takes a query and the replies a Generator returns as input and parses them into `GeneratedAnswer` objects. Optionally, it also takes documents and metadata from the Generator as inputs to enrich the `GeneratedAnswer` objects.
|
||||
|
||||
The `AnswerBuilder` works with both Chat and non-Chat Generators.
|
||||
|
||||
The optional `pattern` parameter defines how to extract answer texts from replies. It needs to be a regular expression with a maximum of one capture group. If a capture group is present, the text matched by the capture group is used as the answer. If no capture group is present, the whole match is used as the answer. If no `pattern` is set, the whole reply is used as the answer text.
|
||||
|
||||
The optional `reference_pattern` parameter can be set to a regular expression that parses referenced documents from the replies so that only those referenced documents are listed in the `GeneratedAnswer` objects. Haystack assumes that documents are referenced by their index in the list of input documents and that indices start at 1. For example, if you set the `reference_pattern` to _`\\[(\\d+)\\]`,_ it finds “1” in a string "This is an answer[1]". If `reference_pattern` is not set, all input documents are listed in the `GeneratedAnswer` objects.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Below is an example where we’re using the `AnswerBuilder` to parse a string that could be the reply received from a Generator using a custom regular expression. Any text other than the answer will not be included in the `GeneratedAnswer` object constructed by the builder.
|
||||
|
||||
```python
|
||||
from haystack.components.builders import AnswerBuilder
|
||||
|
||||
builder = AnswerBuilder(pattern="Answer: (.*)")
|
||||
builder.run(
|
||||
query="What's the answer?",
|
||||
replies=["This is an argument. Answer: This is the answer."],
|
||||
)
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Below is an example of a RAG pipeline where we use an `AnswerBuilder` to create `GeneratedAnswer` objects from the replies returned by a Generator. In addition to the text of the reply, these objects also hold the query, the referenced docs, and metadata returned by the Generator.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
|
||||
from haystack.components.builders.answer_builder import AnswerBuilder
|
||||
from haystack.utils import Secret
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.dataclasses import Document
|
||||
|
||||
prompt_template = [
|
||||
ChatMessage.from_system("You are a helpful assistant."),
|
||||
ChatMessage.from_user(
|
||||
"Given these documents, answer the question.\nDocuments:\n"
|
||||
"{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
|
||||
"Question: {{query}}\nAnswer:",
|
||||
),
|
||||
]
|
||||
|
||||
docs = [
|
||||
Document(content="The capital of France is Paris"),
|
||||
Document(content="The capital of England is London"),
|
||||
]
|
||||
document_store = InMemoryDocumentStore()
|
||||
document_store.write_documents(docs)
|
||||
|
||||
p = Pipeline()
|
||||
p.add_component(
|
||||
instance=InMemoryBM25Retriever(document_store=document_store),
|
||||
name="retriever",
|
||||
)
|
||||
p.add_component(
|
||||
instance=ChatPromptBuilder(
|
||||
template=prompt_template,
|
||||
required_variables={"query", "documents"},
|
||||
),
|
||||
name="prompt_builder",
|
||||
)
|
||||
p.add_component(
|
||||
instance=OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
|
||||
name="llm",
|
||||
)
|
||||
p.add_component(instance=AnswerBuilder(), name="answer_builder")
|
||||
p.connect("retriever", "prompt_builder.documents")
|
||||
p.connect("prompt_builder", "llm.messages")
|
||||
p.connect("llm.replies", "answer_builder.replies")
|
||||
p.connect("retriever", "answer_builder.documents")
|
||||
|
||||
query = "What is the capital of France?"
|
||||
result = p.run(
|
||||
{
|
||||
"retriever": {"query": query},
|
||||
"prompt_builder": {"query": query},
|
||||
"answer_builder": {"query": query},
|
||||
},
|
||||
)
|
||||
|
||||
print(result)
|
||||
```
|
||||
+411
@@ -0,0 +1,411 @@
|
||||
---
|
||||
title: "ChatPromptBuilder"
|
||||
id: chatpromptbuilder
|
||||
slug: "/chatpromptbuilder"
|
||||
description: "This component constructs prompts dynamically by processing chat messages."
|
||||
---
|
||||
|
||||
# ChatPromptBuilder
|
||||
|
||||
This component constructs prompts dynamically by processing chat messages.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [Generator](../generators.mdx) |
|
||||
| **Mandatory init variables** | `template`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects or a special string template. Needs to be provided either during init or run. |
|
||||
| **Mandatory run variables** | `**kwargs`: Any strings that should be used to render the prompt template. See [Variables](#variables) section for more details. |
|
||||
| **Output variables** | `prompt`: A dynamically constructed prompt |
|
||||
| **API reference** | [Builders](/reference/builders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/chat_prompt_builder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `ChatPromptBuilder` component creates prompts using static or dynamic templates written in [Jinja2](https://palletsprojects.com/p/jinja/) syntax, by processing a list of chat messages or a special string template. The templates contain placeholders like `{{ variable }}` that are filled with values provided during runtime. You can use it for static prompts set at initialization or change the templates and variables dynamically while running.
|
||||
|
||||
To use it, start by providing a list of `ChatMessage` objects or a special string as the template.
|
||||
|
||||
[`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) is a data class that includes message content, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
|
||||
|
||||
The builder looks for placeholders in the template and identifies the required variables. You can also list these variables manually. During runtime, the `run` method takes the template and the variables, fills in the placeholders, and returns the completed prompt. If required variables are missing. If the template is invalid, the builder raises an error.
|
||||
|
||||
For example, you can create a simple translation prompt:
|
||||
|
||||
```python
|
||||
template = [ChatMessage.from_user("Translate to {{ target_language }}: {{ text }}")]
|
||||
builder = ChatPromptBuilder(template=template)
|
||||
result = builder.run(target_language="French", text="Hello, how are you?")
|
||||
```
|
||||
|
||||
Or you can also replace the template at runtime with a new one:
|
||||
|
||||
```python
|
||||
new_template = [
|
||||
ChatMessage.from_user("Summarize in {{ target_language }}: {{ content }}"),
|
||||
]
|
||||
result = builder.run(
|
||||
template=new_template,
|
||||
target_language="English",
|
||||
content="A detailed paragraph.",
|
||||
)
|
||||
```
|
||||
|
||||
### Variables
|
||||
|
||||
The template variables found in the init template are used as input types for the component. If there are no `required_vairables` set, all variables are considered optional by default. In this case, any missing variables are replaced with empty strings, which can lead to unintended behavior, especially in complex pipelines.
|
||||
|
||||
Use `required_variables` and `variables` to specify the input types and required variables:
|
||||
|
||||
- `required_variables`
|
||||
- Defines which template variables must be provided when the component runs.
|
||||
- If any required variable is missing, the component raises an error and halts execution.
|
||||
- You can:
|
||||
- Pass a list of required variable names (such as `["name"]`), or
|
||||
- Use `"*"` to mark all variables in the template as required.
|
||||
|
||||
- `variables`
|
||||
- Lists all variables that can appear in the template, whether required or optional.
|
||||
- Optional variables that aren't provided are replaced with an empty string in the rendered prompt.
|
||||
- This allows partial prompts to be constructed without errors, unless a variable is marked as required.
|
||||
|
||||
In the example below, only _name_ is required to run the component, while _topic_ is only an optional variable:
|
||||
|
||||
```python
|
||||
template = [
|
||||
ChatMessage.from_user("Hello, {{ name }}. How can I assist you with {{ topic }}?"),
|
||||
]
|
||||
|
||||
builder = ChatPromptBuilder(
|
||||
template=template,
|
||||
required_variables=["name"],
|
||||
variables=["name", "topic"],
|
||||
)
|
||||
|
||||
result = builder.run(name="Alice")
|
||||
## Output: "Hello, Alice. How can I assist you with ?"
|
||||
```
|
||||
|
||||
The component only waits for the required inputs before running.
|
||||
|
||||
### Roles
|
||||
|
||||
A [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) represents a single message in the conversation and can have one of three class methods that build the chat messages: `from_user`, `from_system`, or `from_assistant`. `from_user` messages are inputs provided by the user, such as a query or request. `from_system` messages provide context or instructions to guide the LLM’s behavior, such as setting a tone or purpose for the conversation. `from_assistant` defines the expected or actual response from the LLM.
|
||||
|
||||
Here’s how the roles work together in a `ChatPromptBuilder`:
|
||||
|
||||
```python
|
||||
system_message = ChatMessage.from_system(
|
||||
"You are an assistant helping tourists in {{ language }}.",
|
||||
)
|
||||
|
||||
user_message = ChatMessage.from_user("What are the best places to visit in {{ city }}?")
|
||||
|
||||
assistant_message = ChatMessage.from_assistant(
|
||||
"The best places to visit in {{ city }} include the Eiffel Tower, Louvre Museum, and Montmartre.",
|
||||
)
|
||||
```
|
||||
|
||||
### String Templates
|
||||
|
||||
Instead of a list of `ChatMessage` objects, you can also express the template as a special string.
|
||||
|
||||
This template format allows you to define `ChatMessage` sequences using Jinja2 syntax. Each `{% message %}` block defines a single message with a specific role, and you can insert dynamic content using `{{ variables }}`.
|
||||
|
||||
Compared to using a list of `ChatMessage`s, this format is more flexible and allows including structured parts like images in the templatized `ChatMessage`; to better understand this use case, check out the [multimodal example](#multimodal) in the Usage section below.
|
||||
|
||||
### Jinja2 Time Extension
|
||||
|
||||
`PromptBuilder` supports the Jinja2 TimeExtension, which allows you to work with datetime formats.
|
||||
|
||||
The Time Extension provides two main features:
|
||||
|
||||
1. A `now` tag that gives you access to the current time,
|
||||
2. Date/time formatting capabilities through Python's datetime module.
|
||||
|
||||
To use the Jinja2 TimeExtension, you need to install a dependency with:
|
||||
|
||||
```shell
|
||||
pip install arrow>=1.3.0
|
||||
```
|
||||
|
||||
#### The `now` Tag
|
||||
|
||||
The `now` tag creates a datetime object representing the current time, which you can then store in a variable:
|
||||
|
||||
```jinja2
|
||||
{% now 'utc' as current_time %}
|
||||
The current UTC time is: {{ current_time }}
|
||||
```
|
||||
|
||||
You can specify different timezones:
|
||||
|
||||
```jinja2
|
||||
{% now 'America/New_York' as ny_time %}
|
||||
The time in New York is: {{ ny_time }}
|
||||
```
|
||||
|
||||
If you don't specify a timezone, your system's local timezone will be used:
|
||||
|
||||
```jinja2
|
||||
{% now as local_time %}
|
||||
Local time: {{ local_time }}
|
||||
```
|
||||
|
||||
#### Date Formatting
|
||||
|
||||
You can format the datetime objects using Python's `strftime` syntax:
|
||||
|
||||
```jinja2
|
||||
{% now as current_time %}
|
||||
Formatted date: {{ current_time.strftime('%Y-%m-%d %H:%M:%S') }}
|
||||
```
|
||||
|
||||
The common format codes are:
|
||||
|
||||
- `%Y`: 4-digit year (for example, 2025)
|
||||
- `%m`: Month as a zero-padded number (01-12)
|
||||
- `%d`: Day as a zero-padded number (01-31)
|
||||
- `%H`: Hour (24-hour clock) as a zero-padded number (00-23)
|
||||
- `%M`: Minute as a zero-padded number (00-59)
|
||||
- `%S`: Second as a zero-padded number (00-59)
|
||||
|
||||
#### Example
|
||||
|
||||
```python
|
||||
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
template = [
|
||||
ChatMessage.from_user("Current date is: {% now 'UTC' %}"),
|
||||
ChatMessage.from_assistant("Thank you for providing the date"),
|
||||
ChatMessage.from_user("Yesterday was: {% now 'UTC' - 'days=1' %}"),
|
||||
]
|
||||
builder = ChatPromptBuilder(template=template)
|
||||
|
||||
result = builder.run()["prompt"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
#### With static template
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
template = [
|
||||
ChatMessage.from_user(
|
||||
"Translate to {{ target_language }}. Context: {{ snippet }}; Translation:",
|
||||
),
|
||||
]
|
||||
builder = ChatPromptBuilder(template=template)
|
||||
builder.run(target_language="spanish", snippet="I can't speak spanish.")
|
||||
```
|
||||
|
||||
#### With special string template
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
template = """
|
||||
{% message role="user" %}
|
||||
Hello, my name is {{name}}!
|
||||
{% endmessage %}
|
||||
"""
|
||||
|
||||
builder = ChatPromptBuilder(template=template)
|
||||
result = builder.run(name="John")
|
||||
|
||||
assert result["prompt"] == [ChatMessage.from_user("Hello, my name is John!")]
|
||||
```
|
||||
|
||||
#### Specifying name and meta in a ChatMessage
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
template = """
|
||||
{% message role="user" name="John" meta={"key": "value"} %}
|
||||
Hello from {{country}}!
|
||||
{% endmessage %}
|
||||
"""
|
||||
|
||||
builder = ChatPromptBuilder(template=template)
|
||||
result = builder.run(country="Italy")
|
||||
assert result["prompt"] == [
|
||||
ChatMessage.from_user("Hello from Italy!", name="John", meta={"key": "value"}),
|
||||
]
|
||||
```
|
||||
|
||||
#### Multiple ChatMessages with different roles
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
template = """
|
||||
{% message role="system" %}
|
||||
You are a {{adjective}} assistant.
|
||||
{% endmessage %}
|
||||
|
||||
{% message role="user" %}
|
||||
Hello, my name is {{name}}!
|
||||
{% endmessage %}
|
||||
|
||||
{% message role="assistant" %}
|
||||
Hello, {{name}}! How can I help you today?
|
||||
{% endmessage %}
|
||||
"""
|
||||
|
||||
builder = ChatPromptBuilder(template=template)
|
||||
result = builder.run(name="John", adjective="helpful")
|
||||
assert result["prompt"] == [
|
||||
ChatMessage.from_system("You are a helpful assistant."),
|
||||
ChatMessage.from_user("Hello, my name is John!"),
|
||||
ChatMessage.from_assistant("Hello, John! How can I help you today?"),
|
||||
]
|
||||
```
|
||||
|
||||
#### Overriding static template at runtime
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
template = [
|
||||
ChatMessage.from_user(
|
||||
"Translate to {{ target_language }}. Context: {{ snippet }}; Translation:",
|
||||
),
|
||||
]
|
||||
builder = ChatPromptBuilder(template=template)
|
||||
builder.run(target_language="spanish", snippet="I can't speak spanish.")
|
||||
|
||||
summary_template = [
|
||||
ChatMessage.from_user(
|
||||
"Translate to {{ target_language }} and summarize. Context: {{ snippet }}; Summary:",
|
||||
),
|
||||
]
|
||||
builder.run(
|
||||
target_language="spanish",
|
||||
snippet="I can't speak spanish.",
|
||||
template=summary_template,
|
||||
)
|
||||
```
|
||||
|
||||
#### Multimodal
|
||||
|
||||
The `| templatize_part` filter in the example below tells the template engine to insert structured (non-text) objects, such as images, into the message content. These are treated differently from plain text and are rendered as special content parts in the final `ChatMessage`.
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.dataclasses import ChatMessage, ImageContent
|
||||
|
||||
template = """
|
||||
{% message role="user" %}
|
||||
Hello! I am {{user_name}}. What's the difference between the following images?
|
||||
{% for image in images %}
|
||||
{{ image | templatize_part }}
|
||||
{% endfor %}
|
||||
{% endmessage %}
|
||||
"""
|
||||
builder = ChatPromptBuilder(template=template)
|
||||
images = [
|
||||
ImageContent.from_file_path("apple.jpg"),
|
||||
ImageContent.from_file_path("kiwi.jpg"),
|
||||
]
|
||||
result = builder.run(user_name="John", images=images)
|
||||
|
||||
assert result["prompt"] == [
|
||||
ChatMessage.from_user(
|
||||
content_parts=[
|
||||
"Hello! I am John. What's the difference between the following images?",
|
||||
*images,
|
||||
],
|
||||
),
|
||||
]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack import Pipeline
|
||||
from haystack.utils import Secret
|
||||
|
||||
## no parameter init, we don't use any runtime template variables
|
||||
prompt_builder = ChatPromptBuilder()
|
||||
llm = OpenAIChatGenerator()
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("prompt_builder", prompt_builder)
|
||||
pipe.add_component("llm", llm)
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
location = "Berlin"
|
||||
language = "English"
|
||||
system_message = ChatMessage.from_system(
|
||||
"You are an assistant giving information to tourists in {{language}}",
|
||||
)
|
||||
messages = [system_message, ChatMessage.from_user("Tell me about {{location}}")]
|
||||
|
||||
res = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"location": location, "language": language},
|
||||
"template": messages,
|
||||
},
|
||||
},
|
||||
)
|
||||
print(res)
|
||||
```
|
||||
|
||||
Then, you could ask about the weather forecast for the said location. The `ChatPromptBuilder` fills in the template with the new `day_count` variable and passes it to an LLM once again:
|
||||
|
||||
```python
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack import Pipeline
|
||||
from haystack.utils import Secret
|
||||
|
||||
## no parameter init, we don't use any runtime template variables
|
||||
prompt_builder = ChatPromptBuilder()
|
||||
llm = OpenAIChatGenerator()
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("prompt_builder", prompt_builder)
|
||||
pipe.add_component("llm", llm)
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
location = "Berlin"
|
||||
|
||||
messages = [
|
||||
system_message,
|
||||
ChatMessage.from_user(
|
||||
"What's the weather forecast for {{location}} in the next {{day_count}} days?",
|
||||
),
|
||||
]
|
||||
res = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"location": location, "day_count": "5"},
|
||||
"template": messages,
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
print(res)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Advanced Prompt Customization for Anthropic](https://haystack.deepset.ai/cookbook/prompt_customization_for_anthropic)
|
||||
+265
@@ -0,0 +1,265 @@
|
||||
---
|
||||
title: "PromptBuilder"
|
||||
id: promptbuilder
|
||||
slug: "/promptbuilder"
|
||||
description: "Use this component in pipelines before a Generator to render a prompt template and fill in variable values."
|
||||
---
|
||||
|
||||
# PromptBuilder
|
||||
|
||||
Use this component in pipelines before a Generator to render a prompt template and fill in variable values.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | In a querying pipeline, before a [Generator](../generators.mdx) |
|
||||
| **Mandatory init variables** | `template`: A prompt template string that uses Jinja2 syntax |
|
||||
| **Mandatory run variables** | `**kwargs`: Any strings that should be used to render the prompt template. See [Variables](#variables) section for more details. |
|
||||
| **Output variables** | `prompt`: A string that represents the rendered prompt template |
|
||||
| **API reference** | [Builders](/reference/builders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/prompt_builder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`PromptBuilder` is initialized with a prompt template and renders it by filling in parameters passed through keyword arguments, `kwargs`. With `kwargs`, you can pass a variable number of keyword arguments so that any variable used in the prompt template can be specified with the desired value. Values for all variables appearing in the prompt template need to be provided through the `kwargs`.
|
||||
|
||||
The template that is provided to the `PromptBuilder` during initialization needs to conform to the [Jinja2](https://palletsprojects.com/p/jinja/) template language.
|
||||
|
||||
### Variables
|
||||
|
||||
The template variables found in the init template are used as input types for the component. If there are no `required_variables` set, all variables are considered optional by default. In this case, any missing variables are replaced with empty strings, which can lead to unintended behavior, especially in complex pipelines.
|
||||
|
||||
Use `required_variables` and `variables` to specify the input types and required variables:
|
||||
|
||||
- `required_variables`
|
||||
- Defines which template variables must be provided when the component runs.
|
||||
- If any required variable is missing, the component raises an error and halts execution.
|
||||
- You can:
|
||||
- Pass a list of required variable names (such as `["query"]`), or
|
||||
- Use `"*"` to mark all variables in the template as required.
|
||||
|
||||
- `variables`
|
||||
- Lists all variables that can appear in the template, whether required or optional.
|
||||
- Optional variables that aren't provided are replaced with an empty string in the rendered prompt.
|
||||
- This allows partial prompts to be constructed without errors, unless a variable is marked as required.
|
||||
|
||||
```python
|
||||
from haystack.components.builders import PromptBuilder
|
||||
|
||||
## All variables optional (default to empty string)
|
||||
builder = PromptBuilder(
|
||||
template="Hello {{name}}! {{greeting}}",
|
||||
required_variables=[], # or omit this parameter entirely
|
||||
)
|
||||
|
||||
## Some variables required
|
||||
builder = PromptBuilder(
|
||||
template="Hello {{name}}! {{greeting}}",
|
||||
required_variables=["name"], # 'greeting' remains optional
|
||||
)
|
||||
```
|
||||
|
||||
The component only waits for the required inputs before running.
|
||||
|
||||
### Jinja2 Time Extension
|
||||
|
||||
`PromptBuilder` supports the Jinja2 TimeExtension, which allows you to work with datetime formats.
|
||||
|
||||
The Time Extension provides two main features:
|
||||
|
||||
1. A `now` tag that gives you access to the current time,
|
||||
2. Date/time formatting capabilities through Python's datetime module.
|
||||
|
||||
To use the Jinja2 TimeExtension, you need to install a dependency with:
|
||||
|
||||
```shell
|
||||
pip install arrow>=1.3.0
|
||||
```
|
||||
|
||||
#### The `now` Tag
|
||||
|
||||
The `now` tag creates a datetime object representing the current time, which you can then store in a variable:
|
||||
|
||||
```jinja2
|
||||
{% now 'utc' as current_time %}
|
||||
The current UTC time is: {{ current_time }}
|
||||
```
|
||||
|
||||
You can specify different timezones:
|
||||
|
||||
```jinja2
|
||||
{% now 'America/New_York' as ny_time %}
|
||||
The time in New York is: {{ ny_time }}
|
||||
```
|
||||
|
||||
If you don't specify a timezone, your system's local timezone will be used:
|
||||
|
||||
```jinja2
|
||||
{% now as local_time %}
|
||||
Local time: {{ local_time }}
|
||||
```
|
||||
|
||||
#### Date Formatting
|
||||
|
||||
You can format the datetime objects using Python's `strftime` syntax:
|
||||
|
||||
```jinja2
|
||||
{% now as current_time %}
|
||||
Formatted date: {{ current_time.strftime('%Y-%m-%d %H:%M:%S') }}
|
||||
```
|
||||
|
||||
The common format codes are:
|
||||
|
||||
- `%Y`: 4-digit year (for example, 2025)
|
||||
- `%m`: Month as a zero-padded number (01-12)
|
||||
- `%d`: Day as a zero-padded number (01-31)
|
||||
- `%H`: Hour (24-hour clock) as a zero-padded number (00-23)
|
||||
- `%M`: Minute as a zero-padded number (00-59)
|
||||
- `%S`: Second as a zero-padded number (00-59)
|
||||
|
||||
#### Example
|
||||
|
||||
```python
|
||||
from haystack.components.builders import PromptBuilder
|
||||
|
||||
## Define template using Jinja-style formatting
|
||||
template = """
|
||||
Current date is: {% now 'UTC' %}
|
||||
Thank you for providing the date
|
||||
Yesterday was: {% now 'UTC' - 'days=1' %}
|
||||
"""
|
||||
|
||||
builder = PromptBuilder(template=template)
|
||||
|
||||
result = builder.run()["prompt"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Below is an example of using the `PromptBuilder` to render a prompt template and fill it with `target_language` and `snippet`. The PromptBuilder returns a prompt with the string `Translate the following context to spanish. Context: I can't speak spanish.; Translation:`.
|
||||
|
||||
```python
|
||||
from haystack.components.builders import PromptBuilder
|
||||
|
||||
template = "Translate the following context to {{ target_language }}. Context: {{ snippet }}; Translation:"
|
||||
builder = PromptBuilder(template=template)
|
||||
builder.run(target_language="spanish", snippet="I can't speak spanish.")
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Below is an example of a RAG pipeline where we use a `PromptBuilder` to render a custom prompt template and fill it with the contents of retrieved documents and a query. The rendered prompt is then sent to a Generator.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline, Document
|
||||
from haystack.utils import Secret
|
||||
from haystack.components.generators import OpenAIGenerator
|
||||
from haystack.components.builders.prompt_builder import PromptBuilder
|
||||
|
||||
## in a real world use case documents could come from a retriever, web, or any other source
|
||||
documents = [
|
||||
Document(content="Joe lives in Berlin"),
|
||||
Document(content="Joe is a software engineer"),
|
||||
]
|
||||
prompt_template = """
|
||||
Given these documents, answer the question.\nDocuments:
|
||||
{% for doc in documents %}
|
||||
{{ doc.content }}
|
||||
{% endfor %}
|
||||
|
||||
\nQuestion: {{query}}
|
||||
\nAnswer:
|
||||
"""
|
||||
p = Pipeline()
|
||||
p.add_component(instance=PromptBuilder(template=prompt_template), name="prompt_builder")
|
||||
p.add_component(
|
||||
instance=OpenAIGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
|
||||
name="llm",
|
||||
)
|
||||
p.connect("prompt_builder", "llm")
|
||||
|
||||
question = "Where does Joe live?"
|
||||
result = p.run({"prompt_builder": {"documents": documents, "query": question}})
|
||||
print(result)
|
||||
```
|
||||
|
||||
#### Changing the template at runtime (Prompt Engineering)
|
||||
|
||||
`PromptBuilder` allows you to switch the prompt template of an existing pipeline. The example below builds on top of the existing pipeline in the previous section. We are invoking the existing pipeline with a new prompt template:
|
||||
|
||||
```python
|
||||
documents = [
|
||||
Document(content="Joe lives in Berlin", meta={"name": "doc1"}),
|
||||
Document(content="Joe is a software engineer", meta={"name": "doc1"}),
|
||||
]
|
||||
new_template = """
|
||||
You are a helpful assistant.
|
||||
Given these documents, answer the question.
|
||||
Documents:
|
||||
{% for doc in documents %}
|
||||
Document {{ loop.index }}:
|
||||
Document name: {{ doc.meta['name'] }}
|
||||
{{ doc.content }}
|
||||
{% endfor %}
|
||||
|
||||
Question: {{ query }}
|
||||
Answer:
|
||||
"""
|
||||
p.run(
|
||||
{
|
||||
"prompt_builder": {
|
||||
"documents": documents,
|
||||
"query": question,
|
||||
"template": new_template,
|
||||
},
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
If you want to use different variables during prompt engineering than in the default template, you can do so by setting `PromptBuilder`'s variables init parameter accordingly.
|
||||
|
||||
#### Overwriting variables at runtime
|
||||
|
||||
In case you want to overwrite the values of variables, you can use `template_variables` during runtime, as shown below:
|
||||
|
||||
```python
|
||||
language_template = """
|
||||
You are a helpful assistant.
|
||||
Given these documents, answer the question.
|
||||
Documents:
|
||||
{% for doc in documents %}
|
||||
Document {{ loop.index }}:
|
||||
Document name: {{ doc.meta['name'] }}
|
||||
{{ doc.content }}
|
||||
{% endfor %}
|
||||
|
||||
Question: {{ query }}
|
||||
Please provide your answer in {{ answer_language | default('English') }}
|
||||
Answer:
|
||||
"""
|
||||
p.run(
|
||||
{
|
||||
"prompt_builder": {
|
||||
"documents": documents,
|
||||
"query": question,
|
||||
"template": language_template,
|
||||
"template_variables": {"answer_language": "German"},
|
||||
},
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
Note that `language_template` introduces `answer_language` variable which is not bound to any pipeline variable. If not set otherwise, it would use its default value, "English". In this example, we overwrite its value to "German".
|
||||
The `template_variables` allows you to overwrite pipeline variables (such as documents) as well.
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbooks:
|
||||
|
||||
- [Advanced Prompt Customization for Anthropic](https://haystack.deepset.ai/cookbook/prompt_customization_for_anthropic)
|
||||
- [Prompt Optimization with DSPy](https://haystack.deepset.ai/cookbook/prompt_optimization_with_dspy)
|
||||
+105
@@ -0,0 +1,105 @@
|
||||
---
|
||||
title: "CacheChecker"
|
||||
id: cachechecker
|
||||
slug: "/cachechecker"
|
||||
description: "This component checks for the presence of documents in a Document Store based on a specified cache field."
|
||||
---
|
||||
|
||||
# CacheChecker
|
||||
|
||||
This component checks for the presence of documents in a Document Store based on a specified cache field.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Flexible |
|
||||
| **Mandatory init variables** | `document_store`: A Document Store instance <br /> <br />`cache_field`: Name of the document's metadata field |
|
||||
| **Mandatory run variables** | `items`: A list of values associated with the `cache_field` in documents |
|
||||
| **Output variables** | `hits`: A list of documents that were found with the specified value in cache <br /> <br />`misses`: A list of values that could not be found |
|
||||
| **API reference** | [Caching](/reference/caching-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/caching/cache_checker.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`CacheChecker` checks if a Document Store contains any document with a value in the `cache_field` that matches any of the values provided in the `items` input variable. It returns a dictionary with two keys: `"hits"` and `"misses"`. The values are lists of documents that were found in the cache and items that were not, respectively.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.caching import CacheChecker
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
my_doc_store = InMemoryDocumentStore()
|
||||
|
||||
## For URL-based caching
|
||||
cache_checker = CacheChecker(document_store=my_doc_store, cache_field="url")
|
||||
cache_check_results = cache_checker.run(
|
||||
items=[
|
||||
"https://example.com/resource",
|
||||
"https://another_example.com/other_resources",
|
||||
],
|
||||
)
|
||||
print(
|
||||
cache_check_results["hits"],
|
||||
) # List of Documents that were found in the cache: all of these have 'url': <one of the above> in the metadata
|
||||
print(
|
||||
cache_check_results["misses"],
|
||||
) # URLs that were not found in the cache, like ["https://example.com/resource"]
|
||||
|
||||
## For caching based on a custom identifier
|
||||
cache_checker = CacheChecker(document_store=my_doc_store, cache_field="metadata_field")
|
||||
cache_check_results = cache_checker.run(items=["12345", "ABCDE"])
|
||||
print(
|
||||
cache_check_results["hits"],
|
||||
) # Documents that were found in the cache: all of these have 'metadata_field': <one of the above> in the metadata
|
||||
print(
|
||||
cache_check_results["misses"],
|
||||
) # Values that were not found in the cache, like: ["ABCDE"]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters import TextFileToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.caching import CacheChecker
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
pipeline = Pipeline()
|
||||
document_store = InMemoryDocumentStore()
|
||||
pipeline.add_component(
|
||||
instance=CacheChecker(document_store, cache_field="meta.file_path"),
|
||||
name="cache_checker",
|
||||
)
|
||||
pipeline.add_component(instance=TextFileToDocument(), name="text_file_converter")
|
||||
pipeline.add_component(instance=DocumentCleaner(), name="cleaner")
|
||||
pipeline.add_component(
|
||||
instance=DocumentSplitter(split_by="sentence", split_length=250, split_overlap=30),
|
||||
name="splitter",
|
||||
)
|
||||
pipeline.add_component(
|
||||
instance=DocumentWriter(document_store=document_store),
|
||||
name="writer",
|
||||
)
|
||||
pipeline.connect("cache_checker.misses", "text_file_converter.sources")
|
||||
pipeline.connect("text_file_converter.documents", "cleaner.documents")
|
||||
pipeline.connect("cleaner.documents", "splitter.documents")
|
||||
pipeline.connect("splitter.documents", "writer.documents")
|
||||
|
||||
pipeline.draw("pipeline.png")
|
||||
|
||||
## Take the current directory as input and run the pipeline
|
||||
result = pipeline.run({"cache_checker": {"items": ["code_of_conduct_1.txt"]}})
|
||||
print(result)
|
||||
|
||||
## The second execution skips the files that were already processed
|
||||
result = pipeline.run({"cache_checker": {"items": ["code_of_conduct_1.txt"]}})
|
||||
print(result)
|
||||
```
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
title: "Classifiers"
|
||||
id: classifiers
|
||||
slug: "/classifiers"
|
||||
description: "Use Classifiers to classify your documents by specific traits and update the metadata."
|
||||
---
|
||||
|
||||
# Classifiers
|
||||
|
||||
Use Classifiers to classify your documents by specific traits and update the metadata.
|
||||
|
||||
| Classifier | Description |
|
||||
| --- | --- |
|
||||
| [DocumentLanguageClassifier](classifiers/documentlanguageclassifier.mdx) | Classify documents by language. |
|
||||
| [TransformersZeroShotDocumentClassifier](classifiers/transformerszeroshotdocumentclassifier.mdx) | Classify the documents based on the provided labels. |
|
||||
+117
@@ -0,0 +1,117 @@
|
||||
---
|
||||
title: "DocumentLanguageClassifier"
|
||||
id: documentlanguageclassifier
|
||||
slug: "/documentlanguageclassifier"
|
||||
description: "Use this component to classify documents by language and add language information to metadata."
|
||||
---
|
||||
|
||||
# DocumentLanguageClassifier
|
||||
|
||||
Use this component to classify documents by language and add language information to metadata.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [`MetadataRouter`](../routers/metadatarouter.mdx) |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Classifiers](/reference/classifiers-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/classifiers/document_language_classifier.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`DocumentLanguageClassifier` classifies the language of documents and adds the detected language to their metadata. If a document's text does not match any of the languages specified at initialization, it is classified as "unmatched". By default, the classifier classifies for English (”en”) documents, with the rest being classified as “unmatched”.
|
||||
|
||||
The set of supported languages can be specified in the init method with the `languages` variable, using ISO codes.
|
||||
|
||||
To route your documents to various branches of the pipeline based on the language, use `MetadataRouter` component right after `DocumentLanguageClassifier`.
|
||||
|
||||
For classifying and then routing plain text using the same logic, use the `TextLanguageRouter` component instead.
|
||||
|
||||
## Usage
|
||||
|
||||
Install the `langdetect`package to use the `DocumentLanguageClassifier`component:
|
||||
|
||||
```shell shell
|
||||
pip install langdetect
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
Below, we are using the `DocumentLanguageClassifier` to classify English and German documents:
|
||||
|
||||
```python
|
||||
from haystack.components.classifiers import DocumentLanguageClassifier
|
||||
from haystack import Document
|
||||
|
||||
documents = [
|
||||
Document(content="Mein Name ist Jean und ich wohne in Paris."),
|
||||
Document(content="Mein Name ist Mark und ich wohne in Berlin."),
|
||||
Document(content="Mein Name ist Giorgio und ich wohne in Rome."),
|
||||
Document(content="My name is Pierre and I live in Paris"),
|
||||
Document(content="My name is Paul and I live in Berlin."),
|
||||
Document(content="My name is Alessia and I live in Rome."),
|
||||
]
|
||||
|
||||
document_classifier = DocumentLanguageClassifier(languages=["en", "de"])
|
||||
document_classifier.run(documents=documents)
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Below, we are using the `DocumentLanguageClassifier` in an indexing pipeline that indexes English and German documents into two difference indexes in an `InMemoryDocumentStore`, using embedding models for each language.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack import Document
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.classifiers import DocumentLanguageClassifier
|
||||
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.routers import MetadataRouter
|
||||
|
||||
document_store_en = InMemoryDocumentStore()
|
||||
document_store_de = InMemoryDocumentStore()
|
||||
|
||||
document_classifier = DocumentLanguageClassifier(languages=["en", "de"])
|
||||
metadata_router = MetadataRouter(
|
||||
rules={"en": {"language": {"$eq": "en"}}, "de": {"language": {"$eq": "de"}}},
|
||||
)
|
||||
english_embedder = SentenceTransformersDocumentEmbedder()
|
||||
german_embedder = SentenceTransformersDocumentEmbedder(
|
||||
model="PM-AI/bi-encoder_msmarco_bert-base_german",
|
||||
)
|
||||
en_writer = DocumentWriter(document_store=document_store_en)
|
||||
de_writer = DocumentWriter(document_store=document_store_de)
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component(
|
||||
instance=document_classifier,
|
||||
name="document_classifier",
|
||||
)
|
||||
indexing_pipeline.add_component(instance=metadata_router, name="metadata_router")
|
||||
indexing_pipeline.add_component(instance=english_embedder, name="english_embedder")
|
||||
indexing_pipeline.add_component(instance=german_embedder, name="german_embedder")
|
||||
indexing_pipeline.add_component(instance=en_writer, name="en_writer")
|
||||
indexing_pipeline.add_component(instance=de_writer, name="de_writer")
|
||||
|
||||
indexing_pipeline.connect("document_classifier.documents", "metadata_router.documents")
|
||||
indexing_pipeline.connect("metadata_router.en", "english_embedder.documents")
|
||||
indexing_pipeline.connect("metadata_router.de", "german_embedder.documents")
|
||||
indexing_pipeline.connect("english_embedder", "en_writer")
|
||||
indexing_pipeline.connect("german_embedder", "de_writer")
|
||||
|
||||
indexing_pipeline.run(
|
||||
{
|
||||
"document_classifier": {
|
||||
"documents": [
|
||||
Document(content="This is an English sentence."),
|
||||
Document(content="Dies ist ein deutscher Satz."),
|
||||
],
|
||||
},
|
||||
},
|
||||
)
|
||||
```
|
||||
+107
@@ -0,0 +1,107 @@
|
||||
---
|
||||
title: "TransformersZeroShotDocumentClassifier"
|
||||
id: transformerszeroshotdocumentclassifier
|
||||
slug: "/transformerszeroshotdocumentclassifier"
|
||||
description: "Classifies the documents based on the provided labels and adds them to their metadata."
|
||||
---
|
||||
|
||||
# TransformersZeroShotDocumentClassifier
|
||||
|
||||
Classifies the documents based on the provided labels and adds them to their metadata.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [MetadataRouter](../routers/metadatarouter.mdx) |
|
||||
| **Mandatory init variables** | `model`: The name or path of a Hugging Face model for zero shot document classification <br /> <br />`labels`: The set of possible class labels to classify each document into, for example, [`positive`, `negative`]. The labels depend on the selected model. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to classify |
|
||||
| **Output variables** | `documents`: A list of processed documents with an added `classification` metadata field |
|
||||
| **API reference** | [Classifiers](/reference/classifiers-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/classifiers/zero_shot_document_classifier.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `TransformersZeroShotDocumentClassifier` component performs zero-shot classification of documents based on the labels that you set and adds the predicted label to their metadata.
|
||||
|
||||
The component uses a Hugging Face pipeline for zero-shot classification.
|
||||
To initialize the component, provide the model and the set of labels to be used for categorization.
|
||||
You can additionally configure the component to allow multiple labels to be true with the `multi_label` boolean set to True.
|
||||
|
||||
Classification is run on the document's content field by default. If you want it to run on another field, set the`classification_field` to one of the document's metadata fields.
|
||||
|
||||
The classification results are stored in the `classification` dictionary within each document's metadata. If `multi_label` is set to `True`, you will find the scores for each label under the `details` key within the `classification` dictionary.
|
||||
|
||||
Available models for the task of zero-shot-classification are:
|
||||
- `valhalla/distilbart-mnli-12-3`
|
||||
- `cross-encoder/nli-distilroberta-base`
|
||||
- `cross-encoder/nli-deberta-v3-xsmall`
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.classifiers import TransformersZeroShotDocumentClassifier
|
||||
|
||||
documents = [
|
||||
Document(id="0", content="Cats don't get teeth cavities."),
|
||||
Document(id="1", content="Cucumbers can be grown in water."),
|
||||
]
|
||||
|
||||
document_classifier = TransformersZeroShotDocumentClassifier(
|
||||
model="cross-encoder/nli-deberta-v3-xsmall",
|
||||
labels=["animals", "food"],
|
||||
)
|
||||
|
||||
document_classifier.warm_up()
|
||||
document_classifier.run(documents=documents)
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The following is a pipeline that classifies documents based on predefined classification labels
|
||||
retrieved from a search pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.core.pipeline import Pipeline
|
||||
from haystack.components.classifiers import TransformersZeroShotDocumentClassifier
|
||||
|
||||
documents = [
|
||||
Document(id="0", content="Today was a nice day!"),
|
||||
Document(id="1", content="Yesterday was a bad day!"),
|
||||
]
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
retriever = InMemoryBM25Retriever(document_store=document_store)
|
||||
document_classifier = TransformersZeroShotDocumentClassifier(
|
||||
model="cross-encoder/nli-deberta-v3-xsmall",
|
||||
labels=["positive", "negative"],
|
||||
)
|
||||
|
||||
document_store.write_documents(documents)
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component(instance=retriever, name="retriever")
|
||||
pipeline.add_component(instance=document_classifier, name="document_classifier")
|
||||
pipeline.connect("retriever", "document_classifier")
|
||||
|
||||
queries = ["How was your day today?", "How was your day yesterday?"]
|
||||
expected_predictions = ["positive", "negative"]
|
||||
|
||||
for idx, query in enumerate(queries):
|
||||
result = pipeline.run({"retriever": {"query": query, "top_k": 1}})
|
||||
assert result["document_classifier"]["documents"][0].to_dict()["id"] == str(idx)
|
||||
assert (
|
||||
result["document_classifier"]["documents"][0].to_dict()["classification"][
|
||||
"label"
|
||||
]
|
||||
== expected_predictions[idx]
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
title: "Connectors"
|
||||
id: connectors
|
||||
slug: "/connectors"
|
||||
description: "These are Haystack integrations that connect your pipelines to services by external providers."
|
||||
---
|
||||
|
||||
# Connectors
|
||||
|
||||
These are Haystack integrations that connect your pipelines to services by external providers.
|
||||
|
||||
| Component | Description |
|
||||
| --- | --- |
|
||||
| [GitHubFileEditor](connectors/githubfileeditor.mdx) | Enables editing files in GitHub repositories through the GitHub API. |
|
||||
| [GitHubIssueCommenter](connectors/githubissuecommenter.mdx) | Enables posting comments to GitHub issues using the GitHub API. |
|
||||
| [GitHubIssueViewer](connectors/githubissueviewer.mdx) | Enables fetching and parsing GitHub issues into Haystack documents. |
|
||||
| [GitHubPRCreator](connectors/githubprcreator.mdx) | Enables creating pull requests from a fork back to the original repository through the GitHub API. |
|
||||
| [GitHubRepoForker](connectors/githubrepoforker.mdx) | Enables forking a GitHub repository from an issue URL through the GitHub API. |
|
||||
| [GitHubRepoViewer](connectors/githubrepoviewer.mdx) | Enables navigating and fetching content from GitHub repositories through the GitHub API. |
|
||||
| [JinaReaderConnector](connectors/jinareaderconnector.mdx) | Use Jina AI’s Reader API with Haystack. |
|
||||
| [LangfuseConnector](connectors/langfuseconnector.mdx) | Enables tracing in Haystack pipelines using Langfuse. |
|
||||
| [OpenAPIConnector](connectors/openapiconnector.mdx) | Acts as an interface between the Haystack ecosystem and OpenAPI services, using explicit input arguments. |
|
||||
| [OpenAPIServiceConnector](connectors/openapiserviceconnector.mdx) | Acts as an interface between the Haystack ecosystem and OpenAPI services. |
|
||||
| [WeaveConnector](connectors/weaveconnector.mdx) | Connects you to Weights & Biases Weave framework for tracing and monitoring your pipeline components. |
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
---
|
||||
title: "External Integrations"
|
||||
id: external-integrations-connectors
|
||||
slug: "/external-integrations-connectors"
|
||||
description: "External integrations that connect your pipelines to services by external providers."
|
||||
---
|
||||
|
||||
# External Integrations
|
||||
|
||||
External integrations that connect your pipelines to services by external providers.
|
||||
|
||||
| Name | Description |
|
||||
| --- | --- |
|
||||
| [Arize AI](https://haystack.deepset.ai/integrations/arize) | Trace and evaluate your Haystack pipelines with Arize AI. |
|
||||
| [Arize Phoenix](https://haystack.deepset.ai/integrations/arize-phoenix) | Trace and evaluate your Haystack pipelines with Arize Phoenix. |
|
||||
| [Context AI](https://haystack.deepset.ai/integrations/context-ai) | Log conversations for analytics by Context.ai |
|
||||
| [Opik](https://haystack.deepset.ai/integrations/opik) | Trace and evaluate your Haystack pipelines with Opik platform. |
|
||||
| [Traceloop](https://haystack.deepset.ai/integrations/traceloop) | Evaluate and monitor the quality of your LLM apps and agents |
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
---
|
||||
title: "GitHubFileEditor"
|
||||
id: githubfileeditor
|
||||
slug: "/githubfileeditor"
|
||||
description: "This is a component for editing files in GitHub repositories through the GitHub API."
|
||||
---
|
||||
|
||||
# GitHubFileEditor
|
||||
|
||||
This is a component for editing files in GitHub repositories through the GitHub API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | After a Chat Generator, or right at the beginning of a pipeline |
|
||||
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
|
||||
| **Mandatory run variables** | `command`: Operation type (edit, create, delete, undo) <br /> <br />`payload`: Command-specific parameters |
|
||||
| **Output variables** | `result`: String that indicates the operation result |
|
||||
| **API reference** | [GitHub](/reference/integrations-github) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`GitHubFileEditor` supports multiple file operations, including editing existing files, creating new files, deleting files, and undoing recent changes.
|
||||
|
||||
There are four main commands:
|
||||
|
||||
- **EDIT**: Edit an existing file by replacing specific content
|
||||
- **CREATE**: Create a new file with specified content
|
||||
- **DELETE**: Delete an existing file
|
||||
- **UNDO**: Revert the last commit if made by the same user
|
||||
|
||||
### Authorization
|
||||
|
||||
This component requires GitHub authentication with a personal access token. You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
|
||||
|
||||
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens). Make sure to grant the appropriate permissions for repository access and content management.
|
||||
|
||||
### Installation
|
||||
|
||||
Install the GitHub integration with pip:
|
||||
|
||||
```shell
|
||||
pip install github-haystack
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
:::info[Repository Placeholder]
|
||||
|
||||
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
|
||||
:::
|
||||
|
||||
### On its own
|
||||
|
||||
Editing an existing file:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.github import GitHubFileEditor, Command
|
||||
|
||||
editor = GitHubFileEditor(repo="owner/repo", branch="main")
|
||||
|
||||
result = editor.run(
|
||||
command=Command.EDIT,
|
||||
payload={
|
||||
"path": "src/example.py",
|
||||
"original": "def old_function():",
|
||||
"replacement": "def new_function():",
|
||||
"message": "Renamed function for clarity",
|
||||
},
|
||||
)
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
```bash
|
||||
{'result': 'Edit successful'}
|
||||
```
|
||||
|
||||
Creating a new file:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.github import GitHubFileEditor, Command
|
||||
|
||||
editor = GitHubFileEditor(repo="owner/repo")
|
||||
|
||||
result = editor.run(
|
||||
command=Command.CREATE,
|
||||
payload={
|
||||
"path": "docs/new_file.md",
|
||||
"content": "# New Documentation\n\nThis is a new file.",
|
||||
"message": "Add new documentation file",
|
||||
},
|
||||
)
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
```bash
|
||||
{'result': 'File created successfully'}
|
||||
```
|
||||
+132
@@ -0,0 +1,132 @@
|
||||
---
|
||||
title: "GitHubIssueCommenter"
|
||||
id: githubissuecommenter
|
||||
slug: "/githubissuecommenter"
|
||||
description: "This component posts comments to GitHub issues using the GitHub API."
|
||||
---
|
||||
|
||||
# GitHubIssueCommenter
|
||||
|
||||
This component posts comments to GitHub issues using the GitHub API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | After a Chat Generator that provides the comment text to post or right at the beginning of a pipeline |
|
||||
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
|
||||
| **Mandatory run variables** | `url`: A GitHub issue URL <br /> <br />`comment`: Comment text to post |
|
||||
| **Output variables** | `success`: Boolean indicating whether the comment was posted successfully |
|
||||
| **API reference** | [GitHub](/reference/integrations-github) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`GitHubIssueCommenter` takes a GitHub issue URL and comment text, then posts the comment to the specified issue.
|
||||
|
||||
The component requires authentication with a GitHub personal access token since posting comments is an authenticated operation.
|
||||
|
||||
### Authorization
|
||||
|
||||
This component requires GitHub authentication with a personal access token. You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
|
||||
|
||||
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens). Make sure to grant the appropriate permissions for repository access and issue management.
|
||||
|
||||
### Installation
|
||||
|
||||
Install the GitHub integration with pip:
|
||||
|
||||
```shell
|
||||
pip install github-haystack
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
:::info[Repository Placeholder]
|
||||
|
||||
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
|
||||
:::
|
||||
|
||||
### On its own
|
||||
|
||||
Basic usage with environment variable authentication:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.github import GitHubIssueCommenter
|
||||
|
||||
commenter = GitHubIssueCommenter()
|
||||
result = commenter.run(
|
||||
url="https://github.com/owner/repo/issues/123",
|
||||
comment="Thanks for reporting this issue! We'll look into it.",
|
||||
)
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
```bash
|
||||
{'success': True}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The following pipeline analyzes a GitHub issue and automatically posts a response:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
|
||||
from haystack.components.converters import OutputAdapter
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack_integrations.components.connectors.github import (
|
||||
GitHubIssueViewer,
|
||||
GitHubIssueCommenter,
|
||||
)
|
||||
|
||||
issue_viewer = GitHubIssueViewer()
|
||||
issue_commenter = GitHubIssueCommenter()
|
||||
|
||||
prompt_template = [
|
||||
ChatMessage.from_system(
|
||||
"You are a helpful assistant that analyzes GitHub issues and creates appropriate responses.",
|
||||
),
|
||||
ChatMessage.from_user(
|
||||
"Based on the following GitHub issue:\n"
|
||||
"{% for document in documents %}"
|
||||
"{% if document.meta.type == 'issue' %}"
|
||||
"**Issue Title:** {{ document.meta.title }}\n"
|
||||
"**Issue Description:** {{ document.content }}\n"
|
||||
"{% endif %}"
|
||||
"{% endfor %}\n"
|
||||
"Generate a helpful response comment for this issue. Keep it professional and concise.",
|
||||
),
|
||||
]
|
||||
|
||||
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
|
||||
llm = OpenAIChatGenerator(model="gpt-4o-mini")
|
||||
adapter = OutputAdapter(template="{{ replies[-1].text }}", output_type=str)
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("issue_viewer", issue_viewer)
|
||||
pipeline.add_component("prompt_builder", prompt_builder)
|
||||
pipeline.add_component("llm", llm)
|
||||
pipeline.add_component("adapter", adapter)
|
||||
pipeline.add_component("issue_commenter", issue_commenter)
|
||||
|
||||
pipeline.connect("issue_viewer.documents", "prompt_builder.documents")
|
||||
pipeline.connect("prompt_builder.prompt", "llm.messages")
|
||||
pipeline.connect("llm.replies", "adapter.replies")
|
||||
pipeline.connect("adapter", "issue_commenter.comment")
|
||||
|
||||
issue_url = "https://github.com/owner/repo/issues/123"
|
||||
result = pipeline.run(
|
||||
data={"issue_viewer": {"url": issue_url}, "issue_commenter": {"url": issue_url}},
|
||||
)
|
||||
|
||||
print(f"Comment posted successfully: {result['issue_commenter']['success']}")
|
||||
```
|
||||
|
||||
```
|
||||
Comment posted successfully: True
|
||||
```
|
||||
+126
@@ -0,0 +1,126 @@
|
||||
---
|
||||
title: "GitHubIssueViewer"
|
||||
id: githubissueviewer
|
||||
slug: "/githubissueviewer"
|
||||
description: "This component fetches and parses GitHub issues into Haystack documents."
|
||||
---
|
||||
|
||||
# GitHubIssueViewer
|
||||
|
||||
This component fetches and parses GitHub issues into Haystack documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Right at the beginning of a pipeline and before a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) that expects the content of a GitHub issue as input |
|
||||
| **Mandatory run variables** | `url`: A GitHub issue URL |
|
||||
| **Output variables** | `documents`: A list of documents containing the main issue and its comments |
|
||||
| **API reference** | [GitHub](/reference/integrations-github) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`GitHubIssueViewer` takes a GitHub issue URL and returns a list of documents where:
|
||||
|
||||
- The first document contains the main issue content
|
||||
- Subsequent documents contain the issue comments (if any)
|
||||
|
||||
Each document includes rich metadata such as the issue title, number, state, creation date, author, and more.
|
||||
|
||||
### Authorization
|
||||
|
||||
The component can work without authentication for public repositories, but for private repositories or to avoid rate limiting, you can provide a GitHub personal access token.
|
||||
|
||||
You can set the token using the `GITHUB_API_KEY` environment variable, or pass it directly during initialization via the `github_token` parameter.
|
||||
|
||||
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens).
|
||||
|
||||
### Installation
|
||||
|
||||
Install the GitHub integration with pip:
|
||||
|
||||
```shell
|
||||
pip install github-haystack
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
:::info[Repository Placeholder]
|
||||
|
||||
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
|
||||
:::
|
||||
|
||||
### On its own
|
||||
|
||||
Basic usage without authentication:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.github import GitHubIssueViewer
|
||||
|
||||
viewer = GitHubIssueViewer()
|
||||
result = viewer.run(url="https://github.com/deepset-ai/haystack/issues/123")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
```bash
|
||||
{'documents': [Document(id=3989459bbd8c2a8420a9ba7f3cd3cf79bb41d78bd0738882e57d509e1293c67a, content: 'sentence-transformers = 0.2.6.1
|
||||
haystack = latest
|
||||
farm = 0.4.3 latest branch
|
||||
|
||||
In the call to Emb...', meta: {'type': 'issue', 'title': 'SentenceTransformer no longer accepts \'gpu" as argument', 'number': 123, 'state': 'closed', 'created_at': '2020-05-28T04:49:31Z', 'updated_at': '2020-05-28T07:11:43Z', 'author': 'predoctech', 'url': 'https://github.com/deepset-ai/haystack/issues/123'}), Document(id=a8a56b9ad119244678804d5873b13da0784587773d8f839e07f644c4d02c167a, content: 'Thanks for reporting!
|
||||
Fixed with #124 ', meta: {'type': 'comment', 'issue_number': 123, 'created_at': '2020-05-28T07:11:42Z', 'updated_at': '2020-05-28T07:11:42Z', 'author': 'tholor', 'url': 'https://github.com/deepset-ai/haystack/issues/123#issuecomment-635153940'})]}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The following pipeline fetches a GitHub issue, extracts relevant information, and generates a summary:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack_integrations.components.connectors.github import GitHubIssueViewer
|
||||
|
||||
## Initialize components
|
||||
issue_viewer = GitHubIssueViewer()
|
||||
|
||||
prompt_template = [
|
||||
ChatMessage.from_system("You are a helpful assistant that analyzes GitHub issues."),
|
||||
ChatMessage.from_user(
|
||||
"Based on the following GitHub issue and comments:\n"
|
||||
"{% for document in documents %}"
|
||||
"{% if document.meta.type == 'issue' %}"
|
||||
"**Issue Title:** {{ document.meta.title }}\n"
|
||||
"**Issue Description:** {{ document.content }}\n"
|
||||
"{% else %}"
|
||||
"**Comment by {{ document.meta.author }}:** {{ document.content }}\n"
|
||||
"{% endif %}"
|
||||
"{% endfor %}\n"
|
||||
"Please provide a summary of the issue and suggest potential solutions.",
|
||||
),
|
||||
]
|
||||
|
||||
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
|
||||
llm = OpenAIChatGenerator(model="gpt-4o-mini")
|
||||
|
||||
## Create pipeline
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("issue_viewer", issue_viewer)
|
||||
pipeline.add_component("prompt_builder", prompt_builder)
|
||||
pipeline.add_component("llm", llm)
|
||||
|
||||
## Connect components
|
||||
pipeline.connect("issue_viewer.documents", "prompt_builder.documents")
|
||||
pipeline.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
## Run pipeline
|
||||
issue_url = "https://github.com/deepset-ai/haystack/issues/123"
|
||||
result = pipeline.run(data={"issue_viewer": {"url": issue_url}})
|
||||
|
||||
print(result["llm"]["replies"][0])
|
||||
```
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
---
|
||||
title: "GitHubPRCreator"
|
||||
id: githubprcreator
|
||||
slug: "/githubprcreator"
|
||||
description: "This component creates pull requests from a fork back to the original repository through the GitHub API."
|
||||
---
|
||||
|
||||
# GitHubPRCreator
|
||||
|
||||
This component creates pull requests from a fork back to the original repository through the GitHub API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | At the end of a pipeline, after [GitHubRepoForker](githubrepoforker.mdx), [GitHubFileEditor](githubfileeditor.mdx) and other components that prepare changes for submission |
|
||||
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
|
||||
| **Mandatory run variables** | `issue_url`: GitHub issue URL <br /> <br />`title`: PR title <br /> <br />`branch`: Source branch <br /> <br />`base`: Target branch |
|
||||
| **Output variables** | `result`: String indicating the pull request creation result |
|
||||
| **API reference** | [GitHub](/reference/integrations-github) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`GitHubPRCreator` takes a GitHub issue URL and creates a pull request from your fork to the original repository, automatically linking it to the specified issue. It's designed to work with existing forks and assumes you have already made changes in a branch.
|
||||
|
||||
Key features:
|
||||
|
||||
- **Cross-repository PRs**: Creates pull requests from your fork to the original repository
|
||||
- **Issue linking**: Automatically links the PR to the specified GitHub issue
|
||||
- **Draft support**: Option to create draft pull requests
|
||||
- **Fork validation**: Checks that the required fork exists before creating the PR
|
||||
|
||||
As optional parameters, you can set `body` to provide a pull request description and the boolean parameter `draft` to open a draft pull request.
|
||||
|
||||
### Authorization
|
||||
|
||||
This component requires GitHub authentication with a personal access token from the fork owner. You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
|
||||
|
||||
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens). Make sure to grant the appropriate permissions for repository access and pull request creation.
|
||||
|
||||
### Installation
|
||||
|
||||
Install the GitHub integration with pip:
|
||||
|
||||
```shell
|
||||
pip install github-haystack
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
:::info[Repository Placeholder]
|
||||
|
||||
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
|
||||
:::
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.github import GitHubPRCreator
|
||||
|
||||
pr_creator = GitHubPRCreator()
|
||||
result = pr_creator.run(
|
||||
issue_url="https://github.com/owner/repo/issues/123",
|
||||
title="Fix issue #123",
|
||||
body="This PR addresses issue #123 by implementing the requested changes.",
|
||||
branch="fix-123", # Branch in your fork with the changes
|
||||
base="main", # Branch in original repo to merge into
|
||||
)
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
```bash
|
||||
{'result': 'Pull request #456 created successfully and linked to issue #123'}
|
||||
```
|
||||
+70
@@ -0,0 +1,70 @@
|
||||
---
|
||||
title: "GitHubRepoForker"
|
||||
id: githubrepoforker
|
||||
slug: "/githubrepoforker"
|
||||
description: "This component forks a GitHub repository from an issue URL through the GitHub API."
|
||||
---
|
||||
|
||||
# GitHubRepoForker
|
||||
|
||||
This component forks a GitHub repository from an issue URL through the GitHub API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Right at the beginning of a pipeline and before an [Agent](../agents-1/agent.mdx) component that expects the name of a GitHub branch as input |
|
||||
| **Mandatory init variables** | `github_token`: GitHub personal access token. Can be set with `GITHUB_TOKEN` env var. |
|
||||
| **Mandatory run variables** | `url`: The URL of a GitHub issue in the repository that should be forked |
|
||||
| **Output variables** | `repo`: Fork repository path <br /> <br />`issue_branch`: Issue-specific branch name (if created) |
|
||||
| **API reference** | [GitHub](/reference/integrations-github) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`GitHubRepoForker` takes a GitHub issue URL, extracts the repository information, creates or syncs a fork of that repository, and optionally creates an issue-specific branch. It's particularly useful for automated workflows that need to create pull requests or work with repository forks.
|
||||
|
||||
Key features:
|
||||
|
||||
- **Auto-sync**: Automatically syncs existing forks with the upstream repository
|
||||
- **Branch creation**: Creates issue-specific branches (e.g., "fix-123" for issue #123)
|
||||
- **Completion waiting**: Optionally waits for fork creation to complete
|
||||
- **Fork management**: Handles existing forks intelligently
|
||||
|
||||
### Authorization
|
||||
|
||||
This component requires GitHub authentication with a personal access token. You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
|
||||
|
||||
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens). Make sure to grant the appropriate permissions for repository forking and management.
|
||||
|
||||
### Installation
|
||||
|
||||
Install the GitHub integration with pip:
|
||||
|
||||
```shell
|
||||
pip install github-haystack
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
:::info[Repository Placeholder]
|
||||
|
||||
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
|
||||
:::
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.github import GitHubRepoForker
|
||||
|
||||
forker = GitHubRepoForker()
|
||||
result = forker.run(url="https://github.com/owner/repo/issues/123")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
```bash
|
||||
{'repo': 'owner/repo', 'issue_branch': 'fix-123'}
|
||||
```
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
---
|
||||
title: "GitHubRepoViewer"
|
||||
id: githubrepoviewer
|
||||
slug: "/githubrepoviewer"
|
||||
description: "This component navigates and fetches content from GitHub repositories through the GitHub API."
|
||||
---
|
||||
|
||||
# GitHubRepoViewer
|
||||
|
||||
This component navigates and fetches content from GitHub repositories through the GitHub API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Right at the beginning of a pipeline and before a [ChatPromptBuilder](../builders/chatpromptbuilder.mdx) that expects the content of GitHub files as input |
|
||||
| **Mandatory run variables** | `path`: Repository path to view <br /> <br />`repo`: Repository in owner/repo format |
|
||||
| **Output variables** | `documents`: A list of documents containing repository contents |
|
||||
| **API reference** | [GitHub](/reference/integrations-github) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/github |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`GitHubRepoViewer` provides different behavior based on the path type:
|
||||
|
||||
- **For directories**: Returns a list of documents, one for each item (files and subdirectories),
|
||||
- **For files**: Returns a single document containing the file content.
|
||||
|
||||
Each document includes rich metadata such as the path, type, size, and URL.
|
||||
|
||||
### Authorization
|
||||
|
||||
The component can work without authentication for public repositories, but for private repositories or to avoid rate limiting, you can provide a GitHub personal access token.
|
||||
|
||||
You can set the token using the `GITHUB_TOKEN` environment variable, or pass it directly during initialization via the `github_token` parameter.
|
||||
|
||||
To create a personal access token, visit [GitHub's token settings page](https://github.com/settings/tokens).
|
||||
|
||||
### Installation
|
||||
|
||||
Install the GitHub integration with pip:
|
||||
|
||||
```shell
|
||||
pip install github-haystack
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
:::info[Repository Placeholder]
|
||||
|
||||
To run the following code snippets, you need to replace the `owner/repo` with your own GitHub repository name.
|
||||
:::
|
||||
|
||||
### On its own
|
||||
|
||||
Viewing a directory listing:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.github import GitHubRepoViewer
|
||||
|
||||
viewer = GitHubRepoViewer()
|
||||
result = viewer.run(
|
||||
repo="deepset-ai/haystack",
|
||||
path="haystack/components",
|
||||
branch="main",
|
||||
)
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
```bash
|
||||
{'documents': [Document(id=..., content: 'agents', meta: {'path': 'haystack/components/agents', 'type': 'dir', 'size': 0, 'url': 'https://github.com/deepset-ai/haystack/tree/main/haystack/components/agents'}), ...]}
|
||||
```
|
||||
|
||||
Viewing a specific file:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.github import GitHubRepoViewer
|
||||
|
||||
viewer = GitHubRepoViewer(repo="deepset-ai/haystack", branch="main")
|
||||
result = viewer.run(path="README.md")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
```bash
|
||||
{'documents': [Document(id=..., content: '<div align="center">
|
||||
<a href="https://haystack.deepset.ai/"><img src="https://raw.githubuserconten...', meta: {'path': 'README.md', 'type': 'file_content', 'size': 11979, 'url': 'https://github.com/deepset-ai/haystack/blob/main/README.md'})]}
|
||||
```
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
---
|
||||
title: "JinaReaderConnector"
|
||||
id: jinareaderconnector
|
||||
slug: "/jinareaderconnector"
|
||||
description: "Use Jina AI’s Reader API with Haystack."
|
||||
---
|
||||
|
||||
# JinaReaderConnector
|
||||
|
||||
Use Jina AI’s Reader API with Haystack.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | As the first component in a pipeline that passes the resulting document downstream |
|
||||
| **Mandatory init variables** | `mode`: The operation mode for the reader (`read`, `search`, or `ground`) <br /> <br />`api_key`: The Jina API key. Can be set with `JINA_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `query`: A query string |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Jina](/reference/integrations-jina) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`JinaReaderConnector` interacts with Jina AI’s Reader API to process queries and output documents.
|
||||
|
||||
You need to select one of the following modes of operations when initializing the component:
|
||||
|
||||
- `read`: Processes a URL and extracts the textual content.
|
||||
- `search`: Searches the web and returns textual content from the most relevant pages.
|
||||
- `ground`: Performs fact-checking using a grounding engine.
|
||||
|
||||
You can find more information on these modes in the [Jina Reader documentation](https://jina.ai/reader/).
|
||||
|
||||
You can additionally control the response format from the Jina Reader API using the component’s `json_response` parameter:
|
||||
|
||||
- `True` (default) requests a JSON response for documents enriched with structured metadata.
|
||||
- `False` requests a raw response, resulting in one document with minimal metadata.
|
||||
|
||||
### Authorization
|
||||
|
||||
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass a Jina API key at initialization with `api_key` like this:
|
||||
|
||||
```python
|
||||
ranker = JinaRanker(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
To get your API key, head to Jina AI’s [website](https://jina.ai/reranker/).
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install jina-haystack
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Read mode:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.jina import JinaReaderConnector
|
||||
|
||||
reader = JinaReaderConnector(mode="read")
|
||||
query = "https://example.com"
|
||||
result = reader.run(query=query)
|
||||
|
||||
print(result)
|
||||
## {'documents': [Document(id=fa3e51e4ca91828086dca4f359b6e1ea2881e358f83b41b53c84616cb0b2f7cf,
|
||||
## content: 'This domain is for use in illustrative examples in documents. You may use this domain in literature ...',
|
||||
## meta: {'title': 'Example Domain', 'description': '', 'url': 'https://example.com/', 'usage': {'tokens': 42}})]}
|
||||
```
|
||||
|
||||
Search mode:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.jina import JinaReaderConnector
|
||||
|
||||
reader = JinaReaderConnector(mode="search")
|
||||
query = "UEFA Champions League 2024"
|
||||
result = reader.run(query=query)
|
||||
|
||||
print(result)
|
||||
## {'documents': Document(id=6a71abf9955594232037321a476d39a835c0cb7bc575d886ee0087c973c95940,
|
||||
## content: '2024/25 UEFA Champions League: Matches, draw, final, key dates | UEFA Champions League | UEFA.com...',
|
||||
## meta: {'title': '2024/25 UEFA Champions League: Matches, draw, final, key dates',
|
||||
## 'description': 'What are the match dates? Where is the 2025 final? How will the competition work?',
|
||||
## 'url': 'https://www.uefa.com/uefachampionsleague/news/...',
|
||||
## 'usage': {'tokens': 5581}}), ...]}
|
||||
```
|
||||
|
||||
Ground mode:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.connectors.jina import JinaReaderConnector
|
||||
|
||||
reader = JinaReaderConnector(mode="ground")
|
||||
query = "ChatGPT was launched in 2017"
|
||||
result = reader.run(query=query)
|
||||
|
||||
print(result)
|
||||
## {'documents': [Document(id=f0c964dbc1ebb2d6584c8032b657150b9aa6e421f714cc1b9f8093a159127f0c,
|
||||
## content: 'The statement that ChatGPT was launched in 2017 is incorrect. Multiple references confirm that ChatG...',
|
||||
## meta: {'factuality': 0, 'result': False, 'references': [
|
||||
## {'url': 'https://en.wikipedia.org/wiki/ChatGPT',
|
||||
## 'keyQuote': 'ChatGPT is a generative artificial intelligence (AI) chatbot developed by OpenAI and launched in 2022.',
|
||||
## 'isSupportive': False}, ...],
|
||||
## 'usage': {'tokens': 10188}})]}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
**Query pipeline with search mode**
|
||||
|
||||
The following pipeline example, the `JinaReaderConnector` first searches for relevant documents, then feeds them along with a user query into a prompt template, and finally generates a response based on the retrieved context.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.utils import Secret
|
||||
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack_integrations.components.connectors.jina import JinaReaderConnector
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
reader_connector = JinaReaderConnector(mode="search")
|
||||
|
||||
prompt_template = [
|
||||
ChatMessage.from_system("You are a helpful assistant."),
|
||||
ChatMessage.from_user(
|
||||
"Given the information below:\n"
|
||||
"{% for document in documents %}{{ document.content }}{% endfor %}\n"
|
||||
"Answer question: {{ query }}.\nAnswer:",
|
||||
),
|
||||
]
|
||||
|
||||
prompt_builder = ChatPromptBuilder(
|
||||
template=prompt_template,
|
||||
required_variables={"query", "documents"},
|
||||
)
|
||||
llm = OpenAIChatGenerator(
|
||||
model="gpt-4o-mini",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("reader_connector", reader_connector)
|
||||
pipe.add_component("prompt_builder", prompt_builder)
|
||||
pipe.add_component("llm", llm)
|
||||
|
||||
pipe.connect("reader_connector.documents", "prompt_builder.documents")
|
||||
pipe.connect("prompt_builder.messages", "llm.messages")
|
||||
|
||||
query = "What is the most famous landmark in Berlin?"
|
||||
|
||||
result = pipe.run(
|
||||
data={"reader_connector": {"query": query}, "prompt_builder": {"query": query}},
|
||||
)
|
||||
print(result)
|
||||
|
||||
## {'llm': {'replies': ['The most famous landmark in Berlin is the **Brandenburg Gate**. It is considered the symbol of the city and represents reunification.'], 'meta': [{'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 27, 'prompt_tokens': 4479, 'total_tokens': 4506, 'completion_tokens_details': CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), 'prompt_tokens_details': PromptTokensDetails(audio_tokens=0, cached_tokens=0)}}]}}
|
||||
```
|
||||
|
||||
The same component in search mode could also be used in an indexing pipeline.
|
||||
+233
@@ -0,0 +1,233 @@
|
||||
---
|
||||
title: "LangfuseConnector"
|
||||
id: langfuseconnector
|
||||
slug: "/langfuseconnector"
|
||||
description: "Learn how to work with Langfuse in Haystack."
|
||||
---
|
||||
|
||||
# LangfuseConnector
|
||||
|
||||
Learn how to work with Langfuse in Haystack.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Anywhere, as it’s not connected to other components |
|
||||
| **Mandatory init variables** | `name`: The name of the pipeline or component to identify the tracing run |
|
||||
| **Output variables** | `name`: The name of the tracing component <br /> <br />`trace_url`: A link to the tracing data |
|
||||
| **API reference** | [langfuse](/reference/integrations-langfuse) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langfuse |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`LangfuseConnector` integrates tracing capabilities into Haystack pipelines using [Langfuse](https://langfuse.com/). It captures detailed information about pipeline runs, like API calls, context data, prompts, and more. Use this component to:
|
||||
|
||||
- Monitor model performance, such as token usage and cost.
|
||||
- Find areas for pipeline improvement by identifying low-quality outputs and collecting user feedback.
|
||||
- Create datasets for fine-tuning and testing from your pipeline executions.
|
||||
|
||||
To work with the integration, add the `LangfuseConnector` to your pipeline, run the pipeline, and then view the tracing data on the Langfuse website. Don’t connect this component to any other – `LangfuseConnector` will simply run in your pipeline’s background.
|
||||
|
||||
You can optionally define two more parameters when working with this component:
|
||||
|
||||
- `httpx_client`: An optional custom `httpx.Client` instance for Langfuse API calls. Note that custom clients are discarded when deserializing a pipeline from YAML, as HTTPX clients cannot be serialized. In such cases, Langfuse creates a default client.
|
||||
- `span_handler`: An optional custom handler for processing spans. If not provided, the `DefaultSpanHandler` is used. The span handler defines how spans are created and processed, enabling customization of span types based on component types and post-processing of spans. See more details in the [Advanced Usage section](#advanced-usage) below.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
These are the things that you need before working with LangfuseConnector:
|
||||
|
||||
1. Make sure you have an active Langfuse [account](https://cloud.langfuse.com/).
|
||||
2. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` – this will enable tracing in your pipelines.
|
||||
3. Set the `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` environment variables with your Langfuse secret and public keys found in your account profile.
|
||||
|
||||
### Installation
|
||||
|
||||
First, install `langfuse-haystack` package to use the `LangfuseConnector`:
|
||||
|
||||
```shell
|
||||
pip install langfuse-haystack
|
||||
```
|
||||
|
||||
<br />
|
||||
|
||||
:::info[Usage Notice]
|
||||
|
||||
To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import. In the example below, we first set the environmental variables and then import the relevant Haystack components.
|
||||
|
||||
Alternatively, an even better practice is to set these environment variables in your shell before running the script. This approach keeps configuration separate from code and allows for easier management of different environments.
|
||||
:::
|
||||
|
||||
## Usage
|
||||
|
||||
In the example below, we are adding `LangfuseConnector` to the pipeline as a _tracer_. Each pipeline run will produce one trace that includes the entire execution context, including prompts, completions, and metadata.
|
||||
|
||||
You can then view the trace by following a URL link printed in the output.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com"
|
||||
os.environ["TOKENIZERS_PARALLELISM"] = "false"
|
||||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||||
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack import Pipeline
|
||||
|
||||
from haystack_integrations.components.connectors.langfuse import LangfuseConnector
|
||||
|
||||
if __name__ == "__main__":
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("tracer", LangfuseConnector("Chat example"))
|
||||
pipe.add_component("prompt_builder", ChatPromptBuilder())
|
||||
pipe.add_component("llm", OpenAIChatGenerator())
|
||||
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
messages = [
|
||||
ChatMessage.from_system(
|
||||
"Always respond in German even if some input data is in other languages.",
|
||||
),
|
||||
ChatMessage.from_user("Tell me about {{location}}"),
|
||||
]
|
||||
|
||||
response = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"location": "Berlin"},
|
||||
"template": messages,
|
||||
},
|
||||
},
|
||||
)
|
||||
print(response["llm"]["replies"][0])
|
||||
print(response["tracer"]["trace_url"])
|
||||
```
|
||||
|
||||
### With an Agent
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com"
|
||||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||||
|
||||
from typing import Annotated
|
||||
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.tools import tool
|
||||
from haystack import Pipeline
|
||||
|
||||
from haystack_integrations.components.connectors.langfuse import LangfuseConnector
|
||||
|
||||
|
||||
@tool
|
||||
def get_weather(city: Annotated[str, "The city to get weather for"]) -> str:
|
||||
"""Get current weather information for a city."""
|
||||
weather_data = {
|
||||
"Berlin": "18°C, partly cloudy",
|
||||
"New York": "22°C, sunny",
|
||||
"Tokyo": "25°C, clear skies",
|
||||
}
|
||||
return weather_data.get(city, f"Weather information for {city} not available")
|
||||
|
||||
|
||||
@tool
|
||||
def calculate(
|
||||
operation: Annotated[
|
||||
str,
|
||||
"Mathematical operation: add, subtract, multiply, divide",
|
||||
],
|
||||
a: Annotated[float, "First number"],
|
||||
b: Annotated[float, "Second number"],
|
||||
) -> str:
|
||||
"""Perform basic mathematical calculations."""
|
||||
if operation == "add":
|
||||
result = a + b
|
||||
elif operation == "subtract":
|
||||
result = a - b
|
||||
elif operation == "multiply":
|
||||
result = a * b
|
||||
elif operation == "divide":
|
||||
if b == 0:
|
||||
return "Error: Division by zero"
|
||||
else:
|
||||
result = a / b
|
||||
else:
|
||||
return f"Error: Unknown operation '{operation}'"
|
||||
|
||||
return f"The result of {a} {operation} {b} is {result}"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
## Create components
|
||||
chat_generator = OpenAIChatGenerator()
|
||||
|
||||
agent = Agent(
|
||||
chat_generator=chat_generator,
|
||||
tools=[get_weather, calculate],
|
||||
system_prompt="You are a helpful assistant with access to weather and calculator tools. Use them when needed.",
|
||||
exit_conditions=["text"],
|
||||
)
|
||||
|
||||
langfuse_connector = LangfuseConnector("Agent Example")
|
||||
|
||||
## Create and run pipeline
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("tracer", langfuse_connector)
|
||||
pipe.add_component("agent", agent)
|
||||
|
||||
response = pipe.run(
|
||||
data={
|
||||
"agent": {
|
||||
"messages": [
|
||||
ChatMessage.from_user(
|
||||
"What's the weather in Berlin and calculate 15 + 27?",
|
||||
),
|
||||
],
|
||||
},
|
||||
"tracer": {"invocation_context": {"test": "agent_with_tools"}},
|
||||
},
|
||||
)
|
||||
|
||||
print(response["agent"]["last_message"].text)
|
||||
print(response["tracer"]["trace_url"])
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Customizing Langfuse Traces with SpanHandler
|
||||
|
||||
The `SpanHandler` interface in Haystack allows you to customize how spans are created and processed for Langfuse trace creation. This enables you to log custom metrics, add tags, or integrate metadata.
|
||||
|
||||
By extending `SpanHandler` or its default implementation, `DefaultSpanHandler`, you can define custom logic for span processing, providing precise control over what data is logged to Langfuse for tracking and analyzing pipeline executions.
|
||||
|
||||
Here's an example:
|
||||
|
||||
```python
|
||||
from haystack_integrations.tracing.langfuse import (
|
||||
LangfuseConnector,
|
||||
DefaultSpanHandler,
|
||||
LangfuseSpan,
|
||||
)
|
||||
from typing import Optional
|
||||
|
||||
|
||||
class CustomSpanHandler(DefaultSpanHandler):
|
||||
def handle(self, span: LangfuseSpan, component_type: Optional[str]) -> None:
|
||||
# Custom logic to add metadata or modify span
|
||||
if component_type == "OpenAIChatGenerator":
|
||||
output = span._data.get("haystack.component.output", {})
|
||||
if len(output.get("text", "")) < 10:
|
||||
span._span.update(level="WARNING", status_message="Response too short")
|
||||
|
||||
|
||||
## Add the custom handler to the LangfuseConnector
|
||||
connector = LangfuseConnector(span_handler=CustomSpanHandler())
|
||||
```
|
||||
+106
@@ -0,0 +1,106 @@
|
||||
---
|
||||
title: "OpenAPIConnector"
|
||||
id: openapiconnector
|
||||
slug: "/openapiconnector"
|
||||
description: "`OpenAPIConnector` is a component that acts as an interface between the Haystack ecosystem and OpenAPI services."
|
||||
---
|
||||
|
||||
# OpenAPIConnector
|
||||
|
||||
`OpenAPIConnector` is a component that acts as an interface between the Haystack ecosystem and OpenAPI services.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Anywhere, after components providing input for its run parameters |
|
||||
| **Mandatory init variables** | `openapi_spec`: The OpenAPI specification for the service. Can be a URL, file path, or raw string. |
|
||||
| **Mandatory run variables** | `operation_id`: The operationId from the OpenAPI spec to invoke. |
|
||||
| **Output variables** | `response`: A REST service response |
|
||||
| **API reference** | [Connectors](/reference/connectors-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/connectors/openapi.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `OpenAPIConnector` is a component within the Haystack ecosystem that allows direct invocation of REST endpoints defined in an OpenAPI (formerly Swagger) specification. It acts as a bridge between Haystack pipelines and any REST API that follows the OpenAPI standard, enabling dynamic method calls, authentication, and parameter handling.
|
||||
|
||||
To use the `OpenAPIConnector`, ensure that you have the `openapi-llm` dependency installed:
|
||||
|
||||
```shell
|
||||
pip install openapi-llm
|
||||
```
|
||||
|
||||
Unlike [OpenAPIServiceConnector](openapiserviceconnector.mdx), which works with LLMs, `OpenAPIConnector` directly calls REST endpoints using explicit input arguments.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
You can initialize and use the `OpenAPIConnector` on its own by passing an OpenAPI specification and other parameters:
|
||||
|
||||
```python
|
||||
from haystack.utils import Secret
|
||||
from haystack.components.connectors.openapi import OpenAPIConnector
|
||||
|
||||
connector = OpenAPIConnector(
|
||||
openapi_spec="https://bit.ly/serperdev_openapi",
|
||||
credentials=Secret.from_env_var("SERPERDEV_API_KEY"),
|
||||
service_kwargs={"config_factory": my_custom_config_factory},
|
||||
)
|
||||
|
||||
response = connector.run(
|
||||
operation_id="search",
|
||||
arguments={"q": "Who was Nikola Tesla?"},
|
||||
)
|
||||
```
|
||||
|
||||
#### Output
|
||||
|
||||
The `OpenAPIConnector` returns a dictionary containing the service response:
|
||||
|
||||
```json
|
||||
{
|
||||
"response": { // here goes REST endpoint response JSON
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The `OpenAPIConnector` can be integrated into a Haystack pipeline to interact with OpenAPI services. For example, here’s how you can link the `OpenAPIConnector` to a pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.connectors.openapi import OpenAPIConnector
|
||||
from haystack.dataclasses.chat_message import ChatMessage
|
||||
from haystack.utils import Secret
|
||||
|
||||
## Initialize the OpenAPIConnector
|
||||
connector = OpenAPIConnector(
|
||||
openapi_spec="https://bit.ly/serperdev_openapi",
|
||||
credentials=Secret.from_env_var("SERPERDEV_API_KEY"),
|
||||
)
|
||||
|
||||
## Create a ChatMessage from the user
|
||||
user_message = ChatMessage.from_user(text="Who was Nikola Tesla?")
|
||||
|
||||
## Define the pipeline
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("openapi_connector", connector)
|
||||
|
||||
## Run the pipeline
|
||||
response = pipeline.run(
|
||||
data={
|
||||
"openapi_connector": {
|
||||
"operation_id": "search",
|
||||
"arguments": {"q": user_message.text},
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
## Extract the answer from the response
|
||||
answer = response.get("openapi_connector", {}).get("response", {})
|
||||
print(answer)
|
||||
```
|
||||
+111
@@ -0,0 +1,111 @@
|
||||
---
|
||||
title: "OpenAPIServiceConnector"
|
||||
id: openapiserviceconnector
|
||||
slug: "/openapiserviceconnector"
|
||||
description: "`OpenAPIServiceConnector` is a component that acts as an interface between the Haystack ecosystem and OpenAPI services."
|
||||
---
|
||||
|
||||
# OpenAPIServiceConnector
|
||||
|
||||
`OpenAPIServiceConnector` is a component that acts as an interface between the Haystack ecosystem and OpenAPI services.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Flexible |
|
||||
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects where the last message is expected to carry parameter invocation payload. <br /> <br />`service_openapi_spec`: OpenAPI specification of the service being invoked. It can be YAML/JSON, and all ref values must be resolved. <br /> <br />`service_credentials`: Authentication credentials for the service. We currently support two OpenAPI spec v3 security schemes: <br /> <br />1. http – for Basic, Bearer, and other HTTP authentication schemes; <br />2. apiKey – for API keys and cookie authentication. |
|
||||
| **Output variables** | `service_response`: A dictionary that is a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects where each message corresponds to a function invocation. <br />If a user specifies multiple function calling requests, there will be multiple responses. |
|
||||
| **API reference** | [Connectors](/reference/connectors-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/connectors/openapi_service.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`OpenAPIServiceConnector` acts as a bridge between Haystack ecosystem and OpenAPI services. This component works by using information from a `ChatMessage` to dynamically invoke service methods. It handles parameter payload parsing from `ChatMessage`, service authentication, method invocation, and response formatting, making it easier to integrate OpenAPI services.
|
||||
|
||||
To use `OpenAPIServiceConnector`, you need to install the optional `openapi3` dependency with:
|
||||
|
||||
```shell
|
||||
pip install openapi3
|
||||
```
|
||||
|
||||
`OpenAPIServiceConnector` component doesn’t have any init parameters.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
This component is primarily meant to be used in pipelines, as [`OpenAPIServiceToFunctions`](../converters/openapiservicetofunctions.mdx), in tandem with the function calling model, resolves the actual function calling parameters that are injected as invocation parameters for `OpenAPIServiceConnector`.
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Let's say we're linking the Serper search engine to a pipeline. Here, `OpenAPIServiceConnector` uses the abilities of `OpenAPIServiceToFunctions`. `OpenAPIServiceToFunctions` first fetches and changes the [Serper's OpenAPI specification](https://bit.ly/serper_dev_spec) into a format that OpenAI's function calling mechanism can understand. Then, `OpenAPIServiceConnector` activates the Serper service using this specification.
|
||||
|
||||
More precisely, `OpenAPIServiceConnector` dynamically calls methods defined in the Serper OpenAPI specification. This involves reading chat messages or other inputs to extract function call parameters, handling authentication with the Serper service, and making the right API calls. The connector makes sure that the method call follows the Serper API requirements, such as correct formatting requests and handling responses.
|
||||
|
||||
Note that we used Serper just as an example here. This could be any OpenAPI-compliant service.
|
||||
|
||||
:::info
|
||||
To run the following code snippet, note that you have to have your own Serper and OpenAI API keys.
|
||||
:::
|
||||
|
||||
```python
|
||||
import json
|
||||
import requests
|
||||
|
||||
from typing import Dict, Any, List
|
||||
from haystack import Pipeline
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
from haystack.components.converters import OpenAPIServiceToFunctions, OutputAdapter
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.connectors import OpenAPIServiceConnector
|
||||
from haystack.components.fetchers import LinkContentFetcher
|
||||
from haystack.dataclasses import ChatMessage, ByteStream
|
||||
from haystack.utils import Secret
|
||||
|
||||
def prepare_fc_params(openai_functions_schema: Dict[str, Any]) -> Dict[str, Any]:
|
||||
return {
|
||||
"tools": [{
|
||||
"type": "function",
|
||||
"function": openai_functions_schema
|
||||
}],
|
||||
"tool_choice": {
|
||||
"type": "function",
|
||||
"function": {"name": openai_functions_schema["name"]}
|
||||
}
|
||||
}
|
||||
|
||||
system_prompt = requests.get("https://bit.ly/serper_dev_system_prompt").text
|
||||
serper_spec = requests.get("https://bit.ly/serper_dev_spec").text
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("spec_to_functions", OpenAPIServiceToFunctions())
|
||||
pipe.add_component("functions_llm", OpenAIChatGenerator(api_key=Secret.from_token(llm_api_key), model="gpt-3.5-turbo-0613"))
|
||||
pipe.add_component("openapi_container", OpenAPIServiceConnector())
|
||||
pipe.add_component("a1", OutputAdapter("{{functions[0] | prepare_fc}}", Dict[str, Any], {"prepare_fc": prepare_fc_params}))
|
||||
pipe.add_component("a2", OutputAdapter("{{specs[0]}}", Dict[str, Any]))
|
||||
pipe.add_component("a3", OutputAdapter("{{system_message + service_response}}", List[ChatMessage]))
|
||||
pipe.add_component("llm", OpenAIChatGenerator(api_key=Secret.from_token(llm_api_key), model="gpt-4-1106-preview", streaming_callback=print_streaming_chunk))
|
||||
|
||||
pipe.connect("spec_to_functions.functions", "a1.functions")
|
||||
pipe.connect("spec_to_functions.openapi_specs", "a2.specs")
|
||||
pipe.connect("a1", "functions_llm.generation_kwargs")
|
||||
pipe.connect("functions_llm.replies", "openapi_container.messages")
|
||||
pipe.connect("a2", "openapi_container.service_openapi_spec")
|
||||
pipe.connect("openapi_container.service_response", "a3.service_response")
|
||||
pipe.connect("a3", "llm.messages")
|
||||
|
||||
user_prompt = "Why was Sam Altman ousted from OpenAI?"
|
||||
|
||||
result = pipe.run(data={"functions_llm": {"messages":[ChatMessage.from_system("Only do function calling"), ChatMessage.from_user(user_prompt)]},
|
||||
"openapi_container": {"service_credentials": serper_dev_key},
|
||||
"spec_to_functions": {"sources": [ByteStream.from_string(serper_spec)]},
|
||||
"a3": {"system_message": [ChatMessage.from_system(system_prompt)]}})
|
||||
|
||||
>Sam Altman was ousted from OpenAI on November 17, 2023, following
|
||||
>a "deliberative review process" by the board of directors. The board concluded
|
||||
>that he was not "consistently candid in his communications". However, he
|
||||
>returned as CEO just days after his ouster.
|
||||
```
|
||||
+183
@@ -0,0 +1,183 @@
|
||||
---
|
||||
title: "WeaveConnector"
|
||||
id: weaveconnector
|
||||
slug: "/weaveconnector"
|
||||
description: "Learn how to use Weights & Biases Weave framework for tracing and monitoring your pipeline components."
|
||||
---
|
||||
|
||||
# WeaveConnector
|
||||
|
||||
Learn how to use Weights & Biases Weave framework for tracing and monitoring your pipeline components.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Anywhere, as it’s not connected to other components |
|
||||
| **Mandatory init variables** | `pipeline_name`: The name of your pipeline, which will also show up in Weaver dashboard. |
|
||||
| **Output variables** | `pipeline_name`: The name of the pipeline that just run |
|
||||
| **API reference** | [Weave](/reference/integrations-weave) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weave |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
This integration allows you to trace and visualize your pipeline execution in [Weights & Biases](https://wandb.ai/site/).
|
||||
|
||||
Information captured by the Haystack tracing tool, such as API calls, context data, and prompts, is sent to Weights & Biases, where you can see the complete trace of your pipeline execution.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
You need a Weave account to use this feature. You can sign up for free at [Weights & Biases website](https://wandb.ai/site).
|
||||
|
||||
You will then need to set the `WANDB_API_KEY` environment variable with your Weights & Biases API key. Once logged in, you can find your API key on [your home page](https://wandb.ai/home).
|
||||
|
||||
Then go to `https://wandb.ai/<user_name>/projects` and see the full trace for your pipeline under the pipeline name you specified when creating the `WeaveConnector`.
|
||||
|
||||
You will also need to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable set to `true`.
|
||||
|
||||
## Usage
|
||||
|
||||
First, install the `weights_biases-haystack` package to use this connector:
|
||||
|
||||
```shell
|
||||
pip install weights_biases-haystack
|
||||
```
|
||||
|
||||
Then, add it to your pipeline without any connections, and it will automatically start sending traces to Weights & Biases:
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from haystack import Pipeline
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
from haystack_integrations.components.connectors.weave import WeaveConnector
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("prompt_builder", ChatPromptBuilder())
|
||||
pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
connector = WeaveConnector(pipeline_name="test_pipeline")
|
||||
pipe.add_component("weave", connector)
|
||||
|
||||
messages = [
|
||||
ChatMessage.from_system(
|
||||
"Always respond in German even if some input data is in other languages.",
|
||||
),
|
||||
ChatMessage.from_user("Tell me about {{location}}"),
|
||||
]
|
||||
|
||||
response = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"location": "Berlin"},
|
||||
"template": messages,
|
||||
},
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
You can then see the complete trace for your pipeline at `https://wandb.ai/<user_name>/projects` under the pipeline name you specified when creating the `WeaveConnector`.
|
||||
|
||||
### With an Agent
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
## Enable Haystack content tracing
|
||||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||||
|
||||
from typing import Annotated
|
||||
|
||||
from haystack.components.agents import Agent
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
from haystack.tools import tool
|
||||
from haystack import Pipeline
|
||||
|
||||
from haystack_integrations.components.connectors.weave import WeaveConnector
|
||||
|
||||
|
||||
@tool
|
||||
def get_weather(city: Annotated[str, "The city to get weather for"]) -> str:
|
||||
"""Get current weather information for a city."""
|
||||
weather_data = {
|
||||
"Berlin": "18°C, partly cloudy",
|
||||
"New York": "22°C, sunny",
|
||||
"Tokyo": "25°C, clear skies",
|
||||
}
|
||||
return weather_data.get(city, f"Weather information for {city} not available")
|
||||
|
||||
|
||||
@tool
|
||||
def calculate(
|
||||
operation: Annotated[
|
||||
str,
|
||||
"Mathematical operation: add, subtract, multiply, divide",
|
||||
],
|
||||
a: Annotated[float, "First number"],
|
||||
b: Annotated[float, "Second number"],
|
||||
) -> str:
|
||||
"""Perform basic mathematical calculations."""
|
||||
if operation == "add":
|
||||
result = a + b
|
||||
elif operation == "subtract":
|
||||
result = a - b
|
||||
elif operation == "multiply":
|
||||
result = a * b
|
||||
elif operation == "divide":
|
||||
if b == 0:
|
||||
return "Error: Division by zero"
|
||||
result = a / b
|
||||
else:
|
||||
return f"Error: Unknown operation '{operation}'"
|
||||
|
||||
return f"The result of {a} {operation} {b} is {result}"
|
||||
|
||||
|
||||
## Create the chat generator
|
||||
chat_generator = OpenAIChatGenerator()
|
||||
|
||||
## Create the agent with tools
|
||||
agent = Agent(
|
||||
chat_generator=chat_generator,
|
||||
tools=[get_weather, calculate],
|
||||
system_prompt="You are a helpful assistant with access to weather and calculator tools. Use them when needed.",
|
||||
exit_conditions=["text"],
|
||||
)
|
||||
|
||||
## Create the WeaveConnector for tracing
|
||||
weave_connector = WeaveConnector(pipeline_name="Agent Example")
|
||||
|
||||
## Build the pipeline
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("tracer", weave_connector)
|
||||
pipe.add_component("agent", agent)
|
||||
|
||||
## Run the pipeline
|
||||
response = pipe.run(
|
||||
data={
|
||||
"agent": {
|
||||
"messages": [
|
||||
ChatMessage.from_user(
|
||||
"What's the weather in Berlin and calculate 15 + 27?",
|
||||
),
|
||||
],
|
||||
},
|
||||
"tracer": {},
|
||||
},
|
||||
)
|
||||
|
||||
## Display results
|
||||
print("Agent Response:")
|
||||
print(response["agent"]["last_message"].text)
|
||||
print(f"\nPipeline Name: {response['tracer']['pipeline_name']}")
|
||||
print(
|
||||
"\nCheck your Weights & Biases dashboard at https://wandb.ai/<user_name>/projects to see the traces!",
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: "Converters"
|
||||
id: converters
|
||||
slug: "/converters"
|
||||
description: "Use various Converters to extract data from files in different formats and cast it into the unified document format. There are several converters available for converting PDFs, images, DOCX files, and more."
|
||||
---
|
||||
|
||||
# Converters
|
||||
|
||||
Use various Converters to extract data from files in different formats and cast it into the unified document format. There are several converters available for converting PDFs, images, DOCX files, and more.
|
||||
|
||||
| Converter | Description |
|
||||
| --- | --- |
|
||||
| [AzureOCRDocumentConverter](converters/azureocrdocumentconverter.mdx) | Converts PDF (both searchable and image-only), JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, and HTML to documents. |
|
||||
| [CSVToDocument](converters/csvtodocument.mdx) | Converts CSV files to documents. |
|
||||
| [DocumentToImageContent](converters/documenttoimagecontent.mdx) | Extracts visual data from image or PDF file-based documents and converts them into `ImageContent` objects. |
|
||||
| [DOCXToDocument](converters/docxtodocument.mdx) | Convert DOCX files to documents. |
|
||||
| [HTMLToDocument](converters/htmltodocument.mdx) | Converts HTML files to documents. |
|
||||
| [ImageFileToDocument](converters/imagefiletodocument.mdx) | Converts image file references into empty `Document` objects with associated metadata. |
|
||||
| [ImageFileToImageContent](converters/imagefiletoimagecontent.mdx) | Reads local image files and converts them into `ImageContent` objects. |
|
||||
| [JSONConverter](converters/jsonconverter.mdx) | Converts JSON files to text documents. |
|
||||
| [MarkdownToDocument](converters/markdowntodocument.mdx) | Converts markdown files to documents. |
|
||||
| [MistralOCRDocumentConverter](converters/mistralocrdocumentconverter.mdx) | Extracts text from documents using Mistral's OCR API, with optional structured annotations. |
|
||||
| [MSGToDocument](converters/msgtodocument.mdx) | Converts Microsoft Outlook .msg files to documents. |
|
||||
| [MultiFileConverter](converters/multifileconverter.mdx) | Converts CSV, DOCX, HTML, JSON, MD, PPTX, PDF, TXT, and XSLX files to documents. |
|
||||
| [OpenAPIServiceToFunctions](converters/openapiservicetofunctions.mdx) | Transforms OpenAPI service specifications into a format compatible with OpenAI's function calling mechanism. |
|
||||
| [OutputAdapter](converters/outputadapter.mdx) | Helps the output of one component fit into the input of another. |
|
||||
| [PaddleOCRVLDocumentConverter](converters/paddleocrvldocumentconverter.mdx) | Extracts text from documents using PaddleOCR's large model document parsing API. |
|
||||
| [PDFMinerToDocument](converters/pdfminertodocument.mdx) | Converts complex PDF files to documents using pdfminer arguments. |
|
||||
| [PDFToImageContent](converters/pdftoimagecontent.mdx) | Reads local PDF files and converts them into `ImageContent` objects. |
|
||||
| [PPTXToDocument](converters/pptxtodocument.mdx) | Converts PPTX files to documents. |
|
||||
| [PyPDFToDocument](converters/pypdftodocument.mdx) | Converts PDF files to documents. |
|
||||
| [TikaDocumentConverter](converters/tikadocumentconverter.mdx) | Converts various file types to documents using Apache Tika. |
|
||||
| [TextFileToDocument](converters/textfiletodocument.mdx) | Converts text files to documents. |
|
||||
| [UnstructuredFileConverter](converters/unstructuredfileconverter.mdx) | Converts text files and directories to a document. |
|
||||
| [XLSXToDocument](converters/xlsxtodocument.mdx) | Converts Excel files into documents. |
|
||||
+92
@@ -0,0 +1,92 @@
|
||||
---
|
||||
title: "AzureOCRDocumentConverter"
|
||||
id: azureocrdocumentconverter
|
||||
slug: "/azureocrdocumentconverter"
|
||||
description: "`AzureOCRDocumentConverter` converts files to documents using Azure's Document Intelligence service. It supports the following file formats: PDF (both searchable and image-only), JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, and HTML."
|
||||
---
|
||||
|
||||
# AzureOCRDocumentConverter
|
||||
|
||||
`AzureOCRDocumentConverter` converts files to documents using Azure's Document Intelligence service. It supports the following file formats: PDF (both searchable and image-only), JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, and HTML.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory init variables** | `endpoint`: The endpoint of your Azure resource <br /> <br />`api_key`: The API key of your Azure resource. Can be set with `AZURE_AI_API_KEY` environment variable. |
|
||||
| **Mandatory run variables** | `sources`: A list of file paths |
|
||||
| **Output variables** | `documents`: A list of documents <br /> <br />`raw_azure_response`: A list of raw responses from Azure |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/azure.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`AzureOCRDocumentConverter` takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and uses Azure services to convert the files to a list of documents. Optionally, metadata can be attached to the documents through the `meta` input parameter. You need an active Azure account and a Document Intelligence or Cognitive Services resource to use this integration. Follow the steps described in the Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/quickstarts/get-started-sdks-rest-api) to set up your resource.
|
||||
|
||||
The component uses an `AZURE_AI_API_KEY` environment variable by default. Otherwise, you can pass an `api_key` at initialization – see code examples below.
|
||||
|
||||
When you initialize the component, you can optionally set the `model_id`, which refers to the model you want to use. Please refer to [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/choose-model-feature) for a list of available models. The default model is `"prebuilt-read"`.
|
||||
|
||||
The `AzureOCRDocumentConverter` doesn’t extract the tables from a file as plain text but generates separate `Document` objects of type `table` that maintain the two-dimensional structure of the tables.
|
||||
|
||||
## Usage
|
||||
|
||||
You need to install `azure-ai-formrecognizer` package to use the `AzureOCRDocumentConverter`:
|
||||
|
||||
```shell
|
||||
pip install "azure-ai-formrecognizer>=3.2.0b2"
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from haystack.components.converters import AzureOCRDocumentConverter
|
||||
from haystack.utils import Secret
|
||||
|
||||
converter = AzureOCRDocumentConverter(
|
||||
endpoint="azure_resource_url",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
|
||||
converter.run(sources=[Path("my_file.pdf")])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import AzureOCRDocumentConverter
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.utils import Secret
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component(
|
||||
"converter",
|
||||
AzureOCRDocumentConverter(
|
||||
endpoint="azure_resource_url",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
),
|
||||
)
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
file_names = ["my_file.pdf"]
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
---
|
||||
title: "CSVToDocument"
|
||||
id: csvtodocument
|
||||
slug: "/csvtodocument"
|
||||
description: "Converts CSV files to documents."
|
||||
---
|
||||
|
||||
# CSVToDocument
|
||||
|
||||
Converts CSV files to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: A list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/csv.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`CSVToDocument` converts one or more CSV files into a text document.
|
||||
|
||||
The component uses UTF-8 encoding by default, but you may specify a different encoding if needed during initialization.
|
||||
You can optionally attach metadata to each document with a `meta` parameter when running the component.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters.csv import CSVToDocument
|
||||
|
||||
converter = CSVToDocument()
|
||||
results = converter.run(
|
||||
sources=["sample.csv"],
|
||||
meta={"date_added": datetime.now().isoformat()},
|
||||
)
|
||||
documents = results["documents"]
|
||||
|
||||
print(documents[0].content)
|
||||
## 'col1,col2\now1,row1\nrow2row2\n'
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import CSVToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", CSVToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
+153
@@ -0,0 +1,153 @@
|
||||
---
|
||||
title: "DocumentToImageContent"
|
||||
id: documenttoimagecontent
|
||||
slug: "/documenttoimagecontent"
|
||||
description: "`DocumentToImageContent` extracts visual data from image or PDF file-based documents and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image question-answering and captioning."
|
||||
---
|
||||
|
||||
# DocumentToImageContent
|
||||
|
||||
`DocumentToImageContent` extracts visual data from image or PDF file-based documents and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image question-answering and captioning.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a `ChatPromptBuilder` in a query pipeline |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to process. Each document should have metadata containing at minimum a 'file_path_meta_field' key. PDF documents additionally require a 'page_number' key to specify which page to convert. |
|
||||
| **Output variables** | `image_contents`: A list of `ImageContent` objects |
|
||||
| **API reference** | [Image Converters](/reference/image-converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/image/document_to_image.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`DocumentToImageContent` processes a list of documents containing image or PDF file paths and converts them into `ImageContent` objects.
|
||||
|
||||
- For images, it reads and encodes the file directly.
|
||||
- For PDFs, it extracts the specified page (through `page_number` in metadata) and converts it to an image.
|
||||
|
||||
By default, it looks for the file path in the `file_path` metadata field. You can customize this with the `file_path_meta_field` parameter. The `root_path` lets you specify a common base directory for file resolution.
|
||||
|
||||
This component is typically used in query pipelines right before a `ChatPromptBuilder` when you would like to add Images to your user prompt.
|
||||
|
||||
If `size` is provided, the images will be resized while maintaining aspect ratio. This reduces file size, memory usage, and processing time, which is beneficial when working with models that have resolution constraints or when transmitting images to remote services.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.converters.image.document_to_image import (
|
||||
DocumentToImageContent,
|
||||
)
|
||||
|
||||
converter = DocumentToImageContent(
|
||||
file_path_meta_field="file_path",
|
||||
root_path="/data/documents",
|
||||
detail="high",
|
||||
size=(800, 600),
|
||||
)
|
||||
|
||||
documents = [
|
||||
Document(content="Photo of a mountain", meta={"file_path": "mountain.jpg"}),
|
||||
Document(
|
||||
content="First page of a report",
|
||||
meta={"file_path": "report.pdf", "page_number": 1},
|
||||
),
|
||||
]
|
||||
|
||||
result = converter.run(documents)
|
||||
image_contents = result["image_contents"]
|
||||
print(image_contents)
|
||||
|
||||
## [
|
||||
## ImageContent(
|
||||
## base64_image="/9j/4A...", mime_type="image/jpeg", detail="high",
|
||||
## meta={"file_path": "mountain.jpg"}
|
||||
## ),
|
||||
## ImageContent(
|
||||
## base64_image="/9j/4A...", mime_type="image/jpeg", detail="high",
|
||||
## meta={"file_path": "report.pdf", "page_number": 1}
|
||||
## )
|
||||
## ]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
You can use `DocumentToImageContent` in multimodal indexing pipelines before passing to an Embedder or captioning model.
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.converters.image.document_to_image import (
|
||||
DocumentToImageContent,
|
||||
)
|
||||
|
||||
## Query pipeline
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("image_converter", DocumentToImageContent(detail="auto"))
|
||||
pipeline.add_component(
|
||||
"chat_prompt_builder",
|
||||
ChatPromptBuilder(
|
||||
required_variables=["question"],
|
||||
template="""{% message role="system" %}
|
||||
You are a friendly assistant that answers questions based on provided images.
|
||||
{% endmessage %}
|
||||
|
||||
{%- message role="user" -%}
|
||||
Only provide an answer to the question using the images provided.
|
||||
|
||||
Question: {{ question }}
|
||||
Answer:
|
||||
|
||||
{%- for img in image_contents -%}
|
||||
{{ img | templatize_part }}
|
||||
{%- endfor -%}
|
||||
{%- endmessage -%}
|
||||
""",
|
||||
),
|
||||
)
|
||||
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
|
||||
|
||||
pipeline.connect("image_converter", "chat_prompt_builder.image_contents")
|
||||
pipeline.connect("chat_prompt_builder", "llm")
|
||||
|
||||
documents = [
|
||||
Document(content="Cat image", meta={"file_path": "cat.jpg"}),
|
||||
Document(content="Doc intro", meta={"file_path": "paper.pdf", "page_number": 1}),
|
||||
]
|
||||
|
||||
result = pipeline.run(
|
||||
data={
|
||||
"image_converter": {"documents": documents},
|
||||
"chat_prompt_builder": {"question": "What color is the cat?"},
|
||||
},
|
||||
)
|
||||
print(result)
|
||||
|
||||
## {
|
||||
## "llm": {
|
||||
## "replies": [
|
||||
## ChatMessage(
|
||||
## _role=<ChatRole.ASSISTANT: 'assistant'>,
|
||||
## _content=[TextContent(text="The cat is orange with some black.")],
|
||||
## _name=None,
|
||||
## _meta={
|
||||
## "model": "gpt-4o-mini-2024-07-18",
|
||||
## "index": 0,
|
||||
## "finish_reason": "stop",
|
||||
## "usage": {...},
|
||||
## },
|
||||
## )
|
||||
## ]
|
||||
## }
|
||||
## }
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Introduction to Multimodality](https://haystack.deepset.ai/cookbook/multimodal_intro)
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
---
|
||||
title: "DOCXToDocument"
|
||||
id: docxtodocument
|
||||
slug: "/docxtodocument"
|
||||
description: "Convert DOCX files to documents."
|
||||
---
|
||||
|
||||
# DOCXToDocument
|
||||
|
||||
Convert DOCX files to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: DOCX file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/docx.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `DOCXToDocument` component converts DOCX files into documents. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. By defining the table format (CSV or Markdown), you can use this component to extract tables in your DOCX files. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
## Usage
|
||||
|
||||
First, install the`python-docx` package to start using this converter:
|
||||
|
||||
```shell
|
||||
pip install python-docx
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters.docx import DOCXToDocument, DOCXTableFormat
|
||||
|
||||
converter = DOCXToDocument()
|
||||
## or define the table format
|
||||
converter = DOCXToDocument(table_format=DOCXTableFormat.CSV)
|
||||
|
||||
results = converter.run(
|
||||
sources=["sample.docx"],
|
||||
meta={"date_added": datetime.now().isoformat()},
|
||||
)
|
||||
documents = results["documents"]
|
||||
|
||||
print(documents[0].content)
|
||||
|
||||
## 'This is the text from the DOCX file.'
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import DOCXToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", DOCXToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
---
|
||||
title: "External Integrations"
|
||||
id: external-integrations-converters
|
||||
slug: "/external-integrations-converters"
|
||||
description: "External integrations that enable extracting data from files in different formats and cast it into the unified document format."
|
||||
---
|
||||
|
||||
# External Integrations
|
||||
|
||||
External integrations that enable extracting data from files in different formats and cast it into the unified document format.
|
||||
|
||||
| Name | Description |
|
||||
| --- | --- |
|
||||
| [Docling](https://haystack.deepset.ai/integrations/docling/) | Parse PDF, DOCX, HTML, and other document formats into a rich standardized representation (such as layout, tables..), which it can then export to Markdown, JSON, and other formats. |
|
||||
+70
@@ -0,0 +1,70 @@
|
||||
---
|
||||
title: "HTMLToDocument"
|
||||
id: htmltodocument
|
||||
slug: "/htmltodocument"
|
||||
description: "A component that converts HTML files to documents."
|
||||
---
|
||||
|
||||
# HTMLToDocument
|
||||
|
||||
A component that converts HTML files to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: A list of HTML file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/html.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `HTMLToDocument` component converts HTML files into documents. It can be used in an indexing pipeline to index the contents of an HTML file into a Document Store or even in a querying pipeline after the [`LinkContentFetcher`](../fetchers/linkcontentfetcher.mdx). The `HTMLToDocument` component takes a list of HTML file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and converts the files to a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
When you initialize the component, you can optionally set `extraction_kwargs`, a dictionary containing keyword arguments to customize the extraction process. These are passed to the underlying Trafilatura `extract` function. For the full list of available arguments, see the [Trafilatura documentation](https://trafilatura.readthedocs.io/en/latest/corefunctions.html#extract).
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from haystack.components.converters import HTMLToDocument
|
||||
|
||||
converter = HTMLToDocument()
|
||||
|
||||
docs = converter.run(sources=[Path("saved_page.html")])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Here's an example of an indexing pipeline that writes the contents of an HTML file into an `InMemoryDocumentStore`:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import HTMLToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", HTMLToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
---
|
||||
title: "ImageFileToDocument"
|
||||
id: imagefiletodocument
|
||||
slug: "/imagefiletodocument"
|
||||
description: "Converts image file references into empty `Document` objects with associated metadata."
|
||||
---
|
||||
|
||||
# ImageFileToDocument
|
||||
|
||||
Converts image file references into empty `Document` objects with associated metadata.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a component that processes images, like `SentenceTransformersImageDocumentEmbedder` or `LLMDocumentContentExtractor` |
|
||||
| **Mandatory run variables** | `sources`: A list of image file paths or ByteStreams |
|
||||
| **Output variables** | `documents`: A list of empty Document objects with associated metadata |
|
||||
| **API reference** | [Image Converters](/reference/image-converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/image/file_to_document.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`ImageFileToDocument` converts image file sources into empty `Document` objects with associated metadata.
|
||||
|
||||
This component is useful in pipelines where image file paths need to be wrapped in `Document` objects to be processed by downstream components such as `SentenceTransformersImageDocumentEmbedder` or `LLMDocumentContentExtractor`.
|
||||
|
||||
It _does not_ extract any content from the image files, but instead creates `Document` objects with `None` as their content and attaches metadata such as file path and any user-provided values.
|
||||
|
||||
Each source can be:
|
||||
|
||||
- A file path (string or `Path`), or
|
||||
- A `ByteStream` object.
|
||||
|
||||
Optionally, you can provide metadata using the `meta` parameter. This can be a single dictionary (applied to all documents) or a list matching the length of `sources`.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
This component is primarily meant to be used in pipelines.
|
||||
|
||||
```python
|
||||
|
||||
from haystack.components.converters.image import ImageFileToDocument
|
||||
|
||||
converter = ImageFileToDocument()
|
||||
|
||||
sources = ["image.jpg", "another_image.png"]
|
||||
|
||||
result = converter.run(sources=sources)
|
||||
documents = result["documents"]
|
||||
|
||||
print(documents)
|
||||
|
||||
## [Document(id=..., content=None, meta={'file_path': 'image.jpg'}),
|
||||
## Document(id=..., content=None, meta={'file_path': 'another_image.png'})]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
In the following Pipeline, image documents are created using the `ImageFileToDocument` component, then they are enriched with image embeddings and saved in the Document Store.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters.image import ImageFileToDocument
|
||||
from haystack.components.embedders.image import (
|
||||
SentenceTransformersDocumentImageEmbedder,
|
||||
)
|
||||
from haystack.components.writers.document_writer import DocumentWriter
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
## Create our document store
|
||||
doc_store = InMemoryDocumentStore()
|
||||
|
||||
## Define pipeline with components
|
||||
indexing_pipe = Pipeline()
|
||||
indexing_pipe.add_component(
|
||||
"image_converter",
|
||||
ImageFileToDocument(store_full_path=True),
|
||||
)
|
||||
indexing_pipe.add_component(
|
||||
"image_doc_embedder",
|
||||
SentenceTransformersDocumentImageEmbedder(),
|
||||
)
|
||||
indexing_pipe.add_component("document_writer", DocumentWriter(doc_store))
|
||||
|
||||
indexing_pipe.connect("image_converter.documents", "image_doc_embedder.documents")
|
||||
indexing_pipe.connect("image_doc_embedder.documents", "document_writer.documents")
|
||||
|
||||
indexing_result = indexing_pipe.run(
|
||||
data={"image_converter": {"sources": ["apple.jpg", "kiwi.png"]}},
|
||||
)
|
||||
|
||||
indexed_documents = doc_store.filter_documents()
|
||||
print(f"Indexed {len(indexed_documents)} documents")
|
||||
## Indexed 2 documents
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Introduction to Multimodality](https://haystack.deepset.ai/cookbook/multimodal_intro)
|
||||
+128
@@ -0,0 +1,128 @@
|
||||
---
|
||||
title: "ImageFileToImageContent"
|
||||
id: imagefiletoimagecontent
|
||||
slug: "/imagefiletoimagecontent"
|
||||
description: "`ImageFileToImageContent` reads local image files and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image captioning, visual QA, or prompt-based generation."
|
||||
---
|
||||
|
||||
# ImageFileToImageContent
|
||||
|
||||
`ImageFileToImageContent` reads local image files and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image captioning, visual QA, or prompt-based generation.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a `ChatPromptBuilder` in a query pipeline |
|
||||
| **Mandatory run variables** | `sources`: A list of image file paths or ByteStreams |
|
||||
| **Output variables** | `image_contents`: A list of ImageContent objects |
|
||||
| **API reference** | [Image Converters](/reference/image-converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/image/file_to_image.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`ImageFileToImageContent` processes a list of image sources and converts them into `ImageContent` objects. These can be used in multimodal pipelines that require base64-encoded image input.
|
||||
|
||||
Each source can be:
|
||||
|
||||
- A file path (string or `Path`), or
|
||||
- A `ByteStream` object.
|
||||
|
||||
Optionally, you can provide metadata using the `meta` parameter. This can be a single dictionary (applied to all images) or a list matching the length of `sources`.
|
||||
|
||||
Use the `size` parameter to resize images while preserving aspect ratio. This reduces memory usage and transmission size, which is helpful when working with remote models or limited-resource environments.
|
||||
|
||||
This component is often used in query pipelines just before a `ChatPromptBuilder`.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
|
||||
from haystack.components.converters.image import ImageFileToImageContent
|
||||
|
||||
converter = ImageFileToImageContent(detail="high", size=(800, 600))
|
||||
|
||||
sources = ["cat.jpg", "scenery.png"]
|
||||
|
||||
result = converter.run(sources=sources)
|
||||
image_contents = result["image_contents"]
|
||||
print(image_contents)
|
||||
|
||||
## [
|
||||
## ImageContent(
|
||||
## base64_image="/9j/4A...", mime_type="image/jpeg", detail="high",
|
||||
## meta={"file_path": "cat.jpg"}
|
||||
## ),
|
||||
## ImageContent(
|
||||
## base64_image="/9j/4A...", mime_type="image/png", detail="high",
|
||||
## meta={"file_path": "scenery.png"}
|
||||
## )
|
||||
## ]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Use `ImageFileToImageContent` to supply image data to a `ChatPromptBuilder` for multimodal QA or captioning with an LLM.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.converters.image import ImageFileToImageContent
|
||||
|
||||
## Query pipeline
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("image_converter", ImageFileToImageContent(detail="auto"))
|
||||
pipeline.add_component(
|
||||
"chat_prompt_builder",
|
||||
ChatPromptBuilder(
|
||||
required_variables=["question"],
|
||||
template="""{% message role="system" %}
|
||||
You are a helpful assistant that answers questions using the provided images.
|
||||
{% endmessage %}
|
||||
|
||||
{% message role="user" %}
|
||||
Question: {{ question }}
|
||||
|
||||
{% for img in image_contents %}
|
||||
{{ img | templatize_part }}
|
||||
{% endfor %}
|
||||
{% endmessage %}
|
||||
""",
|
||||
),
|
||||
)
|
||||
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
|
||||
|
||||
pipeline.connect("image_converter", "chat_prompt_builder.image_contents")
|
||||
pipeline.connect("chat_prompt_builder", "llm")
|
||||
|
||||
sources = ["apple.jpg", "haystack-logo.png"]
|
||||
|
||||
result = pipeline.run(
|
||||
data={
|
||||
"image_converter": {"sources": sources},
|
||||
"chat_prompt_builder": {"question": "Describe the Haystack logo."},
|
||||
},
|
||||
)
|
||||
print(result)
|
||||
|
||||
## {
|
||||
## "llm": {
|
||||
## "replies": [
|
||||
## ChatMessage(
|
||||
## _role=<ChatRole.ASSISTANT: 'assistant'>,
|
||||
## _content=[TextContent(text="The Haystack logo features...")],
|
||||
## ...
|
||||
## )
|
||||
## ]
|
||||
## }
|
||||
## }
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Introduction to Multimodality](https://haystack.deepset.ai/cookbook/multimodal_intro)
|
||||
+118
@@ -0,0 +1,118 @@
|
||||
---
|
||||
title: "JSONConverter"
|
||||
id: jsonconverter
|
||||
slug: "/jsonconverter"
|
||||
description: "Converts JSON files to text documents."
|
||||
---
|
||||
|
||||
# JSONConverter
|
||||
|
||||
Converts JSON files to text documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory init variables** | ONE OF, OR BOTH: <br /> <br />`jq_schema`: A jq filter string to extract content <br /> <br />`content_key`: A key string to extract document content |
|
||||
| **Mandatory run variables** | `sources`: A list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/json.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`JSONConverter` converts one or more JSON files into a text document.
|
||||
|
||||
### Parameters Overview
|
||||
|
||||
To initialize `JSONConverter`, you must provide either `jq_schema`, or `content_key` parameter, or both.
|
||||
|
||||
`jq_schema` parameter filter extracts nested data from JSON files. Refer to the [jq documentation](https://jqlang.github.io/jq/) for filter syntax. If not set, the entire JSON file is used.
|
||||
|
||||
The `content_key` parameter lets you specify which key in the extracted data will be the document's content.
|
||||
|
||||
- If both `jq_schema` and `content_key` are set, the `content_key` is searched in the data extracted by `jq_schema`. Non-object data will be skipped.
|
||||
- If only `jq_schema` is set, the extracted value must be scalar; objects or arrays will be skipped.
|
||||
- If only `content_key` is set, the source must be a JSON object, or it will be skipped.
|
||||
|
||||
Check out the [API reference](../converters.mdx) for the full list of parameters.
|
||||
|
||||
## Usage
|
||||
|
||||
You need to install the `jq` package to use this Converter:
|
||||
|
||||
```shell
|
||||
pip install jq
|
||||
```
|
||||
|
||||
### Example
|
||||
|
||||
Here is an example of simple component usage:
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
from haystack.components.converters import JSONConverter
|
||||
from haystack.dataclasses import ByteStream
|
||||
|
||||
source = ByteStream.from_string(
|
||||
json.dumps({"text": "This is the content of my document"}),
|
||||
)
|
||||
|
||||
converter = JSONConverter(content_key="text")
|
||||
results = converter.run(sources=[source])
|
||||
documents = results["documents"]
|
||||
print(documents[0].content)
|
||||
## 'This is the content of my document'
|
||||
```
|
||||
|
||||
In the following more complex example, we provide a `jq_schema` string to filter the JSON source files and `extra_meta_fields` to extract from the filtered data:
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
from haystack.components.converters import JSONConverter
|
||||
from haystack.dataclasses import ByteStream
|
||||
|
||||
data = {
|
||||
"laureates": [
|
||||
{
|
||||
"firstname": "Enrico",
|
||||
"surname": "Fermi",
|
||||
"motivation": "for his demonstrations of the existence of new radioactive elements produced "
|
||||
"by neutron irradiation, and for his related discovery of nuclear reactions brought about by"
|
||||
" slow neutrons",
|
||||
},
|
||||
{
|
||||
"firstname": "Rita",
|
||||
"surname": "Levi-Montalcini",
|
||||
"motivation": "for their discoveries of growth factors",
|
||||
},
|
||||
],
|
||||
}
|
||||
source = ByteStream.from_string(json.dumps(data))
|
||||
converter = JSONConverter(
|
||||
jq_schema=".laureates[]",
|
||||
content_key="motivation",
|
||||
extra_meta_fields={"firstname", "surname"},
|
||||
)
|
||||
|
||||
results = converter.run(sources=[source])
|
||||
documents = results["documents"]
|
||||
print(documents[0].content)
|
||||
## 'for his demonstrations of the existence of new radioactive elements produced by
|
||||
## neutron irradiation, and for his related discovery of nuclear reactions brought
|
||||
## about by slow neutrons'
|
||||
|
||||
print(documents[0].meta)
|
||||
## {'firstname': 'Enrico', 'surname': 'Fermi'}
|
||||
|
||||
print(documents[1].content)
|
||||
## 'for their discoveries of growth factors'
|
||||
|
||||
print(documents[1].meta)
|
||||
## {'firstname': 'Rita', 'surname': 'Levi-Montalcini'}
|
||||
```
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
---
|
||||
title: "MarkdownToDocument"
|
||||
id: markdowntodocument
|
||||
slug: "/markdowntodocument"
|
||||
description: "A component that converts Markdown files to documents."
|
||||
---
|
||||
|
||||
# MarkdownToDocument
|
||||
|
||||
A component that converts Markdown files to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: Markdown file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/markdown.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `MarkdownToDocument` component converts Markdown files into documents. You can use it in an indexing pipeline to index the contents of a Markdown file into a Document Store. It takes a list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
When you initialize the component, you can optionally turn off progress bars by setting `progress_bar` to `False`. If you want to convert the contents of tables into a single line, you can enable that through the `table_to_single_line` parameter.
|
||||
|
||||
## Usage
|
||||
|
||||
You need to install `markdown-it-py` and `mdit_plain packages` to use the `MarkdownToDocument` component:
|
||||
|
||||
```shell
|
||||
pip install markdown-it-py mdit_plain
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters import MarkdownToDocument
|
||||
|
||||
converter = MarkdownToDocument()
|
||||
|
||||
docs = converter.run(sources=Path("my_file.md"))
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import MarkdownToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", MarkdownToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
:notebook: Tutorial: [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline)
|
||||
+189
@@ -0,0 +1,189 @@
|
||||
---
|
||||
title: "MistralOCRDocumentConverter"
|
||||
id: mistralocrdocumentconverter
|
||||
slug: "/mistralocrdocumentconverter"
|
||||
description: "`MistralOCRDocumentConverter` extracts text from documents using Mistral's OCR API, with optional structured annotations for both individual image regions and full documents. It supports various input formats including local files, URLs, and Mistral file IDs."
|
||||
---
|
||||
|
||||
# MistralOCRDocumentConverter
|
||||
|
||||
`MistralOCRDocumentConverter` extracts text from documents using Mistral's OCR API, with optional structured annotations for both individual image regions and full documents. It supports various input formats including local files, URLs, and Mistral file IDs.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Mistral API key. Can be set with `MISTRAL_API_KEY` environment variable. |
|
||||
| **Mandatory run variables** | `sources`: A list of document sources (file paths, ByteStreams, URLs, or Mistral chunks) |
|
||||
| **Output variables** | `documents`: A list of documents <br /> <br />`raw_mistral_response`: A list of raw OCR responses from Mistral API |
|
||||
| **API reference** | [Mistral](/reference/integrations-mistral) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `MistralOCRDocumentConverter` takes a list of document sources and uses Mistral's OCR API to extract text from images and PDFs. It supports multiple input formats:
|
||||
|
||||
- **Local files**: File paths (str or Path) or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects
|
||||
- **Remote resources**: Document URLs, image URLs using Mistral's `DocumentURLChunk` and `ImageURLChunk`
|
||||
- **Mistral storage**: File IDs using Mistral's `FileChunk` for files previously uploaded to Mistral
|
||||
|
||||
The component returns one Haystack [`Document`](../../concepts/data-classes.mdx#document) per source, with all pages concatenated using form feed characters (`\f`) as separators. This format ensures compatibility with Haystack's [`DocumentSplitter`](../preprocessors/documentsplitter.mdx) for accurate page-wise splitting and overlap handling. The content is returned in markdown format, with images represented as `` tags.
|
||||
|
||||
By default, the component uses the `MISTRAL_API_KEY` environment variable for authentication. You can also pass an `api_key` at initialization. Local files are automatically uploaded to Mistral's storage for processing and deleted afterward (configurable with `cleanup_uploaded_files`).
|
||||
|
||||
When you initialize the component, you can optionally specify which pages to process, set limits on image extraction, configure minimum image sizes, or include base64-encoded images in the response. The default model is `"mistral-ocr-2505"`. See the [Mistral models documentation](https://docs.mistral.ai/getting-started/models/models_overview/) for available models.
|
||||
|
||||
### Structured Annotations
|
||||
|
||||
A unique feature of `MistralOCRDocumentConverter` is its support for structured annotations using Pydantic schemas:
|
||||
|
||||
- **Bounding box annotations** (`bbox_annotation_schema`): Annotate individual image regions with structured data (for example, image type, description, summary). These annotations are inserted inline after the corresponding image tags in the markdown content.
|
||||
- **Document annotations** (`document_annotation_schema`): Annotate the full document with structured data (for example, language, chapter titles, URLs). These annotations are unpacked into the document's metadata with a `source_` prefix (for example, `source_language`, `source_chapter_titles`).
|
||||
|
||||
When annotation schemas are provided, the OCR model first extracts text and structure, then a Vision LLM analyzes the content and generates structured annotations according to your defined Pydantic schemas. Note that document annotation is limited to a maximum of 8 pages. For more details, see the [Mistral documentation on annotations](https://docs.mistral.ai/capabilities/document_ai/annotations/).
|
||||
|
||||
## Usage
|
||||
|
||||
You need to install the `mistral-haystack` integration to use `MistralOCRDocumentConverter`:
|
||||
|
||||
```shell
|
||||
pip install mistral-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
Basic usage with a local file:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from haystack.utils import Secret
|
||||
from haystack_integrations.components.converters.mistral import (
|
||||
MistralOCRDocumentConverter,
|
||||
)
|
||||
|
||||
converter = MistralOCRDocumentConverter(
|
||||
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
|
||||
model="mistral-ocr-2505",
|
||||
)
|
||||
|
||||
result = converter.run(sources=[Path("my_document.pdf")])
|
||||
documents = result["documents"]
|
||||
```
|
||||
|
||||
Processing multiple sources with different types:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from haystack.utils import Secret
|
||||
from haystack_integrations.components.converters.mistral import (
|
||||
MistralOCRDocumentConverter,
|
||||
)
|
||||
from mistralai.models import DocumentURLChunk, ImageURLChunk
|
||||
|
||||
converter = MistralOCRDocumentConverter(
|
||||
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
|
||||
model="mistral-ocr-2505",
|
||||
)
|
||||
|
||||
sources = [
|
||||
Path("local_document.pdf"),
|
||||
DocumentURLChunk(document_url="https://example.com/document.pdf"),
|
||||
ImageURLChunk(image_url="https://example.com/receipt.jpg"),
|
||||
]
|
||||
|
||||
result = converter.run(sources=sources)
|
||||
documents = result["documents"] # List of 3 Documents
|
||||
raw_responses = result["raw_mistral_response"] # List of 3 raw responses
|
||||
```
|
||||
|
||||
Using structured annotations:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
from pydantic import BaseModel, Field
|
||||
from haystack.utils import Secret
|
||||
from haystack_integrations.components.converters.mistral import (
|
||||
MistralOCRDocumentConverter,
|
||||
)
|
||||
from mistralai.models import DocumentURLChunk
|
||||
|
||||
|
||||
# Define schema for image region annotations
|
||||
class ImageAnnotation(BaseModel):
|
||||
image_type: str = Field(..., description="The type of image content")
|
||||
short_description: str = Field(
|
||||
...,
|
||||
description="Short natural-language description",
|
||||
)
|
||||
summary: str = Field(..., description="Detailed summary of the image content")
|
||||
|
||||
|
||||
# Define schema for document-level annotations
|
||||
class DocumentAnnotation(BaseModel):
|
||||
language: str = Field(..., description="Primary language of the document")
|
||||
chapter_titles: List[str] = Field(
|
||||
...,
|
||||
description="Detected chapter or section titles",
|
||||
)
|
||||
urls: List[str] = Field(..., description="URLs found in the text")
|
||||
|
||||
|
||||
converter = MistralOCRDocumentConverter(
|
||||
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
|
||||
model="mistral-ocr-2505",
|
||||
)
|
||||
|
||||
sources = [DocumentURLChunk(document_url="https://example.com/report.pdf")]
|
||||
result = converter.run(
|
||||
sources=sources,
|
||||
bbox_annotation_schema=ImageAnnotation,
|
||||
document_annotation_schema=DocumentAnnotation,
|
||||
)
|
||||
|
||||
documents = result["documents"]
|
||||
# Document metadata will include:
|
||||
# - source_language: extracted from DocumentAnnotation
|
||||
# - source_chapter_titles: extracted from DocumentAnnotation
|
||||
# - source_urls: extracted from DocumentAnnotation
|
||||
# Document content will include inline image annotations
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Here's an example of an indexing pipeline that processes PDFs with OCR and writes them to a Document Store:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.utils import Secret
|
||||
from haystack_integrations.components.converters.mistral import (
|
||||
MistralOCRDocumentConverter,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component(
|
||||
"converter",
|
||||
MistralOCRDocumentConverter(
|
||||
api_key=Secret.from_env_var("MISTRAL_API_KEY"),
|
||||
model="mistral-ocr-2505",
|
||||
),
|
||||
)
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component("splitter", DocumentSplitter(split_by="page", split_length=1))
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
file_paths = ["invoice.pdf", "receipt.jpg", "contract.pdf"]
|
||||
pipeline.run({"converter": {"sources": file_paths}})
|
||||
```
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
---
|
||||
title: "MSGToDocument"
|
||||
id: msgtodocument
|
||||
slug: "/msgtodocument"
|
||||
description: "Converts Microsoft Outlook .msg files to documents."
|
||||
---
|
||||
|
||||
# MSGToDocument
|
||||
|
||||
Converts Microsoft Outlook .msg files to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: A list of .msg file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents <br /> <br />`attachments`: A list of ByteStream objects representing file attachments |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/msg.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `MSGToDocument` component converts Microsoft Outlook `.msg` files into documents. This component extracts the email metadata (such as sender, recipients, CC, BCC, subject) and body content. Additionally, any file attachments within the `.msg` file are extracted as `ByteStream` objects.
|
||||
|
||||
## Usage
|
||||
|
||||
First, install the `python-oxmsg` package to start using this converter:
|
||||
|
||||
```
|
||||
pip install python-oxmsg
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters.msg import MSGToDocument
|
||||
from datetime import datetime
|
||||
|
||||
converter = MSGToDocument()
|
||||
results = converter.run(
|
||||
sources=["sample.msg"],
|
||||
meta={"date_added": datetime.now().isoformat()},
|
||||
)
|
||||
documents = results["documents"]
|
||||
attachments = results["attachments"]
|
||||
|
||||
print(documents[0].content)
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The following setup enables efficient extraction, preprocessing, and indexing of `.msg` email files within a Haystack pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.routers import FileTypeRouter
|
||||
from haystack.components.converters import MSGToDocument
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
router = FileTypeRouter(mime_types=["application/vnd.ms-outlook"])
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("router", router)
|
||||
pipeline.add_component("converter", MSGToDocument())
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
|
||||
pipeline.connect("router.application/vnd.ms-outlook", "converter.sources")
|
||||
pipeline.connect("converter.documents", "writer.documents")
|
||||
|
||||
file_names = ["email1.msg", "email2.msg"]
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
+79
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: "MultiFileConverter"
|
||||
id: multifileconverter
|
||||
slug: "/multifileconverter"
|
||||
description: "Converts CSV, DOCX, HTML, JSON, MD, PPTX, PDF, TXT, and XSLX files to documents."
|
||||
---
|
||||
|
||||
# MultiFileConverter
|
||||
|
||||
Converts CSV, DOCX, HTML, JSON, MD, PPTX, PDF, TXT, and XSLX files to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before PreProcessors , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: A list of file paths or ByteStream objects |
|
||||
| **Output variables** | `documents`: A list of converted documents <br /> <br />`unclassified`: A list of uncategorized file paths or byte streams |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/multi_file_converter.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`MultiFileConverter` converts input files of various file types into documents.
|
||||
|
||||
It is a SuperComponent that combines a [`FileTypeRouter`](../routers/filetyperouter.mdx), nine converters and a [`DocumentJoiner`](../joiners/documentjoiner.mdx) into a single component.
|
||||
|
||||
### Parameters
|
||||
|
||||
To initialize `MultiFileConverter`, there are no mandatory parameters. Optionally, you can provide `encoding` and `json_content_key` parameters.
|
||||
|
||||
The `json_content_key` parameter lets you specify for the JSON files which key in the extracted data will be the document's content. The parameter is passed on to the underlying [`JSONConverter`](jsonconverter.mdx) component.
|
||||
|
||||
The `encoding` parameter lets you specify the default encoding of the TXT, CSV, and MD files. If you don't provide any value, the component uses `utf-8` by default. Note that if the encoding is specified in the metadata of an input ByteStream, it will override this parameter's setting. The parameter is passed on to the underlying [`TextFileToDocument`](textfiletodocument.mdx) and [`CSVToDocument`](csvtodocument.mdx) components.
|
||||
|
||||
## Usage
|
||||
|
||||
Install dependencies for all supported file types to use the `MultiFileConverter`:
|
||||
|
||||
```shell
|
||||
pip install pypdf markdown-it-py mdit_plain trafilatura python-pptx python-docx jq openpyxl tabulate pandas
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters import MultiFileConverter
|
||||
|
||||
converter = MultiFileConverter()
|
||||
converter.run(sources=["test.txt", "test.pdf"], meta={})
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
You can also use `MultiFileConverter` in your indexing pipeline.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters import MultiFileConverter
|
||||
from haystack.components.preprocessors import DocumentPreprocessor
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", MultiFileConverter())
|
||||
pipeline.add_component("preprocessor", DocumentPreprocessor())
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "preprocessor")
|
||||
pipeline.connect("preprocessor", "writer")
|
||||
|
||||
result = pipeline.run(data={"sources": ["test.txt", "test.pdf"]})
|
||||
|
||||
print(result)
|
||||
## {'writer': {'documents_written': 3}}
|
||||
```
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "OpenAPIServiceToFunctions"
|
||||
id: openapiservicetofunctions
|
||||
slug: "/openapiservicetofunctions"
|
||||
description: "`OpenAPIServiceToFunctions` is a component that transforms OpenAPI service specifications into a format compatible with OpenAI's function calling mechanism."
|
||||
---
|
||||
|
||||
# OpenAPIServiceToFunctions
|
||||
|
||||
`OpenAPIServiceToFunctions` is a component that transforms OpenAPI service specifications into a format compatible with OpenAI's function calling mechanism.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Flexible |
|
||||
| **Mandatory run variables** | `sources`: A list of OpenAPI specification sources, which can be file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `functions`: A list of JSON OpenAI function calling definitions objects. For each path definition in OpenAPI specification, a corresponding OpenAI function calling definitions is generated. <br /> <br />`openapi_specs`: A list of JSON/YAML objects with references resolved. Such OpenAPI spec (with references resolved) can, in turn, be used as input to OpenAPIServiceConnector. |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/openapi_functions.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`OpenAPIServiceToFunctions` transforms OpenAPI service specifications into an OpenAI function calling format. It takes an OpenAPI specification, processes it to extract function definitions, and formats these definitions to be compatible with OpenAI's function calling JSON format.
|
||||
|
||||
`OpenAPIServiceToFunctions` is valuable when used together with [`OpenAPIServiceConnector`](../connectors/openapiserviceconnector.mdx) component. It converts OpenAPI specifications into definitions suitable for OpenAI's function calls, allowing `OpenAPIServiceConnector` to handle input parameters for the OpenAPI specification and facilitate their use in REST API calls through `OpenAPIServiceConnector`.
|
||||
|
||||
To use `OpenAPIServiceToFunctions`, you need to install an optional `jsonref` dependency with:
|
||||
|
||||
```shell
|
||||
pip install jsonref
|
||||
```
|
||||
|
||||
`OpenAPIServiceToFunctions` component doesn’t have any init parameters.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
This component is primarily meant to be used in pipelines. Using this component alone is useful when you want to convert OpenAPI specification into OpenAI's function call specification and then perhaps save it in a file and subsequently use it in function calling.
|
||||
|
||||
### In a pipeline
|
||||
|
||||
In a pipeline context, `OpenAPIServiceToFunctions` is most valuable when used alongside `OpenAPIServiceConnector`. For instance, let’s consider integrating [serper.dev](http://serper.dev/) search engine bridge into a pipeline. `OpenAPIServiceToFunctions` retrieves the OpenAPI specification of Serper from https://bit.ly/serper_dev_spec, converts this specification into a format that OpenAI's function calling mechanism can understand, and then seamlessly passes this translated specification as `generation_kwargs` for LLM function calling invocation.
|
||||
|
||||
:::info
|
||||
To run the following code snippet, note that you have to have your own Serper and OpenAI API keys.
|
||||
:::
|
||||
|
||||
```python
|
||||
import json
|
||||
import requests
|
||||
|
||||
from typing import Dict, Any, List
|
||||
from haystack import Pipeline
|
||||
from haystack.components.generators.utils import print_streaming_chunk
|
||||
from haystack.components.converters import OpenAPIServiceToFunctions, OutputAdapter
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.connectors import OpenAPIServiceConnector
|
||||
from haystack.components.fetchers import LinkContentFetcher
|
||||
from haystack.dataclasses import ChatMessage, ByteStream
|
||||
from haystack.utils import Secret
|
||||
|
||||
def prepare_fc_params(openai_functions_schema: Dict[str, Any]) -> Dict[str, Any]:
|
||||
return {
|
||||
"tools": [{
|
||||
"type": "function",
|
||||
"function": openai_functions_schema
|
||||
}],
|
||||
"tool_choice": {
|
||||
"type": "function",
|
||||
"function": {"name": openai_functions_schema["name"]}
|
||||
}
|
||||
}
|
||||
|
||||
system_prompt = requests.get("https://bit.ly/serper_dev_system_prompt").text
|
||||
serper_spec = requests.get("https://bit.ly/serper_dev_spec").text
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("spec_to_functions", OpenAPIServiceToFunctions())
|
||||
pipe.add_component("functions_llm", OpenAIChatGenerator(api_key=Secret.from_token(llm_api_key), model="gpt-3.5-turbo-0613"))
|
||||
pipe.add_component("openapi_container", OpenAPIServiceConnector())
|
||||
pipe.add_component("a1", OutputAdapter("{{functions[0] | prepare_fc}}", Dict[str, Any], {"prepare_fc": prepare_fc_params}))
|
||||
pipe.add_component("a2", OutputAdapter("{{specs[0]}}", Dict[str, Any]))
|
||||
pipe.add_component("a3", OutputAdapter("{{system_message + service_response}}", List[ChatMessage]))
|
||||
pipe.add_component("llm", OpenAIChatGenerator(api_key=Secret.from_token(llm_api_key), model="gpt-4-1106-preview", streaming_callback=print_streaming_chunk))
|
||||
|
||||
pipe.connect("spec_to_functions.functions", "a1.functions")
|
||||
pipe.connect("spec_to_functions.openapi_specs", "a2.specs")
|
||||
pipe.connect("a1", "functions_llm.generation_kwargs")
|
||||
pipe.connect("functions_llm.replies", "openapi_container.messages")
|
||||
pipe.connect("a2", "openapi_container.service_openapi_spec")
|
||||
pipe.connect("openapi_container.service_response", "a3.service_response")
|
||||
pipe.connect("a3", "llm.messages")
|
||||
|
||||
user_prompt = "Why was Sam Altman ousted from OpenAI?"
|
||||
|
||||
result = pipe.run(data={"functions_llm": {"messages":[ChatMessage.from_system("Only do function calling"), ChatMessage.from_user(user_prompt)]},
|
||||
"openapi_container": {"service_credentials": serper_dev_key},
|
||||
"spec_to_functions": {"sources": [ByteStream.from_string(serper_spec)]},
|
||||
"a3": {"system_message": [ChatMessage.from_system(system_prompt)]}})
|
||||
|
||||
>Sam Altman was ousted from OpenAI on November 17, 2023, following
|
||||
>a "deliberative review process" by the board of directors. The board concluded
|
||||
>that he was not "consistently candid in his communications". However, he
|
||||
>returned as CEO just days after his ouster.
|
||||
```
|
||||
+134
@@ -0,0 +1,134 @@
|
||||
---
|
||||
title: "OutputAdapter"
|
||||
id: outputadapter
|
||||
slug: "/outputadapter"
|
||||
description: "This component helps the output of one component fit smoothly into the input of another. It uses Jinja expressions to define how this adaptation occurs."
|
||||
---
|
||||
|
||||
# OutputAdapter
|
||||
|
||||
This component helps the output of one component fit smoothly into the input of another. It uses Jinja expressions to define how this adaptation occurs.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Flexible |
|
||||
| **Mandatory init variables** | `template`: A Jinja template string that defines how to adapt the data <br /> <br />`output_type`: Type alias that this instance will return |
|
||||
| **Mandatory run variables** | `**kwargs`: Input variables to be used in Jinja expression. See [Variables](#variables) section for more details. |
|
||||
| **Output variables** | The output is specified under the `output` key dictionary |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/output_adapter.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
To use `OutputAdapter`, you need to specify the adaptation rule that includes:
|
||||
|
||||
- `template`: A Jinja template string that defines how to adapt the input data.
|
||||
- `output_type`: The type of the output data (such as `str`, `List[int]`..). This doesn't change the actual output type and is only needed to validate connection with other components.
|
||||
- `custom_filters`: An optional dictionary of custom Jinja filters to be used in the template.
|
||||
|
||||
### Variables
|
||||
|
||||
The `OutputAdapter` requires all template variables to be present before running and raises an error if any template variable is missing at pipeline connect time.
|
||||
|
||||
```python
|
||||
from haystack.components.converters import OutputAdapter
|
||||
|
||||
adapter = OutputAdapter(template="Hello {{name}}!", output_type=str)
|
||||
```
|
||||
|
||||
### Unsafe behavior
|
||||
|
||||
The `OutputAdapter` internally renders the `template` using Jinja, and by default, this is safe behavior. However, it limits the output types to strings, bytes, numbers, tuples, lists, dicts, sets, booleans, `None`, and `Ellipsis` (`...`), as well as any combination of these structures.
|
||||
|
||||
If you want to use other types such as `ChatMessage`, `Document`, or `Answer`, you must enable unsafe template rendering by setting the `unsafe` init argument to `True`.
|
||||
|
||||
Be cautious, as enabling this can be unsafe and may lead to remote code execution if the `template` is a string customizable by the end user.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
This component is primarily meant to be used in pipelines.
|
||||
|
||||
In this example, `OutputAdapter` simply outputs the content field of the first document in the arrays of documents:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.converters import OutputAdapter
|
||||
|
||||
adapter = OutputAdapter(template="{{ documents[0].content }}", output_type=str)
|
||||
input_data = {"documents": [Document(content="Test content")]}
|
||||
expected_output = {"output": "Test content"}
|
||||
assert adapter.run(**input_data) == expected_output
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
The example below demonstrates a straightforward pipeline that uses the `OutputAdapter` to capitalize the first document in the list. If needed, you can also utilize the predefined Jinja [filters](https://jinja.palletsprojects.com/en/3.1.x/templates/#builtin-filters).
|
||||
|
||||
```python
|
||||
from haystack import Pipeline, component, Document
|
||||
from haystack.components.converters import OutputAdapter
|
||||
|
||||
|
||||
@component
|
||||
class DocumentProducer:
|
||||
@component.output_types(documents=dict)
|
||||
def run(self):
|
||||
return {"documents": [Document(content="haystack")]}
|
||||
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component(
|
||||
name="output_adapter",
|
||||
instance=OutputAdapter(
|
||||
template="{{ documents[0].content | capitalize}}",
|
||||
output_type=str,
|
||||
),
|
||||
)
|
||||
pipe.add_component(name="document_producer", instance=DocumentProducer())
|
||||
pipe.connect("document_producer", "output_adapter")
|
||||
result = pipe.run(data={})
|
||||
|
||||
assert result["output_adapter"]["output"] == "Haystack"
|
||||
```
|
||||
|
||||
You can also define your own custom filters, which can then be added to an `OutputAdapter` instance through its init method and used in templates. Here’s an example of this approach:
|
||||
|
||||
```python
|
||||
|
||||
from haystack import Pipeline, component, Document
|
||||
from haystack.components.converters import OutputAdapter
|
||||
|
||||
|
||||
def reverse_string(s):
|
||||
return s[::-1]
|
||||
|
||||
|
||||
@component
|
||||
class DocumentProducer:
|
||||
@component.output_types(documents=dict)
|
||||
def run(self):
|
||||
return {"documents": [Document(content="haystack")]}
|
||||
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component(
|
||||
name="output_adapter",
|
||||
instance=OutputAdapter(
|
||||
template="{{ documents[0].content | reverse_string}}",
|
||||
output_type=str,
|
||||
custom_filters={"reverse_string": reverse_string},
|
||||
),
|
||||
)
|
||||
|
||||
pipe.add_component(name="document_producer", instance=DocumentProducer())
|
||||
pipe.connect("document_producer", "output_adapter")
|
||||
result = pipe.run(data={})
|
||||
|
||||
assert result["output_adapter"]["output"] == "kcatsyah"
|
||||
```
|
||||
+97
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: "PaddleOCRVLDocumentConverter"
|
||||
id: paddleocrvldocumentconverter
|
||||
slug: "/paddleocrvldocumentconverter"
|
||||
description: "`PaddleOCRVLDocumentConverter` extracts text from documents using PaddleOCR's large model document parsing API."
|
||||
---
|
||||
|
||||
# PaddleOCRVLDocumentConverter
|
||||
|
||||
`PaddleOCRVLDocumentConverter` extracts text from documents using PaddleOCR's large model document parsing API. PaddleOCR-VL is used behind the scenes. For more information, please refer to the [PaddleOCR-VL documentation](https://www.paddleocr.ai/latest/en/version3.x/algorithm/PaddleOCR-VL/PaddleOCR-VL.html).
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx), or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_url`: The URL of the PaddleOCR-VL API. <br /> <br /> `access_token`: The AI Studio access token. Can be set with `AISTUDIO_ACCESS_TOKEN` environment variable. |
|
||||
| **Mandatory run variables** | `sources`: A list of image or PDF file paths or ByteStream objects. |
|
||||
| **Output variables** | `documents`: A list of documents. <br /> <br />`raw_paddleocr_responses`: A list of raw OCR responses from PaddleOCR API. |
|
||||
| **API reference** | [PaddleOCR](/reference/integrations-paddleocr) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/paddleocr |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `PaddleOCRVLDocumentConverter` takes a list of document sources and uses PaddleOCR's large model document parsing API to extract text from images and PDFs. It supports both images and PDF files.
|
||||
|
||||
The component returns one Haystack [`Document`](../../concepts/data-classes.mdx#document) per source, with all pages concatenated using form feed characters (`\f`) as separators. This format ensures compatibility with Haystack's [`DocumentSplitter`](../preprocessors/documentsplitter.mdx) for accurate page-wise splitting and overlap handling. The content is returned in markdown format, with images represented as `` tags.
|
||||
|
||||
The component takes `api_url` as a required parameter. To obtain the API URL, visit the [PaddleOCR official website](https://aistudio.baidu.com/paddleocr/task), click the **API** button in the upper-left corner, choose the example code for **Large Model document parsing(PaddleOCR-VL)**, and copy the `API_URL`.
|
||||
|
||||
By default, the component uses the `AISTUDIO_ACCESS_TOKEN` environment variable for authentication. You can also pass an `access_token` at initialization. The AI Studio access token can be obtained from [this page](https://aistudio.baidu.com/account/accessToken).
|
||||
|
||||
## Usage
|
||||
|
||||
You need to install the `paddleocr-haystack` integration to use `PaddleOCRVLDocumentConverter`:
|
||||
|
||||
```shell
|
||||
pip install paddleocr-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
Basic usage with a local file:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from haystack.utils import Secret
|
||||
from haystack_integrations.components.converters.paddleocr import (
|
||||
PaddleOCRVLDocumentConverter,
|
||||
)
|
||||
|
||||
converter = PaddleOCRVLDocumentConverter(
|
||||
api_url="<your-api-url>",
|
||||
access_token=Secret.from_env_var("AISTUDIO_ACCESS_TOKEN"),
|
||||
)
|
||||
|
||||
result = converter.run(sources=[Path("my_document.pdf")])
|
||||
documents = result["documents"]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Here's an example of an indexing pipeline that processes PDFs with OCR and writes them to a Document Store:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.utils import Secret
|
||||
from haystack_integrations.components.converters.paddleocr import (
|
||||
PaddleOCRVLDocumentConverter,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component(
|
||||
"converter",
|
||||
PaddleOCRVLDocumentConverter(
|
||||
api_url="<your-api-url>",
|
||||
access_token=Secret.from_env_var("AISTUDIO_ACCESS_TOKEN"),
|
||||
),
|
||||
)
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component("splitter", DocumentSplitter(split_by="page", split_length=1))
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
file_paths = ["invoice.pdf", "receipt.jpg", "contract.pdf"]
|
||||
pipeline.run({"converter": {"sources": file_paths}})
|
||||
```
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
---
|
||||
title: "PDFMinerToDocument"
|
||||
id: pdfminertodocument
|
||||
slug: "/pdfminertodocument"
|
||||
description: "A component that converts complex PDF files to documents using pdfminer arguments."
|
||||
---
|
||||
|
||||
# PDFMinerToDocument
|
||||
|
||||
A component that converts complex PDF files to documents using pdfminer arguments.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: PDF file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/pdfminer.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `PDFMinerToDocument` component converts PDF files into documents using [PDFMiner](https://pdfminersix.readthedocs.io/en/latest/) extraction tool arguments.
|
||||
|
||||
You can use it in an indexing pipeline to index the contents of a PDF file in a Document Store. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream)objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
When initializing the component, you can adjust several parameters to fit your PDF. See the full parameter list and descriptions in our [API reference](/reference/converters-api#pdfminertodocument).
|
||||
|
||||
## Usage
|
||||
|
||||
First, install `pdfminer` package to start using this converter:
|
||||
|
||||
```shell
|
||||
pip install pdfminer.six
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters import PDFMinerToDocument
|
||||
|
||||
converter = PDFMinerToDocument()
|
||||
results = converter.run(
|
||||
sources=["sample.pdf"],
|
||||
meta={"date_added": datetime.now().isoformat()},
|
||||
)
|
||||
documents = results["documents"]
|
||||
|
||||
print(documents[0].content)
|
||||
|
||||
## 'This is a text from the PDF file.'
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import PDFMinerToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", PDFMinerToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
+116
@@ -0,0 +1,116 @@
|
||||
---
|
||||
title: "PDFToImageContent"
|
||||
id: pdftoimagecontent
|
||||
slug: "/pdftoimagecontent"
|
||||
description: "`PDFToImageContent` reads local PDF files and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image captioning, visual QA, or prompt-based generation."
|
||||
---
|
||||
|
||||
# PDFToImageContent
|
||||
|
||||
`PDFToImageContent` reads local PDF files and converts them into `ImageContent` objects. These are ready for multimodal AI pipelines, including tasks like image captioning, visual QA, or prompt-based generation.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a `ChatPromptBuilder` in a query pipeline |
|
||||
| **Mandatory run variables** | `sources`: A list of PDF file paths or ByteStreams |
|
||||
| **Output variables** | `image_contents`: A list of ImageContent objects |
|
||||
| **API reference** | [Image Converters](/reference/image-converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/image/pdf_to_image.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`PDFToImageContent` processes a list of PDF sources and converts them into `ImageContent` objects, one for each page of the PDF. These can be used in multimodal pipelines that require base64-encoded image input.
|
||||
|
||||
Each source can be:
|
||||
|
||||
- A file path (string or `Path`), or
|
||||
- A `ByteStream` object.
|
||||
|
||||
Optionally, you can provide metadata using the `meta` parameter. This can be a single dictionary (applied to all images) or a list matching the length of `sources`.
|
||||
|
||||
Use the `size` parameter to resize images while preserving aspect ratio. This reduces memory usage and transmission size, which is helpful when working with remote models or limited-resource environments.
|
||||
|
||||
This component is often used in query pipelines just before a `ChatPromptBuilder`.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters.image import PDFToImageContent
|
||||
|
||||
converter = PDFToImageContent()
|
||||
|
||||
sources = ["file.pdf", "another_file.pdf"]
|
||||
|
||||
image_contents = converter.run(sources=sources)["image_contents"]
|
||||
print(image_contents)
|
||||
|
||||
## [ImageContent(base64_image='...',
|
||||
## mime_type='application/pdf',
|
||||
## detail=None,
|
||||
## meta={'file_path': 'file.pdf', 'page_number': 1}),
|
||||
## ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Use `ImageFileToImageContent` to supply image data to a `ChatPromptBuilder` for multimodal QA or captioning with an LLM.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.components.converters.image import PDFToImageContent
|
||||
|
||||
## Query pipeline
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("image_converter", PDFToImageContent(detail="auto"))
|
||||
pipeline.add_component(
|
||||
"chat_prompt_builder",
|
||||
ChatPromptBuilder(
|
||||
required_variables=["question"],
|
||||
template="""{% message role="system" %}
|
||||
You are a helpful assistant that answers questions using the provided images.
|
||||
{% endmessage %}
|
||||
|
||||
{% message role="user" %}
|
||||
Question: {{ question }}
|
||||
|
||||
{% for img in image_contents %}
|
||||
{{ img | templatize_part }}
|
||||
{% endfor %}
|
||||
{% endmessage %}
|
||||
""",
|
||||
),
|
||||
)
|
||||
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
|
||||
|
||||
pipeline.connect("image_converter", "chat_prompt_builder.image_contents")
|
||||
pipeline.connect("chat_prompt_builder", "llm")
|
||||
|
||||
sources = ["flan_paper.pdf"]
|
||||
|
||||
result = pipeline.run(
|
||||
data={
|
||||
"image_converter": {"sources": ["flan_paper.pdf"], "page_range": "9"},
|
||||
"chat_prompt_builder": {"question": "What is the main takeaway of Figure 6?"},
|
||||
},
|
||||
)
|
||||
print(result["replies"][0].text)
|
||||
|
||||
## ('The main takeaway of Figure 6 is that Flan-PaLM demonstrates improved '
|
||||
## 'performance in zero-shot reasoning tasks when utilizing chain-of-thought '
|
||||
## '(CoT) reasoning, as indicated by higher accuracy across different model '
|
||||
## 'sizes compared to PaLM without finetuning. This highlights the importance of '
|
||||
## 'instruction finetuning combined with CoT for enhancing reasoning '
|
||||
## 'capabilities in models.')
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Introduction to Multimodality](https://haystack.deepset.ai/cookbook/multimodal_intro)
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
---
|
||||
title: "PPTXToDocument"
|
||||
id: pptxtodocument
|
||||
slug: "/pptxtodocument"
|
||||
description: "Convert PPTX files to documents."
|
||||
---
|
||||
|
||||
# PPTXToDocument
|
||||
|
||||
Convert PPTX files to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: PPTX file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/pptx.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `PPTXToDocument` component converts PPTX files into documents. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
## Usage
|
||||
|
||||
First, install the`python-pptx` package to start using this converter:
|
||||
|
||||
```shell
|
||||
pip install python-pptx
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters import PPTXToDocument
|
||||
|
||||
converter = PPTXToDocument()
|
||||
results = converter.run(
|
||||
sources=["sample.pptx"],
|
||||
meta={"date_added": datetime.now().isoformat()},
|
||||
)
|
||||
documents = results["documents"]
|
||||
|
||||
print(documents[0].content)
|
||||
|
||||
## 'This is the text from the PPTX file.'
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import PPTXToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", PPTXToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
---
|
||||
title: "PyPDFToDocument"
|
||||
id: pypdftodocument
|
||||
slug: "/pypdftodocument"
|
||||
description: "A component that converts PDF files to Documents."
|
||||
---
|
||||
|
||||
# PyPDFToDocument
|
||||
|
||||
A component that converts PDF files to Documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: PDF file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/pypdf.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `PyPDFToDocument` component converts PDF files into documents. You can use it in an indexing pipeline to index the contents of a PDF file into a Document Store. It takes a list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
## Usage
|
||||
|
||||
You need to install `pypdf` package to use the `PyPDFToDocument` converter:
|
||||
|
||||
```shell
|
||||
pip install pypdf
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from haystack.components.converters import PyPDFToDocument
|
||||
|
||||
converter = PyPDFToDocument()
|
||||
|
||||
docs = converter.run(sources=[Path("my_file.pdf")])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import PyPDFToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", PyPDFToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)
|
||||
|
||||
📓 Tutorial: [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline)
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
---
|
||||
title: "TextFileToDocument"
|
||||
id: textfiletodocument
|
||||
slug: "/textfiletodocument"
|
||||
description: "Converts text files to documents."
|
||||
---
|
||||
|
||||
# TextFileToDocument
|
||||
|
||||
Converts text files to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: A list of paths to text files you want to convert |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/txt.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `TextFileToDocument` component converts text files into documents. You can use it in an indexing pipeline to index the contents of text files into a Document Store. It takes a list of file paths or [ByteStream](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
When you initialize the component, you can optionally set the default encoding of the text files through the `encoding` parameter. If you don't provide any value, the component uses `"utf-8"` by default. Note that if the encoding is specified in the metadata of an input ByteStream, it will override this parameter's setting.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from haystack.components.converters import TextFileToDocument
|
||||
|
||||
converter = TextFileToDocument()
|
||||
|
||||
docs = converter.run(sources=[Path("my_file.txt")])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import TextFileToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", TextFileToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
:notebook: Tutorial: [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline)
|
||||
+79
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: "TikaDocumentConverter"
|
||||
id: tikadocumentconverter
|
||||
slug: "/tikadocumentconverter"
|
||||
description: "An integration for converting files of different types (PDF, DOCX, HTML, and more) to documents."
|
||||
---
|
||||
|
||||
# TikaDocumentConverter
|
||||
|
||||
An integration for converting files of different types (PDF, DOCX, HTML, and more) to documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) , or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: File paths |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/tika.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `TikaDocumentConverter` component converts files of different types (pdf, docx, html, and others) into documents. You can use it in an indexing pipeline to index the contents of files into a Document Store. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
This integration uses [Apache Tika](https://tika.apache.org/) to parse the files and requires a running Tika server.
|
||||
|
||||
The easiest way to run Tika is by using Docker: `docker run -d -p 127.0.0.1:9998:9998 apache/tika:latest`.
|
||||
For more options on running Tika on Docker, see the [Tika documentation](https://github.com/apache/tika-docker/blob/main/README.md#usage).
|
||||
|
||||
When you initialize the `TikaDocumentConverter` component, you can specify a custom URL of the Tika server you are using through the parameter `tika_url`. The default URL is `"http://localhost:9998/tika"`.
|
||||
|
||||
## Usage
|
||||
|
||||
You need to install `tika` package to use the `TikaDocumentConverter` component:
|
||||
|
||||
```shell
|
||||
pip install tika
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters import TikaDocumentConverter
|
||||
from pathlib import Path
|
||||
|
||||
converter = TikaDocumentConverter()
|
||||
|
||||
converter.run(sources=[Path("my_file.pdf")])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import TikaDocumentConverter
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", TikaDocumentConverter())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_paths}})
|
||||
```
|
||||
+115
@@ -0,0 +1,115 @@
|
||||
---
|
||||
title: "UnstructuredFileConverter"
|
||||
id: unstructuredfileconverter
|
||||
slug: "/unstructuredfileconverter"
|
||||
description: "Use this component to convert text files and directories to a document."
|
||||
---
|
||||
|
||||
# UnstructuredFileConverter
|
||||
|
||||
Use this component to convert text files and directories to a document.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `paths`: A union of lists of paths |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Unstructured](/reference/integrations-unstructured) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/unstructured |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`UnstructuredFileConverter` converts files and directories into documents using the Unstructured API.
|
||||
|
||||
[Unstructured](https://docs.unstructured.io/) provides a series of tools to do ETL for LLMs. The `UnstructuredFileConverter` calls the Unstructured API that extracts text and other information from a vast range of file [formats](https://docs.unstructured.io/api-reference/api-services/overview#supported-file-types).
|
||||
|
||||
This Converter supports different modes for creating documents from the elements returned by Unstructured:
|
||||
|
||||
- `"one-doc-per-file"`: One Haystack document per file. All elements are concatenated into one text field.
|
||||
- `"one-doc-per-page"`: One Haystack document per page. All elements on a page are concatenated into one text field.
|
||||
- `"one-doc-per-element"`: One Haystack document per element. Each element is converted to a Haystack document.
|
||||
|
||||
## Usage
|
||||
|
||||
Install the Unstructured integration to use `UnstructuredFileConverter`component:
|
||||
|
||||
```shell
|
||||
pip install unstructured-fileconverter-haystack
|
||||
```
|
||||
|
||||
There are free and paid versions of Unstructured API: **Free Unstructured API** and **Unstructured Serverless API**.
|
||||
|
||||
1. **Free Unstructured API**:
|
||||
- API URL: `https://api.unstructured.io/general/v0/general`
|
||||
- This version is free, but comes with certain limitations.
|
||||
|
||||
2. **Unstructured Serverless API**:
|
||||
- You'll find your unique API URL in your Unstructured account after signing up for the paid version.
|
||||
- This is a full-tier paid version of Unstructured.
|
||||
|
||||
For more details about the two tiers refer to Unstructured [FAQ](https://docs.unstructured.io/faq/faq).
|
||||
|
||||
> ❗️ The API keys for the free and paid versions are different and cannot be used interchangeably.
|
||||
|
||||
Regardless of the chosen tier, we recommend to set the Unstructured API key as an environment variable `UNSTRUCTURED_API_KEY`:
|
||||
|
||||
```shell
|
||||
export UNSTRUCTURED_API_KEY=your_api_key
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
import os
|
||||
from haystack_integrations.components.converters.unstructured import (
|
||||
UnstructuredFileConverter,
|
||||
)
|
||||
|
||||
converter = UnstructuredFileConverter()
|
||||
documents = converter.run(paths=["a/file/path.pdf", "a/directory/path"])["documents"]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
import os
|
||||
from haystack import Pipeline
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.converters.unstructured import (
|
||||
UnstructuredFileConverter,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
indexing = Pipeline()
|
||||
indexing.add_component("converter", UnstructuredFileConverter())
|
||||
indexing.add_component("writer", DocumentWriter(document_store))
|
||||
indexing.connect("converter", "writer")
|
||||
|
||||
indexing.run({"converter": {"paths": ["a/file/path.pdf", "a/directory/path"]}})
|
||||
```
|
||||
|
||||
### With Docker
|
||||
|
||||
To use `UnstructuredFileConverter` through Docker, first, set up an Unstructured Docker container:
|
||||
|
||||
```
|
||||
docker run -p 8000:8000 -d --rm --name unstructured-api quay.io/unstructured-io/unstructured-api:latest --port 8000 --host 0.0.0.0
|
||||
```
|
||||
|
||||
When initializing the component, specify the localhost URL:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.converters.unstructured import (
|
||||
UnstructuredFileConverter,
|
||||
)
|
||||
|
||||
converter = UnstructuredFileConverter(
|
||||
api_url="http://localhost:8000/general/v0/general",
|
||||
)
|
||||
```
|
||||
+79
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: "XLSXToDocument"
|
||||
id: xlsxtodocument
|
||||
slug: "/xlsxtodocument"
|
||||
description: "Converts Excel files into documents."
|
||||
---
|
||||
|
||||
# XLSXToDocument
|
||||
|
||||
Converts Excel files into documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before [PreProcessors](../preprocessors.mdx) or right at the beginning of an indexing pipeline |
|
||||
| **Mandatory run variables** | `sources`: File paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects |
|
||||
| **Output variables** | `documents`: A list of documents |
|
||||
| **API reference** | [Converters](/reference/converters-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/converters/xlsx.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The `XLSXToDocument` component converts XLSX files into Haystack Documents with a CSV (default) or Markdown format. It takes a list of file paths or [`ByteStream`](../../concepts/data-classes.mdx#bytestream) objects as input and outputs the converted result as a list of documents. Optionally, you can attach metadata to the documents through the `meta` input parameter.
|
||||
|
||||
To see the additional parameters that you can specify with the component initialization, check out the [API Reference](/reference/converters-api#xlsxtodocument).
|
||||
|
||||
## Usage
|
||||
|
||||
First, install the openpyxl and tabulate packages to start using this converter:
|
||||
|
||||
```shell
|
||||
pip install pandas openpyxl
|
||||
pip install tabulate
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.converters import XLSXToDocument
|
||||
|
||||
converter = XLSXToDocument()
|
||||
results = converter.run(
|
||||
sources=["sample.xlsx"],
|
||||
meta={"date_added": datetime.now().isoformat()},
|
||||
)
|
||||
documents = results["documents"]
|
||||
print(documents[0].content)
|
||||
## ",A,B\n1,col_a,col_b\n2,1.5,test\n"
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.converters import XLSXToDocument
|
||||
from haystack.components.preprocessors import DocumentCleaner
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("converter", XLSXToDocument())
|
||||
pipeline.add_component("cleaner", DocumentCleaner())
|
||||
pipeline.add_component(
|
||||
"splitter",
|
||||
DocumentSplitter(split_by="sentence", split_length=5),
|
||||
)
|
||||
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
pipeline.connect("converter", "cleaner")
|
||||
pipeline.connect("cleaner", "splitter")
|
||||
pipeline.connect("splitter", "writer")
|
||||
|
||||
pipeline.run({"converter": {"sources": file_names}})
|
||||
```
|
||||
+278
@@ -0,0 +1,278 @@
|
||||
---
|
||||
title: "S3Downloader"
|
||||
id: s3downloader
|
||||
slug: "/s3downloader"
|
||||
description: "`S3Downloader` downloads files from AWS S3 buckets to the local filesystem and enriches documents with the local file path."
|
||||
---
|
||||
|
||||
# S3Downloader
|
||||
|
||||
`S3Downloader` downloads files from AWS S3 buckets to the local filesystem and enriches documents with the local file path.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before File Converters or Routers that need local file paths |
|
||||
| **Mandatory init variables** | `file_root_path`: Path where files will be downloaded. Can be set with `FILE_ROOT_PATH` env var. <br /> <br />`aws_access_key_id`: AWS access key ID. Can be set with AWS_ACCESS_KEY_ID env var. <br /> <br />`aws_secret_access_key`: AWS secret access key. Can be set with AWS_SECRET_ACCESS_KEY env var. <br /> <br />`aws_region_name`: AWS region name. Can be set with AWS_DEFAULT_REGION env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents containing name of the file to download in metadata. |
|
||||
| **Output variables** | `documents`: A list of documents enriched with the local file path in `meta['file_path']` |
|
||||
| **API reference** | [S3Downloader](/reference/integrations-amazon-bedrock) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`S3Downloader` downloads files from AWS S3 buckets to your local filesystem and enriches Document objects with the local file path. This component is useful for pipelines that need to process files stored in S3, such as PDFs, images, or text files.
|
||||
|
||||
The component supports AWS authentication through environment variables by default. You can set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION` environment variables. Alternatively, you can pass credentials directly at initialization using the [Secret API](../../concepts/secret-management.mdx):
|
||||
|
||||
```python
|
||||
from haystack.utils import Secret
|
||||
from haystack_integrations.components.downloaders.s3 import S3Downloader
|
||||
|
||||
downloader = S3Downloader(
|
||||
aws_access_key_id=Secret.from_token("<your-access-key-id>"),
|
||||
aws_secret_access_key=Secret.from_token("<your-secret-access-key>"),
|
||||
aws_region_name=Secret.from_token("<your-region>"),
|
||||
file_root_path="/path/to/download/directory",
|
||||
)
|
||||
```
|
||||
|
||||
The component downloads multiple files in parallel using the `max_workers` parameter (default is 32 workers) to speed up processing of large document sets. Downloaded files are cached locally, and when the cache exceeds `max_cache_size` (default is 100 files), least recently accessed files are automatically removed. Already downloaded files are touched to update their access time without re-downloading.
|
||||
|
||||
:::info[Required Configuration]
|
||||
|
||||
The component requires two critical configurations:
|
||||
|
||||
1. `file_root_path` parameter or `FILE_ROOT_PATH` environment variable: Specifies where files will be downloaded. This directory will be created if it doesn't exist when `warm_up()` is called.
|
||||
2. `S3_DOWNLOADER_BUCKET` environment variable: Specifies which S3 bucket to download files from.
|
||||
:::
|
||||
|
||||
The optional environment variable `S3_DOWNLOADER_PREFIX` can be set to add a prefix of the files to all generated S3 keys.
|
||||
|
||||
### File Extension Filtering
|
||||
|
||||
You can use the `file_extensions` parameter to download only specific file types, reducing unnecessary downloads and processing time. For example, `file_extensions=[".pdf", ".txt"]` downloads only PDF and TXT files while skipping others.
|
||||
|
||||
### Custom S3 Key Generation
|
||||
|
||||
By default, the component uses the `file_name` from Document metadata as the S3 key. If your S3 file structure doesn't match the file names in metadata, you can provide an optional `s3_key_generation_function` to customize how S3 keys are generated from Document metadata.
|
||||
|
||||
## Usage
|
||||
|
||||
You need to install the `amazon-bedrock-haystack` package to use `S3Downloader`:
|
||||
|
||||
```shell
|
||||
pip install amazon-bedrock-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
Before running the examples, ensure you have set the required environment variables:
|
||||
|
||||
```shell
|
||||
export AWS_ACCESS_KEY_ID="<your-access-key-id>"
|
||||
export AWS_SECRET_ACCESS_KEY="<your-secret-access-key>"
|
||||
export AWS_DEFAULT_REGION="<your-region>"
|
||||
export S3_DOWNLOADER_BUCKET="<your-bucket-name>"
|
||||
```
|
||||
|
||||
Here's how to use `S3Downloader` to download files from S3:
|
||||
|
||||
```python
|
||||
from haystack.dataclasses import Document
|
||||
from haystack_integrations.components.downloaders.s3 import S3Downloader
|
||||
|
||||
## Create documents with file names in metadata
|
||||
documents = [
|
||||
Document(meta={"file_name": "report.pdf"}),
|
||||
Document(meta={"file_name": "data.txt"}),
|
||||
]
|
||||
|
||||
## Initialize the downloader
|
||||
downloader = S3Downloader(file_root_path="/tmp/s3_downloads")
|
||||
|
||||
## Warm up the component
|
||||
downloader.warm_up()
|
||||
|
||||
## Download the files
|
||||
result = downloader.run(documents=documents)
|
||||
|
||||
## Access the downloaded files
|
||||
for doc in result["documents"]:
|
||||
print(f"File downloaded to: {doc.meta['file_path']}")
|
||||
```
|
||||
|
||||
With file extension filtering:
|
||||
|
||||
```python
|
||||
from haystack.dataclasses import Document
|
||||
from haystack_integrations.components.downloaders.s3 import S3Downloader
|
||||
|
||||
documents = [
|
||||
Document(meta={"file_name": "report.pdf"}),
|
||||
Document(meta={"file_name": "image.png"}),
|
||||
Document(meta={"file_name": "data.txt"}),
|
||||
]
|
||||
|
||||
## Only download PDF files
|
||||
downloader = S3Downloader(file_root_path="/tmp/s3_downloads", file_extensions=[".pdf"])
|
||||
|
||||
downloader.warm_up()
|
||||
|
||||
result = downloader.run(documents=documents)
|
||||
|
||||
## Only report.pdf is downloaded
|
||||
print(f"Downloaded {len(result['documents'])} file(s)")
|
||||
## Output: Downloaded 1 file(s)
|
||||
```
|
||||
|
||||
With custom S3 key generation:
|
||||
|
||||
```python
|
||||
from haystack.dataclasses import Document
|
||||
from haystack_integrations.components.downloaders.s3 import S3Downloader
|
||||
|
||||
|
||||
def custom_s3_key_function(document: Document) -> str:
|
||||
"""Generate S3 key from custom metadata."""
|
||||
folder = document.meta.get("folder", "default")
|
||||
file_name = document.meta.get("file_name")
|
||||
if not file_name:
|
||||
raise ValueError("Document must have 'file_name' in metadata")
|
||||
return f"{folder}/{file_name}"
|
||||
|
||||
|
||||
documents = [
|
||||
Document(meta={"file_name": "report.pdf", "folder": "reports/2025"}),
|
||||
]
|
||||
|
||||
downloader = S3Downloader(
|
||||
file_root_path="/tmp/s3_downloads",
|
||||
s3_key_generation_function=custom_s3_key_function,
|
||||
)
|
||||
|
||||
downloader.warm_up()
|
||||
result = downloader.run(documents=documents)
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Here's an example of using `S3Downloader` in a document processing pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters import PDFMinerToDocument
|
||||
from haystack.components.routers import DocumentTypeRouter
|
||||
from haystack.dataclasses import Document
|
||||
|
||||
from haystack_integrations.components.downloaders.s3 import S3Downloader
|
||||
|
||||
## Create a pipeline
|
||||
pipe = Pipeline()
|
||||
|
||||
## Add S3Downloader to download files from S3
|
||||
pipe.add_component(
|
||||
"downloader",
|
||||
S3Downloader(file_root_path="/tmp/s3_downloads", file_extensions=[".pdf", ".txt"]),
|
||||
)
|
||||
|
||||
## Route documents by file type
|
||||
pipe.add_component(
|
||||
"router",
|
||||
DocumentTypeRouter(
|
||||
file_path_meta_field="file_path",
|
||||
mime_types=["application/pdf", "text/plain"],
|
||||
),
|
||||
)
|
||||
|
||||
## Convert PDFs to documents
|
||||
pipe.add_component("pdf_converter", PDFMinerToDocument())
|
||||
|
||||
## Connect components
|
||||
pipe.connect("downloader.documents", "router.documents")
|
||||
pipe.connect("router.application/pdf", "pdf_converter.documents")
|
||||
|
||||
## Create documents with S3 file names
|
||||
documents = [
|
||||
Document(meta={"file_name": "report.pdf"}),
|
||||
Document(meta={"file_name": "summary.txt"}),
|
||||
]
|
||||
|
||||
## Run the pipeline
|
||||
result = pipe.run({"downloader": {"documents": documents}})
|
||||
```
|
||||
|
||||
For a more complex example with image processing and LLM:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.converters.image import DocumentToImageContent
|
||||
from haystack.components.routers import DocumentTypeRouter
|
||||
from haystack.dataclasses import Document
|
||||
|
||||
from haystack_integrations.components.downloaders.s3 import S3Downloader
|
||||
from haystack_integrations.components.generators.amazon_bedrock import (
|
||||
AmazonBedrockChatGenerator,
|
||||
)
|
||||
|
||||
## Create documents with file names
|
||||
documents = [
|
||||
Document(meta={"file_name": "chart.png"}),
|
||||
Document(meta={"file_name": "report.pdf"}),
|
||||
]
|
||||
|
||||
## Create pipeline
|
||||
pipe = Pipeline()
|
||||
|
||||
## Download files from S3
|
||||
pipe.add_component("downloader", S3Downloader(file_root_path="/tmp/s3_downloads"))
|
||||
|
||||
## Route by document type
|
||||
pipe.add_component(
|
||||
"router",
|
||||
DocumentTypeRouter(
|
||||
file_path_meta_field="file_path",
|
||||
mime_types=["image/png", "application/pdf"],
|
||||
),
|
||||
)
|
||||
|
||||
## Convert images for LLM
|
||||
pipe.add_component("image_converter", DocumentToImageContent(detail="auto"))
|
||||
|
||||
## Create chat prompt with template
|
||||
template = """{% message role="user" %}
|
||||
Answer the question based on the provided images.
|
||||
|
||||
Question: {{ question }}
|
||||
|
||||
{% for image in image_contents %}
|
||||
{{ image | templatize_part }}
|
||||
{% endfor %}
|
||||
{% endmessage %}"""
|
||||
|
||||
pipe.add_component("prompt_builder", ChatPromptBuilder(template=template))
|
||||
|
||||
## Generate response
|
||||
pipe.add_component(
|
||||
"llm",
|
||||
AmazonBedrockChatGenerator(model="anthropic.claude-3-haiku-20240307-v1:0"),
|
||||
)
|
||||
|
||||
## Connect components
|
||||
pipe.connect("downloader.documents", "router.documents")
|
||||
pipe.connect("router.image/png", "image_converter.documents")
|
||||
pipe.connect("image_converter.image_contents", "prompt_builder.image_contents")
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
## Run pipeline
|
||||
result = pipe.run(
|
||||
{
|
||||
"downloader": {"documents": documents},
|
||||
"prompt_builder": {"question": "What information is shown in the chart?"},
|
||||
},
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
title: "Embedders"
|
||||
id: embedders
|
||||
slug: "/embedders"
|
||||
description: "Embedders in Haystack transform texts or documents into vector representations using pre-trained models. You can then use the embedding for tasks like question answering, information retrieval, and more."
|
||||
---
|
||||
|
||||
# Embedders
|
||||
|
||||
Embedders in Haystack transform texts or documents into vector representations using pre-trained models. You can then use the embedding for tasks like question answering, information retrieval, and more.
|
||||
|
||||
:::info
|
||||
For general guidance on how to choose an Embedder that would be right for you, read our [Choosing the Right Embedder](embedders/choosing-the-right-embedder.mdx) page.
|
||||
:::
|
||||
|
||||
These are the Embedders available in Haystack:
|
||||
|
||||
| Embedder | Description |
|
||||
| --- | --- |
|
||||
| [AmazonBedrockTextEmbedder](embedders/amazonbedrocktextembedder.mdx) | Computes embeddings for text (such as a query) using models through Amazon Bedrock API. |
|
||||
| [AmazonBedrockDocumentEmbedder](embedders/amazonbedrockdocumentembedder.mdx) | Computes embeddings for documents using models through Amazon Bedrock API. |
|
||||
| [AmazonBedrockDocumentImageEmbedder](embedders/amazonbedrockdocumentimageembedder.mdx) | Computes image embeddings for a document. |
|
||||
| [AzureOpenAITextEmbedder](embedders/azureopenaitextembedder.mdx) | Computes embeddings for text (such as a query) using OpenAI models deployed through Azure. |
|
||||
| [AzureOpenAIDocumentEmbedder](embedders/azureopenaidocumentembedder.mdx) | Computes embeddings for documents using OpenAI models deployed through Azure. |
|
||||
| [CohereTextEmbedder](embedders/coheretextembedder.mdx) | Embeds a simple string (such as a query) with a Cohere model. Requires an API key from Cohere |
|
||||
| [CohereDocumentEmbedder](embedders/coheredocumentembedder.mdx) | Embeds a list of documents with a Cohere model. Requires an API key from Cohere. |
|
||||
| [CohereDocumentImageEmbedder](embedders/coheredocumentimageembedder.mdx) | Computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. |
|
||||
| [FastembedTextEmbedder](embedders/fastembedtextembedder.mdx) | Computes the embeddings of a string using embedding models supported by Fastembed. |
|
||||
| [FastembedDocumentEmbedder](embedders/fastembeddocumentembedder.mdx) | Computes the embeddings of a list of documents using the models supported by Fastembed. |
|
||||
| [FastembedSparseTextEmbedder](embedders/fastembedsparsetextembedder.mdx) | Embeds a simple string (such as a query) into a sparse vector using the models supported by Fastembed. |
|
||||
| [FastembedSparseDocumentEmbedder](embedders/fastembedsparsedocumentembedder.mdx) | Enriches a list of documents with their sparse embeddings using the models supported by Fastembed. |
|
||||
| [GoogleGenAITextEmbedder](embedders/googlegenaitextembedder.mdx) | Embeds a simple string (such as a query) with a Google AI model. Requires an API key from Google. |
|
||||
| [GoogleGenAIDocumentEmbedder](embedders/googlegenaidocumentembedder.mdx) | Embeds a list of documents with a Google AI model. Requires an API key from Google. |
|
||||
| [HuggingFaceAPIDocumentEmbedder](embedders/huggingfaceapidocumentembedder.mdx) | Computes document embeddings using various Hugging Face APIs. |
|
||||
| [HuggingFaceAPITextEmbedder](embedders/huggingfaceapitextembedder.mdx) | Embeds strings using various Hugging Face APIs. |
|
||||
| [JinaTextEmbedder](embedders/jinatextembedder.mdx) | Embeds a simple string (such as a query) with a Jina AI Embeddings model. Requires an API key from Jina AI. |
|
||||
| [JinaDocumentEmbedder](embedders/jinadocumentembedder.mdx) | Embeds a list of documents with a Jina AI Embeddings model. Requires an API key from Jina AI. |
|
||||
| [JinaDocumentImageEmbedder](embedders/jinadocumentimageembedder.mdx) | Computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. |
|
||||
| [MistralTextEmbedder](embedders/mistraltextembedder.mdx) | Transforms a string into a vector using the Mistral API and models. |
|
||||
| [MistralDocumentEmbedder](embedders/mistraldocumentembedder.mdx) | Computes the embeddings of a list of documents using the Mistral API and models. |
|
||||
| [NvidiaTextEmbedder](embedders/nvidiatextembedder.mdx) | Embeds a simple string (such as a query) into a vector. |
|
||||
| [NvidiaDocumentEmbedder](embedders/nvidiadocumentembedder.mdx) | Enriches the metadata of documents with an embedding of their content. |
|
||||
| [OllamaTextEmbedder](embedders/ollamatextembedder.mdx) | Computes the embeddings of a string using embedding models compatible with the Ollama Library. |
|
||||
| [OllamaDocumentEmbedder](embedders/ollamadocumentembedder.mdx) | Computes the embeddings of a list of documents using embedding models compatible with the Ollama Library. |
|
||||
| [OpenAIDocumentEmbedder](embedders/openaidocumentembedder.mdx) | Embeds a list of documents with an OpenAI embedding model. Requires an API key from an active OpenAI account. |
|
||||
| [OpenAITextEmbedder](embedders/openaitextembedder.mdx) | Embeds a simple string (such as a query) with an OpenAI embedding model. Requires an API key from an active OpenAI account. |
|
||||
| [OptimumTextEmbedder](embedders/optimumtextembedder.mdx) | Embeds text using models loaded with the Hugging Face Optimum library. |
|
||||
| [OptimumDocumentEmbedder](embedders/optimumdocumentembedder.mdx) | Computes documents’ embeddings using models loaded with the Hugging Face Optimum library. |
|
||||
| [SentenceTransformersTextEmbedder](embedders/sentencetransformerstextembedder.mdx) | Embeds a simple string (such as a query) using a Sentence Transformer model. |
|
||||
| [SentenceTransformersDocumentEmbedder](embedders/sentencetransformersdocumentembedder.mdx) | Embeds a list of documents with a Sentence Transformer model. |
|
||||
| [SentenceTransformersDocumentImageEmbedder](embedders/sentencetransformersdocumentimageembedder.mdx) | Computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. |
|
||||
| [SentenceTransformersSparseTextEmbedder](embedders/sentencetransformerssparsetextembedder.mdx) | Embeds a simple string (such as a query) into a sparse vector using Sentence Transformers models. |
|
||||
| [SentenceTransformersSparseDocumentEmbedder](embedders/sentencetransformerssparsedocumentembedder.mdx) | Enriches a list of documents with their sparse embeddings using Sentence Transformers models. |
|
||||
| [STACKITTextEmbedder](embedders/stackittextembedder.mdx) | Enables text embedding using the STACKIT API. |
|
||||
| [STACKITDocumentEmbedder](embedders/stackitdocumentembedder.mdx) | Enables document embedding using the STACKIT API. |
|
||||
| [VertexAITextEmbedder](embedders/vertexaitextembedder.mdx) | Computes embeddings for text (such as a query) using models through VertexAI Embeddings API. **_This integration will be deprecated soon. We recommend using [GoogleGenAITextEmbedder](embedders/googlegenaitextembedder.mdx) integration instead._** |
|
||||
| [VertexAIDocumentEmbedder](embedders/vertexaidocumentembedder.mdx) | Computes embeddings for documents using models through VertexAI Embeddings API. **_This integration will be deprecated soon. We recommend using [GoogleGenAIDocumentEmbedder](embedders/googlegenaidocumentembedder.mdx) integration instead._** |
|
||||
| [WatsonxTextEmbedder](embedders/watsonxtextembedder.mdx) | Computes embeddings for text (such as a query) using IBM Watsonx models. |
|
||||
| [WatsonxDocumentEmbedder](embedders/watsonxdocumentembedder.mdx) | Computes embeddings for documents using IBM Watsonx models. |
|
||||
+171
@@ -0,0 +1,171 @@
|
||||
---
|
||||
title: "AmazonBedrockDocumentEmbedder"
|
||||
id: amazonbedrockdocumentembedder
|
||||
slug: "/amazonbedrockdocumentembedder"
|
||||
description: "This component computes embeddings for documents using models through Amazon Bedrock API."
|
||||
---
|
||||
|
||||
# AmazonBedrockDocumentEmbedder
|
||||
|
||||
This component computes embeddings for documents using models through Amazon Bedrock API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `model`: The embedding model to use <br /> <br />`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var. <br /> <br />`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var. <br /> <br />`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
|
||||
| **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes language models from leading AI startups and Amazon available for your use through a unified API.
|
||||
|
||||
Supported models are `amazon.titan-embed-text-v1`, `cohere.embed-english-v3`, `cohere.embed-multilingual-v3`, and `amazon.titan-embed-text-v2:0`.
|
||||
|
||||
:::info[Batch Inference]
|
||||
|
||||
Note that only Cohere models support batch inference – computing embeddings for more documents with the same request.
|
||||
:::
|
||||
|
||||
This component should be used to embed a list of documents. To embed a string, you should use the [`AmazonBedrockTextEmbedder`](amazonbedrocktextembedder.mdx).
|
||||
|
||||
### Authentication
|
||||
|
||||
`AmazonBedrockDocumentEmbedder` uses AWS for authentication. You can either provide credentials as parameters directly to the component or use the AWS CLI and authenticate through your IAM. For more information on how to set up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
|
||||
To initialize `AmazonBedrockDocumentEmbedder` and authenticate by providing credentials, provide the `model_name`, as well as `aws_access_key_id`, `aws_secret_access_key` and `aws_region_name`. Other parameters are optional. You can check them out in our [API reference](/reference/integrations-amazon-bedrock#amazonbedrockdocumentembedder).
|
||||
|
||||
### Model-specific parameters
|
||||
|
||||
Even if Haystack provides a unified interface, each model offered by Bedrock can accept specific parameters. You can pass these parameters at initialization.
|
||||
|
||||
For example, Cohere models support `input_type` and `truncate`, as seen in [Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockDocumentEmbedder,
|
||||
)
|
||||
|
||||
embedder = AmazonBedrockDocumentEmbedder(
|
||||
model="cohere.embed-english-v3",
|
||||
input_type="search_document",
|
||||
truncate="LEFT",
|
||||
)
|
||||
```
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this easily by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = AmazonBedrockDocumentEmbedder(
|
||||
model="cohere.embed-english-v3",
|
||||
meta_fields_to_embed=["title"],
|
||||
)
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Installation
|
||||
|
||||
You need to install `amazon-bedrock-haystack` package to use the `AmazonBedrockTextEmbedder`:
|
||||
|
||||
```shell
|
||||
pip install amazon-bedrock-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
Basic usage:
|
||||
|
||||
```python
|
||||
import os
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import AmazonBedrockDocumentEmbedder
|
||||
from haystack.dataclasses import DOcument
|
||||
|
||||
os.environ["AWS_ACCESS_KEY_ID"] = "..."
|
||||
os.environ["AWS_SECRET_ACCESS_KEY"] = "..."
|
||||
os.environ["AWS_DEFAULT_REGION"] = "us-east-1" # just an example
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
embedder = AmazonBedrockDocumentEmbedder(model="cohere.embed-english-v3",
|
||||
input_type="search_document"
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result['documents'][0].embedding)
|
||||
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
In a RAG pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockDocumentEmbedder,
|
||||
AmazonBedrockTextEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component(
|
||||
"embedder",
|
||||
AmazonBedrockDocumentEmbedder(model="cohere.embed-english-v3"),
|
||||
)
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"text_embedder",
|
||||
AmazonBedrockTextEmbedder(model="cohere.embed-english-v3"),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)
|
||||
+164
@@ -0,0 +1,164 @@
|
||||
---
|
||||
title: "AmazonBedrockDocumentImageEmbedder"
|
||||
id: amazonbedrockdocumentimageembedder
|
||||
slug: "/amazonbedrockdocumentimageembedder"
|
||||
description: "`AmazonBedrockDocumentImageEmbedder` computes image embeddings for documents using models exposed through the Amazon Bedrock API. It stores the obtained vectors in the embedding field of each document."
|
||||
---
|
||||
|
||||
# AmazonBedrockDocumentImageEmbedder
|
||||
|
||||
`AmazonBedrockDocumentImageEmbedder` computes image embeddings for documents using models exposed through the Amazon Bedrock API. It stores the obtained vectors in the embedding field of each document.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `model`: The multimodal embedding model to use. <br /> <br />`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var. <br /> <br />`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var. <br /> <br />`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
|
||||
| **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
Amazon Bedrock is a fully managed service that provides access to foundation models through a unified API.
|
||||
|
||||
`AmazonBedrockDocumentImageEmbedder` expects a list of documents containing an image or a PDF file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
|
||||
|
||||
The embedder efficiently loads the images, computes the embeddings using selected Bedrock model, and stores each of them in the `embedding` field of the document.
|
||||
|
||||
Supported models are `amazon.titan-embed-image-v1`, `cohere.embed-english-v3` , and `cohere.embed-multilingual-v3`.
|
||||
|
||||
`AmazonBedrockDocumentImageEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with `AmazonBedrockTextEmbedder` to embed the query, before using an Embedding Retriever.
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install amazon-bedrock-haystack
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
`AmazonBedrockDocumentImageEmbedder` uses AWS for authentication. You can either provide credentials as parameters directly to the component or use the AWS CLI and authenticate through your IAM. For more information on how to set up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
|
||||
|
||||
To initialize `AmazonBedrockDocumentImageEmbedder` and authenticate by providing credentials, provide the `model` name, as well as `aws_access_key_id`, `aws_secret_access_key`, and `aws_region_name`. Other parameters are optional, you can check them out in our [API reference](/reference/integrations-amazon-bedrock#amazonbedrocktextembedder).
|
||||
|
||||
### Model-specific parameters
|
||||
|
||||
Even if Haystack provides a unified interface, each model offered by Bedrock can accept specific parameters. You can pass these parameters at initialization.
|
||||
|
||||
- **Amazon Titan**: Use `embeddingConfig` to control embedding behavior.
|
||||
- **Cohere v3**: Use `embedding_types` to select a single embedding type for images.
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockDocumentImageEmbedder,
|
||||
)
|
||||
|
||||
embedder = AmazonBedrockDocumentImageEmbedder(
|
||||
model="cohere.embed-english-v3",
|
||||
embedding_types=["float"], # single value only
|
||||
)
|
||||
```
|
||||
|
||||
Note that only _one_ value in `embedding_types` is supported by this component. Passing multiple values raises an error.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
import os
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockDocumentImageEmbedder,
|
||||
)
|
||||
|
||||
os.environ["AWS_ACCESS_KEY_ID"] = "..."
|
||||
os.environ["AWS_SECRET_ACCESS_KEY"] = "..."
|
||||
os.environ["AWS_DEFAULT_REGION"] = "us-east-1" # example
|
||||
|
||||
## Point Documents to image/PDF files via metadata (default key: "file_path")
|
||||
documents = [
|
||||
Document(content="A photo of a cat", meta={"file_path": "cat.jpg"}),
|
||||
Document(
|
||||
content="Invoice page",
|
||||
meta={
|
||||
"file_path": "invoice.pdf",
|
||||
"mime_type": "application/pdf",
|
||||
"page_number": 1,
|
||||
},
|
||||
),
|
||||
]
|
||||
|
||||
embedder = AmazonBedrockDocumentImageEmbedder(
|
||||
model="amazon.titan-embed-image-v1",
|
||||
image_size=(1024, 1024), # optional downscaling
|
||||
)
|
||||
|
||||
result = embedder.run(documents=documents)
|
||||
embedded_docs = result["documents"]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
In this example, we can see an indexing pipeline with 3 components:
|
||||
|
||||
- `ImageFileToDocument` Converter that creates empty documents with a reference to an image in the `meta.file_path` field;
|
||||
- `AmazonBedrockDocumentImageEmbedder` that loads the images, computes embeddings and stores them in documents;
|
||||
- `DocumentWriter` that write the documents in the `InMemoryDocumentStore`.
|
||||
|
||||
There is also a multimodal retrieval pipeline, composed of an `AmazonBedrockTextEmbedder` (using the same model as before) and an `InMemoryEmbeddingRetriever`.
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockDocumentImageEmbedder,
|
||||
AmazonBedrockTextEmbedder,
|
||||
)
|
||||
|
||||
## Document store using vector similarity for retrieval
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
## Sample corpus with file paths in metadata
|
||||
documents = [
|
||||
Document(content="A sketch of a horse", meta={"file_path": "horse.png"}),
|
||||
Document(content="A city map", meta={"file_path": "map.jpg"}),
|
||||
]
|
||||
|
||||
## Indexing pipeline: image embeddings -> write to store
|
||||
indexing = Pipeline()
|
||||
indexing.add_component(
|
||||
"image_embedder",
|
||||
AmazonBedrockDocumentImageEmbedder(model="cohere.embed-english-v3"),
|
||||
)
|
||||
indexing.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing.connect("image_embedder", "writer")
|
||||
indexing.run({"image_embedder": {"documents": documents}})
|
||||
|
||||
## Query pipeline: text -> embedding -> vector retriever
|
||||
query = Pipeline()
|
||||
query.add_component(
|
||||
"text_embedder",
|
||||
AmazonBedrockTextEmbedder(model="cohere.embed-english-v3"),
|
||||
)
|
||||
query.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
res = query.run({"text_embedder": {"text": "Which document shows a horse?"}})
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
:notebook: Tutorial: [Creating Vision+Text RAG Pipelines](https://haystack.deepset.ai/tutorials/46_multimodal_rag)
|
||||
+139
@@ -0,0 +1,139 @@
|
||||
---
|
||||
title: "AmazonBedrockTextEmbedder"
|
||||
id: amazonbedrocktextembedder
|
||||
slug: "/amazonbedrocktextembedder"
|
||||
description: "This component computes embeddings for text (such as a query) using models through Amazon Bedrock API."
|
||||
---
|
||||
|
||||
# AmazonBedrockTextEmbedder
|
||||
|
||||
This component computes embeddings for text (such as a query) using models through Amazon Bedrock API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `model`: The embedding model to use <br /> <br />`aws_access_key_id`: AWS access key ID. Can be set with `AWS_ACCESS_KEY_ID` env var. <br /> <br />`aws_secret_access_key`: AWS secret access key. Can be set with `AWS_SECRET_ACCESS_KEY` env var. <br /> <br />`aws_region_name`: AWS region name. Can be set with `AWS_DEFAULT_REGION` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers (vector) |
|
||||
| **API reference** | [Amazon Bedrock](/reference/integrations-amazon-bedrock) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/amazon_bedrock |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes language models from leading AI startups and Amazon available for your use through a unified API.
|
||||
|
||||
Supported models are `amazon.titan-embed-text-v1`, `cohere.embed-english-v3` and `cohere.embed-multilingual-v3`.
|
||||
|
||||
Use `AmazonBedrockTextEmbedder` to embed a simple string (such as a query) into a vector. Use the [`AmazonBedrockDocumentEmbedder`](amazonbedrockdocumentembedder.mdx) to enrich the documents with the computed embedding, also known as vector.
|
||||
|
||||
### Authentication
|
||||
|
||||
`AmazonBedrockTextEmbedder` uses AWS for authentication. You can either provide credentials as parameters directly to the component or use the AWS CLI and authenticate through your IAM. For more information on how to set up an IAM identity-based policy, see the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html).
|
||||
To initialize `AmazonBedrockTextEmbedder` and authenticate by providing credentials, provide the `model` name, as well as `aws_access_key_id`, `aws_secret_access_key`, and `aws_region_name`. Other parameters are optional, you can check them out in our [API reference](/reference/integrations-amazon-bedrock#amazonbedrocktextembedder).
|
||||
|
||||
### Model-specific parameters
|
||||
|
||||
Even if Haystack provides a unified interface, each model offered by Bedrock can accept specific parameters. You can pass these parameters at initialization.
|
||||
|
||||
For example, the Cohere models support `input_type` and `truncate`, as seen in [Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockTextEmbedder,
|
||||
)
|
||||
|
||||
embedder = AmazonBedrockTextEmbedder(
|
||||
model="cohere.embed-english-v3",
|
||||
input_type="search_query",
|
||||
truncate="LEFT",
|
||||
)
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Installation
|
||||
|
||||
You need to install `amazon-bedrock-haystack` package to use the `AmazonBedrockTextEmbedder`:
|
||||
|
||||
```shell
|
||||
pip install amazon-bedrock-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
Basic usage:
|
||||
|
||||
```python
|
||||
import os
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockTextEmbedder,
|
||||
)
|
||||
|
||||
os.environ["AWS_ACCESS_KEY_ID"] = "..."
|
||||
os.environ["AWS_SECRET_ACCESS_KEY"] = "..."
|
||||
os.environ["AWS_DEFAULT_REGION"] = "us-east-1" # just an example
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = AmazonBedrockTextEmbedder(
|
||||
model="cohere.embed-english-v3",
|
||||
input_type="search_query",
|
||||
)
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
## {'embedding': [-0.453125, 1.2236328, 2.0058594, 0.67871094...]}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
In a RAG pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.amazon_bedrock import (
|
||||
AmazonBedrockDocumentEmbedder,
|
||||
AmazonBedrockTextEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = AmazonBedrockDocumentEmbedder(model="cohere.embed-english-v3")
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"text_embedder",
|
||||
AmazonBedrockTextEmbedder(model="cohere.embed-english-v3"),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)
|
||||
+127
@@ -0,0 +1,127 @@
|
||||
---
|
||||
title: "AzureOpenAIDocumentEmbedder"
|
||||
id: azureopenaidocumentembedder
|
||||
slug: "/azureopenaidocumentembedder"
|
||||
description: "This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Azure cognitive services for text and document embedding with models deployed on Azure."
|
||||
---
|
||||
|
||||
# AzureOpenAIDocumentEmbedder
|
||||
|
||||
This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Azure cognitive services for text and document embedding with models deployed on Azure.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) |
|
||||
| **Mandatory init variables** | `api_key`: The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var. <br />`azure_endpoint`: The endpoint of the model deployed on Azure. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/azure_document_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
To see the list of compatible embedding models, head over to Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?source=recommendations). The default model for `AzureOpenAITextEmbedder` is `text-embedding-ada-002`.
|
||||
|
||||
This component should be used to embed a list of documents. To embed a string, you should use the [`AzureOpenAITextEmbedder`](azureopenaitextembedder.mdx).
|
||||
|
||||
To work with Azure components, you will need an Azure OpenAI API key, as well as an Azure OpenAI Endpoint. You can learn more about them in Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
|
||||
|
||||
The component uses `AZURE_OPENAI_API_KEY` or `AZURE_OPENAI_AD_TOKEN` environment variables by default. Otherwise, you can pass `api_key` or `azure_ad_token` at initialization:
|
||||
|
||||
```python
|
||||
client = AzureOpenAIDocumentEmbedder(
|
||||
azure_endpoint="<Your Azure endpoint e.g. `https://your-company.azure.openai.com/>",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
azure_deployment="<a model name>",
|
||||
)
|
||||
```
|
||||
|
||||
:::info
|
||||
We recommend using environment variables instead of initialization parameters.
|
||||
:::
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this easily by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.embedders import AzureOpenAIDocumentEmbedder
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = AzureOpenAIDocumentEmbedder(meta_fields_to_embed=["title"])
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.embedders import AzureOpenAIDocumentEmbedder
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = AzureOpenAIDocumentEmbedder()
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.embedders import (
|
||||
AzureOpenAITextEmbedder,
|
||||
AzureOpenAIDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("embedder", AzureOpenAIDocumentEmbedder())
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", AzureOpenAITextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "AzureOpenAITextEmbedder"
|
||||
id: azureopenaitextembedder
|
||||
slug: "/azureopenaitextembedder"
|
||||
description: "When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents."
|
||||
---
|
||||
|
||||
# AzureOpenAITextEmbedder
|
||||
|
||||
When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Azure OpenAI API key. Can be set with `AZURE_OPENAI_API_KEY` env var. <br />`azure_endpoint`: The endpoint of the model deployed on Azure. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/azure_text_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`AzureOpenAITextEmbedder` transforms a string into a vector that captures its semantics using an OpenAI embedding model. It uses Azure cognitive services for text and document embedding with models deployed on Azure.
|
||||
|
||||
To see the list of compatible embedding models, head over to Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?source=recommendations). The default model for `AzureOpenAITextEmbedder` is `text-embedding-ada-002`.
|
||||
|
||||
Use `AzureOpenAITextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`AzureOpenAIDocumentEmbedder`](azureopenaidocumentembedder.mdx), which enriches the documents with the computed embedding, also known as vector.
|
||||
|
||||
To work with Azure components, you will need an Azure OpenAI API key, as well as an Azure OpenAI Endpoint. You can learn more about them in Azure [documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).
|
||||
|
||||
The component uses `AZURE_OPENAI_API_KEY` or `AZURE_OPENAI_AD_TOKEN` environment variables by default. Otherwise, you can pass `api_key` or `azure_ad_token` at initialization:
|
||||
|
||||
```python
|
||||
client = AzureOpenAITextEmbedder(
|
||||
azure_endpoint="<Your Azure endpoint e.g. `https://your-company.azure.openai.com/>",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
azure_deployment="<a model name>",
|
||||
)
|
||||
```
|
||||
|
||||
:::info
|
||||
We recommend using environment variables instead of initialization parameters.
|
||||
:::
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import AzureOpenAITextEmbedder
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = AzureOpenAITextEmbedder()
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
|
||||
## {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
|
||||
## 'meta': {'model': 'text-embedding-ada-002-v2',
|
||||
## 'usage': {'prompt_tokens': 4, 'total_tokens': 4}}}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.embedders import (
|
||||
AzureOpenAITextEmbedder,
|
||||
AzureOpenAIDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = AzureOpenAIDocumentEmbedder()
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", AzureOpenAITextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
---
|
||||
title: "Choosing the Right Embedder"
|
||||
id: choosing-the-right-embedder
|
||||
slug: "/choosing-the-right-embedder"
|
||||
description: "This page provides information on choosing the right Embedder when working with Haystack. It explains the distinction between Text and Document Embedders and discusses API-based Embedders and Embedders with models running on-premise."
|
||||
---
|
||||
|
||||
# Choosing the Right Embedder
|
||||
|
||||
This page provides information on choosing the right Embedder when working with Haystack. It explains the distinction between Text and Document Embedders and discusses API-based Embedders and Embedders with models running on-premise.
|
||||
|
||||
Embedders in Haystack transform texts or documents into vector representations using pre-trained models. The embeddings produced by Haystack Embedders are fixed-length vectors. They capture contextual information and semantic relationships within the text.
|
||||
|
||||
Embeddings in isolation are only used for information retrieval purposes (to do semantic search/vector search). You can use the embeddings in your pipeline for tasks like question answering. The QA pipeline with embedding retrieval would then include the following steps:
|
||||
|
||||
1. Transform the query into a vector/embedding.
|
||||
2. Find similar documents based on the embedding similarity.
|
||||
3. Pass the query and the retrieved documents to a Language Model, which can be extractive or generative.
|
||||
|
||||
## Text and Document Embedders
|
||||
|
||||
There are two types of Embedders: text and document.
|
||||
|
||||
Text Embedders work with text strings and are most often used at the beginning of query pipelines. They convert query text into vector embeddings and send them to a Retriever.
|
||||
|
||||
Document Embedders embed Document objects and are most often used in indexing pipelines, after Converters, and before a DocumentWriter. They preserve the Document object format and add an embedding field with a list of float numbers.
|
||||
|
||||
You must use the same embedding model for text and documents. This means that if you use CohereDocumentEmbedder in your indexing pipeline, you must then use CohereTextEmbedder with the same model in your query pipeline.
|
||||
|
||||
## API-Based Embedders
|
||||
|
||||
These Embedders use external APIs to generate embeddings. They give you access to powerful models without needing to handle the computing yourself.
|
||||
|
||||
The costs associated with these solutions can vary. Depending on the solution you choose, you pay for the tokens consumed, both sent and generated, or for the hosting of the model, often billed per hour. Refer to the individual providers’ websites for detailed information.
|
||||
|
||||
Haystack supports the models offered by a variety of providers: **OpenAI**, **Cohere**, **Jina**, **Azure**, **Mistral**, and **Amazon Bedrock**, with more being added constantly.
|
||||
|
||||
Additionally, you could use Haystack’s **Hugging Face API Embedders** for prototyping with [HF Serverless Inference API](https://huggingface.co/docs/api-inference/en/index) or the [paid HF Inference Endpoints](https://huggingface.co/inference-endpoints/dedicated).
|
||||
|
||||
## On-Premise Embedders
|
||||
|
||||
On-premise Embedders allow you to host open models on your machine/infrastructure. This choice is ideal for local experimentation.
|
||||
|
||||
When you self-host an embedder, you can choose the model from plenty of open model options. The [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) can be a good reference point for understanding retrieval performance and model size.
|
||||
|
||||
It is suitable in production scenarios where data privacy concerns drive the decision not to transmit data to external providers and you have ample computational resources (CPU or GPU).
|
||||
|
||||
Here are some options available in Haystack:
|
||||
|
||||
- **Sentence Transformers**: This library mostly uses PyTorch, so it can be a fast-running option if you’re using a GPU. On the other hand, Sentence Transformers are progressively adding support for more efficient backends, which do not require GPU.
|
||||
- **Hugging Face Text Embedding Inference**: This is a library for efficiently serving open embedding models on both CPU and GPU. In Haystack, it can be used via HuggingFace API Embedders.
|
||||
- **Hugging Face Optimum:** These Embedders are designed to run models faster on targeted hardware. They implement optimizations that are specific for a certain hardware, such as Intel IPEX.
|
||||
- **Fastembed**: Fastembed is optimized for running on standard machines even with low resources. It supports several types of embeddings, including sparse techniques (BM25, SPLADE) and classic dense embeddings.
|
||||
- **Ollama:** These Embedders run quantized models on CPU(+GPU). Embedding quality might be lower due to the quantization of regular models. However, this makes these models run efficiently on standard machines.
|
||||
- **Nvidia**: Nvidia Embedders are built on Nvidia's NIM and hosted on their optimized cloud platform. They give you both options: using models through their API or deploying models locally with Nvidia NIM.
|
||||
|
||||
***
|
||||
|
||||
:::info
|
||||
See the full list of Embedders available in Haystack on the main [Embedders](../embedders.mdx) page.
|
||||
:::
|
||||
+135
@@ -0,0 +1,135 @@
|
||||
---
|
||||
title: "CohereDocumentEmbedder"
|
||||
id: coheredocumentembedder
|
||||
slug: "/coheredocumentembedder"
|
||||
description: "This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Cohere embedding models."
|
||||
---
|
||||
|
||||
# CohereDocumentEmbedder
|
||||
|
||||
This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Cohere embedding models.
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata strings |
|
||||
| **API reference** | [Cohere](/reference/integrations-cohere) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`CohereDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, you should use the [`CohereTextEmbedder`](coheretextembedder.mdx).
|
||||
|
||||
The component supports the following Cohere models:
|
||||
`"embed-english-v3.0"`, `"embed-english-light-v3.0"`, `"embed-multilingual-v3.0"`,
|
||||
`"embed-multilingual-light-v3.0"`, `"embed-english-v2.0"`, `"embed-english-light-v2.0"`,
|
||||
`"embed-multilingual-v2.0"`. The default model is `embed-english-v2.0`. This list of all supported models can be found in Cohere’s [model documentation](https://docs.cohere.com/docs/models#representation).
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install cohere-haystack
|
||||
```
|
||||
|
||||
The component uses a `COHERE_API_KEY` or `CO_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
|
||||
|
||||
```python
|
||||
embedder = CohereDocumentEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
To get a Cohere API key, head over to https://cohere.com/.
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from cohere_haystack.embedders.document_embedder import CohereDocumentEmbedder
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = CohereDocumentEmbedder(api_key=Secret.from_token("<your-api-key>", meta_fields_to_embed=["title"])
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Remember to set `COHERE_API_KEY` as an environment variable first, or pass it in directly.
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.cohere.document_embedder import (
|
||||
CohereDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
embedder = CohereDocumentEmbedder()
|
||||
|
||||
result = embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
## [-0.453125, 1.2236328, 2.0058594, 0.67871094...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
from haystack_integrations.components.embedders.cohere.document_embedder import (
|
||||
CohereDocumentEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.cohere.text_embedder import (
|
||||
CohereTextEmbedder,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("embedder", CohereDocumentEmbedder())
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", CohereTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+166
@@ -0,0 +1,166 @@
|
||||
---
|
||||
title: "CohereDocumentImageEmbedder"
|
||||
id: coheredocumentimageembedder
|
||||
slug: "/coheredocumentimageembedder"
|
||||
description: "`CohereDocumentImageEmbedder` computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Cohere embedding models with the ability to embed text and images into the same vector space."
|
||||
---
|
||||
|
||||
# CohereDocumentImageEmbedder
|
||||
|
||||
`CohereDocumentImageEmbedder` computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Cohere embedding models with the ability to embed text and images into the same vector space.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
|
||||
| **API reference** | [Cohere](/reference/integrations-cohere) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`CohereDocumentImageEmbedder` expects a list of documents containing an image or a PDF file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
|
||||
|
||||
The embedder efficiently loads the images, computes the embeddings using a Cohere model, and stores each of them in the `embedding` field of the document.
|
||||
|
||||
`CohereDocumentImageEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with a `CohereTextEmbedder` to embed the query, before using an Embedding Retriever.
|
||||
|
||||
This component is compatible with Cohere Embed models v3 and later. For a complete list of supported models, see the [Cohere documentation](https://docs.cohere.com/docs/models#embed).
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install cohere-haystack
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
The component uses a `COHERE_API_KEY` or `CO_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` method:
|
||||
|
||||
```python
|
||||
embedder = CohereTextEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
To get a Cohere API key, head over to https://cohere.com/.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Remember to set `COHERE_API_KEY` as an environment variable first.
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.cohere import (
|
||||
CohereDocumentImageEmbedder,
|
||||
)
|
||||
|
||||
embedder = CohereDocumentImageEmbedder(model="embed-v4.0")
|
||||
embedder.warm_up()
|
||||
|
||||
documents = [
|
||||
Document(content="A photo of a cat", meta={"file_path": "cat.jpg"}),
|
||||
Document(content="A photo of a dog", meta={"file_path": "dog.jpg"}),
|
||||
]
|
||||
|
||||
result = embedder.run(documents=documents)
|
||||
documents_with_embeddings = result["documents"]
|
||||
print(documents_with_embeddings)
|
||||
|
||||
## [Document(id=...,
|
||||
## content='A photo of a cat',
|
||||
## meta={'file_path': 'cat.jpg',
|
||||
## 'embedding_source': {'type': 'image', 'file_path_meta_field': 'file_path'}},
|
||||
## embedding=vector of size 1536),
|
||||
## ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
In this example, we can see an indexing pipeline with three components:
|
||||
|
||||
- `ImageFileToDocument` converter that creates empty documents with a reference to an image in the `meta.file_path` field;
|
||||
- `CohereDocumentImageEmbedder` that loads the images, computes embeddings and store them in documents;
|
||||
- `DocumentWriter` that writes the documents in the `InMemoryDocumentStore`.
|
||||
|
||||
There is also a multimodal retrieval pipeline, composed of a `CohereTextEmbedder` (using the same model as before) and an `InMemoryEmbeddingRetriever`.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters.image import ImageFileToDocument
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
from haystack_integrations.components.embedders.cohere import (
|
||||
CohereDocumentImageEmbedder,
|
||||
CohereTextEmbedder,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
## Indexing pipeline
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("image_converter", ImageFileToDocument())
|
||||
indexing_pipeline.add_component(
|
||||
"embedder",
|
||||
CohereDocumentImageEmbedder(model="embed-v4.0"),
|
||||
)
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("image_converter", "embedder")
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run(data={"image_converter": {"sources": ["dog.jpg", "hyena.jpeg"]}})
|
||||
|
||||
## Multimodal retrieval pipeline
|
||||
retrieval_pipeline = Pipeline()
|
||||
retrieval_pipeline.add_component("embedder", CohereTextEmbedder(model="embed-v4.0"))
|
||||
retrieval_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store, top_k=2),
|
||||
)
|
||||
retrieval_pipeline.connect("embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
result = retrieval_pipeline.run(data={"text": "man's best friend"})
|
||||
print(result)
|
||||
|
||||
## {
|
||||
## 'retriever': {
|
||||
## 'documents': [
|
||||
## Document(
|
||||
## id=0c96...,
|
||||
## meta={
|
||||
## 'file_path': 'dog.jpg',
|
||||
## 'embedding_source': {
|
||||
## 'type': 'image',
|
||||
## 'file_path_meta_field': 'file_path'
|
||||
## }
|
||||
## },
|
||||
## score=0.288
|
||||
## ),
|
||||
## Document(
|
||||
## id=5e76...,
|
||||
## meta={
|
||||
## 'file_path': 'hyena.jpeg',
|
||||
## 'embedding_source': {
|
||||
## 'type': 'image',
|
||||
## 'file_path_meta_field': 'file_path'
|
||||
## }
|
||||
## },
|
||||
## score=0.248
|
||||
## )
|
||||
## ]
|
||||
## }
|
||||
## }
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
:notebook: Tutorial: [Creating Vision+Text RAG Pipelines](https://haystack.deepset.ai/tutorials/46_multimodal_rag)
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "CohereTextEmbedder"
|
||||
id: coheretextembedder
|
||||
slug: "/coheretextembedder"
|
||||
description: "This component transforms a string into a vector that captures its semantics using a Cohere embedding model. When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents."
|
||||
---
|
||||
|
||||
# CohereTextEmbedder
|
||||
|
||||
This component transforms a string into a vector that captures its semantics using a Cohere embedding model. When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Cohere API key. Can be set with `COHERE_API_KEY` or `CO_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers (vectors) <br /> <br />`meta`: A dictionary of metadata strings |
|
||||
| **API reference** | [Cohere](/reference/integrations-cohere) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`CohereTextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the use the [`CohereDocumentEmbedder`](coheredocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
|
||||
|
||||
The component supports the following Cohere models:
|
||||
`"embed-english-v3.0"`, `"embed-english-light-v3.0"`, `"embed-multilingual-v3.0"`,
|
||||
`"embed-multilingual-light-v3.0"`, `"embed-english-v2.0"`, `"embed-english-light-v2.0"`,
|
||||
`"embed-multilingual-v2.0"`. The default model is `embed-english-v2.0`. This list of all supported models can be found in Cohere’s [model documentation](https://docs.cohere.com/docs/models#representation).
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install cohere-haystack
|
||||
```
|
||||
|
||||
The component uses a `COHERE_API_KEY` or `CO_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
|
||||
|
||||
```python
|
||||
embedder = CohereTextEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
To get a Cohere API key, head over to https://cohere.com/.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own. You’ll need to pass in your Cohere API key via Secret or set it as an environment variable called `COHERE_API_KEY`. The examples below assume you've set the environment variable.
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.cohere.text_embedder import (
|
||||
CohereTextEmbedder,
|
||||
)
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = CohereTextEmbedder()
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
## {'embedding': [-0.453125, 1.2236328, 2.0058594, 0.67871094...],
|
||||
## 'meta': {'api_version': {'version': '1'}, 'billed_units': {'input_tokens': 4}}}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.cohere.text_embedder import (
|
||||
CohereTextEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.cohere.document_embedder import (
|
||||
CohereDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = CohereDocumentEmbedder()
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", CohereTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
---
|
||||
title: "External Integrations"
|
||||
id: external-integrations-embedders
|
||||
slug: "/external-integrations-embedders"
|
||||
description: "External integrations that enable transforming texts or documents into vector representations using pre-trained models."
|
||||
---
|
||||
|
||||
# External Integrations
|
||||
|
||||
External integrations that enable transforming texts or documents into vector representations using pre-trained models.
|
||||
|
||||
| Name | Description |
|
||||
| --- | --- |
|
||||
| [mixedbread ai](https://haystack.deepset.ai/integrations/mixedbread-ai) | Compute embeddings for text and documents using mixedbread's API. |
|
||||
| [Voyage AI](https://haystack.deepset.ai/integrations/voyage) | Computing embeddings for text and documents using Voyage AI embedding models. |
|
||||
+172
@@ -0,0 +1,172 @@
|
||||
---
|
||||
title: "FastembedDocumentEmbedder"
|
||||
id: fastembeddocumentembedder
|
||||
slug: "/fastembeddocumentembedder"
|
||||
description: "This component computes the embeddings of a list of documents using the models supported by FastEmbed."
|
||||
---
|
||||
|
||||
# FastembedDocumentEmbedder
|
||||
|
||||
This component computes the embeddings of a list of documents using the models supported by FastEmbed.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
|
||||
| **API reference** | [FastEmbed](/reference/fastembed-embedders) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed |
|
||||
|
||||
</div>
|
||||
|
||||
This component should be used to embed a list of documents. To embed a string, use the [`FastembedTextEmbedder`](fastembedtextembedder.mdx).
|
||||
|
||||
## Overview
|
||||
|
||||
`FastembedDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses embedding [models supported by FastEmbed](https://qdrant.github.io/fastembed/examples/Supported_Models/).
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents in order to find the most similar or relevant documents.
|
||||
|
||||
### Compatible models
|
||||
|
||||
You can find the original models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/).
|
||||
|
||||
Nowadays, most of the models in the [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) are compatible with FastEmbed. You can look for compatibility in the [supported model list](https://qdrant.github.io/fastembed/examples/Supported_Models/).
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install fastembed-haystack
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single `onnxruntime` session can use.
|
||||
|
||||
```python
|
||||
cache_dir= "/your_cacheDirectory"
|
||||
embedder = FastembedDocumentEmbedder(
|
||||
*model="*BAAI/bge-large-en-v1.5",
|
||||
cache_dir=cache_dir,
|
||||
threads=2
|
||||
)
|
||||
```
|
||||
|
||||
If you want to use the data parallel encoding, you can set the parameters `parallel` and `batch_size`.
|
||||
|
||||
- If parallel > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
|
||||
- If parallel is 0, use all available cores.
|
||||
- If None, don't use data-parallel processing; use default `onnxruntime` threading instead.
|
||||
|
||||
:::tip
|
||||
If you create a Text Embedder and a Document Embedder based on the same model, Haystack uses the same resource behind the scenes to save resources.
|
||||
:::
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this easily by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack.preview import Document
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(
|
||||
text="some text",
|
||||
metadata={"title": "relevant title", "page number": 18},
|
||||
)
|
||||
|
||||
embedder = FastembedDocumentEmbedder(
|
||||
model="BAAI/bge-small-en-v1.5",
|
||||
batch_size=256,
|
||||
metadata_fields_to_embed=["title"],
|
||||
)
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.dataclasses import Document
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedDocumentEmbedder,
|
||||
)
|
||||
|
||||
document_list = [
|
||||
Document(content="I love pizza!"),
|
||||
Document(content="I like spaghetti"),
|
||||
]
|
||||
|
||||
doc_embedder = FastembedDocumentEmbedder()
|
||||
doc_embedder.warm_up()
|
||||
|
||||
result = doc_embedder.run(document_list)
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [-0.04235665127635002, 0.021791068837046623, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.document_stores.types import DuplicatePolicy
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedDocumentEmbedder,
|
||||
FastembedTextEmbedder,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
Document(content="fastembed is supported by and maintained by Qdrant."),
|
||||
]
|
||||
|
||||
document_embedder = FastembedDocumentEmbedder()
|
||||
writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE)
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("document_embedder", document_embedder)
|
||||
indexing_pipeline.add_component("writer", writer)
|
||||
indexing_pipeline.connect("document_embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"document_embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", FastembedTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who supports fastembed?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0]) # noqa: T201
|
||||
|
||||
## Document(id=...,
|
||||
## content: 'fastembed is supported by and maintained by Qdrant.',
|
||||
## score: 0.758..)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [RAG Pipeline Using FastEmbed for Embeddings Generation](https://haystack.deepset.ai/cookbook/rag_fastembed)
|
||||
+191
@@ -0,0 +1,191 @@
|
||||
---
|
||||
title: "FastembedSparseDocumentEmbedder"
|
||||
id: fastembedsparsedocumentembedder
|
||||
slug: "/fastembedsparsedocumentembedder"
|
||||
description: "Use this component to enrich a list of documents with their sparse embeddings."
|
||||
---
|
||||
|
||||
# FastembedSparseDocumentEmbedder
|
||||
|
||||
Use this component to enrich a list of documents with their sparse embeddings.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with sparse embeddings) |
|
||||
| **API reference** | [FastEmbed](/reference/fastembed-embedders) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed |
|
||||
|
||||
</div>
|
||||
|
||||
To compute a sparse embedding for a string, use the [`FastembedSparseTextEmbedder`](fastembedsparsetextembedder.mdx).
|
||||
|
||||
## Overview
|
||||
|
||||
`FastembedSparseDocumentEmbedder` computes the sparse embeddings of a list of documents and stores the obtained vectors in the `sparse_embedding` field of each document. It uses sparse embedding [models](https://qdrant.github.io/fastembed/examples/Supported_Models/#supported-sparse-text-embedding-models) supported by FastEmbed.
|
||||
|
||||
The vectors calculated by this component are necessary for performing sparse embedding retrieval on a set of documents. During retrieval, the sparse vector representing the query is compared to those of the documents to identify the most similar or relevant ones.
|
||||
|
||||
### Compatible models
|
||||
|
||||
You can find the supported models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/examples/Supported_Models/#supported-sparse-text-embedding-models).
|
||||
|
||||
Currently, supported models are based on SPLADE, a technique for producing sparse representations for text, where each non-zero value in the embedding is the importance weight of a term in the BERT WordPiece vocabulary. For more information, see [our docs](../retrievers.mdx#sparse-embedding-based-retrievers) that explain sparse embedding-based Retrievers further.
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install fastembed-haystack
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single `onnxruntime` session can use:
|
||||
|
||||
```python
|
||||
cache_dir = "/your_cacheDirectory"
|
||||
embedder = FastembedSparseDocumentEmbedder(
|
||||
model="prithivida/Splade_PP_en_v1",
|
||||
cache_dir=cache_dir,
|
||||
threads=2,
|
||||
)
|
||||
```
|
||||
|
||||
If you want to use the data parallel encoding, you can set the parameters `parallel` and `batch_size`.
|
||||
|
||||
- If `parallel` > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
|
||||
- If `parallel` is 0, use all available cores.
|
||||
- If None, don't use data-parallel processing; use default `onnxruntime` threading instead.
|
||||
|
||||
:::tip
|
||||
If you create both a Sparse Text Embedder and a Sparse Document Embedder based on the same model, Haystack utilizes a shared resource behind the scenes to conserve resources.
|
||||
:::
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often include metadata. If the metadata is distinctive and semantically meaningful, you can embed it along with the document's text to improve retrieval.
|
||||
|
||||
You can do this easily by using the sparse Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack.preview import Document
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedSparseDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(
|
||||
text="some text",
|
||||
metadata={"title": "relevant title", "page number": 18},
|
||||
)
|
||||
|
||||
embedder = FastembedSparseDocumentEmbedder(
|
||||
model="prithivida/Splade_PP_en_v1",
|
||||
metadata_fields_to_embed=["title"],
|
||||
)
|
||||
|
||||
docs_w_sparse_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.dataclasses import Document
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedSparseDocumentEmbedder,
|
||||
)
|
||||
|
||||
document_list = [
|
||||
Document(content="I love pizza!"),
|
||||
Document(content="I like spaghetti"),
|
||||
]
|
||||
|
||||
doc_embedder = FastembedSparseDocumentEmbedder()
|
||||
doc_embedder.warm_up()
|
||||
|
||||
result = doc_embedder.run(document_list)
|
||||
print(result["documents"][0])
|
||||
|
||||
## Document(id=...,
|
||||
## content: 'I love pizza!',
|
||||
## sparse_embedding: vector with 24 non-zero elements)
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Currently, sparse embedding retrieval is only supported by `QdrantDocumentStore`.
|
||||
First, install the package with:
|
||||
|
||||
```shell
|
||||
pip install qdrant-haystack
|
||||
```
|
||||
|
||||
Then, try out this pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack_integrations.components.retrievers.qdrant import (
|
||||
QdrantSparseEmbeddingRetriever,
|
||||
)
|
||||
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
|
||||
from haystack.document_stores.types import DuplicatePolicy
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedDocumentEmbedder,
|
||||
FastembedTextEmbedder,
|
||||
)
|
||||
|
||||
document_store = QdrantDocumentStore(
|
||||
":memory:",
|
||||
recreate_index=True,
|
||||
use_sparse_embeddings=True,
|
||||
)
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
Document(content="fastembed is supported by and maintained by Qdrant."),
|
||||
]
|
||||
|
||||
sparse_document_embedder = FastembedSparseDocumentEmbedder()
|
||||
writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE)
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("sparse_document_embedder", sparse_document_embedder)
|
||||
indexing_pipeline.add_component("writer", writer)
|
||||
indexing_pipeline.connect("sparse_document_embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"sparse_document_embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("sparse_text_embedder", FastembedSparseTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"sparse_retriever",
|
||||
QdrantSparseEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect(
|
||||
"sparse_text_embedder.sparse_embedding",
|
||||
"sparse_retriever.query_sparse_embedding",
|
||||
)
|
||||
|
||||
query = "Who supports fastembed?"
|
||||
|
||||
result = query_pipeline.run({"sparse_text_embedder": {"text": query}})
|
||||
|
||||
print(result["sparse_retriever"]["documents"][0]) # noqa: T201
|
||||
|
||||
## Document(id=...,
|
||||
## content: 'fastembed is supported by and maintained by Qdrant.',
|
||||
## score: 0.758..)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Sparse Embedding Retrieval with Qdrant and FastEmbed](https://haystack.deepset.ai/cookbook/sparse_embedding_retrieval)
|
||||
+156
@@ -0,0 +1,156 @@
|
||||
---
|
||||
title: "FastembedSparseTextEmbedder"
|
||||
id: fastembedsparsetextembedder
|
||||
slug: "/fastembedsparsetextembedder"
|
||||
description: "Use this component to embed a simple string (such as a query) into a sparse vector."
|
||||
---
|
||||
|
||||
# FastembedSparseTextEmbedder
|
||||
|
||||
Use this component to embed a simple string (such as a query) into a sparse vector.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a sparse embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object |
|
||||
| **API reference** | [FastEmbed](/reference/fastembed-embedders) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed |
|
||||
|
||||
</div>
|
||||
|
||||
For embedding lists of documents, use the [`FastembedSparseDocumentEmbedder`](fastembedsparsedocumentembedder.mdx), which enriches the document with the computed sparse embedding.
|
||||
|
||||
## Overview
|
||||
|
||||
`FastembedSparseTextEmbedder` transforms a string into a sparse vector using sparse embedding [models](https://qdrant.github.io/fastembed/examples/Supported_Models/#supported-sparse-text-embedding-models) supported by FastEmbed.
|
||||
|
||||
When you perform sparse embedding retrieval, use this component first to transform your query into a sparse vector. Then, the sparse embedding Retriever will use the vector to search for similar or relevant documents.
|
||||
|
||||
### Compatible Models
|
||||
|
||||
You can find the supported models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/examples/Supported_Models/#supported-sparse-text-embedding-models).
|
||||
|
||||
Currently, supported models are based on SPLADE, a technique for producing sparse representations for text, where each non-zero value in the embedding is the importance weight of a term in the BERT WordPiece vocabulary. For more information, see [our docs](../retrievers.mdx#sparse-embedding-based-retrievers) that explain sparse embedding-based Retrievers further.
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install fastembed-haystack
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single `onnxruntime` session can use:
|
||||
|
||||
```python
|
||||
cache_dir = "/your_cacheDirectory"
|
||||
embedder = FastembedSparseTextEmbedder(
|
||||
model="prithivida/Splade_PP_en_v1",
|
||||
cache_dir=cache_dir,
|
||||
threads=2,
|
||||
)
|
||||
```
|
||||
|
||||
If you want to use the data parallel encoding, you can set the `parallel` parameter.
|
||||
|
||||
- If `parallel` > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
|
||||
- If `parallel` is 0, use all available cores.
|
||||
- If None, don't use data-parallel processing; use the default `onnxruntime` threading instead.
|
||||
|
||||
:::tip
|
||||
If you create both a Sparse Text Embedder and a Sparse Document Embedder based on the same model, Haystack utilizes a shared resource behind the scenes to conserve resources.
|
||||
:::
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedSparseTextEmbedder,
|
||||
)
|
||||
|
||||
text = """It clearly says online this will work on a Mac OS system.
|
||||
The disk comes and it does not, only Windows.
|
||||
Do Not order this if you have a Mac!!"""
|
||||
|
||||
text_embedder = FastembedSparseTextEmbedder(model="prithivida/Splade_PP_en_v1")
|
||||
text_embedder.warm_up()
|
||||
|
||||
sparse_embedding = text_embedder.run(text)["sparse_embedding"]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Currently, sparse embedding retrieval is only supported by `QdrantDocumentStore`.
|
||||
First, install the package with:
|
||||
|
||||
```shell
|
||||
pip install qdrant-haystack
|
||||
```
|
||||
|
||||
Then, try out this pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
|
||||
from haystack_integrations.components.retrievers.qdrant import (
|
||||
QdrantSparseEmbeddingRetriever,
|
||||
)
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedSparseTextEmbedder,
|
||||
FastembedSparseDocumentEmbedder,
|
||||
FastembedTextEmbedder,
|
||||
)
|
||||
|
||||
document_store = QdrantDocumentStore(
|
||||
":memory:",
|
||||
recreate_index=True,
|
||||
use_sparse_embeddings=True,
|
||||
)
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
Document(content="fastembed is supported by and maintained by Qdrant."),
|
||||
]
|
||||
|
||||
sparse_document_embedder = FastembedSparseDocumentEmbedder(
|
||||
model="prithivida/Splade_PP_en_v1",
|
||||
)
|
||||
|
||||
sparse_document_embedder.warm_up()
|
||||
documents_with_sparse_embeddings = sparse_document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_sparse_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("sparse_text_embedder", FastembedSparseTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"sparse_retriever",
|
||||
QdrantSparseEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect(
|
||||
"sparse_text_embedder.sparse_embedding",
|
||||
"sparse_retriever.query_sparse_embedding",
|
||||
)
|
||||
|
||||
query = "Who supports fastembed?"
|
||||
|
||||
result = query_pipeline.run({"sparse_text_embedder": {"text": query}})
|
||||
|
||||
print(result["sparse_retriever"]["documents"][0]) # noqa: T201
|
||||
|
||||
## Document(id=...,
|
||||
## content: 'fastembed is supported by and maintained by Qdrant.',
|
||||
## score: 0.561..)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Sparse Embedding Retrieval with Qdrant and FastEmbed](https://haystack.deepset.ai/cookbook/sparse_embedding_retrieval)
|
||||
+144
@@ -0,0 +1,144 @@
|
||||
---
|
||||
title: "FastembedTextEmbedder"
|
||||
id: fastembedtextembedder
|
||||
slug: "/fastembedtextembedder"
|
||||
description: "This component computes the embeddings of a string using embedding models supported by FastEmbed."
|
||||
---
|
||||
|
||||
# FastembedTextEmbedder
|
||||
|
||||
This component computes the embeddings of a string using embedding models supported by FastEmbed.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A vector (list of float numbers) |
|
||||
| **API reference** | [FastEmbed](/reference/fastembed-embedders) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed |
|
||||
|
||||
</div>
|
||||
|
||||
This component should be used to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`FastembedDocumentEmbedder`](fastembeddocumentembedder.mdx), which enriches the document with the computed embedding, known as vector.
|
||||
|
||||
## Overview
|
||||
|
||||
`FastembedTextEmbedder` transforms a string into a vector that captures its semantics using embedding [models supported by FastEmbed](https://qdrant.github.io/fastembed/examples/Supported_Models/).
|
||||
|
||||
When you perform embedding retrieval, use this component first to transform your query into a vector. Then, the embedding Retriever will use the vector to search for similar or relevant documents.
|
||||
|
||||
### Compatible models
|
||||
|
||||
You can find the original models in the [FastEmbed documentation](https://qdrant.github.io/fastembed/).
|
||||
|
||||
Currently, most of the models in the [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) are compatible with FastEmbed. You can look for compatibility in the [supported model list](https://qdrant.github.io/fastembed/examples/Supported_Models/).
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```bash
|
||||
pip install fastembed-haystack
|
||||
```
|
||||
|
||||
### Instructions
|
||||
|
||||
Some recent models that you can find in MTEB require prepending the text with an instruction to work better for retrieval.
|
||||
For example, if you use `[BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5#model-list)` model, you should prefix your query with the `instruction: “passage:”`.
|
||||
|
||||
This is how it works with `FastembedTextEmbedder`:
|
||||
|
||||
```python
|
||||
instruction = "passage:"
|
||||
embedder = FastembedTextEmbedder(
|
||||
*model="*BAAI/bge-large-en-v1.5",
|
||||
prefix=instruction)
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single `onnxruntime` session can use.
|
||||
|
||||
```python
|
||||
cache_dir= "/your_cacheDirectory"
|
||||
embedder = FastembedTextEmbedder(
|
||||
*model="*BAAI/bge-large-en-v1.5",
|
||||
cache_dir=cache_dir,
|
||||
threads=2
|
||||
)
|
||||
```
|
||||
|
||||
If you want to use the data parallel encoding, you can set the parameters `parallel` and `batch_size`.
|
||||
|
||||
- If parallel > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
|
||||
- If parallel is 0, use all available cores.
|
||||
- If None, don't use data-parallel processing; use default `onnxruntime` threading instead.
|
||||
|
||||
:::tip
|
||||
If you create a Text Embedder and a Document Embedder based on the same model, Haystack uses the same resource behind the scenes to save resources.
|
||||
:::
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.fastembed import FastembedTextEmbedder
|
||||
|
||||
text = """It clearly says online this will work on a Mac OS system.
|
||||
The disk comes and it does not, only Windows.
|
||||
Do Not order this if you have a Mac!!"""
|
||||
text_embedder = FastembedTextEmbedder(model="BAAI/bge-small-en-v1.5")
|
||||
text_embedder.warm_up()
|
||||
embedding = text_embedder.run(text)["embedding"]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.fastembed import (
|
||||
FastembedDocumentEmbedder,
|
||||
FastembedTextEmbedder,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
Document(content="fastembed is supported by and maintained by Qdrant."),
|
||||
]
|
||||
|
||||
document_embedder = FastembedDocumentEmbedder()
|
||||
document_embedder.warm_up()
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", FastembedTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who supports FastEmbed?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0]) # noqa: T201
|
||||
|
||||
## Document(id=...,
|
||||
## content: 'FastEmbed is supported by and maintained by Qdrant.',
|
||||
## score: 0.758..)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [RAG Pipeline Using FastEmbed for Embeddings Generation](https://haystack.deepset.ai/cookbook/rag_fastembed)
|
||||
+182
@@ -0,0 +1,182 @@
|
||||
---
|
||||
title: "GoogleGenAIDocumentEmbedder"
|
||||
id: googlegenaidocumentembedder
|
||||
slug: "/googlegenaidocumentembedder"
|
||||
description: "The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents."
|
||||
---
|
||||
|
||||
# GoogleGenAIDocumentEmbedder
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [DocumentWriter](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Google API key. Can be set with `GOOGLE_API_KEY` or `GEMINI_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Google AI](/reference/integrations-google-genai) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`GoogleGenAIDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, you should use the [`GoogleGenAITextEmbedder`](googlegenaitextembedder.mdx).
|
||||
|
||||
The component supports the following Google AI models:
|
||||
|
||||
- `text-embedding-004` (default)
|
||||
- `text-embedding-004-v2`
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install google-genai-haystack
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.
|
||||
|
||||
To use this component with the Gemini Developer API and get an API key, visit [Google AI Studio](https://aistudio.google.com/).
|
||||
To use this component with the Vertex AI API, visit [Google Cloud > Vertex AI](https://cloud.google.com/vertex-ai).
|
||||
|
||||
The component uses a `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
|
||||
|
||||
```python
|
||||
embedder = GoogleGenAIDocumentEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
The following examples show how to use the component with the Gemini Developer API and the Vertex AI API.
|
||||
|
||||
#### Gemini Developer API (API Key Authentication)
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAIDocumentEmbedder,
|
||||
)
|
||||
|
||||
## set the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY)
|
||||
chat_generator = GoogleGenAIDocumentEmbedder()
|
||||
```
|
||||
|
||||
#### Vertex AI (Application Default Credentials)
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAIDocumentEmbedder,
|
||||
)
|
||||
|
||||
## Using Application Default Credentials (requires gcloud auth setup)
|
||||
chat_generator = GoogleGenAIDocumentEmbedder(
|
||||
api="vertex",
|
||||
vertex_ai_project="my-project",
|
||||
vertex_ai_location="us-central1",
|
||||
)
|
||||
```
|
||||
|
||||
#### Vertex AI (API Key Authentication)
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAIDocumentEmbedder,
|
||||
)
|
||||
|
||||
## set the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY)
|
||||
chat_generator = GoogleGenAIDocumentEmbedder(api="vertex")
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAIDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = GoogleGenAIDocumentEmbedder(
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
meta_fields_to_embed=["title"],
|
||||
)
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own. You'll need to pass in your Google API key via Secret or set it as an environment variable called `GOOGLE_API_KEY` or `GEMINI_API_KEY`. The examples below assume you've set the environment variable.
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAIDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = GoogleGenAIDocumentEmbedder()
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAITextEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAIDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("embedder", GoogleGenAIDocumentEmbedder())
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", GoogleGenAITextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+154
@@ -0,0 +1,154 @@
|
||||
---
|
||||
title: "GoogleGenAITextEmbedder"
|
||||
id: googlegenaitextembedder
|
||||
slug: "/googlegenaitextembedder"
|
||||
description: "This component transforms a string into a vector that captures its semantics using a Google AI embedding models. When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents."
|
||||
---
|
||||
|
||||
# GoogleGenAITextEmbedder
|
||||
|
||||
This component transforms a string into a vector that captures its semantics using a Google AI embedding models. When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Google API key. Can be set with `GOOGLE_API_KEY` or `GEMINI_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Google AI](/reference/integrations-google-genai) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_genai |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`GoogleGenAITextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the [`GoogleGenAIDocumentEmbedder`](googlegenaidocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
|
||||
|
||||
The component supports the following Google AI models:
|
||||
|
||||
- `text-embedding-004` (default)
|
||||
- `text-embedding-004-v2`
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install google-genai-haystack
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.
|
||||
|
||||
To use this component with the Gemini Developer API and get an API key, visit [Google AI Studio](https://aistudio.google.com/).
|
||||
To use this component with the Vertex AI API, visit [Google Cloud > Vertex AI](https://cloud.google.com/vertex-ai).
|
||||
|
||||
The component uses a `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` static method:
|
||||
|
||||
```python
|
||||
embedder = GoogleGenAITextEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
The following examples show how to use the component with the Gemini Developer API and the Vertex AI API.
|
||||
|
||||
#### Gemini Developer API (API Key Authentication)
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAITextEmbedder,
|
||||
)
|
||||
|
||||
## set the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY)
|
||||
chat_generator = GoogleGenAITextEmbedder()
|
||||
```
|
||||
|
||||
#### Vertex AI (Application Default Credentials)
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAITextEmbedder,
|
||||
)
|
||||
|
||||
## Using Application Default Credentials (requires gcloud auth setup)
|
||||
chat_generator = GoogleGenAITextEmbedder(
|
||||
api="vertex",
|
||||
vertex_ai_project="my-project",
|
||||
vertex_ai_location="us-central1",
|
||||
)
|
||||
```
|
||||
|
||||
#### Vertex AI (API Key Authentication)
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAITextEmbedder,
|
||||
)
|
||||
|
||||
## set the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY)
|
||||
chat_generator = GoogleGenAITextEmbedder(api="vertex")
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own. You'll need to pass in your Google API key with a Secret or set it as an environment variable called `GOOGLE_API_KEY` or `GEMINI_API_KEY`. The examples below assume you've set the environment variable.
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAITextEmbedder,
|
||||
)
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = GoogleGenAITextEmbedder()
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
## {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
|
||||
## 'meta': {'model': 'text-embedding-004',
|
||||
## 'usage': {'prompt_tokens': 4, 'total_tokens': 4}}}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAITextEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.google_genai import (
|
||||
GoogleGenAIDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = GoogleGenAIDocumentEmbedder()
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", GoogleGenAITextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+181
@@ -0,0 +1,181 @@
|
||||
---
|
||||
title: "HuggingFaceAPIDocumentEmbedder"
|
||||
id: huggingfaceapidocumentembedder
|
||||
slug: "/huggingfaceapidocumentembedder"
|
||||
description: "Use this component to compute document embeddings using various Hugging Face APIs."
|
||||
---
|
||||
|
||||
# HuggingFaceAPIDocumentEmbedder
|
||||
|
||||
Use this component to compute document embeddings using various Hugging Face APIs.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_type`: The type of Hugging Face API to use <br /> <br />`api_params`: A dictionary with one of the following keys: <br /> <br />- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_EMBEDDINGS_INFERENCE`. <br /> <br />`token`: The Hugging Face API token. Can be set with `HF_API_TOKEN` or `HF_TOKEN` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents to be embedded (enriched with embeddings) |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/hugging_face_api_document_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`HuggingFaceAPIDocumentEmbedder` can be used to compute document embeddings using different Hugging Face APIs:
|
||||
|
||||
- [Free Serverless Inference API](https://huggingface.co/inference-api)
|
||||
- [Paid Inference Endpoints](https://huggingface.co/inference-endpoints)
|
||||
- [Self-hosted Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference)
|
||||
|
||||
:::info
|
||||
This component should be used to embed a list of documents. To embed a string, use [`HuggingFaceAPITextEmbedder`](huggingfaceapitextembedder.mdx).
|
||||
:::
|
||||
|
||||
The component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token` – see code examples below.
|
||||
The token is needed:
|
||||
|
||||
- If you use the Serverless Inference API, or
|
||||
- If you use the Inference Endpoints.
|
||||
|
||||
## Usage
|
||||
|
||||
Similarly to other Document Embedders, this component allows adding prefixes (and postfixes) to include instruction and embedding metadata.
|
||||
For more fine-grained details, refer to the component’s [API reference](/reference/embedders-api#huggingfaceapidocumentembedder).
|
||||
|
||||
### On its own
|
||||
|
||||
#### Using Free Serverless Inference API
|
||||
|
||||
Formerly known as (free) Hugging Face Inference API, this API allows you to quickly experiment with many models hosted on the Hugging Face Hub, offloading the inference to Hugging Face servers. It’s rate-limited and not meant for production.
|
||||
|
||||
To use this API, you need a [free Hugging Face token](https://huggingface.co/settings/tokens).
|
||||
The Embedder expects the `model` in `api_params`.
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import HuggingFaceAPIDocumentEmbedder
|
||||
from haystack.utils import Secret
|
||||
from haystack.dataclasses import Document
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = HuggingFaceAPIDocumentEmbedder(
|
||||
api_type="serverless_inference_api",
|
||||
api_params={"model": "BAAI/bge-small-en-v1.5"},
|
||||
token=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
#### Using Paid Inference Endpoints
|
||||
|
||||
In this case, a private instance of the model is deployed by Hugging Face, and you typically pay per hour.
|
||||
|
||||
To understand how to spin up an Inference Endpoint, visit [Hugging Face documentation](https://huggingface.co/inference-endpoints/dedicated).
|
||||
|
||||
Additionally, in this case, you need to provide your Hugging Face token.
|
||||
The Embedder expects the `url` of your endpoint in `api_params`.
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import HuggingFaceAPIDocumentEmbedder
|
||||
from haystack.utils import Secret
|
||||
from haystack.dataclasses import Document
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = HuggingFaceAPIDocumentEmbedder(
|
||||
api_type="inference_endpoints",
|
||||
api_params={"url": "<your-inference-endpoint-url>"},
|
||||
token=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
#### Using Self-Hosted Text Embeddings Inference (TEI)
|
||||
|
||||
[Hugging Face Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference) is a toolkit for efficiently deploying and serving text embedding models.
|
||||
|
||||
While it powers the most recent versions of Serverless Inference API and Inference Endpoints, it can be used easily on-premise through Docker.
|
||||
|
||||
For example, you can run a TEI container as follows:
|
||||
|
||||
```shell
|
||||
model=BAAI/bge-large-en-v1.5
|
||||
revision=refs/pr/5
|
||||
volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run
|
||||
|
||||
docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.2 --model-id $model --revision $revision
|
||||
```
|
||||
|
||||
For more information, refer to the [official TEI repository](https://github.com/huggingface/text-embeddings-inference).
|
||||
|
||||
The Embedder expects the `url` of your TEI instance in `api_params`.
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import HuggingFaceAPIDocumentEmbedder
|
||||
from haystack.dataclasses import Document
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = HuggingFaceAPIDocumentEmbedder(
|
||||
api_type="text_embeddings_inference",
|
||||
api_params={"url": "http://localhost:8080"},
|
||||
)
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.embedders import HuggingFaceAPITextEmbedder, HuggingFaceAPIDocumentEmbedder
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities")]
|
||||
|
||||
document_embedder = HuggingFaceAPIDocumentEmbedder(api_type="serverless_inference_api",
|
||||
api_params={"model": "BAAI/bge-small-en-v1.5"})
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("document_embedder", document_embedder)
|
||||
indexing_pipeline.add_component("doc_writer", DocumentWriter(document_store=document_store)
|
||||
indexing_pipeline.connect("document_embedder", "doc_writer")
|
||||
indexing_pipeline.run({"document_embedder": {"documents": documents}})
|
||||
|
||||
text_embedder = HuggingFaceAPITextEmbedder(api_type="serverless_inference_api",
|
||||
api_params={"model": "BAAI/bge-small-en-v1.5"})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", text_embedder)
|
||||
query_pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store))
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder":{"text": query}})
|
||||
|
||||
print(result['retriever']['documents'][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin', ...)
|
||||
```
|
||||
+177
@@ -0,0 +1,177 @@
|
||||
---
|
||||
title: "HuggingFaceAPITextEmbedder"
|
||||
id: huggingfaceapitextembedder
|
||||
slug: "/huggingfaceapitextembedder"
|
||||
description: "Use this component to embed strings using various Hugging Face APIs."
|
||||
---
|
||||
|
||||
# HuggingFaceAPITextEmbedder
|
||||
|
||||
Use this component to embed strings using various Hugging Face APIs.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_type`: The type of Hugging Face API to use <br /> <br />`api_params`: A dictionary with one of the following keys: <br /> <br />- `model`: Hugging Face model ID. Required when `api_type` is `SERVERLESS_INFERENCE_API`.**OR** - `url`: URL of the inference endpoint. Required when `api_type` is `INFERENCE_ENDPOINTS` or `TEXT_EMBEDDINGS_INFERENCE`. <br /> <br />`token`: The Hugging Face API token. Can be set with `HF_API_TOKEN` or `HF_TOKEN` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/hugging_face_api_text_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`HuggingFaceAPITextEmbedder` can be used to embed strings using different Hugging Face APIs:
|
||||
|
||||
- [Free Serverless Inference API](https://huggingface.co/inference-api)
|
||||
- [Paid Inference Endpoints](https://huggingface.co/inference-endpoints)
|
||||
- [Self-hosted Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference)
|
||||
|
||||
:::info
|
||||
This component should be used to embed plain text. To embed a list of documents, use [`HuggingFaceAPIDocumentEmbedder`](huggingfaceapidocumentembedder.mdx).
|
||||
:::
|
||||
|
||||
The component uses a `HF_API_TOKEN` environment variable by default. Otherwise, you can pass a Hugging Face API token at initialization with `token` – see code examples below.
|
||||
The token is needed:
|
||||
|
||||
- If you use the Serverless Inference API, or
|
||||
- If you use the Inference Endpoints.
|
||||
|
||||
## Usage
|
||||
|
||||
Similarly to other text Embedders, this component allows adding prefixes (and postfixes) to include instructions.
|
||||
For more fine-grained details, refer to the component’s [API reference](/reference/embedders-api#huggingfaceapitextembedder).
|
||||
|
||||
### On its own
|
||||
|
||||
#### Using Free Serverless Inference API
|
||||
|
||||
Formerly known as (free) Hugging Face Inference API, this API allows you to quickly experiment with many models hosted on the Hugging Face Hub, offloading the inference to Hugging Face servers. It’s rate-limited and not meant for production.
|
||||
|
||||
To use this API, you need a [free Hugging Face token](https://huggingface.co/settings/tokens).
|
||||
The Embedder expects the `model` in `api_params`.
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import HuggingFaceAPITextEmbedder
|
||||
from haystack.utils import Secret
|
||||
|
||||
text_embedder = HuggingFaceAPITextEmbedder(
|
||||
api_type="serverless_inference_api",
|
||||
api_params={"model": "BAAI/bge-small-en-v1.5"},
|
||||
token=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
|
||||
print(text_embedder.run("I love pizza!"))
|
||||
|
||||
## {'embedding': [0.017020374536514282, -0.023255806416273117, ...]}
|
||||
```
|
||||
|
||||
#### Using Paid Inference Endpoints
|
||||
|
||||
In this case, a private instance of the model is deployed by Hugging Face, and you typically pay per hour.
|
||||
|
||||
To understand how to spin up an Inference Endpoint, visit [Hugging Face documentation](https://huggingface.co/inference-endpoints/dedicated).
|
||||
|
||||
Additionally, in this case, you need to provide your Hugging Face token.
|
||||
The Embedder expects the `url` of your endpoint in `api_params`.
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import HuggingFaceAPITextEmbedder
|
||||
from haystack.utils import Secret
|
||||
|
||||
text_embedder = HuggingFaceAPITextEmbedder(
|
||||
api_type="inference_endpoints",
|
||||
api_params={"model": "BAAI/bge-small-en-v1.5"},
|
||||
token=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
|
||||
print(text_embedder.run("I love pizza!"))
|
||||
|
||||
## {'embedding': [0.017020374536514282, -0.023255806416273117, ...]}
|
||||
```
|
||||
|
||||
#### Using Self-Hosted Text Embeddings Inference (TEI)
|
||||
|
||||
[Hugging Face Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference) is a toolkit for efficiently deploying and serving text embedding models.
|
||||
|
||||
While it powers the most recent versions of Serverless Inference API and Inference Endpoints, it can be used easily on-premise through Docker.
|
||||
|
||||
For example, you can run a TEI container as follows:
|
||||
|
||||
```shell
|
||||
model=BAAI/bge-large-en-v1.5
|
||||
revision=refs/pr/5
|
||||
volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run
|
||||
|
||||
docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.2 --model-id $model --revision $revision
|
||||
```
|
||||
|
||||
For more information, refer to the [official TEI repository](https://github.com/huggingface/text-embeddings-inference).
|
||||
|
||||
The Embedder expects the `url` of your TEI instance in `api_params`.
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import HuggingFaceAPITextEmbedder
|
||||
from haystack.utils import Secret
|
||||
|
||||
text_embedder = HuggingFaceAPITextEmbedder(
|
||||
api_type="text_embeddings_inference",
|
||||
api_params={"url": "http://localhost:8080"},
|
||||
)
|
||||
|
||||
print(text_embedder.run("I love pizza!"))
|
||||
|
||||
## {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.embedders import (
|
||||
HuggingFaceAPITextEmbedder,
|
||||
HuggingFaceAPIDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = HuggingFaceAPIDocumentEmbedder(
|
||||
api_type="serverless_inference_api",
|
||||
api_params={"model": "BAAI/bge-small-en-v1.5"},
|
||||
)
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
text_embedder = HuggingFaceAPITextEmbedder(
|
||||
api_type="serverless_inference_api",
|
||||
api_params={"model": "BAAI/bge-small-en-v1.5"},
|
||||
)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", text_embedder)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin', ...)
|
||||
```
|
||||
+137
@@ -0,0 +1,137 @@
|
||||
---
|
||||
title: "JinaDocumentEmbedder"
|
||||
id: jinadocumentembedder
|
||||
slug: "/jinadocumentembedder"
|
||||
description: "This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Jina AI Embeddings models. The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents."
|
||||
---
|
||||
|
||||
# JinaDocumentEmbedder
|
||||
|
||||
This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Jina AI Embeddings models. The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Jina API key. Can be set with `JINA_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Jina](/reference/integrations-jina) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`JinaDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, you should use the [`JinaTextEmbedder`](jinatextembedder.mdx). To see the list of compatible Jina Embeddings models, head to Jina AI’s [website](https://jina.ai/embeddings/). The default model for `JinaDocumentEmbedder` is `jina-embeddings-v2-base-en`.
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install jina-haystack
|
||||
```
|
||||
|
||||
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
|
||||
|
||||
```python
|
||||
embedder = JinaDocumentEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
To get a Jina Embeddings API key, head to https://jina.ai/embeddings/.
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this easily by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.jina import JinaDocumentEmbedder
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = JinaDocumentEmbedder(
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
meta_fields_to_embed=["title"],
|
||||
)
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.jina import JinaDocumentEmbedder
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = JinaDocumentEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
:::info
|
||||
We recommend setting JINA_API_KEY as an environment variable instead of setting it as a parameter.
|
||||
:::
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.jina import JinaDocumentEmbedder
|
||||
from haystack_integrations.components.embedders.jina import JinaTextEmbedder
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component(
|
||||
"embedder",
|
||||
JinaDocumentEmbedder(api_key=Secret.from_token("<your-api-key>")),
|
||||
)
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"text_embedder",
|
||||
JinaTextEmbedder(api_key=Secret.from_token("<your-api-key>")),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Using the Jina-embeddings-v2-base-en model in a Haystack RAG pipeline for legal document analysis](https://haystack.deepset.ai/cookbook/jina-embeddings-v2-legal-analysis-rag)
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
---
|
||||
title: "JinaDocumentImageEmbedder"
|
||||
id: jinadocumentimageembedder
|
||||
slug: "/jinadocumentimageembedder"
|
||||
description: "`JinaDocumentImageEmbedder` computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Jina embedding models with the ability to embed text and images into the same vector space."
|
||||
---
|
||||
|
||||
# JinaDocumentImageEmbedder
|
||||
|
||||
`JinaDocumentImageEmbedder` computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Jina embedding models with the ability to embed text and images into the same vector space.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Jina API key. Can be set with `JINA_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
|
||||
| **API reference** | [Jina](/reference/integrations-jina) |
|
||||
| **GitHub link** | [https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cohere) |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`JinaDocumentImageEmbedder` expects a list of documents containing an image or a PDF file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
|
||||
|
||||
The embedder efficiently loads the images, computes the embeddings using a Jina model, and stores each of them in the `embedding` field of the document.
|
||||
|
||||
`JinaDocumentImageEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with a `JinaTextEmbedder` to embed the query, before using an Embedding Retriever.
|
||||
|
||||
This component is compatible with Jina multimodal embedding models:
|
||||
|
||||
- `jina-clip-v1`
|
||||
- `jina-clip-v2` (default)
|
||||
- `jina-embeddings-v4` (non-commercial research only)
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install jina-haystack
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with a [Secret](../../concepts/secret-management.mdx) and `Secret.from_token` method:
|
||||
|
||||
```python
|
||||
embedder = JinaDocumentImageEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
To get a Cohere API key, head over to https://jina.ai/embeddings/.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Remember to set `JINA_API_KEY` as an environment variable first.
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.jina import JinaDocumentImageEmbedder
|
||||
|
||||
embedder = JinaDocumentImageEmbedder(model="jina-clip-v2")
|
||||
embedder.warm_up()
|
||||
|
||||
documents = [
|
||||
Document(content="A photo of a cat", meta={"file_path": "cat.jpg"}),
|
||||
Document(content="A photo of a dog", meta={"file_path": "dog.jpg"}),
|
||||
]
|
||||
|
||||
result = embedder.run(documents=documents)
|
||||
documents_with_embeddings = result["documents"]
|
||||
print(documents_with_embeddings)
|
||||
|
||||
## [Document(id=...,
|
||||
## content='A photo of a cat',
|
||||
## meta={'file_path': 'cat.jpg',
|
||||
## 'embedding_source': {'type': 'image', 'file_path_meta_field': 'file_path'}},
|
||||
## embedding=vector of size 1024),
|
||||
## ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
In this example, we can see an indexing pipeline with 3 components:
|
||||
|
||||
- `ImageFileToDocument` Converter that creates empty documents with a reference to an image in the `meta.file_path` field.
|
||||
- `JinaDocumentImageEmbedder` that loads the images, computes embeddings and store them in documents. Here, we set the `image_size` parameter to resize the image to fit within the specified dimensions while maintaining aspect ratio. This reduces API usage.
|
||||
- `DocumentWriter` that writes the documents in the `InMemoryDocumentStore`.
|
||||
|
||||
There is also a multimodal retrieval pipeline, composed of a `JinaTextEmbedder` (using the same model as before) and an `InMemoryEmbeddingRetriever`.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters.image import ImageFileToDocument
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
from haystack_integrations.components.embedders.jina import (
|
||||
JinaDocumentImageEmbedder,
|
||||
JinaTextEmbedder,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
## Indexing pipeline
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("image_converter", ImageFileToDocument())
|
||||
indexing_pipeline.add_component(
|
||||
"embedder",
|
||||
JinaDocumentImageEmbedder(model="jina-clip-v2", image_size=(200, 200)),
|
||||
)
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("image_converter", "embedder")
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run(data={"image_converter": {"sources": ["dog.jpg", "cat.jpg"]}})
|
||||
|
||||
## Multimodal retrieval pipeline
|
||||
retrieval_pipeline = Pipeline()
|
||||
retrieval_pipeline.add_component("embedder", JinaTextEmbedder(model="jina-clip-v2"))
|
||||
retrieval_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store, top_k=2),
|
||||
)
|
||||
retrieval_pipeline.connect("embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
result = retrieval_pipeline.run(data={"text": "man's best friend"})
|
||||
print(result)
|
||||
|
||||
## {
|
||||
## 'retriever': {
|
||||
## 'documents': [
|
||||
## Document(
|
||||
## id=0c96...,
|
||||
## meta={
|
||||
## 'file_path': 'dog.jpg',
|
||||
## 'embedding_source': {
|
||||
## 'type': 'image',
|
||||
## 'file_path_meta_field': 'file_path'
|
||||
## }
|
||||
## },
|
||||
## score=0.246
|
||||
## ),
|
||||
## Document(
|
||||
## id=5e76...,
|
||||
## meta={
|
||||
## 'file_path': 'cat.jpg',
|
||||
## 'embedding_source': {
|
||||
## 'type': 'image',
|
||||
## 'file_path_meta_field': 'file_path'
|
||||
## }
|
||||
## },
|
||||
## score=0.199
|
||||
## )
|
||||
## ]
|
||||
## }
|
||||
## }
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
:notebook: Tutorial: [Creating Vision+Text RAG Pipelines](https://haystack.deepset.ai/tutorials/46_multimodal_rag)
|
||||
+112
@@ -0,0 +1,112 @@
|
||||
---
|
||||
title: "JinaTextEmbedder"
|
||||
id: jinatextembedder
|
||||
slug: "/jinatextembedder"
|
||||
description: "This component transforms a string into a vector that captures its semantics using a Jina Embeddings model. When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents."
|
||||
---
|
||||
|
||||
# JinaTextEmbedder
|
||||
|
||||
This component transforms a string into a vector that captures its semantics using a Jina Embeddings model. When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Jina API key. Can be set with `JINA_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Jina](/reference/integrations-jina) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/jina |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`JinaTextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the use the [`JinaDocumentEmbedder`](jinadocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector. To see the list of compatible Jina Embeddings models, head to Jina AI’s [website](https://jina.ai/embeddings/). The default model for `JinaTextEmbedder` is `jina-embeddings-v2-base-en`.
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install jina-haystack
|
||||
```
|
||||
|
||||
The component uses a `JINA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
|
||||
|
||||
```python
|
||||
embedder = JinaTextEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
To get a Jina Embeddings API key, head to https://jina.ai/embeddings/.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.jina import JinaTextEmbedder
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = JinaTextEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
|
||||
## {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
|
||||
## 'meta': {'model': 'text-embedding-ada-002-v2',
|
||||
## 'usage': {'prompt_tokens': 4, 'total_tokens': 4}}}
|
||||
```
|
||||
|
||||
:::info
|
||||
We recommend setting JINA_API_KEY as an environment variable instead of setting it as a parameter.
|
||||
:::
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.jina import JinaDocumentEmbedder
|
||||
from haystack_integrations.components.embedders.jina import JinaTextEmbedder
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = JinaDocumentEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"text_embedder",
|
||||
JinaTextEmbedder(api_key=Secret.from_token("<your-api-key>")),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Using the Jina-embeddings-v2-base-en model in a Haystack RAG pipeline for legal document analysis](https://haystack.deepset.ai/cookbook/jina-embeddings-v2-legal-analysis-rag)
|
||||
+110
@@ -0,0 +1,110 @@
|
||||
---
|
||||
title: "MistralDocumentEmbedder"
|
||||
id: mistraldocumentembedder
|
||||
slug: "/mistraldocumentembedder"
|
||||
description: "This component computes the embeddings of a list of documents using the Mistral API and models."
|
||||
---
|
||||
|
||||
# MistralDocumentEmbedder
|
||||
|
||||
This component computes the embeddings of a list of documents using the Mistral API and models.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Mistral API key. Can be set with `MISTRAL_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata strings |
|
||||
| **API reference** | [Mistral](/reference/integrations-mistral) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral |
|
||||
|
||||
</div>
|
||||
|
||||
This component should be used to embed a list of Documents. To embed a string, use the [`MistralTextEmbedder`](mistraltextembedder.mdx).
|
||||
|
||||
## Overview
|
||||
|
||||
`MistralDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses the Mistral API and its embedding models.
|
||||
|
||||
The component currently supports the `mistral-embed` embedding model. The list of all supported models can be found in Mistral’s [embedding models documentation](https://docs.mistral.ai/platform/endpoints/#embedding-models).
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install mistral-haystack
|
||||
```
|
||||
|
||||
`MistralDocumentEmbedder` needs a Mistral API key to work. It uses an `MISTRAL_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
|
||||
|
||||
```python
|
||||
embedder = MistralDocumentEmbedder(
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
model="mistral-embed",
|
||||
)
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Remember first to set the`MISTRAL_API_KEY` as an environment variable or pass it in directly.
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.mistral.document_embedder import (
|
||||
MistralDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
embedder = MistralDocumentEmbedder(
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
model="mistral-embed",
|
||||
)
|
||||
|
||||
result = embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
## [-0.453125, 1.2236328, 2.0058594, 0.67871094...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Below is an example of the `MistralDocumentEmbedder` in an indexing pipeline. We are indexing the contents of a webpage into an `InMemoryDocumentStore`.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters import HTMLToDocument
|
||||
from haystack.components.fetchers import LinkContentFetcher
|
||||
from haystack.components.preprocessors import DocumentSplitter
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.mistral.document_embedder import (
|
||||
MistralDocumentEmbedder,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
fetcher = LinkContentFetcher()
|
||||
converter = HTMLToDocument()
|
||||
chunker = DocumentSplitter()
|
||||
embedder = MistralDocumentEmbedder()
|
||||
writer = DocumentWriter(document_store=document_store)
|
||||
|
||||
indexing = Pipeline()
|
||||
|
||||
indexing.add_component(name="fetcher", instance=fetcher)
|
||||
indexing.add_component(name="converter", instance=converter)
|
||||
indexing.add_component(name="chunker", instance=chunker)
|
||||
indexing.add_component(name="embedder", instance=embedder)
|
||||
indexing.add_component(name="writer", instance=writer)
|
||||
|
||||
indexing.connect("fetcher", "converter")
|
||||
indexing.connect("converter", "chunker")
|
||||
indexing.connect("chunker", "embedder")
|
||||
indexing.connect("embedder", "writer")
|
||||
|
||||
indexing.run(data={"fetcher": {"urls": ["https://mistral.ai/news/la-plateforme/"]}})
|
||||
```
|
||||
+169
@@ -0,0 +1,169 @@
|
||||
---
|
||||
title: "MistralTextEmbedder"
|
||||
id: mistraltextembedder
|
||||
slug: "/mistraltextembedder"
|
||||
description: "This component transforms a string into a vector using the Mistral API and models. Use it for embedding retrieval to transform your query into an embedding."
|
||||
---
|
||||
|
||||
# MistralTextEmbedder
|
||||
|
||||
This component transforms a string into a vector using the Mistral API and models. Use it for embedding retrieval to transform your query into an embedding.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The Mistral API key. Can be set with `MISTRAL_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers (vectors) <br /> <br />`meta`: A dictionary of metadata strings |
|
||||
| **API reference** | [Mistral](/reference/integrations-mistral) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mistral |
|
||||
|
||||
</div>
|
||||
|
||||
Use `MistalTextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`MistralDocumentEmbedder`](mistraldocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
|
||||
|
||||
## Overview
|
||||
|
||||
`MistralTextEmbedder` transforms a string into a vector that captures its semantics using a Mistral embedding model.
|
||||
|
||||
The component currently supports the `mistral-embed` embedding model. The list of all supported models can be found in Mistral’s [embedding models documentation](https://docs.mistral.ai/platform/endpoints/#embedding-models).
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install mistral-haystack
|
||||
```
|
||||
|
||||
`MistralTextEmbedder` needs a Mistral API key to work. It uses a `MISTRAL_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
|
||||
|
||||
```python
|
||||
embedder = MistralTextEmbedder(
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
model="mistral-embed",
|
||||
)
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Remember to set the`MISTRAL_API_KEY` as an environment variable first or pass it in directly.
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
|
||||
from haystack_integrations.components.embedders.mistral.text_embedder import (
|
||||
MistralTextEmbedder,
|
||||
)
|
||||
|
||||
embedder = MistralTextEmbedder(
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
model="mistral-embed",
|
||||
)
|
||||
|
||||
result = embedder.run(text="How can I ise the Mistral embedding models with Haystack?")
|
||||
|
||||
print(result["embedding"])
|
||||
## [-0.0015687942504882812, 0.052154541015625, 0.037109375...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Below is an example of the `MistralTextEmbedder` in a document search pipeline. We are building this pipeline on top of an `InMemoryDocumentStore` where we index the contents of two URLs.
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.utils import Secret
|
||||
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
|
||||
from haystack.components.fetchers import LinkContentFetcher
|
||||
from haystack.components.converters import HTMLToDocument
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.mistral.document_embedder import (
|
||||
MistralDocumentEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.mistral.text_embedder import (
|
||||
MistralTextEmbedder,
|
||||
)
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
## Initialize document store
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
## Indexing components
|
||||
fetcher = LinkContentFetcher()
|
||||
converter = HTMLToDocument()
|
||||
embedder = MistralDocumentEmbedder()
|
||||
writer = DocumentWriter(document_store=document_store)
|
||||
|
||||
indexing = Pipeline()
|
||||
indexing.add_component(name="fetcher", instance=fetcher)
|
||||
indexing.add_component(name="converter", instance=converter)
|
||||
indexing.add_component(name="embedder", instance=embedder)
|
||||
indexing.add_component(name="writer", instance=writer)
|
||||
|
||||
indexing.connect("fetcher", "converter")
|
||||
indexing.connect("converter", "embedder")
|
||||
indexing.connect("embedder", "writer")
|
||||
|
||||
indexing.run(
|
||||
data={
|
||||
"fetcher": {
|
||||
"urls": [
|
||||
"https://docs.mistral.ai/self-deployment/cloudflare/",
|
||||
"https://docs.mistral.ai/platform/endpoints/",
|
||||
],
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
## Retrieval components
|
||||
text_embedder = MistralTextEmbedder()
|
||||
retriever = InMemoryEmbeddingRetriever(document_store=document_store)
|
||||
|
||||
## Define prompt template
|
||||
prompt_template = [
|
||||
ChatMessage.from_system("You are a helpful assistant."),
|
||||
ChatMessage.from_user(
|
||||
"Given the retrieved documents, answer the question.\nDocuments:\n"
|
||||
"{% for document in documents %}{{ document.content }}{% endfor %}\n"
|
||||
"Question: {{ query }}\nAnswer:",
|
||||
),
|
||||
]
|
||||
|
||||
prompt_builder = ChatPromptBuilder(
|
||||
template=prompt_template,
|
||||
required_variables={"query", "documents"},
|
||||
)
|
||||
llm = OpenAIChatGenerator(
|
||||
model="gpt-4o-mini",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
|
||||
doc_search = Pipeline()
|
||||
doc_search.add_component("text_embedder", text_embedder)
|
||||
doc_search.add_component("retriever", retriever)
|
||||
doc_search.add_component("prompt_builder", prompt_builder)
|
||||
doc_search.add_component("llm", llm)
|
||||
|
||||
doc_search.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
doc_search.connect("retriever.documents", "prompt_builder.documents")
|
||||
doc_search.connect("prompt_builder.messages", "llm.messages")
|
||||
|
||||
query = "How can I deploy Mistral models with Cloudflare?"
|
||||
|
||||
result = doc_search.run(
|
||||
{
|
||||
"text_embedder": {"text": query},
|
||||
"retriever": {"top_k": 1},
|
||||
"prompt_builder": {"query": query},
|
||||
},
|
||||
)
|
||||
|
||||
print(result["llm"]["replies"])
|
||||
```
|
||||
+141
@@ -0,0 +1,141 @@
|
||||
---
|
||||
title: "NvidiaDocumentEmbedder"
|
||||
id: nvidiadocumentembedder
|
||||
slug: "/nvidiadocumentembedder"
|
||||
description: "This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document."
|
||||
---
|
||||
|
||||
# NvidiaDocumentEmbedder
|
||||
|
||||
This component computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Nvidia](/reference/integrations-nvidia) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`NvidiaDocumentEmbedder` enriches the metadata of documents with an embedding of their content.
|
||||
|
||||
It can be used with self-hosted models with NVIDIA NIM or models hosted on the [NVIDIA API catalog](https://build.nvidia.com/explore/discover).
|
||||
|
||||
To embed a string, use the [`NvidiaTextEmbedder`](nvidiatextembedder.mdx).
|
||||
|
||||
## Usage
|
||||
|
||||
To start using `NvidiaDocumentEmbedder`, first, install the `nvidia-haystack` package:
|
||||
|
||||
```shell
|
||||
pip install nvidia-haystack
|
||||
```
|
||||
|
||||
You can use the `NvidiaDocumentEmbedder` with all the embedder models available on the [NVIDIA API catalog](https://docs.api.nvidia.com/nim/reference) or using a model deployed with NVIDIA NIM. Follow the [Deploying Text Embedding Models](https://developer.nvidia.com/docs/nemo-microservices/embedding/source/deploy.html) guide to learn how to deploy the model you want on your infrastructure.
|
||||
|
||||
### On its own
|
||||
|
||||
To use LLMs from the NVIDIA API catalog, you need to specify the correct `api_url` and your API key. You can get your API key directly from the [catalog website](https://build.nvidia.com/explore/discover).
|
||||
|
||||
The `NvidiaDocumentEmbedder` needs an Nvidia API key to work. It uses the `NVIDIA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`, as in the following example.
|
||||
|
||||
```python
|
||||
from haystack.utils.auth import Secret
|
||||
from haystack_integrations.components.embedders.nvidia import NvidiaDocumentEmbedder
|
||||
|
||||
embedder = NvidiaDocumentEmbedder(
|
||||
model="nvidia/nv-embedqa-e5-v5",
|
||||
api_url="https://integrate.api.nvidia.com/v1",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
embedder.warm_up()
|
||||
|
||||
result = embedder.run("A transformer is a deep learning architecture")
|
||||
print(result["embedding"])
|
||||
print(result["meta"])
|
||||
```
|
||||
|
||||
To use a locally deployed model, you need to set the `api_url` to your localhost and unset your `api_key`.
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.nvidia import NvidiaDocumentEmbedder
|
||||
|
||||
embedder = NvidiaDocumentEmbedder(
|
||||
model="nvidia/nv-embedqa-e5-v5",
|
||||
api_url="http://0.0.0.0:9999/v1",
|
||||
api_key=None,
|
||||
)
|
||||
embedder.warm_up()
|
||||
|
||||
result = embedder.run("A transformer is a deep learning architecture")
|
||||
print(result["embedding"])
|
||||
print(result["meta"])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Here's an example of a RAG pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline, Document
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.nvidia import (
|
||||
NvidiaTextEmbedder,
|
||||
NvidiaDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component(
|
||||
"embedder",
|
||||
NvidiaDocumentEmbedder(
|
||||
model="nvidia/nv-embedqa-e5-v5",
|
||||
api_url="https://integrate.api.nvidia.com/v1",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
),
|
||||
)
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"text_embedder",
|
||||
NvidiaTextEmbedder(
|
||||
model="nvidia/nv-embedqa-e5-v5",
|
||||
api_url="https://integrate.api.nvidia.com/v1",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Haystack RAG Pipeline with Self-Deployed AI models using NVIDIA NIMs](https://haystack.deepset.ai/cookbook/rag-with-nims)
|
||||
+141
@@ -0,0 +1,141 @@
|
||||
---
|
||||
title: "NvidiaTextEmbedder"
|
||||
id: nvidiatextembedder
|
||||
slug: "/nvidiatextembedder"
|
||||
description: "This component transforms a string into a vector that captures its semantics using Nvidia-hosted models."
|
||||
---
|
||||
|
||||
# NvidiaTextEmbedder
|
||||
|
||||
This component transforms a string into a vector that captures its semantics using Nvidia-hosted models.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_key`: API key for the NVIDIA NIM. Can be set with `NVIDIA_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers (vectors) <br /> <br />`meta`: A dictionary of metadata strings |
|
||||
| **API reference** | [Nvidia](/reference/integrations-nvidia) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/nvidia |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`NvidiaTextEmbedder` embeds a simple string (such as a query) into a vector.
|
||||
|
||||
It can be used with self-hosted models with NVIDIA NIM or models hosted on the [NVIDIA API catalog](https://build.nvidia.com/explore/discover).
|
||||
|
||||
To embed a list of documents, use the [`NvidiaDocumentEmbedder`](nvidiadocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
|
||||
|
||||
## Usage
|
||||
|
||||
To start using `NvidiaTextEmbedder`, first, install the `nvidia-haystack` package:
|
||||
|
||||
```shell
|
||||
pip install nvidia-haystack
|
||||
```
|
||||
|
||||
You can use the `NvidiaTextEmbedder` with all the embedder models available on the [NVIDIA API catalog](https://docs.api.nvidia.com/nim/reference) or using a model deployed with NVIDIA NIM. Follow the [Deploying Text Embedding Models](https://developer.nvidia.com/docs/nemo-microservices/embedding/source/deploy.html) guide to learn how to deploy the model you want on your infrastructure.
|
||||
|
||||
### On its own
|
||||
|
||||
To use LLMs from the NVIDIA API catalog, you need to specify the correct `api_url` and your API key. You can get your API key directly from the [catalog website](https://build.nvidia.com/explore/discover).
|
||||
|
||||
The `NvidiaTextEmbedder` needs an Nvidia API key to work. It uses the `NVIDIA_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`, as in the following example.
|
||||
|
||||
```python
|
||||
from haystack.utils.auth import Secret
|
||||
from haystack_integrations.components.embedders.nvidia import NvidiaTextEmbedder
|
||||
|
||||
embedder = NvidiaTextEmbedder(
|
||||
model="nvidia/nv-embedqa-e5-v5",
|
||||
api_url="https://integrate.api.nvidia.com/v1",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
embedder.warm_up()
|
||||
|
||||
result = embedder.run("A transformer is a deep learning architecture")
|
||||
print(result["embedding"])
|
||||
print(result["meta"])
|
||||
```
|
||||
|
||||
To use a locally deployed model, you need to set the `api_url` to your localhost and unset your `api_key`.
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.nvidia import NvidiaTextEmbedder
|
||||
|
||||
embedder = NvidiaTextEmbedder(
|
||||
model="nvidia/nv-embedqa-e5-v5",
|
||||
api_url="http://0.0.0.0:9999/v1",
|
||||
api_key=None,
|
||||
)
|
||||
embedder.warm_up()
|
||||
|
||||
result = embedder.run("A transformer is a deep learning architecture")
|
||||
print(result["embedding"])
|
||||
print(result["meta"])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Here's an example of a RAG pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline, Document
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.nvidia import (
|
||||
NvidiaTextEmbedder,
|
||||
NvidiaDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component(
|
||||
"embedder",
|
||||
NvidiaDocumentEmbedder(
|
||||
model="nvidia/nv-embedqa-e5-v5",
|
||||
api_url="https://integrate.api.nvidia.com/v1",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
),
|
||||
)
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"text_embedder",
|
||||
NvidiaTextEmbedder(
|
||||
model="nvidia/nv-embedqa-e5-v5",
|
||||
api_url="https://integrate.api.nvidia.com/v1",
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Haystack RAG Pipeline with Self-Deployed AI models using NVIDIA NIMs](https://haystack.deepset.ai/cookbook/rag-with-nims)
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
---
|
||||
title: "OllamaDocumentEmbedder"
|
||||
id: ollamadocumentembedder
|
||||
slug: "/ollamadocumentembedder"
|
||||
description: "This component computes the embeddings of a list of documents using embedding models compatible with the Ollama Library."
|
||||
---
|
||||
|
||||
# OllamaDocumentEmbedder
|
||||
|
||||
This component computes the embeddings of a list of documents using embedding models compatible with the Ollama Library.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata strings |
|
||||
| **API reference** | [Ollama](/reference/integrations-ollama) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama |
|
||||
|
||||
</div>
|
||||
|
||||
`OllamaDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses embedding models compatible with the Ollama Library.
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
## Overview
|
||||
|
||||
`OllamaDocumentEmbedder` should be used to embed a list of documents. For embedding a string only, use the [`OllamaTextEmbedder`](ollamatextembedder.mdx).
|
||||
|
||||
The component uses `http://localhost:11434` as the default URL as most available setups (Mac, Linux, Docker) default to port 11434.
|
||||
|
||||
### Compatible Models
|
||||
|
||||
Unless specified otherwise while initializing this component, the default embedding model is "nomic-embed-text". See other possible pre-built models in Ollama's [library](https://ollama.ai/library). To load your own custom model, follow the [instructions](https://github.com/ollama/ollama/blob/main/docs/modelfile.md) from Ollama.
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install ollama-haystack
|
||||
```
|
||||
|
||||
Make sure that you have a running Ollama model (either through a docker container, or locally hosted). No other configuration is necessary as Ollama has the embedding API built in.
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Most embedded metadata contains information about the model name and type. You can pass [optional arguments](https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values), such as temperature, top_p, and others, to the Ollama generation endpoint.
|
||||
|
||||
The name of the model used will be automatically appended as part of the document metadata. An example payload using the nomic-embed-text model will look like this:
|
||||
|
||||
```python
|
||||
{"meta": {"model": "nomic-embed-text"}}
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.ollama import OllamaDocumentEmbedder
|
||||
|
||||
doc = Document(content="What do llamas say once you have thanked them? No probllama!")
|
||||
document_embedder = OllamaDocumentEmbedder()
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## Calculating embeddings: 100%|██████████| 1/1 [00:02<00:00, 2.82s/it]
|
||||
|
||||
## [-0.16412407159805298, -3.8359334468841553, ... ]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
|
||||
from haystack_integrations.components.embedders.ollama import OllamaDocumentEmbedder
|
||||
|
||||
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
|
||||
|
||||
from haystack.components.converters import PyPDFToDocument
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.document_stores.types import DuplicatePolicy
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
embedder = OllamaDocumentEmbedder(
|
||||
model="nomic-embed-text",
|
||||
url="http://localhost:11434",
|
||||
) # This is the default model and URL
|
||||
|
||||
cleaner = DocumentCleaner()
|
||||
splitter = DocumentSplitter()
|
||||
file_converter = PyPDFToDocument()
|
||||
writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE)
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
|
||||
## Add components to pipeline
|
||||
indexing_pipeline.add_component("embedder", embedder)
|
||||
indexing_pipeline.add_component("converter", file_converter)
|
||||
indexing_pipeline.add_component("cleaner", cleaner)
|
||||
indexing_pipeline.add_component("splitter", splitter)
|
||||
indexing_pipeline.add_component("writer", writer)
|
||||
|
||||
## Connect components in pipeline
|
||||
indexing_pipeline.connect("converter", "cleaner")
|
||||
indexing_pipeline.connect("cleaner", "splitter")
|
||||
indexing_pipeline.connect("splitter", "embedder")
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
## Run Pipeline
|
||||
indexing_pipeline.run({"converter": {"sources": ["files/test_pdf_data.pdf"]}})
|
||||
|
||||
## Calculating embeddings: 100%|██████████| 115/115
|
||||
## {'embedder': {'meta': {'model': 'nomic-embed-text'}}, 'writer': {'documents_written': 115}}
|
||||
```
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "OllamaTextEmbedder"
|
||||
id: ollamatextembedder
|
||||
slug: "/ollamatextembedder"
|
||||
description: "This component computes the embeddings of a string using embedding models compatible with the Ollama Library."
|
||||
---
|
||||
|
||||
# OllamaTextEmbedder
|
||||
|
||||
This component computes the embeddings of a string using embedding models compatible with the Ollama Library.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers (vectors) <br /> <br />`meta`: A dictionary of metadata strings |
|
||||
| **API reference** | [Ollama](/reference/integrations-ollama) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/ollama |
|
||||
|
||||
</div>
|
||||
|
||||
`OllamaDocumentEmbedder` computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses embedding models compatible with the Ollama Library.
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
## Overview
|
||||
|
||||
`OllamaTextEmbedder` should be used to embed a string. For embedding a list of documents, use the [`OllamaDocumentEmbedder`](ollamadocumentembedder.mdx).
|
||||
|
||||
The component uses `http://localhost:11434` as the default URL as most available setups (Mac, Linux, Docker) default to port 11434.
|
||||
|
||||
### Compatible Models
|
||||
|
||||
Unless specified otherwise while initializing this component, the default embedding model is "nomic-embed-text". See other possible pre-built models in Ollama's [library](https://ollama.ai/library). To load your own custom model, follow the [instructions](https://github.com/ollama/ollama/blob/main/docs/modelfile.md) from Ollama.
|
||||
|
||||
### Installation
|
||||
|
||||
To start using this integration with Haystack, install the package with:
|
||||
|
||||
```shell
|
||||
pip install ollama-haystack
|
||||
```
|
||||
|
||||
Make sure that you have a running Ollama model (either through a docker container, or locally hosted). No other configuration is necessary as Ollama has the embedding API built in.
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Most embedded metadata contains information about the model name and type. You can pass [optional arguments](https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values), such as temperature, top_p, and others, to the Ollama generation endpoint.
|
||||
|
||||
The name of the model used will be automatically appended as part of the metadata. An example payload using the nomic-embed-text model will look like this:
|
||||
|
||||
```python
|
||||
{"meta": {"model": "nomic-embed-text"}}
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.ollama import OllamaTextEmbedder
|
||||
|
||||
embedder = OllamaTextEmbedder()
|
||||
|
||||
result = embedder.run(
|
||||
text="What do llamas say once you have thanked them? No probllama!",
|
||||
)
|
||||
|
||||
print(result["embedding"])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from cohere_haystack.embedders.text_embedder import OllamaTextEmbedder
|
||||
from cohere_haystack.embedders.document_embedder import OllamaDocumentEmbedder
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = OllamaDocumentEmbedder()
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", OllamaTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
```
|
||||
+119
@@ -0,0 +1,119 @@
|
||||
---
|
||||
title: "OpenAIDocumentEmbedder"
|
||||
id: openaidocumentembedder
|
||||
slug: "/openaidocumentembedder"
|
||||
description: "OpenAIDocumentEmbedder computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses OpenAI embedding models."
|
||||
---
|
||||
|
||||
# OpenAIDocumentEmbedder
|
||||
|
||||
OpenAIDocumentEmbedder computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses OpenAI embedding models.
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector representing the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/openai_document_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
To see the list of compatible OpenAI embedding models, head over to OpenAI [documentation](https://platform.openai.com/docs/guides/embeddings/embedding-models). The default model for `OpenAIDocumentEmbedder` is `text-embedding-ada-002`. You can specify another model with the `model` parameter when initializing this component.
|
||||
|
||||
This component should be used to embed a list of documents. To embed a string, use the [OpenAITextEmbedder](openaitextembedder.mdx).
|
||||
|
||||
The component uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
|
||||
|
||||
```
|
||||
embedder = OpenAIDocumentEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this easily by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.embedders import OpenAIDocumentEmbedder
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = OpenAIDocumentEmbedder(meta_fields_to_embed=["title"])
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import OpenAIDocumentEmbedder
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = OpenAIDocumentEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
:::info
|
||||
We recommend setting OPENAI_API_KEY as an environment variable instead of setting it as a parameter.
|
||||
:::
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.embedders import OpenAITextEmbedder, OpenAIDocumentEmbedder
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("embedder", OpenAIDocumentEmbedder())
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", OpenAITextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+100
@@ -0,0 +1,100 @@
|
||||
---
|
||||
title: "OpenAITextEmbedder"
|
||||
id: openaitextembedder
|
||||
slug: "/openaitextembedder"
|
||||
description: "OpenAITextEmbedder transforms a string into a vector that captures its semantics using an OpenAI embedding model."
|
||||
---
|
||||
|
||||
# OpenAITextEmbedder
|
||||
|
||||
OpenAITextEmbedder transforms a string into a vector that captures its semantics using an OpenAI embedding model.
|
||||
|
||||
When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_key`: An OpenAI API key. Can be set with `OPENAI_API_KEY` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/openai_text_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
To see the list of compatible OpenAI embedding models, head over to OpenAI [documentation](https://platform.openai.com/docs/guides/embeddings/embedding-models). The default model for `OpenAITextEmbedder` is `text-embedding-ada-002`. You can specify another model with the `model` parameter when initializing this component.
|
||||
|
||||
Use `OpenAITextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [OpenAIDocumentEmbedder](openaidocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
|
||||
|
||||
The component uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`:
|
||||
|
||||
```python
|
||||
embedder = OpenAITextEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import OpenAITextEmbedder
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = OpenAITextEmbedder(api_key=Secret.from_token("<your-api-key>"))
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
|
||||
## {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
|
||||
## 'meta': {'model': 'text-embedding-ada-002-v2',
|
||||
## 'usage': {'prompt_tokens': 4, 'total_tokens': 4}}}
|
||||
```
|
||||
|
||||
:::info
|
||||
We recommend setting OPENAI_API_KEY as an environment variable instead of setting it as a parameter.
|
||||
:::
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.embedders import OpenAITextEmbedder, OpenAIDocumentEmbedder
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = OpenAIDocumentEmbedder()
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", OpenAITextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "OptimumDocumentEmbedder"
|
||||
id: optimumdocumentembedder
|
||||
slug: "/optimumdocumentembedder"
|
||||
description: "A component to compute documents’ embeddings using models loaded with the Hugging Face Optimum library."
|
||||
---
|
||||
|
||||
# OptimumDocumentEmbedder
|
||||
|
||||
A component to compute documents’ embeddings using models loaded with the Hugging Face Optimum library.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents enriched with embeddings |
|
||||
| **API reference** | [Optimum](/reference/integrations-optimum) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/optimum |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`OptimumDocumentEmbedder` embeds text strings using models loaded with the [HuggingFace Optimum](https://huggingface.co/docs/optimum/index) library. It uses the [ONNX runtime](https://onnxruntime.ai/) for high-speed inference.
|
||||
|
||||
The default model is `sentence-transformers/all-mpnet-base-v2`.
|
||||
|
||||
Similarly to other Embedders, this component allows adding prefixes (and suffixes) to include instructions. For more details, refer to the component’s API reference.
|
||||
|
||||
There are three useful parameters specific to the Optimum Embedder that you can control with various modes:
|
||||
|
||||
- [Pooling](/reference/integrations-optimum#optimumembedderpooling): generate a fixed-sized sentence embedding from a variable-sized sentence embedding
|
||||
- [Optimization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/optimization): apply graph optimization to the model and improve inference speed
|
||||
- [Quantization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/quantization): reduce the computational and memory costs
|
||||
|
||||
Find all the available mode details in our Optimum [API Reference](/reference/integrations-optimum).
|
||||
|
||||
### Authentication
|
||||
|
||||
Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
|
||||
|
||||
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
|
||||
|
||||
## Usage
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install optimum-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.dataclasses import Document
|
||||
from haystack_integrations.components.embedders.optimum import OptimumDocumentEmbedder
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = OptimumDocumentEmbedder(
|
||||
model="sentence-transformers/all-mpnet-base-v2",
|
||||
)
|
||||
document_embedder.warm_up()
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [0.017020374536514282, -0.023255806416273117, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.optimum import (
|
||||
OptimumDocumentEmbedder,
|
||||
OptimumEmbedderPooling,
|
||||
OptimumEmbedderOptimizationConfig,
|
||||
OptimumEmbedderOptimizationMode,
|
||||
)
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
embedder = OptimumDocumentEmbedder(
|
||||
model="intfloat/e5-base-v2",
|
||||
normalize_embeddings=True,
|
||||
onnx_execution_provider="CUDAExecutionProvider",
|
||||
optimizer_settings=OptimumEmbedderOptimizationConfig(
|
||||
mode=OptimumEmbedderOptimizationMode.O4,
|
||||
for_gpu=True,
|
||||
),
|
||||
working_dir="/tmp/optimum",
|
||||
pooling_mode=OptimumEmbedderPooling.MEAN,
|
||||
)
|
||||
|
||||
pipeline = Pipeline()
|
||||
pipeline.add_component("embedder", embedder)
|
||||
|
||||
pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
print(results["embedder"]["embedding"])
|
||||
```
|
||||
+106
@@ -0,0 +1,106 @@
|
||||
---
|
||||
title: "OptimumTextEmbedder"
|
||||
id: optimumtextembedder
|
||||
slug: "/optimumtextembedder"
|
||||
description: "A component to embed text using models loaded with the Hugging Face Optimum library."
|
||||
---
|
||||
|
||||
# OptimumTextEmbedder
|
||||
|
||||
A component to embed text using models loaded with the Hugging Face Optimum library.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers (vectors) |
|
||||
| **API reference** | [Optimum](/reference/integrations-optimum) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/optimum |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`OptimumTextEmbedder` embeds text strings using models loaded with the [HuggingFace Optimum](https://huggingface.co/docs/optimum/index) library. It uses the [ONNX runtime](https://onnxruntime.ai/) for high-speed inference.
|
||||
|
||||
The default model is `sentence-transformers/all-mpnet-base-v2`.
|
||||
|
||||
Similarly to other Embedders, this component allows adding prefixes (and suffixes) to include instructions. For more details, refer to the component’s API reference.
|
||||
|
||||
There are three useful parameters specific to the Optimum Embedder that you can control with various modes:
|
||||
|
||||
- [Pooling](/reference/integrations-optimum#optimumembedderpooling): generate a fixed-sized sentence embedding from a variable-sized sentence embedding
|
||||
- [Optimization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/optimization): apply graph optimization to the model and improve inference speed
|
||||
- [Quantization](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/quantization): reduce the computational and memory costs
|
||||
|
||||
Find all the available mode details in our Optimum [API Reference](/reference/integrations-optimum).
|
||||
|
||||
### Authentication
|
||||
|
||||
Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
|
||||
|
||||
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
|
||||
|
||||
## Usage
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install optimum-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.optimum import OptimumTextEmbedder
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = OptimumTextEmbedder(model="sentence-transformers/all-mpnet-base-v2")
|
||||
text_embedder.warm_up()
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
|
||||
## {'embedding': [-0.07804739475250244, 0.1498992145061493,, ...]}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Note that this example requires GPU support to execute.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
|
||||
from haystack_integrations.components.embedders.optimum import (
|
||||
OptimumTextEmbedder,
|
||||
OptimumEmbedderPooling,
|
||||
OptimumEmbedderOptimizationConfig,
|
||||
OptimumEmbedderOptimizationMode,
|
||||
)
|
||||
|
||||
pipeline = Pipeline()
|
||||
embedder = OptimumTextEmbedder(
|
||||
model="intfloat/e5-base-v2",
|
||||
normalize_embeddings=True,
|
||||
onnx_execution_provider="CUDAExecutionProvider",
|
||||
optimizer_settings=OptimumEmbedderOptimizationConfig(
|
||||
mode=OptimumEmbedderOptimizationMode.O4,
|
||||
for_gpu=True,
|
||||
),
|
||||
working_dir="/tmp/optimum",
|
||||
pooling_mode=OptimumEmbedderPooling.MEAN,
|
||||
)
|
||||
pipeline.add_component("embedder", embedder)
|
||||
|
||||
results = pipeline.run(
|
||||
{
|
||||
"embedder": {
|
||||
"text": "Ex profunditate antique doctrinae, Ad caelos supra semper, Hoc incantamentum evoco, draco apparet, Incantamentum iam transactum est",
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
print(results["embedder"]["embedding"])
|
||||
```
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
---
|
||||
title: "SentenceTransformersDocumentEmbedder"
|
||||
id: sentencetransformersdocumentembedder
|
||||
slug: "/sentencetransformersdocumentembedder"
|
||||
description: "SentenceTransformersDocumentEmbedder computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses embedding models compatible with the Sentence Transformers library."
|
||||
---
|
||||
|
||||
# SentenceTransformersDocumentEmbedder
|
||||
|
||||
SentenceTransformersDocumentEmbedder computes the embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses embedding models compatible with the Sentence Transformers library.
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/sentence_transformers_document_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`SentenceTransformersDocumentEmbedder` should be used to embed a list of documents. To embed a string, use the [SentenceTransformersTextEmbedder](sentencetransformerstextembedder.mdx).
|
||||
|
||||
### Authentication
|
||||
|
||||
Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
|
||||
|
||||
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
|
||||
|
||||
```python
|
||||
document_embedder = SentenceTransformersDocumentEmbedder(
|
||||
token=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
```
|
||||
|
||||
### Compatible Models
|
||||
|
||||
The default embedding model is [\`sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)\`. You can specify another model with the `model` parameter when initializing this component.
|
||||
|
||||
See the original models in the Sentence Transformers [documentation](https://www.sbert.net/docs/pretrained_models.html).
|
||||
|
||||
Nowadays, most of the models in the [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) are compatible with Sentence Transformers.
|
||||
You can look for compatibility in the model card: [an example related to BGE models](https://huggingface.co/BAAI/bge-large-en-v1.5#using-sentence-transformers).
|
||||
|
||||
### Instructions
|
||||
|
||||
Some recent models that you can find in MTEB require prepending the text with an instruction to work better for retrieval.
|
||||
For example, if you use [intfloat/e5-large-v2](https://huggingface.co/BAAI/bge-large-en-v1.5#model-list), you should prefix your document with the following instruction: “passage:”
|
||||
|
||||
This is how it works with `SentenceTransformersDocumentEmbedder`:
|
||||
|
||||
```python
|
||||
embedder = SentenceTransformersDocumentEmbedder(
|
||||
model="intfloat/e5-large-v2",
|
||||
prefix="passage",
|
||||
)
|
||||
```
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this easily by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = SentenceTransformersDocumentEmbedder(meta_fields_to_embed=["title"])
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
doc_embedder = SentenceTransformersDocumentEmbedder()
|
||||
doc_embedder.warm_up()
|
||||
|
||||
result = doc_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [-0.07804739475250244, 0.1498992145061493, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.embedders import (
|
||||
SentenceTransformersTextEmbedder,
|
||||
SentenceTransformersDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("embedder", SentenceTransformersDocumentEmbedder())
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
indexing_pipeline.run({"documents": documents})
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
---
|
||||
title: "SentenceTransformersDocumentImageEmbedder"
|
||||
id: sentencetransformersdocumentimageembedder
|
||||
slug: "/sentencetransformersdocumentimageembedder"
|
||||
description: "`SentenceTransformersDocumentImageEmbedder` computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Sentence Transformers embedding models with the ability to embed text and images into the same vector space."
|
||||
---
|
||||
|
||||
# SentenceTransformersDocumentImageEmbedder
|
||||
|
||||
`SentenceTransformersDocumentImageEmbedder` computes the image embeddings of a list of documents and stores the obtained vectors in the embedding field of each document. It uses Sentence Transformers embedding models with the ability to embed text and images into the same vector space.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `token` (only for private models): The Hugging Face API token. Can be set with `HF_API_TOKEN` or `HF_TOKEN` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents, with a meta field containing an image file path |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/image/sentence_transformers_doc_image_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`SentenceTransformersDocumentImageEmbedder` expects a list of documents containing an image or a PDF file path in a meta field. The meta field can be specified with the `file_path_meta_field` init parameter of this component.
|
||||
|
||||
The embedder efficiently loads the images, computes the embeddings using a Sentence Transformers models, and stores each of them in the `embedding` field of the document.
|
||||
|
||||
`SentenceTransformersDocumentImageEmbedder` is commonly used in indexing pipelines. At retrieval time, you need to use the same model with a `SentenceTransformersTextEmbedder` to embed the query before using an Embedding Retriever.
|
||||
|
||||
You can set the `device` parameter to use HF models on your CPU or GPU.
|
||||
|
||||
Additionally, you can select the backend to use for the Sentence Transformers mode with the `backend` parameterl: `torch` (default), `onnx`, or `openvino`. ONNX and OpenVINO allow specific speed optimizations; for more information, read the [Sentence Transformers documentation](https://sbert.net/docs/sentence_transformer/usage/efficiency.html).
|
||||
|
||||
### Authentication
|
||||
|
||||
Authentication with a Hugging Face API Token is only required to access private or gated models.
|
||||
|
||||
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
|
||||
|
||||
### Compatible Models
|
||||
|
||||
To be used with this component, the model must be compatible with Sentence Transformers and
|
||||
|
||||
able to embed images and text into the same vector space. Compatible models include:
|
||||
|
||||
- `sentence-transformers/clip-ViT-B-32` (default)
|
||||
- `sentence-transformers/clip-ViT-L-14`
|
||||
- `sentence-transformers/clip-ViT-B-16`
|
||||
- `sentence-transformers/clip-ViT-B-32-multilingual-v1`
|
||||
- `jinaai/jina-embeddings-v4`
|
||||
- `jinaai/jina-clip-v1`
|
||||
- `jinaai/jina-clip-v2`
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.embedders.image import (
|
||||
SentenceTransformersDocumentImageEmbedder,
|
||||
)
|
||||
|
||||
embedder = SentenceTransformersDocumentImageEmbedder(
|
||||
model="sentence-transformers/clip-ViT-B-32",
|
||||
)
|
||||
embedder.warm_up()
|
||||
|
||||
documents = [
|
||||
Document(content="A photo of a cat", meta={"file_path": "cat.jpg"}),
|
||||
Document(content="A photo of a dog", meta={"file_path": "dog.jpg"}),
|
||||
]
|
||||
|
||||
result = embedder.run(documents=documents)
|
||||
documents_with_embeddings = result["documents"]
|
||||
print(documents_with_embeddings)
|
||||
|
||||
## [Document(id=...,
|
||||
## content='A photo of a cat',
|
||||
## meta={'file_path': 'cat.jpg',
|
||||
## 'embedding_source': {'type': 'image', 'file_path_meta_field': 'file_path'}},
|
||||
## embedding=vector of size 512),
|
||||
## ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
In this example, we can see an indexing pipeline with 3 components:
|
||||
|
||||
- `ImageFileToDocument` Converter that creates empty documents with a reference to an image in the `meta.file_path` field,
|
||||
- `SentenceTransformersDocumentImageEmbedder` that loads the images, computes embeddings and stores them in documents,
|
||||
- `DocumentWriter` that writes the documents in the `InMemoryDocumentStore`
|
||||
|
||||
There is also a multimodal retrieval pipeline, composed by a `SentenceTransformersTextEmbedder` (using the same model as before) and an `InMemoryEmbeddingRetriever`.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.converters.image import ImageFileToDocument
|
||||
from haystack.components.embedders import SentenceTransformersTextEmbedder
|
||||
from haystack.components.embedders.image import (
|
||||
SentenceTransformersDocumentImageEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
## Indexing pipeline
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("image_converter", ImageFileToDocument())
|
||||
indexing_pipeline.add_component(
|
||||
"embedder",
|
||||
SentenceTransformersDocumentImageEmbedder(
|
||||
model="sentence-transformers/clip-ViT-B-32",
|
||||
),
|
||||
)
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("image_converter", "embedder")
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run(data={"image_converter": {"sources": ["dog.jpg", "hyena.jpeg"]}})
|
||||
|
||||
## Multimodal retrieval pipeline
|
||||
retrieval_pipeline = Pipeline()
|
||||
retrieval_pipeline.add_component(
|
||||
"embedder",
|
||||
SentenceTransformersTextEmbedder(model="sentence-transformers/clip-ViT-B-32"),
|
||||
)
|
||||
retrieval_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store, top_k=2),
|
||||
)
|
||||
retrieval_pipeline.connect("embedder", "retriever")
|
||||
|
||||
result = retrieval_pipeline.run(data={"text": "man's best friend"})
|
||||
print(result)
|
||||
|
||||
## {
|
||||
## 'retriever': {
|
||||
## 'documents': [
|
||||
## Document(
|
||||
## id=0c96...,
|
||||
## meta={
|
||||
## 'file_path': 'dog.jpg',
|
||||
## 'embedding_source': {
|
||||
## 'type': 'image',
|
||||
## 'file_path_meta_field': 'file_path'
|
||||
## }
|
||||
## },
|
||||
## score=32.025817780129856
|
||||
## ),
|
||||
## Document(
|
||||
## id=5e76...,
|
||||
## meta={
|
||||
## 'file_path': 'hyena.jpeg',
|
||||
## 'embedding_source': {
|
||||
## 'type': 'image',
|
||||
## 'file_path_meta_field': 'file_path'
|
||||
## }
|
||||
## },
|
||||
## score=20.648225327085242
|
||||
## )
|
||||
## ]
|
||||
## }
|
||||
## }
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [Introduction to Multimodality](https://haystack.deepset.ai/cookbook/multimodal_intro)
|
||||
+186
@@ -0,0 +1,186 @@
|
||||
---
|
||||
title: "SentenceTransformersSparseDocumentEmbedder"
|
||||
id: sentencetransformerssparsedocumentembedder
|
||||
slug: "/sentencetransformerssparsedocumentembedder"
|
||||
description: "Use this component to enrich a list of documents with their sparse embeddings using Sentence Transformers models."
|
||||
---
|
||||
|
||||
# SentenceTransformersSparseDocumentEmbedder
|
||||
|
||||
Use this component to enrich a list of documents with their sparse embeddings using Sentence Transformers models.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory run variables** | `documents`: A list of documents |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with sparse embeddings) |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/sentence_transformers_sparse_document_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
To compute a sparse embedding for a string, use the [`SentenceTransformersSparseTextEmbedder`](sentencetransformerssparsetextembedder.mdx).
|
||||
|
||||
## Overview
|
||||
|
||||
`SentenceTransformersSparseDocumentEmbedder` computes the sparse embeddings of a list of documents and stores the obtained vectors in the `sparse_embedding` field of each document. It uses sparse embedding models supported by the Sentence Transformers library.
|
||||
|
||||
The vectors computed by this component are necessary to perform sparse embedding retrieval on a collection of documents. At retrieval time, the sparse vector representing the query is compared with those of the documents to find the most similar or relevant ones.
|
||||
|
||||
### Compatible Models
|
||||
|
||||
The default embedding model is [`prithivida/Splade_PP_en_v2`](https://huggingface.co/prithivida/Splade_PP_en_v2). You can specify another model with the `model` parameter when initializing this component.
|
||||
|
||||
Compatible models are based on SPLADE (SParse Lexical AnD Expansion), a technique for producing sparse representations for text, where each non-zero value in the embedding is the importance weight of a term in the vocabulary. This approach combines the benefits of learned sparse representations with the efficiency of traditional sparse retrieval methods. For more information, see [our docs](../retrievers.mdx#sparse-embedding-based-retrievers) that explain sparse embedding-based Retrievers further.
|
||||
|
||||
You can find compatible SPLADE models on the [Hugging Face Model Hub](https://huggingface.co/models?search=splade).
|
||||
|
||||
### Authentication
|
||||
|
||||
Authentication with a Hugging Face API Token is only required to access private or gated models.
|
||||
|
||||
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
|
||||
|
||||
```python
|
||||
from haystack.utils import Secret
|
||||
from haystack.components.embedders import SentenceTransformersSparseDocumentEmbedder
|
||||
|
||||
document_embedder = SentenceTransformersSparseDocumentEmbedder(
|
||||
token=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
```
|
||||
|
||||
### Backend Options
|
||||
|
||||
This component supports multiple backends for model execution:
|
||||
|
||||
- **torch** (default): Standard PyTorch backend
|
||||
- **onnx**: Optimized ONNX Runtime backend for faster inference
|
||||
- **openvino**: Intel OpenVINO backend for additional optimizations on Intel hardware
|
||||
|
||||
You can specify the backend during initialization:
|
||||
|
||||
```python
|
||||
embedder = SentenceTransformersSparseDocumentEmbedder(
|
||||
model="prithivida/Splade_PP_en_v2",
|
||||
backend="onnx",
|
||||
)
|
||||
```
|
||||
|
||||
For more information on acceleration and quantization options, refer to the [Sentence Transformers documentation](https://sbert.net/docs/sentence_transformer/usage/efficiency.html).
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often include metadata. If the metadata is distinctive and semantically meaningful, you can embed it along with the document's text to improve retrieval.
|
||||
|
||||
You can do this easily by using the Sparse Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.embedders import SentenceTransformersSparseDocumentEmbedder
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = SentenceTransformersSparseDocumentEmbedder(meta_fields_to_embed=["title"])
|
||||
embedder.warm_up()
|
||||
|
||||
docs_w_sparse_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.embedders import SentenceTransformersSparseDocumentEmbedder
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
doc_embedder = SentenceTransformersSparseDocumentEmbedder()
|
||||
doc_embedder.warm_up()
|
||||
|
||||
result = doc_embedder.run([doc])
|
||||
print(result["documents"][0].sparse_embedding)
|
||||
|
||||
## SparseEmbedding(indices=[999, 1045, ...], values=[0.918, 0.867, ...])
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Currently, sparse embedding retrieval is only supported by `QdrantDocumentStore`.
|
||||
|
||||
First, install the required package:
|
||||
|
||||
```shell
|
||||
pip install qdrant-haystack
|
||||
```
|
||||
|
||||
Then, try out this pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.components.embedders import (
|
||||
SentenceTransformersSparseDocumentEmbedder,
|
||||
SentenceTransformersSparseTextEmbedder,
|
||||
)
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack_integrations.components.retrievers.qdrant import (
|
||||
QdrantSparseEmbeddingRetriever,
|
||||
)
|
||||
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
|
||||
from haystack.document_stores.types import DuplicatePolicy
|
||||
|
||||
document_store = QdrantDocumentStore(
|
||||
":memory:",
|
||||
recreate_index=True,
|
||||
use_sparse_embeddings=True,
|
||||
)
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
Document(content="Sentence Transformers provides sparse embedding models."),
|
||||
]
|
||||
|
||||
## Indexing pipeline
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component(
|
||||
"sparse_document_embedder",
|
||||
SentenceTransformersSparseDocumentEmbedder(),
|
||||
)
|
||||
indexing_pipeline.add_component(
|
||||
"writer",
|
||||
DocumentWriter(document_store=document_store, policy=DuplicatePolicy.OVERWRITE),
|
||||
)
|
||||
indexing_pipeline.connect("sparse_document_embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"sparse_document_embedder": {"documents": documents}})
|
||||
|
||||
## Query pipeline
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"sparse_text_embedder",
|
||||
SentenceTransformersSparseTextEmbedder(),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"sparse_retriever",
|
||||
QdrantSparseEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect(
|
||||
"sparse_text_embedder.sparse_embedding",
|
||||
"sparse_retriever.query_sparse_embedding",
|
||||
)
|
||||
|
||||
query = "Who provides sparse embedding models?"
|
||||
|
||||
result = query_pipeline.run({"sparse_text_embedder": {"text": query}})
|
||||
|
||||
print(result["sparse_retriever"]["documents"][0])
|
||||
|
||||
## Document(id=...,
|
||||
## content: 'Sentence Transformers provides sparse embedding models.',
|
||||
## score: 0.75...)
|
||||
```
|
||||
+175
@@ -0,0 +1,175 @@
|
||||
---
|
||||
title: "SentenceTransformersSparseTextEmbedder"
|
||||
id: sentencetransformerssparsetextembedder
|
||||
slug: "/sentencetransformerssparsetextembedder"
|
||||
description: "Use this component to embed a simple string (such as a query) into a sparse vector using Sentence Transformers models."
|
||||
---
|
||||
|
||||
# SentenceTransformersSparseTextEmbedder
|
||||
|
||||
Use this component to embed a simple string (such as a query) into a sparse vector using Sentence Transformers models.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a sparse embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/sentence_transformers_sparse_text_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
For embedding lists of documents, use the [`SentenceTransformersSparseDocumentEmbedder`](sentencetransformerssparsedocumentembedder.mdx), which enriches the document with the computed sparse embedding.
|
||||
|
||||
## Overview
|
||||
|
||||
`SentenceTransformersSparseTextEmbedder` transforms a string into a sparse vector using sparse embedding models supported by the Sentence Transformers library.
|
||||
|
||||
When you perform sparse embedding retrieval, use this component first to transform your query into a sparse vector. Then, the Retriever will use the sparse vector to search for similar or relevant documents.
|
||||
|
||||
### Compatible Models
|
||||
|
||||
The default embedding model is [`prithivida/Splade_PP_en_v2`](https://huggingface.co/prithivida/Splade_PP_en_v2). You can specify another model with the `model` parameter when initializing this component.
|
||||
|
||||
Compatible models are based on SPLADE (SParse Lexical AnD Expansion), a technique for producing sparse representations for text, where each non-zero value in the embedding is the importance weight of a term in the vocabulary. This approach combines the benefits of learned sparse representations with the efficiency of traditional sparse retrieval methods. For more information, see [our docs](../retrievers.mdx#sparse-embedding-based-retrievers) that explain sparse embedding-based Retrievers further.
|
||||
|
||||
You can find compatible SPLADE models on the [Hugging Face Model Hub](https://huggingface.co/models?search=splade).
|
||||
|
||||
### Authentication
|
||||
|
||||
Authentication with a Hugging Face API Token is only required to access private or gated models.
|
||||
|
||||
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
|
||||
|
||||
```python
|
||||
from haystack.utils import Secret
|
||||
from haystack.components.embedders import SentenceTransformersSparseTextEmbedder
|
||||
|
||||
text_embedder = SentenceTransformersSparseTextEmbedder(
|
||||
token=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
```
|
||||
|
||||
### Backend Options
|
||||
|
||||
This component supports multiple backends for model execution:
|
||||
|
||||
- **torch** (default): Standard PyTorch backend
|
||||
- **onnx**: Optimized ONNX Runtime backend for faster inference
|
||||
- **openvino**: Intel OpenVINO backend for additional optimizations on Intel hardware
|
||||
|
||||
You can specify the backend during initialization:
|
||||
|
||||
```python
|
||||
embedder = SentenceTransformersSparseTextEmbedder(
|
||||
model="prithivida/Splade_PP_en_v2",
|
||||
backend="onnx",
|
||||
)
|
||||
```
|
||||
|
||||
For more information on acceleration and quantization options, refer to the [Sentence Transformers documentation](https://sbert.net/docs/sentence_transformer/usage/efficiency.html).
|
||||
|
||||
### Prefix and Suffix
|
||||
|
||||
Some models may benefit from adding a prefix or suffix to the text before embedding. You can specify these during initialization:
|
||||
|
||||
```python
|
||||
embedder = SentenceTransformersSparseTextEmbedder(
|
||||
model="prithivida/Splade_PP_en_v2",
|
||||
prefix="query: ",
|
||||
suffix="",
|
||||
)
|
||||
```
|
||||
|
||||
:::tip
|
||||
If you create a Sparse Text Embedder and a Sparse Document Embedder based on the same model, Haystack takes care of using the same resource behind the scenes in order to save resources.
|
||||
:::
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import SentenceTransformersSparseTextEmbedder
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = SentenceTransformersSparseTextEmbedder()
|
||||
text_embedder.warm_up()
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
|
||||
## {'sparse_embedding': SparseEmbedding(indices=[999, 1045, ...], values=[0.918, 0.867, ...])}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Currently, sparse embedding retrieval is only supported by `QdrantDocumentStore`.
|
||||
|
||||
First, install the required package:
|
||||
|
||||
```shell
|
||||
pip install qdrant-haystack
|
||||
```
|
||||
|
||||
Then, try out this pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.components.embedders import (
|
||||
SentenceTransformersSparseDocumentEmbedder,
|
||||
SentenceTransformersSparseTextEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.retrievers.qdrant import (
|
||||
QdrantSparseEmbeddingRetriever,
|
||||
)
|
||||
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
|
||||
|
||||
document_store = QdrantDocumentStore(
|
||||
":memory:",
|
||||
recreate_index=True,
|
||||
use_sparse_embeddings=True,
|
||||
)
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
Document(content="Sentence Transformers provides sparse embedding models."),
|
||||
]
|
||||
|
||||
## Embed and write documents
|
||||
sparse_document_embedder = SentenceTransformersSparseDocumentEmbedder(
|
||||
model="prithivida/Splade_PP_en_v2",
|
||||
)
|
||||
sparse_document_embedder.warm_up()
|
||||
documents_with_sparse_embeddings = sparse_document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_sparse_embeddings)
|
||||
|
||||
## Query pipeline
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"sparse_text_embedder",
|
||||
SentenceTransformersSparseTextEmbedder(),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"sparse_retriever",
|
||||
QdrantSparseEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect(
|
||||
"sparse_text_embedder.sparse_embedding",
|
||||
"sparse_retriever.query_sparse_embedding",
|
||||
)
|
||||
|
||||
query = "Who provides sparse embedding models?"
|
||||
|
||||
result = query_pipeline.run({"sparse_text_embedder": {"text": query}})
|
||||
|
||||
print(result["sparse_retriever"]["documents"][0])
|
||||
|
||||
## Document(id=...,
|
||||
## content: 'Sentence Transformers provides sparse embedding models.',
|
||||
## score: 0.56...)
|
||||
```
|
||||
+127
@@ -0,0 +1,127 @@
|
||||
---
|
||||
title: "SentenceTransformersTextEmbedder"
|
||||
id: sentencetransformerstextembedder
|
||||
slug: "/sentencetransformerstextembedder"
|
||||
description: "SentenceTransformersTextEmbedder transforms a string into a vector that captures its semantics using an embedding model compatible with the Sentence Transformers library."
|
||||
---
|
||||
|
||||
# SentenceTransformersTextEmbedder
|
||||
|
||||
SentenceTransformersTextEmbedder transforms a string into a vector that captures its semantics using an embedding model compatible with the Sentence Transformers library.
|
||||
|
||||
When you perform embedding retrieval, use this component first to transform your query into a vector. Then, the embedding Retriever will use the vector to search for similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers |
|
||||
| **API reference** | [Embedders](/reference/embedders-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/embedders/sentence_transformers_text_embedder.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
This component should be used to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [SentenceTransformersDocumentEmbedder](sentencetransformersdocumentembedder.mdx), which enriches the document with the computed embedding, known as vector.
|
||||
|
||||
### Authentication
|
||||
|
||||
Authentication with a Hugging Face API Token is only required to access private or gated models through Serverless Inference API or the Inference Endpoints.
|
||||
|
||||
The component uses an `HF_API_TOKEN` or `HF_TOKEN` environment variable, or you can pass a Hugging Face API token at initialization. See our [Secret Management](../../concepts/secret-management.mdx) page for more information.
|
||||
|
||||
```python
|
||||
text_embedder = SentenceTransformersTextEmbedder(
|
||||
token=Secret.from_token("<your-api-key>"),
|
||||
)
|
||||
```
|
||||
|
||||
### Compatible Models
|
||||
|
||||
The default embedding model is [\`sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)\`. You can specify another model with the `model` parameter when initializing this component.
|
||||
|
||||
See the original models in the Sentence Transformers [documentation](https://www.sbert.net/docs/pretrained_models.html).
|
||||
|
||||
Nowadays, most of the models in the [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) are compatible with Sentence Transformers.
|
||||
You can look for compatibility in the model card: [an example related to BGE models](https://huggingface.co/BAAI/bge-large-en-v1.5#using-sentence-transformers).
|
||||
|
||||
### Instructions
|
||||
|
||||
Some recent models that you can find in MTEB require prepending the text with an instruction to work better for retrieval.
|
||||
For example, if you use [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5#model-list), you should prefix your query with the following instruction: “Represent this sentence for searching relevant passages:”
|
||||
|
||||
This is how it works with `SentenceTransformersTextEmbedder`:
|
||||
|
||||
```python
|
||||
instruction = "Represent this sentence for searching relevant passages:"
|
||||
embedder = SentenceTransformersTextEmbedder(
|
||||
*model="*BAAI/bge-large-en-v1.5",
|
||||
prefix=instruction)
|
||||
```
|
||||
|
||||
:::tip
|
||||
If you create a Text Embedder and a Document Embedder based on the same model, Haystack takes care of using the same resource behind the scenes in order to save resources.
|
||||
:::
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack.components.embedders import SentenceTransformersTextEmbedder
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = SentenceTransformersTextEmbedder()
|
||||
text_embedder.warm_up()
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
|
||||
## {'embedding': [-0.07804739475250244, 0.1498992145061493,, ...]}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.embedders import (
|
||||
SentenceTransformersTextEmbedder,
|
||||
SentenceTransformersDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = SentenceTransformersDocumentEmbedder()
|
||||
document_embedder.warm_up()
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "STACKITDocumentEmbedder"
|
||||
id: stackitdocumentembedder
|
||||
slug: "/stackitdocumentembedder"
|
||||
description: "This component enables document embedding using the STACKIT API."
|
||||
---
|
||||
|
||||
# STACKITDocumentEmbedder
|
||||
|
||||
This component enables document embedding using the STACKIT API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [DocumentWriter](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `model`: The model used through the STACKIT API |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents enriched with embeddings |
|
||||
| **API reference** | [STACKIT](/reference/integrations-stackit) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`STACKITDocumentEmbedder` enables document embedding models served by STACKIT through their API.
|
||||
|
||||
### Parameters
|
||||
|
||||
To use the `STACKITDocumentEmbedder`, ensure you have set a `STACKIT_API_KEY` as an environment variable. Alternatively, provide the API key as an environment variable with a different name or a token by setting `api_key` and using Haystack’s [secret management](../../concepts/secret-management.mdx).
|
||||
|
||||
Set your preferred supported model with the `model` parameter when initializing the component. See the full list of all supported models on the [STACKIT website](https://docs.stackit.cloud/stackit/en/models-licenses-319914532.html).
|
||||
|
||||
Optionally, you can change the default `api_base_url`, which is `"https://api.openai-compat.model-serving.eu01.onstackit.cloud/v1"`.
|
||||
|
||||
You can pass any text generation parameters valid for the STACKIT Chat Completion API directly to this component with the `generation_kwargs` parameter in the init or run methods.
|
||||
|
||||
Then component needs a list of documents as input to operate.
|
||||
|
||||
## Usage
|
||||
|
||||
Install the `stackit-haystack` package to use the `STACKITDocumentEmbedder` and set an environment variable called `STACKIT_API_KEY` to your API key.
|
||||
|
||||
```shell
|
||||
pip install stackit-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.stackit import STACKITDocumentEmbedder
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = STACKITDocumentEmbedder(model="intfloat/e5-mistral-7b-instruct")
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
|
||||
## [0.0215301513671875, 0.01499176025390625, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
You can also use `STACKITDocumentEmbedder` in your pipeline in a following way.
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.stackit import (
|
||||
STACKITTextEmbedder,
|
||||
STACKITDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = STACKITDocumentEmbedder(model="intfloat/e5-mistral-7b-instruct")
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
text_embedder = STACKITTextEmbedder(model="intfloat/e5-mistral-7b-instruct")
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", text_embedder)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Where does Wolfgang live?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin', score: ...)
|
||||
```
|
||||
|
||||
You can find more usage examples in the STACKIT integration [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit/examples) and its [integration page](https://haystack.deepset.ai/integrations/stackit).
|
||||
+106
@@ -0,0 +1,106 @@
|
||||
---
|
||||
title: "STACKITTextEmbedder"
|
||||
id: stackittextembedder
|
||||
slug: "/stackittextembedder"
|
||||
description: "This component enables text embedding using the STACKIT API."
|
||||
---
|
||||
|
||||
# STACKITTextEmbedder
|
||||
|
||||
This component enables text embedding using the STACKIT API.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `model`: The model used through the STACKIT API |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers |
|
||||
| **API reference** | [STACKIT](/reference/integrations-stackit) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`STACKITTextEmbedder` enables text embedding models served by STACKIT through their API.
|
||||
|
||||
### Parameters
|
||||
|
||||
To use the `STACKITTextEmbedder`, ensure you have set a `STACKIT_API_KEY` as an environment variable. Alternatively, provide the API key as an environment variable with a different name or a token by setting `api_key` and using Haystack’s [secret management](../../concepts/secret-management.mdx).
|
||||
|
||||
Set your preferred supported model with the `model` parameter when initializing the component. See the full list of all supported models on the [STACKIT website](https://docs.stackit.cloud/stackit/en/models-licenses-319914532.html).
|
||||
|
||||
Optionally, you can change the default `api_base_url`, which is `"https://api.openai-compat.model-serving.eu01.onstackit.cloud/v1"`.
|
||||
|
||||
You can pass any text generation parameters valid for the STACKIT Chat Completion API directly to this component with the `generation_kwargs` parameter in the init or run methods.
|
||||
|
||||
The component needs a text input to operate.
|
||||
|
||||
## Usage
|
||||
|
||||
Install the `stackit-haystack` package to use the `STACKITTextEmbedder` and set an environment variable called `STACKIT_API_KEY` to your API key.
|
||||
|
||||
```shell
|
||||
pip install stackit-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.stackit import STACKITTextEmbedder
|
||||
|
||||
text_embedder = STACKITTextEmbedder(model="intfloat/e5-mistral-7b-instruct")
|
||||
|
||||
print(text_embedder.run("I love pizza!"))
|
||||
|
||||
## {'embedding': [0.0215301513671875, 0.01499176025390625, ...]}
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
You can also use `STACKITTextEmbedder` in your pipeline.
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.stackit import (
|
||||
STACKITTextEmbedder,
|
||||
STACKITDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore()
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = STACKITDocumentEmbedder(model="intfloat/e5-mistral-7b-instruct")
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
text_embedder = STACKITTextEmbedder(model="intfloat/e5-mistral-7b-instruct")
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", text_embedder)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Where does Wolfgang live?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin', score: ...)
|
||||
```
|
||||
|
||||
You can find more usage examples in the STACKIT integration [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit/examples) and its [integration page](https://haystack.deepset.ai/integrations/stackit).
|
||||
+121
@@ -0,0 +1,121 @@
|
||||
---
|
||||
title: "VertexAIDocumentEmbedder"
|
||||
id: vertexaidocumentembedder
|
||||
slug: "/vertexaidocumentembedder"
|
||||
description: "This component computes embeddings for documents using models through VertexAI Embeddings API."
|
||||
---
|
||||
|
||||
# VertexAIDocumentEmbedder
|
||||
|
||||
This component computes embeddings for documents using models through VertexAI Embeddings API.
|
||||
|
||||
:::warning[Deprecation Notice]
|
||||
|
||||
This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.
|
||||
|
||||
We recommend switching to the new [GoogleGenAIDocumentEmbedder](googlegenaidocumentembedder.mdx) integration instead.
|
||||
:::
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [DocumentWriter](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `model`: The model used through the VertexAI Embeddings API |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents enriched with embeddings |
|
||||
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
|
||||
|
||||
</div>
|
||||
|
||||
`VertexAIDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, use the [`VertexAITextEmbedder`](vertexaitextembedder.mdx).
|
||||
|
||||
To use the `VertexAIDocumentEmbedder`, initialize it with:
|
||||
|
||||
- `model`: The supported models are:
|
||||
- "text-embedding-004"
|
||||
- "text-embedding-005"
|
||||
- "textembedding-gecko-multilingual@001"
|
||||
- "text-multilingual-embedding-002"
|
||||
- "text-embedding-large-exp-03-07"
|
||||
- `task_type`: "RETRIEVAL_DOCUMENT” is the default. You can find all task types in the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype).
|
||||
|
||||
### Authentication
|
||||
|
||||
`VertexAIDocumentEmbedder` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
|
||||
|
||||
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
|
||||
|
||||
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
|
||||
|
||||
## Usage
|
||||
|
||||
Install the `google-vertex-haystack` package to use this Embedder:
|
||||
|
||||
```shell
|
||||
pip install google-vertex-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.google_vertex import (
|
||||
VertexAIDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005")
|
||||
|
||||
result = document_embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
## [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665,
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.google_vertex import (
|
||||
VertexAITextEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.google_vertex import (
|
||||
VertexAIDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005")
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"text_embedder",
|
||||
VertexAITextEmbedder(model="text-embedding-005"),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+121
@@ -0,0 +1,121 @@
|
||||
---
|
||||
title: "VertexAITextEmbedder"
|
||||
id: vertexaitextembedder
|
||||
slug: "/vertexaitextembedder"
|
||||
description: "This component computes embeddings for text (such as a query) using models through VertexAI Embeddings API."
|
||||
---
|
||||
|
||||
# VertexAITextEmbedder
|
||||
|
||||
This component computes embeddings for text (such as a query) using models through VertexAI Embeddings API.
|
||||
|
||||
:::warning[Deprecation Notice]
|
||||
|
||||
This integration uses the deprecated google-generativeai SDK, which will lose support after August 2025.
|
||||
|
||||
We recommend switching to the new [GoogleGenAITextEmbedder](googlegenaitextembedder.mdx) integration instead.
|
||||
:::
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `model`: The model used through the VertexAI Embeddings API |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers |
|
||||
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`VertexAITextEmbedder` embeds a simple string (such as a query) into a vector. For embedding lists of documents, use the [`VertexAIDocumentEmbedder`](vertexaidocumentembedder.mdx) which enriches the document with the computed embedding, also known as vector.
|
||||
|
||||
To start using the `VertexAITextEmbedder`, initialize it with:
|
||||
|
||||
- `model`: The supported models are:
|
||||
- "text-embedding-004"
|
||||
- "text-embedding-005"
|
||||
- "textembedding-gecko-multilingual@001"
|
||||
- "text-multilingual-embedding-002"
|
||||
- "text-embedding-large-exp-03-07"
|
||||
- `task_type`: "RETRIEVAL_QUERY” is the default. You can find all task types in the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype).
|
||||
|
||||
### Authentication
|
||||
|
||||
`VertexAITextEmbedder` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
|
||||
|
||||
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
|
||||
|
||||
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
|
||||
|
||||
## Usage
|
||||
|
||||
Install the `google-vertex-haystack` package to use this Embedder:
|
||||
|
||||
```shell
|
||||
pip install google-vertex-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.google_vertex import (
|
||||
VertexAITextEmbedder,
|
||||
)
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = VertexAITextEmbedder(model="text-embedding-005")
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
## {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.google_vertex import (
|
||||
VertexAITextEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.google_vertex import (
|
||||
VertexAIDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005")
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component(
|
||||
"text_embedder",
|
||||
VertexAITextEmbedder(model="text-embedding-005"),
|
||||
)
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., content: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+146
@@ -0,0 +1,146 @@
|
||||
---
|
||||
title: "WatsonxDocumentEmbedder"
|
||||
id: watsonxdocumentembedder
|
||||
slug: "/watsonxdocumentembedder"
|
||||
description: "The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant documents."
|
||||
---
|
||||
|
||||
# WatsonxDocumentEmbedder
|
||||
|
||||
The vectors computed by this component are necessary to perform embedding retrieval on a collection of documents. At retrieval time, the vector that represents the query is compared with those of the documents to find the most similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before a [`DocumentWriter`](../writers/documentwriter.mdx) in an indexing pipeline |
|
||||
| **Mandatory init variables** | `api_key`: The IBM Cloud API key. Can be set with `WATSONX_API_KEY` env var. <br /> <br />`project_id`: The IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. |
|
||||
| **Mandatory run variables** | `documents`: A list of documents to be embedded |
|
||||
| **Output variables** | `documents`: A list of documents (enriched with embeddings) <br /> <br />`meta`: A dictionary of metadata strings |
|
||||
| **API reference** | [Watsonx](/reference/integrations-watsonx) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
`WatsonxDocumentEmbedder` enriches the metadata of documents with an embedding of their content. To embed a string, you should use the [`WatsonxTextEmbedder`](watsonxtextembedder.mdx).
|
||||
|
||||
The component supports IBM watsonx.ai embedding models such as `ibm/slate-30m-english-rtrvr` and similar. The default model is `ibm/slate-30m-english-rtrvr`. This list of all supported models can be found in IBM's [model documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-embed.html?context=wx).
|
||||
|
||||
To start using this integration with Haystack, install it with:
|
||||
|
||||
```shell
|
||||
pip install watsonx-haystack
|
||||
```
|
||||
|
||||
The component uses `WATSONX_API_KEY` and `WATSONX_PROJECT_ID` environment variables by default. Otherwise, you can pass API credentials at initialization with `api_key` and `project_id`:
|
||||
|
||||
```python
|
||||
embedder = WatsonxDocumentEmbedder(
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
project_id=Secret.from_token("<your-project-id>"),
|
||||
)
|
||||
```
|
||||
|
||||
To get IBM Cloud credentials, head over to https://cloud.ibm.com/.
|
||||
|
||||
### Embedding Metadata
|
||||
|
||||
Text documents often come with a set of metadata. If they are distinctive and semantically meaningful, you can embed them along with the text of the document to improve retrieval.
|
||||
|
||||
You can do this by using the Document Embedder:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.watsonx.document_embedder import (
|
||||
WatsonxDocumentEmbedder,
|
||||
)
|
||||
from haystack.utils import Secret
|
||||
|
||||
doc = Document(content="some text", meta={"title": "relevant title", "page number": 18})
|
||||
|
||||
embedder = WatsonxDocumentEmbedder(
|
||||
api_key=Secret.from_env_var("WATSONX_API_KEY"),
|
||||
project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
|
||||
meta_fields_to_embed=["title"],
|
||||
)
|
||||
|
||||
docs_w_embeddings = embedder.run(documents=[doc])["documents"]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Install the `watsonx-haystack` package to use the `WatsonxDocumentEmbedder`:
|
||||
|
||||
```shell
|
||||
pip install watsonx-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
Remember to set `WATSONX_API_KEY` and `WATSONX_PROJECT_ID` as environment variables first, or pass them in directly.
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack_integrations.components.embedders.watsonx.document_embedder import (
|
||||
WatsonxDocumentEmbedder,
|
||||
)
|
||||
|
||||
doc = Document(content="I love pizza!")
|
||||
|
||||
embedder = WatsonxDocumentEmbedder()
|
||||
|
||||
result = embedder.run([doc])
|
||||
print(result["documents"][0].embedding)
|
||||
## [-0.453125, 1.2236328, 2.0058594, 0.67871094...]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack.components.writers import DocumentWriter
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
from haystack_integrations.components.embedders.watsonx.document_embedder import (
|
||||
WatsonxDocumentEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.watsonx.text_embedder import (
|
||||
WatsonxTextEmbedder,
|
||||
)
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
indexing_pipeline = Pipeline()
|
||||
indexing_pipeline.add_component("embedder", WatsonxDocumentEmbedder())
|
||||
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
|
||||
indexing_pipeline.connect("embedder", "writer")
|
||||
|
||||
indexing_pipeline.run({"embedder": {"documents": documents}})
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", WatsonxTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
+119
@@ -0,0 +1,119 @@
|
||||
---
|
||||
title: "WatsonxTextEmbedder"
|
||||
id: watsonxtextembedder
|
||||
slug: "/watsonxtextembedder"
|
||||
description: "When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents."
|
||||
---
|
||||
|
||||
# WatsonxTextEmbedder
|
||||
|
||||
When you perform embedding retrieval, you use this component to transform your query into a vector. Then, the embedding Retriever looks for similar or relevant documents.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | Before an embedding [Retriever](../retrievers.mdx) in a query/RAG pipeline |
|
||||
| **Mandatory init variables** | `api_key`: An IBM Cloud API key. Can be set with `WATSONX_API_KEY` env var. <br /> <br />`project_id`: An IBM Cloud project ID. Can be set with `WATSONX_PROJECT_ID` env var. |
|
||||
| **Mandatory run variables** | `text`: A string |
|
||||
| **Output variables** | `embedding`: A list of float numbers <br /> <br />`meta`: A dictionary of metadata |
|
||||
| **API reference** | [Watsonx](/reference/integrations-watsonx) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/watsonx |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
To see the list of compatible IBM watsonx.ai embedding models, head over to IBM [documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-embed.html?context=wx). The default model for `WatsonxTextEmbedder` is `ibm/slate-30m-english-rtrvr`. You can specify another model with the `model` parameter when initializing this component.
|
||||
|
||||
Use `WatsonxTextEmbedder` to embed a simple string (such as a query) into a vector. For embedding lists of documents, use the [`WatsonxDocumentEmbedder`](watsonxdocumentembedder.mdx), which enriches the document with the computed embedding, also known as vector.
|
||||
|
||||
The component uses `WATSONX_API_KEY` and `WATSONX_PROJECT_ID` environment variables by default. Otherwise, you can pass API credentials at initialization with `api_key` and `project_id`:
|
||||
|
||||
```python
|
||||
embedder = WatsonxTextEmbedder(
|
||||
api_key=Secret.from_token("<your-api-key>"),
|
||||
project_id=Secret.from_token("<your-project-id>"),
|
||||
)
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Install the `watsonx-haystack` package to use the `WatsonxTextEmbedder`:
|
||||
|
||||
```shell
|
||||
pip install watsonx-haystack
|
||||
```
|
||||
|
||||
### On its own
|
||||
|
||||
Here is how you can use the component on its own:
|
||||
|
||||
```python
|
||||
from haystack_integrations.components.embedders.watsonx.text_embedder import (
|
||||
WatsonxTextEmbedder,
|
||||
)
|
||||
from haystack.utils import Secret
|
||||
|
||||
text_to_embed = "I love pizza!"
|
||||
|
||||
text_embedder = WatsonxTextEmbedder(
|
||||
api_key=Secret.from_env_var("WATSONX_API_KEY"),
|
||||
project_id=Secret.from_env_var("WATSONX_PROJECT_ID"),
|
||||
model="ibm/slate-30m-english-rtrvr",
|
||||
)
|
||||
|
||||
print(text_embedder.run(text_to_embed))
|
||||
|
||||
## {'embedding': [0.017020374536514282, -0.023255806416273117, ...],
|
||||
## 'meta': {'model': 'ibm/slate-30m-english-rtrvr',
|
||||
## 'truncated_input_tokens': 3}}
|
||||
```
|
||||
|
||||
:::info
|
||||
We recommend setting WATSONX_API_KEY and WATSONX_PROJECT_ID as environment variables instead of setting them as parameters.
|
||||
:::
|
||||
|
||||
### In a pipeline
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack import Pipeline
|
||||
from haystack.document_stores.in_memory import InMemoryDocumentStore
|
||||
from haystack_integrations.components.embedders.watsonx.text_embedder import (
|
||||
WatsonxTextEmbedder,
|
||||
)
|
||||
from haystack_integrations.components.embedders.watsonx.document_embedder import (
|
||||
WatsonxDocumentEmbedder,
|
||||
)
|
||||
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
|
||||
|
||||
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
|
||||
|
||||
documents = [
|
||||
Document(content="My name is Wolfgang and I live in Berlin"),
|
||||
Document(content="I saw a black horse running"),
|
||||
Document(content="Germany has many big cities"),
|
||||
]
|
||||
|
||||
document_embedder = WatsonxDocumentEmbedder()
|
||||
documents_with_embeddings = document_embedder.run(documents)["documents"]
|
||||
document_store.write_documents(documents_with_embeddings)
|
||||
|
||||
query_pipeline = Pipeline()
|
||||
query_pipeline.add_component("text_embedder", WatsonxTextEmbedder())
|
||||
query_pipeline.add_component(
|
||||
"retriever",
|
||||
InMemoryEmbeddingRetriever(document_store=document_store),
|
||||
)
|
||||
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
|
||||
|
||||
query = "Who lives in Berlin?"
|
||||
|
||||
result = query_pipeline.run({"text_embedder": {"text": query}})
|
||||
|
||||
print(result["retriever"]["documents"][0])
|
||||
|
||||
## Document(id=..., mimetype: 'text/plain',
|
||||
## text: 'My name is Wolfgang and I live in Berlin')
|
||||
```
|
||||
@@ -0,0 +1,21 @@
|
||||
---
|
||||
title: "Evaluators"
|
||||
id: evaluators
|
||||
slug: "/evaluators"
|
||||
---
|
||||
|
||||
# Evaluators
|
||||
|
||||
| Evaluator | Description |
|
||||
| --- | --- |
|
||||
| [AnswerExactMatchEvaluator](evaluators/answerexactmatchevaluator.mdx) | Evaluates answers predicted by Haystack pipelines using ground truth labels. It checks character by character whether a predicted answer exactly matches the ground truth answer. |
|
||||
| [ContextRelevanceEvaluator](evaluators/contextrelevanceevaluator.mdx) | Uses an LLM to evaluate whether a generated answer can be inferred from the provided contexts. |
|
||||
| [DeepEvalEvaluator](evaluators/deepevalevaluator.mdx) | Use DeepEval to evaluate generative pipelines. |
|
||||
| [DocumentMAPEvaluator](evaluators/documentmapevaluator.mdx) | Evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks to what extent the list of retrieved documents contains only relevant documents as specified in the ground truth labels or also non-relevant documents. |
|
||||
| [DocumentMRREvaluator](evaluators/documentmrrevaluator.mdx) | Evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks at what rank ground truth documents appear in the list of retrieved documents. |
|
||||
| [DocumentNDCGEvaluator](evaluators/documentndcgevaluator.mdx) | Evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks at what rank ground truth documents appear in the list of retrieved documents. This metric is called normalized discounted cumulative gain (NDCG). |
|
||||
| [DocumentRecallEvaluator](evaluators/documentrecallevaluator.mdx) | Evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks how many of the ground truth documents were retrieved. |
|
||||
| [FaithfulnessEvaluator](evaluators/faithfulnessevaluator.mdx) | Uses an LLM to evaluate whether a generated answer can be inferred from the provided contexts. Does not require ground truth labels. |
|
||||
| [LLMEvaluator](evaluators/llmevaluator.mdx) | Uses an LLM to evaluate inputs based on a prompt containing user-defined instructions and examples. |
|
||||
| [RagasEvaluator](evaluators/ragasevaluator.mdx) | Use Ragas framework to evaluate a retrieval-augmented generative pipeline. |
|
||||
| [SASEvaluator](evaluators/sasevaluator.mdx) | Evaluates answers predicted by Haystack pipelines using ground truth labels. It checks the semantic similarity of a predicted answer and the ground truth answer using a fine-tuned language model. |
|
||||
+93
@@ -0,0 +1,93 @@
|
||||
---
|
||||
title: "AnswerExactMatchEvaluator"
|
||||
id: answerexactmatchevaluator
|
||||
slug: "/answerexactmatchevaluator"
|
||||
description: "The `AnswerExactMatchEvaluator` evaluates answers predicted by Haystack pipelines using ground truth labels. It checks character by character whether a predicted answer exactly matches the ground truth answer. This metric is called the exact match."
|
||||
---
|
||||
|
||||
# AnswerExactMatchEvaluator
|
||||
|
||||
The `AnswerExactMatchEvaluator` evaluates answers predicted by Haystack pipelines using ground truth labels. It checks character by character whether a predicted answer exactly matches the ground truth answer. This metric is called the exact match.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
|
||||
| **Mandatory run variables** | `ground_truth_answers`: A list of strings containing the ground truth answers <br /> <br />`predicted_answers`: A list of strings containing the predicted answers to be evaluated |
|
||||
| **Output variables** | A dictionary containing: <br /> <br />\- `score`: A number from 0.0 to 1.0 representing the proportion of questions in which any predicted answer matched the ground truth answers <br /> <br />- `individual_scores`: A list of 0s and 1s, where 1 means that the predicted answer matched one of the ground truths |
|
||||
| **API reference** | [Evaluators](/reference/evaluators-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/answer_exact_match.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
You can use the `AnswerExactMatchEvaluator` component to evaluate answers predicted by a Haystack pipeline, such as an extractive question answering pipeline, against ground truth labels. As the `AnswerExactMatchEvaluator` checks whether a predicted answer exactly matches the ground truth answer. It is not suited to evaluate answers generated by LLMs, for example, in a RAG pipeline. Use `FaithfulnessEvaluator` or `SASEvaluator` instead.
|
||||
|
||||
To initialize an `AnswerExactMatchEvaluator`, there are no parameters required.
|
||||
|
||||
Note that only _one_ predicted answer is compared to _one_ ground truth answer at a time. The component does not support multiple ground truth answers for the same question or multiple answers predicted for the same question.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Below is an example of using an `AnswerExactMatchEvaluator` component to evaluate two answers and compare them to ground truth answers.
|
||||
|
||||
```python
|
||||
from haystack.components.evaluators import AnswerExactMatchEvaluator
|
||||
|
||||
evaluator = AnswerExactMatchEvaluator()
|
||||
result = evaluator.run(
|
||||
ground_truth_answers=["Berlin", "Paris"],
|
||||
predicted_answers=["Berlin", "Lyon"],
|
||||
)
|
||||
|
||||
print(result["individual_scores"])
|
||||
## [1, 0]
|
||||
print(result["score"])
|
||||
## 0.5
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Below is an example where we use an `AnswerExactMatchEvaluator` and a `SASEvaluator` in a pipeline to evaluate two answers and compare them to ground truth answers. Running a pipeline instead of the individual components simplifies calculating more than one metric.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.evaluators import AnswerExactMatchEvaluator
|
||||
from haystack.components.evaluators import SASEvaluator
|
||||
|
||||
pipeline = Pipeline()
|
||||
em_evaluator = AnswerExactMatchEvaluator()
|
||||
sas_evaluator = SASEvaluator()
|
||||
pipeline.add_component("em_evaluator", em_evaluator)
|
||||
pipeline.add_component("sas_evaluator", sas_evaluator)
|
||||
|
||||
ground_truth_answers = ["Berlin", "Paris"]
|
||||
predicted_answers = ["Berlin", "Lyon"]
|
||||
|
||||
result = pipeline.run(
|
||||
{
|
||||
"em_evaluator": {
|
||||
"ground_truth_answers": ground_truth_answers,
|
||||
"predicted_answers": predicted_answers,
|
||||
},
|
||||
"sas_evaluator": {
|
||||
"ground_truth_answers": ground_truth_answers,
|
||||
"predicted_answers": predicted_answers,
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
for evaluator in result:
|
||||
print(result[evaluator]["individual_scores"])
|
||||
## [1, 0]
|
||||
## [array([[0.99999994]], dtype=float32), array([[0.51747656]], dtype=float32)]
|
||||
|
||||
for evaluator in result:
|
||||
print(result[evaluator]["score"])
|
||||
## 0.5
|
||||
## 0.7587383
|
||||
```
|
||||
+136
@@ -0,0 +1,136 @@
|
||||
---
|
||||
title: "ContextRelevanceEvaluator"
|
||||
id: contextrelevanceevaluator
|
||||
slug: "/contextrelevanceevaluator"
|
||||
description: "The `ContextRelevanceEvaluator` uses an LLM to evaluate whether contexts are relevant to a question. It does not require ground truth labels."
|
||||
---
|
||||
|
||||
# ContextRelevanceEvaluator
|
||||
|
||||
The `ContextRelevanceEvaluator` uses an LLM to evaluate whether contexts are relevant to a question. It does not require ground truth labels.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
|
||||
| **Mandatory run variables** | `questions`: A list of questions <br /> <br />`contexts`: A list of a list of contexts, which are the contents of documents. This accounts for one list of contexts per question. |
|
||||
| **Output variables** | A dictionary containing: <br /> <br />\- `score`: A number from 0.0 to 1.0 that represents the mean average precision <br /> <br />- `individual_scores`: A list of the individual average precision scores ranging from 0.0 to 1.0 for each input pair of a list of retrieved documents and a list of ground truth documents <br /> <br />- `results`: A list of dictionaries with keys `statements` and `statement_scores`. They contain the statements extracted by an LLM from each context and the corresponding context relevance scores per statement, which are either 0 or 1. |
|
||||
| **API reference** | [Evaluators](/reference/evaluators-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/context_relevance.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
You can use the `ContextRelevanceEvaluator` component to evaluate documents retrieved by a Haystack pipeline, such as a RAG pipeline, without ground truth labels. The component breaks up the context into multiple statements and checks whether each statement is relevant for answering a question. The final score for the context relevance is a number from 0.0 to 1.0 and represents the proportion of statements that are relevant to the provided question.
|
||||
|
||||
### Parameters
|
||||
|
||||
The default model for this Evaluator is `gpt-4o-mini`. You can override the model using the `chat_generator` parameter during initialization. This needs to be a Chat Generator instance configured to return a JSON object. For example, when using the [`OpenAIChatGenerator`](../generators/openaichatgenerator.mdx), you should pass `{"response_format": {"type": "json_object"}}` in its `generation_kwargs`.
|
||||
|
||||
If you are not initializing the Evaluator with your own Chat Generator other than OpenAI, a valid OpenAI API key must be set as an `OPENAI_API_KEY` environment variable. For details, see our [documentation page on secret management](../../concepts/secret-management.mdx).
|
||||
|
||||
Two optional initialization parameters are:
|
||||
|
||||
- `raise_on_failure`: If True, raise an exception on an unsuccessful API call.
|
||||
- `progress_bar`: Whether to show a progress bar during the evaluation.
|
||||
|
||||
`ContextRelevanceEvaluator` has an optional `examples` parameter that can be used to pass few-shot examples conforming to the expected input and output format of `ContextRelevanceEvaluator`. These examples are included in the prompt that is sent to the LLM. Examples, therefore, increase the number of tokens of the prompt and make each request more costly. Adding examples is helpful if you want to improve the quality of the evaluation at the cost of more tokens.
|
||||
|
||||
Each example must be a dictionary with keys `inputs` and `outputs`.
|
||||
`inputs` must be a dictionary with keys `questions` and `contexts`.
|
||||
`outputs` must be a dictionary with `statements` and `statement_scores`.
|
||||
Here is the expected format:
|
||||
|
||||
```python
|
||||
[
|
||||
{
|
||||
"inputs": {
|
||||
"questions": "What is the capital of Italy?",
|
||||
"contexts": ["Rome is the capital of Italy."],
|
||||
},
|
||||
"outputs": {
|
||||
"statements": [
|
||||
"Rome is the capital of Italy.",
|
||||
"Rome has more than 4 million inhabitants.",
|
||||
],
|
||||
"statement_scores": [1, 0],
|
||||
},
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Below is an example where we use a `ContextRelevanceEvaluator` component to evaluate a response generated based on a provided question and context. The `ContextRelevanceEvaluator` returns a score of 1.0 because it detects one statement in the context, which is relevant to the question.
|
||||
|
||||
```python
|
||||
from haystack.components.evaluators import ContextRelevanceEvaluator
|
||||
|
||||
questions = ["Who created the Python language?"]
|
||||
contexts = [
|
||||
[
|
||||
"Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language. Its design philosophy emphasizes code readability, and its language constructs aim to help programmers write clear, logical code for both small and large-scale software projects.",
|
||||
],
|
||||
]
|
||||
|
||||
evaluator = ContextRelevanceEvaluator()
|
||||
result = evaluator.run(questions=questions, contexts=contexts)
|
||||
print(result["score"])
|
||||
## 1.0
|
||||
print(result["individual_scores"])
|
||||
## [1.0]
|
||||
print(result["results"])
|
||||
## [{'statements': ['Python, created by Guido van Rossum in the late 1980s.'], 'statement_scores': [1], 'score': 1.0}]
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Below is an example where we use a `FaithfulnessEvaluator` and a `ContextRelevanceEvaluator` in a pipeline to evaluate responses and contexts (the content of documents) received by a RAG pipeline based on provided questions. Running a pipeline instead of the individual components simplifies calculating more than one metric.
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack.components.evaluators import (
|
||||
ContextRelevanceEvaluator,
|
||||
FaithfulnessEvaluator,
|
||||
)
|
||||
|
||||
pipeline = Pipeline()
|
||||
context_relevance_evaluator = ContextRelevanceEvaluator()
|
||||
faithfulness_evaluator = FaithfulnessEvaluator()
|
||||
pipeline.add_component("context_relevance_evaluator", context_relevance_evaluator)
|
||||
pipeline.add_component("faithfulness_evaluator", faithfulness_evaluator)
|
||||
|
||||
questions = ["Who created the Python language?"]
|
||||
contexts = [
|
||||
[
|
||||
"Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language. Its design philosophy emphasizes code readability, and its language constructs aim to help programmers write clear, logical code for both small and large-scale software projects.",
|
||||
],
|
||||
]
|
||||
responses = [
|
||||
"Python is a high-level general-purpose programming language that was created by George Lucas.",
|
||||
]
|
||||
|
||||
result = pipeline.run(
|
||||
{
|
||||
"context_relevance_evaluator": {"questions": questions, "contexts": contexts},
|
||||
"faithfulness_evaluator": {
|
||||
"questions": questions,
|
||||
"contexts": contexts,
|
||||
"responses": responses,
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
for evaluator in result:
|
||||
print(result[evaluator]["individual_scores"])
|
||||
## [1.0]
|
||||
## [0.5]
|
||||
for evaluator in result:
|
||||
print(result[evaluator]["score"])
|
||||
## 1.0
|
||||
## 0.5
|
||||
```
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
---
|
||||
title: "DeepEvalEvaluator"
|
||||
id: deepevalevaluator
|
||||
slug: "/deepevalevaluator"
|
||||
description: "The DeepEvalEvaluator evaluates Haystack pipelines using LLM-based metrics. It supports metrics like answer relevancy, faithfulness, contextual relevance, and more."
|
||||
---
|
||||
|
||||
# DeepEvalEvaluator
|
||||
|
||||
The DeepEvalEvaluator evaluates Haystack pipelines using LLM-based metrics. It supports metrics like answer relevancy, faithfulness, contextual relevance, and more.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline has generated the inputs for the Evaluator. |
|
||||
| **Mandatory init variables** | `metric`: One of the DeepEval metrics to use for evaluation |
|
||||
| **Mandatory run variables** | `**inputs`: A keyword arguments dictionary containing the expected inputs. The expected inputs will change based on the metric you are evaluating. See below for more details. |
|
||||
| **Output variables** | `results`: A nested list of metric results. There can be one or more results, depending on the metric. Each result is a dictionary containing: <br /> <br />- `name` - The name of the metric <br />- `score` - The score of the metric <br />- `explanation` - An optional explanation of the score |
|
||||
| **API reference** | [DeepEval](/reference/integrations-deepeval) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/deepeval |
|
||||
|
||||
</div>
|
||||
|
||||
DeepEval is an evaluation framework that provides a number of LLM-based evaluation metrics. You can use the `DeepEvalEvaluator` component to evaluate a Haystack pipeline, such as a retrieval-augmented generated pipeline, against one of the metrics provided by DeepEval.
|
||||
|
||||
## Supported Metrics
|
||||
|
||||
DeepEval supports a number of metrics, which we expose through the [DeepEval metric enumeration.](/reference/integrations-deepeval#deepevalmetric) [`DeepEvalEvaluator`](/reference/integrations-deepeval#deepevalevaluator) in Haystack supports the metrics listed below with the expected `metric_params` while initializing the Evaluator. Many metrics use OpenAI models and require you to set an environment variable `OPENAI_API_KEY`. For a complete guide on these metrics, visit the [DeepEval documentation](https://docs.confident-ai.com/docs/getting-started).
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline has generated the inputs for the Evaluator. |
|
||||
| **Mandatory init variables** | `metric`: One of the DeepEval metrics to use for evaluation |
|
||||
| **Mandatory run variables** | “\*\*inputs”: A keyword arguments dictionary containing the expected inputs. The expected inputs will change based on the metric you are evaluating. See below for more details. |
|
||||
| **Output variables** | `results`: A nested list of metric results. There can be one or more results, depending on the metric. Each result is a dictionary containing: <br /> <br />- `name` - The name of the metric <br />- `score` - The score of the metric <br />- `explanation` - An optional explanation of the score |
|
||||
| **API reference** | [DeepEval](/reference/integrations-deepeval) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/deepeval |
|
||||
|
||||
</div>
|
||||
|
||||
## Parameters Overview
|
||||
|
||||
To initialize a `DeepEvalEvaluator`, you need to provide the following parameters :
|
||||
|
||||
- `metric`: A `DeepEvalMetric`.
|
||||
- `metric_params`: Optionally, if the metric calls for any additional parameters, you should provide them here.
|
||||
|
||||
## Usage
|
||||
|
||||
To use the `DeepEvalEvaluator`, you first need to install the integration:
|
||||
|
||||
```bash
|
||||
pip install deepeval-haystack
|
||||
```
|
||||
|
||||
To use the `DeepEvalEvaluator` you need to follow these steps:
|
||||
|
||||
1. Initialize the `DeepEvalEvaluator` while providing the correct `metric_params` for the metric you are using.
|
||||
2. Run the `DeepEvalEvaluator` on its own or in a pipeline by providing the expected input for the metric you are using.
|
||||
|
||||
### Examples
|
||||
|
||||
**Evaluate Faithfulness**
|
||||
|
||||
To create a faithfulness evaluation pipeline:
|
||||
|
||||
```python
|
||||
from haystack import Pipeline
|
||||
from haystack_integrations.components.evaluators.deepeval import (
|
||||
DeepEvalEvaluator,
|
||||
DeepEvalMetric,
|
||||
)
|
||||
|
||||
pipeline = Pipeline()
|
||||
evaluator = DeepEvalEvaluator(
|
||||
metric=DeepEvalMetric.FAITHFULNESS,
|
||||
metric_params={"model": "gpt-4"},
|
||||
)
|
||||
pipeline.add_component("evaluator", evaluator)
|
||||
```
|
||||
|
||||
To run the evaluation pipeline, you should have the _expected inputs_ for the metric ready at hand. This metric expects a list of `questions` and `contexts`. These should come from the results of the pipeline you want to evaluate.
|
||||
|
||||
```python
|
||||
results = pipeline.run(
|
||||
{
|
||||
"evaluator": {
|
||||
"questions": [
|
||||
"When was the Rhodes Statue built?",
|
||||
"Where is the Pyramid of Giza?",
|
||||
],
|
||||
"contexts": [["Context for question 1"], ["Context for question 2"]],
|
||||
"responses": ["Response for question 1", "response for question 2"],
|
||||
},
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
## Additional References
|
||||
|
||||
🧑🍳 Cookbook: [RAG Pipeline Evaluation Using DeepEval](https://haystack.deepset.ai/cookbook/rag_eval_deep_eval)
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "DocumentMAPEvaluator"
|
||||
id: documentmapevaluator
|
||||
slug: "/documentmapevaluator"
|
||||
description: "The `DocumentMAPEvaluator` evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks to what extent the list of retrieved documents contains only relevant documents as specified in the ground truth labels or also non-relevant documents. This metric is called mean average precision (MAP)."
|
||||
---
|
||||
|
||||
# DocumentMAPEvaluator
|
||||
|
||||
The `DocumentMAPEvaluator` evaluates documents retrieved by Haystack pipelines using ground truth labels. It checks to what extent the list of retrieved documents contains only relevant documents as specified in the ground truth labels or also non-relevant documents. This metric is called mean average precision (MAP).
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Most common position in a pipeline** | On its own or in an evaluation pipeline. To be used after a separate pipeline that has generated the inputs for the Evaluator. |
|
||||
| **Mandatory run variables** | `ground_truth_documents`: A list of a list of ground truth documents. This accounts for one list of ground truth documents per question. <br /> <br />`retrieved_documents`: A list of a list of retrieved documents. This accounts for one list of retrieved documents per question. |
|
||||
| **Output variables** | A dictionary containing: <br /> <br />\- `score`: A number from 0.0 to 1.0 that represents the mean average precision <br /> <br />- `individual_scores`: A list of the individual average precision scores ranging from 0.0 to 1.0 for each input pair of a list of retrieved documents and a list of ground truth documents |
|
||||
| **API reference** | [Evaluators](/reference/evaluators-api) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/evaluators/document_map.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
You can use the `DocumentMAPEvaluator` component to evaluate documents retrieved by a Haystack pipeline, such as a RAG pipeline, against ground truth labels. A higher mean average precision is better, indicating that the list of retrieved documents contains many relevant documents and only a few non-relevant documents or none at all.
|
||||
|
||||
To initialize a `DocumentMAPEvaluator`, there are no parameters required.
|
||||
|
||||
## Usage
|
||||
|
||||
### On its own
|
||||
|
||||
Below is an example where we use a `DocumentMAPEvaluator` component to evaluate documents retrieved for two queries. For the first query, there is one ground truth document and one retrieved document. For the second query, there are two ground truth documents and three retrieved documents.
|
||||
|
||||
```python
|
||||
from haystack import Document
|
||||
from haystack.components.evaluators import DocumentMAPEvaluator
|
||||
|
||||
evaluator = DocumentMAPEvaluator()
|
||||
result = evaluator.run(
|
||||
ground_truth_documents=[
|
||||
[Document(content="France")],
|
||||
[Document(content="9th century"), Document(content="9th")],
|
||||
],
|
||||
retrieved_documents=[
|
||||
[Document(content="France")],
|
||||
[
|
||||
Document(content="9th century"),
|
||||
Document(content="10th century"),
|
||||
Document(content="9th"),
|
||||
],
|
||||
],
|
||||
)
|
||||
print(result["individual_scores"])
|
||||
## [1.0, 0.8333333333333333]
|
||||
print(result["score"])
|
||||
## 0.9166666666666666
|
||||
```
|
||||
|
||||
### In a pipeline
|
||||
|
||||
Below is an example where we use a `DocumentMAPEvaluator` and a `DocumentMRREvaluator` in a pipeline to evaluate two answers and compare them to ground truth answers. Running a pipeline instead of the individual components simplifies calculating more than one metric.
|
||||
|
||||
```python
|
||||
from haystack import Document, Pipeline
|
||||
from haystack.components.evaluators import DocumentMRREvaluator, DocumentMAPEvaluator
|
||||
|
||||
pipeline = Pipeline()
|
||||
mrr_evaluator = DocumentMRREvaluator()
|
||||
map_evaluator = DocumentMAPEvaluator()
|
||||
pipeline.add_component("mrr_evaluator", mrr_evaluator)
|
||||
pipeline.add_component("map_evaluator", map_evaluator)
|
||||
|
||||
ground_truth_documents = [
|
||||
[Document(content="France")],
|
||||
[Document(content="9th century"), Document(content="9th")],
|
||||
]
|
||||
retrieved_documents = [
|
||||
[Document(content="France")],
|
||||
[
|
||||
Document(content="9th century"),
|
||||
Document(content="10th century"),
|
||||
Document(content="9th"),
|
||||
],
|
||||
]
|
||||
|
||||
result = pipeline.run(
|
||||
{
|
||||
"mrr_evaluator": {
|
||||
"ground_truth_documents": ground_truth_documents,
|
||||
"retrieved_documents": retrieved_documents,
|
||||
},
|
||||
"map_evaluator": {
|
||||
"ground_truth_documents": ground_truth_documents,
|
||||
"retrieved_documents": retrieved_documents,
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
for evaluator in result:
|
||||
print(result[evaluator]["individual_scores"])
|
||||
## [1.0, 1.0]
|
||||
## [1.0, 0.8333333333333333]
|
||||
for evaluator in result:
|
||||
print(result[evaluator]["score"])
|
||||
## 1.0
|
||||
## 0.9166666666666666
|
||||
```
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user