wehub/deepset-ai--haystack
1
0
Fork 0
Code Issues Pull Requests Actions 4 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Details
Update Platform Components Table / update (push) Has been cancelled
Details
Docker image release / Build base image (push) Has been cancelled
Details
Sync docs with Docusaurus / sync (push) Has been cancelled
Details
Tests / Check if changed (push) Has been cancelled
Details
Tests / format (push) Has been cancelled
Details
Tests / check-imports (push) Has been cancelled
Details
Tests / Unit / macos-latest (push) Has been cancelled
Details
Tests / Unit / ubuntu-latest (push) Has been cancelled
Details
Tests / Unit / windows-latest (push) Has been cancelled
Details
Tests / mypy (push) Has been cancelled
Details
Tests / Integration / ubuntu-latest (push) Has been cancelled
Details
Tests / Integration / macos-latest (push) Has been cancelled
Details
Tests / Integration / windows-latest (push) Has been cancelled
Details
Tests / notify-slack-on-failure (push) Has been cancelled
Details
Tests / Mark tests as completed (push) Has been cancelled
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:22:28 +08:00
commit c56bef871b
9296 changed files with 1854228 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs-website/versioned_docs/version-2.26/concepts/agents.mdx
+209
View File
@@ -0,0 +1,209 @@
---
title: "Agents"
id: agents
slug: "/agents"
description: "This page explains how to create an AI agent in Haystack capable of retrieving information, generating responses, and taking actions using various Haystack components."
---
# Agents
This page explains how to create an AI agent in Haystack capable of retrieving information, generating responses, and taking actions using various Haystack components.
## What’s an AI Agent?
An AI agent is a system that can:
- Understand user input (text, image, audio, and other queries),
- Retrieve relevant information (documents or structured data),
- Generate intelligent responses (using LLMs like OpenAI or Hugging Face models),
- Perform actions (calling APIs, fetching live data, executing functions).
### Understanding AI Agents
AI agents are autonomous systems that use large language models (LLMs) to make decisions and solve complex tasks. They interact with their environment using tools, memory, and reasoning.
### What Makes an AI Agent
An AI agent is more than a chatbot. It actively plans, chooses the right tools and executes tasks to achieve a goal. Unlike traditional software, it adapts to new information and refines its process as needed.
1. **LLM as the Brain**: The agent's core is an LLM, which understands context, processes natural language and serves as the central intelligence system.
2. **Tools for Interaction**: Agents connect to external tools, APIs, and databases to gather information and take action.
3. **Memory for Context**: Short-term memory helps track conversations, while long-term memory stores knowledge for future interactions.
4. **Reasoning and Planning**: Agents break down complex problems, come up with step-by-step action plans, and adapt based on new data and feedback.
### How AI Agents Work
An AI agent starts with a prompt that defines its role and objectives. It decides when to use tools, gathers data, and refines its approach through loops of reasoning and action. It evaluates progress and adjusts its strategy to improve results.
For example, a customer service agent answers queries using a database. If it lacks an answer, it fetches real-time data, summarizes it, and provides a response. A coding assistant understands project requirements, suggests solutions, and even writes code.
## Key Components
### Agents
Haystack has a universal [Agent](../pipeline-components/agents-1/agent.mdx) component that interacts with chat-based LLMs and tools to solve complex queries. It requires a Chat Generator that supports tools to work and can be customizable according to your needs. Check out the [Agent](../pipeline-components/agents-1/agent.mdx) documentation, or the [example](#tool-calling-agent) below to see how it works.
### Additional Components
You can build an AI agent in Haystack yourself, using the three main elements in a pipeline:
- [Chat Generators](../pipeline-components/generators.mdx) to generate tool calls (with tool name and arguments) or assistant responses with an LLM,
- [`Tool`](../tools/tool.mdx) class that allows the LLM to perform actions such as running a pipeline or calling an external API, connecting to the external world,
- [`ToolInvoker`](../pipeline-components/tools/toolinvoker.mdx) component to execute tool calls generated by an LLM. It parses the LLM's tool-calling responses and invokes the appropriate tool with the correct arguments from the pipeline.
There are three ways of creating a tool in Haystack:
- [`Tool`](../tools/tool.mdx) class – Creates a tool representation for a consistent tool-calling experience across all Generators. It allows for most customization, as you can define its own name and description.
- [`ComponentTool`](../tools/componenttool.mdx) class – Wraps a Haystack component as a callable tool.
- [`@tool`](../tools/tool.mdx#tool-decorator) decorator – Creates tools from Python functions and automatically uses their function name and docstring.
- [Toolset](../tools/toolset.mdx) – A container for grouping multiple tools that can be passed directly to Agents or Generators.
## Example Agents
### Tool-Calling Agent
You can create a similar tool-calling agent with the `Agent` component:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.websearch import SerperDevWebSearch
from haystack.dataclasses import Document, ChatMessage
from haystack.tools.component_tool import ComponentTool
## Create the web search component
web_search = SerperDevWebSearch(top_k=3)
## Create the ComponentTool with simpler parameters
web_tool = ComponentTool(
component=web_search,
name="web_search",
description="Search the web for current information like weather, news, or facts.",
)
## Create the agent with the web tool
tool_calling_agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
system_prompt="""You're a helpful agent. When asked about current information like weather, news, or facts,
use the web_search tool to find the information and then summarize the findings.
When you get web search results, extract the relevant information and present it in a clear,
concise manner.""",
tools=[web_tool],
)
## Run the agent with the user message
user_message = ChatMessage.from_user("How is the weather in Berlin?")
result = tool_calling_agent.run(messages=[user_message])
## Print the result - using .text instead of .content
print(result["messages"][-1].text)
```
Resulting in:
```python
>>> The current weather in Berlin is approximately 60°F. The forecast for today includes clouds in the morning with some sunshine later. The high temperature is expected to be around 65°F, and the low tonight will drop to 40°F.
- **Morning**: 49°F
- **Afternoon**: 57°F
- **Evening**: 47°F
- **Overnight**: 39°F
For more details, you can check the full forecasts on [AccuWeather](https://www.accuweather.com/en/de/berlin/10178/current-weather/178087) or [Weather.com](https://weather.com/weather/today/l/5ca23443513a0fdc1d37ae2ffaf5586162c6fe592a66acc9320a0d0536be1bb9).
```
### Pipeline With Tools
Here’s an example of how you would build a tool-calling agent with the help of `ToolInvoker`.
This is what’s happening in this code example:
1. `OpenAIChatGenerator` uses an LLM to analyze the user's message and determines whether to provide an assistant response or initiate a tool call.
2. `ConditionalRouter` directs the output from the `OpenAIChatGenerator` to `there_are_tool_calls` branch if it’s a tool call or to `final_replies` to return to the user directly.
3. `ToolInvoker` executes the tool call generated by the LLM. `ComponentTool` wraps the `SerperDevWebSearch` component that fetches real-time search results, making it accessible for `ToolInvoker` to execute it as a tool.
4. After the tool provides its output, the `ToolInvoker` sends this information back to the `OpenAIChatGenerator`, along with the original user question stored by the `MessageCollector`.
```python
from haystack import component, Pipeline
from haystack.components.tools import ToolInvoker
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.routers import ConditionalRouter
from haystack.components.websearch import SerperDevWebSearch
from haystack.core.component.types import Variadic
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from typing import Any
## helper component to temporarily store last user query before the tool call
@component()
class MessageCollector:
def __init__(self):
self._messages = []
@component.output_types(messages=list[ChatMessage])
def run(self, messages: Variadic[list[ChatMessage]]) -> dict[str, Any]:
self._messages.extend([msg for inner in messages for msg in inner])
return {"messages": self._messages}
def clear(self):
self._messages = []
## Create a tool from a component
web_tool = ComponentTool(component=SerperDevWebSearch(top_k=3))
## Define routing conditions
routes = [
{
"condition": "{{replies[0].tool_calls | length > 0}}",
"output": "{{replies}}",
"output_name": "there_are_tool_calls",
"output_type": list[ChatMessage],
},
{
"condition": "{{replies[0].tool_calls | length == 0}}",
"output": "{{replies}}",
"output_name": "final_replies",
"output_type": list[ChatMessage],
},
]
## Create the pipeline
tool_agent = Pipeline()
tool_agent.add_component("message_collector", MessageCollector())
tool_agent.add_component(
"generator",
OpenAIChatGenerator(model="gpt-4o-mini", tools=[web_tool]),
)
tool_agent.add_component("router", ConditionalRouter(routes, unsafe=True))
tool_agent.add_component("tool_invoker", ToolInvoker(tools=[web_tool]))
tool_agent.connect("generator.replies", "router")
tool_agent.connect("router.there_are_tool_calls", "tool_invoker")
tool_agent.connect("router.there_are_tool_calls", "message_collector")
tool_agent.connect("tool_invoker.tool_messages", "message_collector")
tool_agent.connect("message_collector", "generator.messages")
messages = [
ChatMessage.from_system(
"You're a helpful agent choosing the right tool when necessary",
),
ChatMessage.from_user("How is the weather in Berlin?"),
]
result = tool_agent.run({"messages": messages})
print(result["router"]["final_replies"][0].text)
```
Resulting in:
```python
>>> The current weather in Berlin is around 46°F (8°C) with cloudy conditions. The high for today is forecasted to reach 48°F (9°C) and the low is expected to be around 37°F (3°C). The humidity is quite high at 92%, and there is a light wind blowing at 4 mph.
For more detailed weather updates, you can check the following links:
- [AccuWeather](https://www.accuweather.com/en/de/berlin/10178/weather-forecast/178087)
- [Weather.com](https://weather.com/weather/today/l/5ca23443513a0fdc1d37ae2ffaf5586162c6fe592a66acc9320a0d0536be1bb9)
```
docs-website/versioned_docs/version-2.26/concepts/agents/state.mdx
+513
View File
@@ -0,0 +1,513 @@
---
title: "State"
id: state
slug: "/state"
description: "`State` is a container for storing shared information during Agent and Tool execution. It provides a structured way to store messages during execution, share data between tools, and store intermediate results throughout an agent's workflow."
---
# State
`State` is a container for storing shared information during Agent and Tool execution. It provides a structured way to store messages during execution, share data between tools, and store intermediate results throughout an agent's workflow.
## Overview
When building agents that use multiple tools, you often need tools to share information with each other. State solves this problem by providing centralized storage that all tools can read from and write to. For example, one tool might retrieve documents while another tool uses those documents to generate an answer.
State uses a schema-based approach where you define:
- What data can be stored,
- The type of each piece of data,
- How values are merged when updated.
### Supported Types
State supports standard Python types:
- Basic types: `str`, `int`, `float`, `bool`, `dict`,
- List types: `list`, `list[str]`, `list[int]`, `list[Document]`,
- Union types: `Union[str, int]`, `Optional[str]`,
- Custom classes and data classes.
### Automatic Message Handling
State automatically includes a `messages` field to store messages during execution. You don't need to define this in your schema.
```python
## State automatically adds messages field
state = State(schema={"user_id": {"type": str}})
## The messages field is available
print("messages" in state.schema) # True
print(state.schema["messages"]["type"]) # list[ChatMessage]
## Access messages
messages = state.get("messages", [])
```
The `messages` field uses `list[ChatMessage]` type and `merge_lists` handler by default, which means new messages are appended during execution.
## Usage
### Creating State
Create State by defining a schema that specifies what data can be stored and their types:
```python
from haystack.components.agents.state import State
## Define the schema
schema = {
"user_name": {"type": str},
"documents": {"type": list},
"count": {"type": int},
}
## Create State with initial data
state = State(schema=schema, data={"user_name": "Alice", "documents": [], "count": 0})
```
### Reading from State
Use the `get()` method to retrieve values:
```python
## Get a value
user_name = state.get("user_name")
## Get a value with a default if key doesn't exist
documents = state.get("documents", [])
## Check if a key exists
if state.has("user_name"):
print(f"User: {state.get('user_name')}")
```
### Writing to State
Use the `set()` method to store or merge values:
```python
## Set a value
state.set("user_name", "Bob")
## Set list values (these are merged by default)
state.set("documents", [{"title": "Doc 1", "content": "Content 1"}])
```
## Schema Definition
The schema defines what data can be stored and how values are updated. Each schema entry consists of:
- `type` (required): The Python type that defines what kind of data can be stored (for example, `str`, `int`, `list`)
- `handler` (optional): A function that determines how new values are merged with existing values when you call `set()`
```python
{
"parameter_name": {
"type": SomeType, # Required: Expected Python type for this field
"handler": Optional[
Callable[[Any, Any], Any]
], # Optional: Function to merge values
},
}
```
If you don't specify a handler, State automatically assigns a default handler based on the type.
### Default Handlers
Handlers control how values are merged when you call `set()` on an existing key. State provides two default handlers:
- `merge_lists`: Combines the lists together (default for list types)
- `replace_values`: Overwrites the existing value (default for non-list types)
```python
from haystack.components.agents.state.state_utils import merge_lists, replace_values
schema = {
"documents": {"type": list}, # Uses merge_lists by default
"user_name": {"type": str}, # Uses replace_values by default
"count": {"type": int}, # Uses replace_values by default
}
state = State(schema=schema)
## Lists are merged by default
state.set("documents", [1, 2])
state.set("documents", [3, 4])
print(state.get("documents")) # Output: [1, 2, 3, 4]
## Other values are replaced
state.set("user_name", "Alice")
state.set("user_name", "Bob")
print(state.get("user_name")) # Output: "Bob"
```
### Custom Handlers
You can define custom handlers for specific merge behavior:
```python
def custom_merge(current_value, new_value):
"""Custom handler that merges and sorts lists."""
current_list = current_value or []
new_list = new_value if isinstance(new_value, list) else [new_value]
return sorted(current_list + new_list)
schema = {"numbers": {"type": list, "handler": custom_merge}}
state = State(schema=schema)
state.set("numbers", [3, 1])
state.set("numbers", [2, 4])
print(state.get("numbers")) # Output: [1, 2, 3, 4]
```
You can also override handlers for individual operations:
```python
def concatenate_strings(current, new):
return f"{current}-{new}" if current else new
schema = {"user_name": {"type": str}}
state = State(schema=schema)
state.set("user_name", "Alice")
state.set("user_name", "Bob", handler_override=concatenate_strings)
print(state.get("user_name")) # Output: "Alice-Bob"
```
## Using State with Agents
To use State with an Agent, define a state schema when creating the Agent. The Agent automatically manages State throughout its execution.
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool
## Define a simple calculation tool
def calculate(expression: str) -> dict:
"""Evaluate a mathematical expression."""
result = eval(expression, {"__builtins__": {}})
return {"result": result}
## Create a tool that writes to state
calculator_tool = Tool(
name="calculator",
description="Evaluate basic math expressions",
parameters={
"type": "object",
"properties": {"expression": {"type": "string"}},
"required": ["expression"],
},
function=calculate,
outputs_to_state={"calc_result": {"source": "result"}},
)
## Create agent with state schema
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[calculator_tool],
state_schema={"calc_result": {"type": int}},
)
## Run the agent
result = agent.run(messages=[ChatMessage.from_user("Calculate 15 + 27")])
## Access the state from results
calc_result = result["calc_result"]
print(calc_result) # Output: 42
```
## Tools and State
Tools interact with State through two mechanisms: `inputs_from_state` and `outputs_to_state`.
### Reading from State: `inputs_from_state`
Tools can automatically read values from State and use them as parameters. The `inputs_from_state` parameter maps state keys to tool parameter names.
```python
def search_documents(query: str, user_context: str) -> dict:
"""Search documents using query and user context."""
return {"results": [f"Found results for '{query}' (user: {user_context})"]}
## Create tool that reads from state
search_tool = Tool(
name="search",
description="Search documents",
parameters={
"type": "object",
"properties": {"query": {"type": "string"}, "user_context": {"type": "string"}},
"required": ["query"],
},
function=search_documents,
inputs_from_state={
"user_name": "user_context",
}, # Maps state's "user_name" to the tool’s input parameter “user_context”
)
## Define agent with state schema including user_name
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[search_tool],
state_schema={"user_name": {"type": str}, "search_results": {"type": list}},
)
## Initialize agent with user context
result = agent.run(
messages=[ChatMessage.from_user("Search for Python tutorials")],
user_name="Alice", # All additional kwargs passed to Agent at runtime are put into State
)
```
When the tool is invoked, the Agent automatically retrieves the value from State and passes it to the tool function.
### Writing to State: `outputs_to_state`
Tools can write their results back to State. The `outputs_to_state` parameter defines mappings from tool outputs to state keys.
The structure of the output is: `{”state_key”: {”source”: “tool_result_key”}}`.
```python
def retrieve_documents(query: str) -> dict:
"""Retrieve documents based on query."""
return {
"documents": [
{"title": "Doc 1", "content": "Content about Python"},
{"title": "Doc 2", "content": "More about Python"},
],
"count": 2,
"query": query,
}
## Create tool that writes to state
retrieval_tool = Tool(
name="retrieve",
description="Retrieve relevant documents",
parameters={
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
function=retrieve_documents,
outputs_to_state={
"documents": {
"source": "documents",
}, # Maps tool's "documents" output to state's "documents"
"result_count": {
"source": "count",
}, # Maps tool's "count" output to state's "result_count"
"last_query": {
"source": "query",
}, # Maps tool's "query" output to state's "last_query"
},
)
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[retrieval_tool],
state_schema={
"documents": {"type": list},
"result_count": {"type": int},
"last_query": {"type": str},
},
)
result = agent.run(messages=[ChatMessage.from_user("Find information about Python")])
## Access state values from result
documents = result["documents"]
result_count = result["result_count"]
last_query = result["last_query"]
print(documents) # List of retrieved documents
print(result_count) # 2
print(last_query) # "Find information about Python"
```
Each mapping can specify:
- `source`: Which field from the tool's output to use
- `handler`: Optional custom function for merging values
If you omit the `source`, the entire tool result is stored:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool
def get_user_info() -> dict:
"""Get user information."""
return {"name": "Alice", "email": "alice@example.com", "role": "admin"}
## Tool that stores entire result
info_tool = Tool(
name="get_info",
description="Get user information",
parameters={"type": "object", "properties": {}},
function=get_user_info,
outputs_to_state={
"user_info": {}, # Stores entire result dict in state's "user_info"
},
)
## Create agent with matching state schema
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[info_tool],
state_schema={
"user_info": {"type": dict}, # Schema must match the tool's output type
},
)
## Run the agent
result = agent.run(messages=[ChatMessage.from_user("Get the user information")])
## Access the complete result from state
user_info = result["user_info"]
print(
user_info,
) # Output: {"name": "Alice", "email": "alice@example.com", "role": "admin"}
print(user_info["name"]) # Output: "Alice"
print(user_info["email"]) # Output: "alice@example.com"
```
### Combining Inputs and Outputs
Tools can both read from and write to State, enabling tool chaining:
```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool
def process_documents(documents: list, max_results: int) -> dict:
"""Process documents and return filtered results."""
processed = documents[:max_results]
return {"processed_docs": processed, "processed_count": len(processed)}
processing_tool = Tool(
name="process",
description="Process retrieved documents",
parameters={
"type": "object",
"properties": {"max_results": {"type": "integer"}},
"required": ["max_results"],
},
function=process_documents,
inputs_from_state={"documents": "documents"}, # Reads documents from state
outputs_to_state={
"final_docs": {"source": "processed_docs"},
"final_count": {"source": "processed_count"},
},
)
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[retrieval_tool, processing_tool], # Chain tools using state
state_schema={
"documents": {"type": list},
"final_docs": {"type": list},
"final_count": {"type": int},
},
)
## Run the agent - tools will chain through state
result = agent.run(
messages=[ChatMessage.from_user("Find and process 3 documents about Python")],
)
## Access the final processed results
final_docs = result["final_docs"]
final_count = result["final_count"]
print(f"Processed {final_count} documents")
print(final_docs)
```
## Complete Example
This example shows a multi-tool agent workflow where tools share data through State:
```python
import math
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool
## Tool 1: Calculate factorial
def factorial(n: int) -> dict:
"""Calculate the factorial of a number."""
result = math.factorial(n)
return {"result": result}
factorial_tool = Tool(
name="factorial",
description="Calculate the factorial of a number",
parameters={
"type": "object",
"properties": {"n": {"type": "integer"}},
"required": ["n"],
},
function=factorial,
outputs_to_state={"factorial_result": {"source": "result"}},
)
## Tool 2: Perform calculation
def calculate(expression: str) -> dict:
"""Evaluate a mathematical expression."""
result = eval(expression, {"__builtins__": {}})
return {"result": result}
calculator_tool = Tool(
name="calculator",
description="Evaluate basic math expressions",
parameters={
"type": "object",
"properties": {"expression": {"type": "string"}},
"required": ["expression"],
},
function=calculate,
outputs_to_state={"calc_result": {"source": "result"}},
)
## Create agent with both tools
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[calculator_tool, factorial_tool],
state_schema={"calc_result": {"type": int}, "factorial_result": {"type": int}},
)
## Run the agent
result = agent.run(
messages=[
ChatMessage.from_user("Calculate the factorial of 5, then multiply it by 2"),
],
)
## Access state values from result
factorial_result = result["factorial_result"]
calc_result = result["calc_result"]
## Access messages from execution
for message in result["messages"]:
print(f"{message.role}: {message.text}")
```
docs-website/versioned_docs/version-2.26/concepts/components.mdx
+59
View File
@@ -0,0 +1,59 @@
---
title: "Components"
id: components
slug: "/components"
description: "Components are the building blocks of a pipeline. They perform tasks such as preprocessing, retrieving, or summarizing text while routing queries through different branches of a pipeline. This page is a summary of all component types available in Haystack."
---
import ClickableImage from "@site/src/components/ClickableImage";
# Components
Components are the building blocks of a pipeline. They perform tasks such as preprocessing, retrieving, or summarizing text while routing queries through different branches of a pipeline. This page is a summary of all component types available in Haystack.
Components are connected to each other using a [pipeline](pipelines.mdx), and they function like building blocks that can be easily switched out for each other. A component can take the selected outputs of other components as input. You can also provide input to a component when you call `pipeline.run()`.
## Stand-Alone or In a Pipeline
You can integrate components in a pipeline to perform a specific task. But you can also use some of them stand-alone, outside of a pipeline. For example, you can run `DocumentWriter` on its own, to write documents into a Document Store. To check how to use a component and if it's usable outside of a pipeline, check the _Usage_ section on the component's documentation page.
Each component has a `run()` method. When you connect components in a pipeline, and you run the pipeline by calling `Pipeline.run()`, it invokes the `run()` method for each component sequentially.
## Input and Output
To connect components in a pipeline, you need to know the names of the inputs and outputs they accept. The output of one component must be compatible with the input the subsequent component accepts. For example, to connect Retriever and Ranker in a pipeline, you must know that the Retriever outputs `documents` and the Ranker accepts `documents` as input.
The mandatory inputs and outputs are listed in a table at the top of each component's documentation page so that you can quickly check them:
<ClickableImage src="/img/3a53f3e-inputs_and_outputs.png" alt="DocumentWriter component specification table showing Name, Folder Path, Position in Pipeline, Inputs (documents list), and Outputs (documents_written integer)" />
You can also look them up in the code in the component`run()` method. Here's an example of the inputs and outputs of `TransformerSimilarityRanker`:
```python
@component.output_types(documents=List[Document]) # "documents" is the output name you need when connecting components in a pipeline
def run(self, query: str, documents: List[Document], top_k: Optional[int] = None):# "query" and "documents" are the mandatory inputs, additionally you can also specify the optional top_k parameter
"""
Returns a list of Documents ranked by their similarity to the given query.
:param query: Query string.
:param documents: List of Documents.
:param top_k: The maximum number of Documents you want the Ranker to return.
:return: List of Documents sorted by their similarity to the query with the most similar Documents appearing first.
"""
```
## Warming Up Components
Components that use heavy resources, like LLMs or embedding models, have a `warm_up()` method that loads the necessary resources (such as models) into memory. This method is automatically called the first time the component runs, so you can use components directly without explicitly calling `warm_up()`:
```python
from haystack import Document
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
doc = Document(content="I love pizza!")
doc_embedder = SentenceTransformersDocumentEmbedder()
result = doc_embedder.run([doc]) # warm_up() is called automatically on first run
print(result["documents"][0].embedding)
```
You can still call `warm_up()` explicitly if you want to control when resources are loaded.
docs-website/versioned_docs/version-2.26/concepts/components/custom-components.mdx
+180
View File
@@ -0,0 +1,180 @@
---
title: "Creating Custom Components"
id: custom-components
slug: "/custom-components"
description: "Create your own components and use them standalone or in pipelines."
---
# Creating Custom Components
Create your own components and use them standalone or in pipelines.
With Haystack, you can easily create any custom components for various tasks, from filtering results to integrating with external software. You can then insert, reuse, and share these components within Haystack or even with an external audience by packaging them and submitting them to [Haystack Integrations](../integrations.mdx)!
## Requirements
Here are the requirements for all custom components:
- `@component`: This decorator marks a class as a component, allowing it to be used in a pipeline.
- `run()`: This is a required method in every component. It accepts input arguments and returns a `dict`. The inputs can either come from the pipeline when it’s executed, or from the output of another component when connected using `connect()`. The `run()` method should be compatible with the input/output definitions declared for the component. See an [Extended Example](#extended-example) below to check how it works.
:::note[Avoid in-place input mutation]
When building custom components, do not change the component's inputs directly. Instead, work on a copy or a new version of the input, modify that, and return it. The reason for this is that the original input values might be reused by other components or by later pipeline steps. Mutating the input directly can lead to unintended side effects and bugs in the pipeline, as other components might rely on the original input values.
When only one or a few fields of the input need to be changed (for example, `meta` on a `Document`), use `dataclasses.replace()` to create a new instance with the updated fields. This is simpler and more efficient than deep-copying the whole object:
```python
from dataclasses import replace
def run(self, documents):
updated = [replace(doc, meta={**doc.meta, "processed": True}) for doc in documents]
return {"documents": updated}
```
When you need to modify nested mutable structures, for example `list` or `dict` attributes, or update many fields of the dataclass instance, use a full deep copy instead:
```python
import copy
def run(self, documents):
documents_copy = copy.deepcopy(documents)
# mutate documents_copy safely here
return {"documents": documents_copy}
```
:::
### Inputs and Outputs
Next, define the inputs and outputs for your component.
#### Inputs
You can choose between three input options:
- `set_input_type`: This method defines or updates a single input socket for a component instance. It’s ideal for adding or modifying a specific input at runtime without affecting others. Use this when you need to dynamically set or modify a single input based on specific conditions.
- `set_input_types`: This method allows you to define multiple input sockets at once, replacing any existing inputs. It’s useful when you know all the inputs the component will need and want to configure them in bulk. Use this when you want to define multiple inputs during initialization.
- Declaring arguments directly in the `run()` method. Use this method when the component’s inputs are static and known at the time of class definition.
#### Outputs
You can choose between two output options:
- `@component.output_types`: This decorator defines the output types and names at the time of class definition. The output names and types must match the `dict` returned by the `run()` method. Use this when the output types are static and known in advance. This decorator is cleaner and more readable for static components.
- `set_output_types`: This method defines or updates multiple output sockets for a component instance at runtime. It’s useful when you need flexibility in configuring outputs dynamically. Use this when the output types need to be set at runtime for greater flexibility.
## Short Example
Here is an example of a simple minimal component setup:
```python
from haystack import component
@component
class WelcomeTextGenerator:
"""
A component generating personal welcome message and making it upper case
"""
@component.output_types(welcome_text=str, note=str)
def run(self, name: str):
return {
"welcome_text": f"Hello {name}, welcome to Haystack!".upper(),
"note": "welcome message is ready",
}
```
Here, the custom component `WelcomeTextGenerator` accepts one input: `name` string and returns two outputs: `welcome_text` and `note`.
## Extended Example
Check out an example below on how to create two custom components and connect them in a Haystack pipeline.
```python
# import necessary dependencies
from haystack import component, Pipeline
# Create two custom components. Note the mandatory @component decorator and @component.output_types, as well as the mandatory run method.
@component
class WelcomeTextGenerator:
"""
A component generating personal welcome message and making it upper case
"""
@component.output_types(welcome_text=str, note=str)
def run(self, name: str):
return {
"welcome_text": (
"Hello {name}, welcome to Haystack!".format(name=name)
).upper(),
"note": "welcome message is ready",
}
@component
class WhitespaceSplitter:
"""
A component for splitting the text by whitespace
"""
@component.output_types(split_text=list[str])
def run(self, text: str):
return {"split_text": text.split()}
# create a pipeline and add the custom components to it
text_pipeline = Pipeline()
text_pipeline.add_component(
name="welcome_text_generator",
instance=WelcomeTextGenerator(),
)
text_pipeline.add_component(name="splitter", instance=WhitespaceSplitter())
# connect the components
text_pipeline.connect(
sender="welcome_text_generator.welcome_text",
receiver="splitter.text",
)
# define the result and run the pipeline
result = text_pipeline.run({"welcome_text_generator": {"name": "Bilge"}})
print(result["splitter"]["split_text"])
```
## Extending the Existing Components
To extend already existing components in Haystack, subclass an existing component and use the `@component` decorator to mark it. Override or extend the `run()` method to process inputs and outputs. Call `super()` with the derived class name from the init of the derived class to avoid initialization issues:
```python
class DerivedComponent(BaseComponent):
def __init__(self):
super(DerivedComponent, self).__init__()
## ...
dc = DerivedComponent() # ok
```
An example of an extended component is Haystack's [FaithfulnessEvaluator](https://github.com/deepset-ai/haystack/blob/e5a80722c22c59eb99416bf0cd712f6de7cd581a/haystack/components/evaluators/faithfulness.py) derived from LLMEvaluator.
## Project Template
If you're building a custom component that you want to package and share, we provide a [GitHub template repository](https://github.com/deepset-ai/custom-component) that gives you a ready-made project structure. It includes the boilerplate for packaging, testing, and distributing your custom component as a standalone Python package. Use it to quickly scaffold a new integration or reusable component without setting up the project from scratch.
## Additional References
🧑‍🍳 Cookbooks:
- [Build quizzes and adventures with Character Codex and llamafile](https://haystack.deepset.ai/cookbook/charactercodex_llamafile/)
- [Run tasks concurrently within a custom component](https://haystack.deepset.ai/cookbook/concurrent_tasks/)
- [Chat With Your SQL Database](https://haystack.deepset.ai/cookbook/chat_with_sql_3_ways/)
- [Hacker News Summaries with Custom Components](https://haystack.deepset.ai/cookbook/hackernews-custom-component-rag/)
docs-website/versioned_docs/version-2.26/concepts/components/supercomponents.mdx
+187
View File
@@ -0,0 +1,187 @@
---
title: "SuperComponents"
id: supercomponents
slug: "/supercomponents"
description: "`SuperComponent` lets you wrap a complete pipeline and use it like a single component. This is helpful when you want to simplify the interface of a complex pipeline, reuse it in different contexts, or expose only the necessary inputs and outputs."
---
# SuperComponents
`SuperComponent` lets you wrap a complete pipeline and use it like a single component. This is helpful when you want to simplify the interface of a complex pipeline, reuse it in different contexts, or expose only the necessary inputs and outputs.
## `@super_component` decorator (recommended)
Haystack now provides a simple `@super_component` decorator for wrapping a pipeline as a component. All you need is to create a class with the decorator, and to include an `pipeline` attribute.
With this decorator, the `to_dict` and `from_dict` serialization is optional, as is the input and output mapping.
### Example
The custom HybridRetriever example SuperComponent below turns your query into embeddings, then runs both a BM25 search and an embedding-based search at the same time. It finally merges those two result sets and returns the combined documents.
```python
## pip install haystack-ai datasets "sentence-transformers>=3.0.0"
from haystack import Document, Pipeline, super_component
from haystack.components.joiners import DocumentJoiner
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers import (
InMemoryBM25Retriever,
InMemoryEmbeddingRetriever,
)
from haystack.document_stores.in_memory import InMemoryDocumentStore
from datasets import load_dataset
@super_component
class HybridRetriever:
def __init__(
self,
document_store: InMemoryDocumentStore,
embedder_model: str = "BAAI/bge-small-en-v1.5",
):
embedding_retriever = InMemoryEmbeddingRetriever(document_store)
bm25_retriever = InMemoryBM25Retriever(document_store)
text_embedder = SentenceTransformersTextEmbedder(embedder_model)
document_joiner = DocumentJoiner()
self.pipeline = Pipeline()
self.pipeline.add_component("text_embedder", text_embedder)
self.pipeline.add_component("embedding_retriever", embedding_retriever)
self.pipeline.add_component("bm25_retriever", bm25_retriever)
self.pipeline.add_component("document_joiner", document_joiner)
self.pipeline.connect("text_embedder", "embedding_retriever")
self.pipeline.connect("bm25_retriever", "document_joiner")
self.pipeline.connect("embedding_retriever", "document_joiner")
dataset = load_dataset("HaystackBot/medrag-pubmed-chunk-with-embeddings", split="train")
docs = [
Document(content=doc["contents"], embedding=doc["embedding"]) for doc in dataset
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)
query = "What treatments are available for chronic bronchitis?"
result = HybridRetriever(document_store).run(text=query, query=query)
print(result)
```
### Input Mapping
You can optionally map the input names of your SuperComponent to the actual sockets inside the pipeline.
```python
input_mapping = {"query": ["retriever.query", "prompt.query"]}
```
### Output Mapping
You can also map the pipeline's output sockets that you want to expose to the SuperComponent's output names.
```python
output_mapping = {"llm.replies": "replies"}
```
If you don’t provide mappings, SuperComponent will try to auto-detect them. So, if multiple components have outputs with the same name, we recommend using `output_mapping` to avoid conflicts.
## SuperComponent class
Haystack also gives you an option to inherit from SuperComponent class. This option requires `to_dict` and `from_dict` serialization, as well as the input and output mapping described above.
### Example
Here is a simple example of initializing a `SuperComponent` with a pipeline:
```python
from haystack import Pipeline, SuperComponent
with open("pipeline.yaml", "r") as file:
pipeline = Pipeline.load(file)
super_component = SuperComponent(pipeline)
```
The example pipeline below retrieves relevant documents based on a user query, builds a custom prompt using those documents, then sends the prompt to an `OpenAIChatGenerator` to create an answer. The `SuperComponent` wraps the pipeline so it can be run with a simple input (`query`) and returns a clean output (`replies`).
```python
from haystack import Pipeline, SuperComponent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders import ChatPromptBuilder
from haystack.components.retrievers import InMemoryBM25Retriever
from haystack.dataclasses.chat_message import ChatMessage
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.dataclasses import Document
document_store = InMemoryDocumentStore()
documents = [
Document(content="Paris is the capital of France."),
Document(content="London is the capital of England."),
]
document_store.write_documents(documents)
prompt_template = [
ChatMessage.from_user(
'''
According to the following documents:
{% for document in documents %}
{{document.content}}
{% endfor %}
Answer the given question: {{query}}
Answer:
'''
)
]
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
pipeline = Pipeline()
pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", OpenAIChatGenerator())
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder.prompt", "llm.messages")
## Create a super component with simplified input/output mapping
wrapper = SuperComponent(
pipeline=pipeline,
input_mapping={
"query": ["retriever.query", "prompt_builder.query"],
},
output_mapping={
"llm.replies": "replies",
"retriever.documents": "documents"
}
)
## Run the pipeline with simplified interface
result = wrapper.run(query="What is the capital of France?")
print(result)
{'replies': [ChatMessage(_role=<ChatRole.ASSISTANT: 'assistant'>,
_content=[TextContent(text='The capital of France is Paris.')],...)
```
## Type Checking and Static Code Analysis
Creating SuperComponents using the @super_component decorator can induce type or linting errors. One way to avoid these issues is to add the exposed public methods to your SuperComponent. Here's an example:
```python
from typing import TYPE_CHECKING
if TYPE_CHECKING:
def run(self, *, documents: list[Document]) -> dict[str, list[Document]]: ...
def warm_up(self) -> None: # noqa: D102
...
```
## Ready-Made SuperComponents
You can see two implementations of SuperComponents already integrated in Haystack:
- [DocumentPreprocessor](../../pipeline-components/preprocessors/documentpreprocessor.mdx)
- [MultiFileConverter](../../pipeline-components/converters/multifileconverter.mdx)
- [OpenSearchHybridRetriever](../../pipeline-components/retrievers/opensearchhybridretriever.mdx)
docs-website/versioned_docs/version-2.26/concepts/concepts-overview.mdx
+56
View File
@@ -0,0 +1,56 @@
---
title: "Haystack Concepts Overview"
id: concepts-overview
slug: "/concepts-overview"
description: "Haystack provides all the tools you need to build custom agents and RAG pipelines with LLMs that work for you. This includes everything from prototyping to deployment. This page discusses the most important concepts Haystack operates on."
---
import ClickableImage from "@site/src/components/ClickableImage";
# Haystack Concepts Overview
Haystack provides all the tools you need to build custom agents and RAG pipelines with LLMs that work for you. This includes everything from prototyping to deployment. This page discusses the most important concepts Haystack operates on.
### Components
Haystack offers various components, each performing different kinds of tasks. You can see the whole variety in the **PIPELINE COMPONENTS** section in the left-side navigation. These are often powered by the latest Large Language Models (LLMs) and transformer models. Code-wise, they are Python classes with methods you can directly call. Most commonly, all you need to do is initialize the component with the required parameters and then run it with a `run()` method.
Working on this level with Haystack components is a hands-on approach. Components define the name and the type of all of their inputs and outputs. The Component API reduces complexity and makes it easier to [create custom components](components/custom-components.mdx), for example, for third-party APIs and databases. Haystack validates the connections between components before running the pipeline and, if needed, generates error messages with instructions on fixing the errors.
#### Generators
[Generators](../pipeline-components/generators.mdx) are responsible for generating text responses after you give them a prompt. They are specific for each LLM technology (OpenAI, Cohere, local models, and others). There are two types of Generators: chat and non-chat:
- The chat ones enable chat completion and are designed for conversational contexts. It expects a list of messages to interact with the user.
- The non-chat Generators use LLMs for simpler text generation (for example, translating or summarizing text).
Read more about various Generators in our [guides](../pipeline-components/generators/guides-to-generators/choosing-the-right-generator.mdx).
#### Retrievers
[Retrievers](../pipeline-components/retrievers.mdx) go through all the documents in a Document Store, select the ones that match the user query, and pass it on to the next component. There are various Retrievers that are customized for specific Document Stores. This means that they can handle specific requirements for each database using customized parameters.
For example, for Elasticsearch Document Store, you will find both the Document Store and Retriever packages in its GitHub [repo](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch).
### Document Stores
[Document Store](document-store.mdx) is an object that stores your documents in Haystack, like an interface to a storage database. It uses specific functions like `write_documents()` or `delete_documents()` to work with data. Various components have access to the Document Store and can interact with it by, for example, reading or writing Documents.
If you are working with more complex pipelines in Haystack, you can use a [`DocumentWriter`](../pipeline-components/writers/documentwriter.mdx) component to write data into Document Stores for you
### Data Classes
You can use different [data classes](data-classes.mdx) in Haystack to carry the data through the system. The data classes are mostly likely to appear as inputs or outputs of your pipelines.
`Document` class contains information to be carried through the pipeline. It can be text, metadata, tables, or binary data. Documents can be written into Document Stores but also written and read by other components.
`Answer` class holds not only the answer generated in a pipeline but also the originating query and metadata.
### Pipelines
Finally, you can combine various components, Document Stores, and integrations into [pipelines](pipelines.mdx) to create powerful and customizable systems. It is a highly flexible system that allows you to have simultaneous flows, standalone components, loops, and other types of connections. You can have the preprocessing, indexing, and querying steps all in one pipeline, or you can split them up according to your needs.
If you want to reuse pipelines, you can save them into a convenient format (YAML, TOML, and more) on a disk or share them around using the [serialization](pipelines/serialization.mdx) process.
Here is a short Haystack pipeline, illustrated:
<ClickableImage src="/img/00f5fe8-Pipeline_Illustrations_2.png" alt="RAG architecture overview showing query flow through retrieval and generation stages, with document stores providing context for the language model" />
docs-website/versioned_docs/version-2.26/concepts/data-classes.mdx
+303
View File
@@ -0,0 +1,303 @@
---
title: "Data Classes"
id: data-classes
slug: "/data-classes"
description: "In Haystack, there are a handful of core classes that are regularly used in many different places. These are classes that carry data through the system and you are likely to interact with these as either the input or output of your pipeline."
---
# Data Classes
In Haystack, there are a handful of core classes that are regularly used in many different places. These are classes that carry data through the system and you are likely to interact with these as either the input or output of your pipeline.
Haystack uses data classes to help components communicate with each other in a simple and modular way. By doing this, data flows seamlessly through the Haystack pipelines. This page goes over the available data classes in Haystack: ByteStream, Answer (along with its variants ExtractedAnswer and GeneratedAnswer), ChatMessage, Document, and StreamingChunk, explaining how they contribute to the Haystack ecosystem.
You can check out the detailed parameters in our [Data Classes](/reference/data-classes-api) API reference.
### Answer
#### Overview
The `Answer` class serves as the base for responses generated within Haystack, containing the answer's data, the originating query, and additional metadata.
#### Key Features
- Adaptable data handling, accommodating any data type (`data`).
- Query tracking for contextual relevance (`query`).
- Extensive metadata support for detailed answer description.
#### Attributes
```python
@dataclass
class Answer:
data: Any
query: str
meta: Dict[str, Any]
```
### ExtractedAnswer
#### Overview
`ExtractedAnswer` is a subclass of `Answer` that deals explicitly with answers derived from Documents, offering more detailed attributes.
#### Key Features
- Includes reference to the originating `Document`.
- Score attribute to quantify the answer's confidence level.
- Optional start and end indices for pinpointing answer location within the source.
#### Attributes
```python
@dataclass
class ExtractedAnswer:
query: str
score: float
data: Optional[str] = None
document: Optional[Document] = None
context: Optional[str] = None
document_offset: Optional["Span"] = None
context_offset: Optional["Span"] = None
meta: Dict[str, Any] = field(default_factory=dict)
```
### GeneratedAnswer
#### Overview
`GeneratedAnswer` extends the `Answer` class to accommodate answers generated from multiple Documents.
#### Key Features
- Handles string-type data.
- Links to a list of `Document` objects, enhancing answer traceability.
#### Attributes
```python
@dataclass
class GeneratedAnswer:
data: str
query: str
documents: List[Document]
meta: Dict[str, Any] = field(default_factory=dict)
```
### ByteStream
#### Overview
`ByteStream` represents binary object abstraction in the Haystack framework and is crucial for handling various binary data formats.
#### Key Features
- Holds binary data and associated metadata.
- Optional MIME type specification for flexibility.
- File interaction methods (`to_file`, `from_file_path`, `from_string`) for easy data manipulation.
#### Attributes
```python
@dataclass(repr=False)
class ByteStream:
data: bytes
meta: Dict[str, Any] = field(default_factory=dict, hash=False)
mime_type: Optional[str] = field(default=None)
```
#### Example
```python
from haystack.dataclasses.byte_stream import ByteStream
image = ByteStream.from_file_path("dog.jpg")
```
### ChatMessage
`ChatMessage` is the central abstraction to represent a message for a LLM. It contains role, metadata and several types of content, including text, tool calls and tool calls results.
Read the detailed documentation for the `ChatMessage` data class on a dedicated [ChatMessage](data-classes/chatmessage.mdx) page.
### Document
#### Overview
`Document` represents a central data abstraction in Haystack, capable of holding text, tables, and binary data.
#### Key Features
- Unique ID for each document.
- Multiple content types are supported: text, binary (`blob`).
- Custom metadata and scoring for advanced document management.
- Optional embedding for AI-based applications.
#### Attributes
```python
@dataclass
class Document(metaclass=_BackwardCompatible):
id: str = field(default="")
content: Optional[str] = field(default=None)
blob: Optional[ByteStream] = field(default=None)
meta: Dict[str, Any] = field(default_factory=dict)
score: Optional[float] = field(default=None)
embedding: Optional[List[float]] = field(default=None)
sparse_embedding: Optional[SparseEmbedding] = field(default=None)
```
#### Example
```python
from haystack import Document
documents = Document(
content="Here are the contents of your document",
embedding=[0.1] * 768,
)
```
### StreamingChunk
#### Overview
`StreamingChunk` represents a partially streamed LLM response, enabling real-time LLM response processing. It encapsulates a segment of streamed content along with associated metadata and provides comprehensive information about the streaming state.
#### Key Features
- String-based content representation for text chunks
- Support for tool calls and tool call results
- Component tracking and metadata management
- Streaming state indicators (start, finish reason)
- Content block indexing for multi-part responses
#### Attributes
```python
@dataclass
class StreamingChunk:
content: str
meta: dict[str, Any] = field(default_factory=dict, hash=False)
component_info: Optional[ComponentInfo] = field(default=None)
index: Optional[int] = field(default=None)
tool_calls: Optional[list[ToolCallDelta]] = field(default=None)
tool_call_result: Optional[ToolCallResult] = field(default=None)
start: bool = field(default=False)
finish_reason: Optional[FinishReason] = field(default=None)
reasoning: Optional[ReasoningContent] = field(default=None)
```
#### Example
```python
from haystack.dataclasses import StreamingChunk, ToolCallDelta, ReasoningContent
## Basic text chunk
chunk = StreamingChunk(
content="Hello world",
start=True,
meta={"model": "gpt-5-mini"},
)
## Tool call chunk
tool_chunk = StreamingChunk(
content="",
tool_calls=[
ToolCallDelta(
index=0,
tool_name="calculator",
arguments='{"operation": "add", "a": 2, "b": 3}',
),
],
index=0,
start=False,
finish_reason="tool_calls",
)
## Reasoning chunk
reasoning_chunk = StreamingChunk(
content="",
reasoning=ReasoningContent(
reasoning_text="Thinking step by step about the answer.",
),
index=0,
start=True,
meta={"model": "gpt-4.1-mini"},
)
```
### ToolCallDelta
#### Overview
`ToolCallDelta` represents a tool call prepared by the model, usually contained in an assistant message during streaming.
#### Attributes
```python
@dataclass
class ToolCallDelta:
index: int
tool_name: Optional[str] = field(default=None)
arguments: Optional[str] = field(default=None)
id: Optional[str] = field(default=None)
extra: Optional[Dict[str, Any]] = field(default=None)
```
### ComponentInfo
#### Overview
The `ComponentInfo` class represents information about a component within a Haystack pipeline. It is used to track the type and name of components that generate or process data, aiding in debugging, tracing, and metadata management throughout the pipeline.
#### Key Features
- Stores the type of the component (including module and class name).
- Optionally stores the name assigned to the component in the pipeline.
- Provides a convenient class method to create a `ComponentInfo` instance from a `Component` object.
#### Attributes
```python
@dataclass
class ComponentInfo:
type: str
name: Optional[str] = field(default=None)
@classmethod
def from_component(cls, component: Component) -> "ComponentInfo": ...
```
#### Example
```python
from haystack.dataclasses.streaming_chunk import ComponentInfo
from haystack.core.component import Component
class MyComponent(Component): ...
component = MyComponent()
info = ComponentInfo.from_component(component)
print(info.type) # e.g., 'my_module.MyComponent'
print(info.name) # Name assigned in the pipeline, if any
```
### SparseEmbedding
#### Overview
The `SparseEmbedding` class represents a sparse embedding: a vector where most values are zeros.
#### Attributes
- `indices`: List of indices of non-zero elements in the embedding.
- `values`: List of values of non-zero elements in the embedding.
### Tool
`Tool` is a data class representing a tool that Language Models can prepare a call for.
Read the detailed documentation for the `Tool` data class on a dedicated [Tool](../tools/tool.mdx) page.
docs-website/versioned_docs/version-2.26/concepts/data-classes/chatmessage.mdx
+413
View File
@@ -0,0 +1,413 @@
---
title: "ChatMessage"
id: chatmessage
slug: "/chatmessage"
description: "`ChatMessage` is the central abstraction to represent a message for a LLM. It contains role, metadata and several types of content, including text, images, tool calls, tool call results, and reasoning content."
---
# ChatMessage
`ChatMessage` is the central abstraction to represent a message for a LLM. It contains role, metadata and several types of content, including text, images, tool calls, tool call results, and reasoning content.
To create a `ChatMessage` instance, use `from_user`, `from_system`, `from_assistant`, and `from_tool` class methods.
The [content](#types-of-content) of the `ChatMessage` can then be inspected using the `text`, `texts`, `image`, `images`, `file`, `files`, `tool_call`, `tool_calls`, `tool_call_result`, `tool_call_results`, `reasoning`, and `reasonings` properties.
If you are looking for the details of this data class methods and parameters, head over to our [API documentation](/reference/data-classes-api#chatmessage).
## Types of Content
`ChatMessage` currently supports `TextContent`, `ImageContent`, `FileContent`, `ToolCall`, `ToolCallResult`, and `ReasoningContent` types of content:
```python
@dataclass
class TextContent:
"""
The textual content of a chat message.
:param text: The text content of the message.
"""
text: str
@dataclass
class ToolCall:
"""
Represents a Tool call prepared by the model, usually contained in an assistant message.
:param tool_name: The name of the Tool to call.
:param arguments: The arguments to call the Tool with.
:param id: The ID of the Tool call.
:param extra: Dictionary of extra information about the Tool call. Use to store provider-specific
information. To avoid serialization issues, values should be JSON serializable.
"""
tool_name: str
arguments: Dict[str, Any]
id: Optional[str] = None # noqa: A003
extra: Optional[Dict[str, Any]] = None
@dataclass
class ToolCallResult:
"""
Represents the result of a Tool invocation.
:param result: The result of the Tool invocation.
:param origin: The Tool call that produced this result.
:param error: Whether the Tool invocation resulted in an error.
"""
result: str | Sequence[TextContent | ImageContent]
origin: ToolCall
error: bool
@dataclass
class ImageContent:
"""
The image content of a chat message.
:param base64_image: A base64 string representing the image.
:param mime_type: The MIME type of the image (e.g. "image/png", "image/jpeg").
Providing this value is recommended, as most LLM providers require it.
If not provided, the MIME type is guessed from the base64 string, which can be slow and not always reliable.
:param detail: Optional detail level of the image (only supported by OpenAI). One of "auto", "high", or "low".
:param meta: Optional metadata for the image.
:param validation: If True (default), a validation process is performed:
- Check whether the base64 string is valid;
- Guess the MIME type if not provided;
- Check if the MIME type is a valid image MIME type.
Set to False to skip validation and speed up initialization.
"""
base64_image: str
mime_type: Optional[str] = None
detail: Optional[Literal["auto", "high", "low"]] = None
meta: Dict[str, Any] = field(default_factory=dict)
validation: bool = True
@dataclass
class FileContent:
"""
The file content of a chat message.
:param base64_data: A base64 string representing the file.
:param mime_type: The MIME type of the file (e.g. "application/pdf").
Providing this value is recommended, as most LLM providers require it.
If not provided, the MIME type is guessed from the base64 string, which can be slow and not always reliable.
:param filename: Optional filename of the file. Some LLM providers use this information.
:param extra: Dictionary of extra information about the file. Can be used to store provider-specific information.
To avoid serialization issues, values should be JSON serializable.
:param validation: If True (default), a validation process is performed:
- Check whether the base64 string is valid;
- Guess the MIME type if not provided.
Set to False to skip validation and speed up initialization.
"""
base64_data: str
mime_type: str | None = None
filename: str | None = None
extra: dict[str, Any] = field(default_factory=dict)
validation: bool = True
@dataclass
class ReasoningContent:
"""
Represents the optional reasoning content prepared by the model, usually contained in an assistant message.
:param reasoning_text: The reasoning text produced by the model.
:param extra: Dictionary of extra information about the reasoning content. Use to store provider-specific
information. To avoid serialization issues, values should be JSON serializable.
"""
reasoning_text: str
extra: Dict[str, Any] = field(default_factory=dict)
```
The `ImageContent` and `FileContent` dataclasses also provide two convenience class methods: `from_file_path` and `from_url`.
For more details, refer to our [API documentation](/reference/data-classes-api).
## Working with a ChatMessage
The following examples demonstrate how to create a `ChatMessage` and inspect its properties.
### from_user with TextContent
```python
from haystack.dataclasses import ChatMessage
user_message = ChatMessage.from_user("What is the capital of Australia?")
print(user_message)
>>> ChatMessage(
>>> _role=<ChatRole.USER: 'user'>,
>>> _content=[TextContent(text='What is the capital of Australia?')],
>>> _name=None,
>>> _meta={}
>>>)
print(user_message.text)
>>> What is the capital of Australia?
print(user_message.texts)
>>> ['What is the capital of Australia?']
```
### from_user with TextContent and ImageContent
```python
from haystack.dataclasses import ChatMessage, ImageContent
lion_image_url = (
"https://images.unsplash.com/photo-1546182990-dffeafbe841d?"
"ixlib=rb-4.0&q=80&w=1080&fit=max"
)
image_content = ImageContent.from_url(lion_image_url, detail="low")
user_message = ChatMessage.from_user(
content_parts=[
"What does the image show?",
image_content
])
print(user_message)
>>> ChatMessage(
>>> _role=<ChatRole.USER: 'user'>,
>>> _content=[
>>> TextContent(text='What does the image show?'),
>>> ImageContent(
>>> base64_image='/9j/4...',
>>> mime_type='image/jpeg',
>>> detail='low',
>>> meta={
>>> 'content_type': 'image/jpeg',
>>> 'url': '...'
>>> }
>>> )
>>> ],
>>> _name=None,
>>> _meta={}
>>> )
print(user_message.text)
>>> What does the image show?
print(user_message.texts)
>>> ['What does the image show?']
print(user_message.image)
>>> ImageContent(
>>> base64_image='/9j/4...',
>>> mime_type='image/jpeg',
>>> detail='low',
>>> meta={
>>> 'content_type': 'image/jpeg',
>>> 'url': '...'
>>> }
>>> )
```
### from_user with TextContent and FileContent
```python
from haystack.dataclasses import ChatMessage, FileContent
paper_url = "https://arxiv.org/pdf/2309.08632"
file_content = FileContent.from_url(paper_url)
user_message = ChatMessage.from_user(
content_parts=[
file_content,
"Summarize this paper in 100 words."
])
print(user_message)
>>> ChatMessage(
>>> _role=<ChatRole.USER: 'user'>,
>>> _content=[
>>> FileContent(
>>> base64_data='JVBERi0...',
>>> mime_type='application/pdf',
>>> filename='2309.08632',
>>> extra={}
>>> ),
>>> TextContent(text='Summarize this paper in 100 words.')
>>> ],
>>> _name=None,
>>> _meta={}
>>> )
print(user_message.text)
>>> Summarize this paper in 100 words.
print(user_message.texts)
>>> ['Summarize this paper in 100 words.']
print(user_message.file)
>>> FileContent(
>>> base64_data='JVBERi0...',
>>> mime_type='application/pdf',
>>> filename='2309.08632',
>>> extra={}
>>> )
```
### from_assistant with TextContent
```python
from haystack.dataclasses import ChatMessage
assistant_message = ChatMessage.from_assistant("How can I assist you today?")
print(assistant_message)
>>> ChatMessage(
>>> _role=<ChatRole.ASSISTANT: 'assistant'>,
>>> _content=[TextContent(text='How can I assist you today?')],
>>> _name=None,
>>> _meta={}
>>>)
print(assistant_message.text)
>>> How can I assist you today?
print(assistant_message.texts)
>>> ['How can I assist you today?']
```
### from_assistant with ToolCall
```python
from haystack.dataclasses import ChatMessage, ToolCall
tool_call = ToolCall(tool_name="weather_tool", arguments={"location": "Rome"})
assistant_message_w_tool_call = ChatMessage.from_assistant(tool_calls=[tool_call])
print(assistant_message_w_tool_call)
>>> ChatMessage(
>>> _role=<ChatRole.ASSISTANT: 'assistant'>,
>>> _content=[ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None)],
>>> _name=None,
>>> _meta={}
>>>)
print(assistant_message_w_tool_call.text)
>>> None
print(assistant_message_w_tool_call.texts)
>>> []
print(assistant_message_w_tool_call.tool_call)
>>> ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None)
print(assistant_message_w_tool_call.tool_calls)
>>> [ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None)]
print(assistant_message_w_tool_call.tool_call_result)
>>> None
print(assistant_message_w_tool_call.tool_call_results)
>>> []
```
### from_tool
```python
from haystack.dataclasses import ChatMessage
tool_message = ChatMessage.from_tool(tool_result="temperature: 25°C", origin=tool_call, error=False)
print(tool_message)
>>> ChatMessage(
>>> _role=<ChatRole.TOOL: 'tool'>,
>>> _content=[ToolCallResult(
>>> result='temperature: 25°C',
>>> origin=ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None),
>>> error=False
>>> )],
>>> _name=None,
>>> _meta={}
>>>)
print(tool_message.text)
>>> None
print(tool_message.texts)
>>> []
print(tool_message.tool_call)
>>> None
print(tool_message.tool_calls)
>>> []
print(tool_message.tool_call_result)
>>> ToolCallResult(
>>> result='temperature: 25°C',
>>> origin=ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None),
>>> error=False
>>> )
print(tool_message.tool_call_results)
>>> [
>>> ToolCallResult(
>>> result='temperature: 25°C',
>>> origin=ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None),
>>> error=False
>>> )
>>> ]
```
## Migrating from Legacy ChatMessage (before v2.9)
In Haystack 2.9, we updated the `ChatMessage` data class for greater flexibility and support for multiple content types: text, tool calls, and tool call results.
There are some breaking changes involved, so we recommend reviewing this guide to migrate smoothly.
### Creating a ChatMessage
You can no longer directly initialize `ChatMessage` using `role`, `content`, and `meta`.
- Use the following class methods instead: `from_assistant`, `from_user`, `from_system`, and `from_tool`.
- Replace the `content` parameter with `text`.
```python
from haystack.dataclasses import ChatMessage
## LEGACY - DOES NOT WORK IN 2.9.0
message = ChatMessage(role=ChatRole.USER, content="Hello!")
## Use the class method instead
message = ChatMessage.from_user("Hello!")
```
### Accessing ChatMessage Attributes
- The legacy `content` attribute is now internal (`_content`).
- Inspect `ChatMessage` attributes using the following properties:
- `role`
- `meta`
- `name`
- `text` and `texts`
- `image` and `images`
- `tool_call` and `tool_calls`
- `tool_call_result` and `tool_call_results`
- `reasoning` and `reasonings`
```python
from haystack.dataclasses import ChatMessage
message = ChatMessage.from_user("Hello!")
## LEGACY - DOES NOT WORK IN 2.9.0
print(message.content)
## Use the appropriate property instead
print(message.text)
```
docs-website/versioned_docs/version-2.26/concepts/device-management.mdx
+127
View File
@@ -0,0 +1,127 @@
---
title: "Device Management"
id: device-management
slug: "/device-management"
description: "This page discusses the concept of device management in the context of Haystack."
---
# Device Management
This page discusses the concept of device management in the context of Haystack.
Many Haystack components, such as `HuggingFaceLocalGenerator` , `AzureOpenAIGenerator`, and others, allow users the ability to pick and choose which language model is to be queried and executed. For components that interface with cloud-based services, the service provider automatically takes care of the details of provisioning the requisite hardware (like GPUs). However, if you wish to use models on your local machine, you’ll need to figure out how to deploy them on your hardware. Further complicating things, different ML libraries have different APIs to launch models on specific devices.
To make the process of running inference on local models as straightforward as possible, Haystack uses a framework-agnostic device management implementation. Exposing devices through this interface means you no longer need to worry about library-specific invocations and device representations.
## Concepts
Haystack’s device management is built on the following abstractions:
- `DeviceType` - An enumeration that lists all the different types of supported devices.
- `Device` - A generic representation of a device composed of a `DeviceType` and a unique identifier. Together, it represents a single device in the group of all available devices.
- `DeviceMap` - A mapping of strings to `Device` instances. The strings represent model-specific identifiers, usually model parameters. This allows us to map specific parts of a model to specific devices.
- `ComponentDevice` - A tagged union of a single `Device` or a `DeviceMap` instance. Components that support local inference will expose an optional `device` parameter of this type in their constructor.
With the above abstractions, Haystack can fully address any supported device that’s part of your local machine and can support the usage of multiple devices at the same time. Every component that supports local inference will internally handle the conversion of these generic representations to their backend-specific representations.
:::info[Source Code]
Find the full code for the abstractions above in the Haystack GitHub [repo](https://github.com/deepset-ai/haystack/blob/6a776e672fb69cc4ee42df9039066200f1baf24e/haystack/utils/device.py).
:::
## Usage
To use a single device for inference, use either the `ComponentDevice.from_single` or `ComponentDevice.from_str` class method:
```python
from haystack.utils import ComponentDevice, Device
device = ComponentDevice.from_single(Device.gpu(id=1))
## Alternatively, use a PyTorch device string
device = ComponentDevice.from_str("cuda:1")
generator = HuggingFaceLocalGenerator(model="llama2", device=device)
```
To use multiple devices, use the `ComponentDevice.from_multiple` class method:
```python
from haystack.utils import ComponentDevice, Device, DeviceMap
device_map = DeviceMap(
{
"encoder.layer1": Device.gpu(id=0),
"decoder.layer2": Device.gpu(id=1),
"self_attention": Device.disk(),
"lm_head": Device.cpu(),
},
)
device = ComponentDevice.from_multiple(device_map)
generator = HuggingFaceLocalGenerator(model="llama2", device=device)
```
### Integrating Devices in Custom Components
Components should expose an optional `device` parameter of type `ComponentDevice`. Once exposed, they can determine what to do with it:
- If `device=None`, the component can pass that to the backend. In this case, the backend decides which device the model will be placed on.
- Alternatively, the component can attempt to automatically pick an available device before passing it to the backend using the `ComponentDevice.resolve_device` class method.
Once the device has been resolved, the component can use the `ComponentDevice.to_*` methods to get the backend-specific representation of the underlying device, which is then passed to the backend.
The `ComponentDevice` instance should be serialized in the component’s `to_dict` and `from_dict` methods.
```python
from haystack.utils import ComponentDevice, Device, DeviceMap
class MyComponent(Component):
def __init__(self, device: Optional[ComponentDevice] = None):
# If device is None, automatically select a device.
self.device = ComponentDevice.resolve_device(device)
def warm_up(self):
# Call the framework-specific conversion method.
self.model = AutoModel.from_pretrained(
"deepset/bert-base-cased-squad2", device=self.device.to_hf()
)
def to_dict(self):
# Serialize the policy like any other (custom) data.
return default_to_dict(
self, device=self.device.to_dict() if self.device else None, ...
)
@classmethod
def from_dict(cls, data):
# Deserialize the device data inplace before passing
# it to the generic from_dict function.
init_params = data["init_parameters"]
init_params["device"] = ComponentDevice.from_dict(init_params["device"])
return default_from_dict(cls, data)
## Automatically selects a device.
c = MyComponent(device=None)
## Uses the first GPU available.
c = MyComponent(device=ComponentDevice.from_str("cuda:0"))
## Uses the CPU.
c = MyComponent(device=ComponentDevice.from_single(Device.cpu()))
## Allow the component to use multiple devices using a device map.
c = MyComponent(device=ComponentDevice.from_multiple(DeviceMap({
"layer1": Device.cpu(),
"layer2": Device.gpu(1),
"layer3": Device.disk()
})))
```
If the component’s backend provides a more specialized API to manage devices, it could add an additional init parameter that acts as a conduit. For instance, `HuggingFaceLocalGenerator` exposes a `huggingface_pipeline_kwargs` parameter through which Hugging Face-specific `device_map` arguments can be passed:
```python
generator = HuggingFaceLocalGenerator(
model="llama2",
huggingface_pipeline_kwargs={"device_map": "balanced"},
)
```
In such cases, ensure that the parameter precedence and selection behavior is clearly documented. In the case of `HuggingFaceLocalGenerator`, the device map passed through the `huggingface_pipeline_kwargs` parameter overrides the explicit `device` parameter and is documented as such.
docs-website/versioned_docs/version-2.26/concepts/document-store.mdx
+103
View File
@@ -0,0 +1,103 @@
---
title: "Document Store"
id: document-store
slug: "/document-store"
description: "You can think of the Document Store as a database that stores your data and provides them to the Retriever at query time. Learn how to use Document Store in a pipeline or how to create your own."
---
# Document Store
You can think of the Document Store as a database that stores your data and provides them to the Retriever at query time. Learn how to use Document Store in a pipeline or how to create your own.
Document Store is an object that stores your documents. In Haystack, a Document Store is different from a component, as it doesn't have the `run()` method. You can think of it as an interface to your database – you put the information there, or you can look through it. This means that a Document Store is not a piece of a pipeline but rather a tool that the components of a pipeline have access to and can interact with.
:::tip[Work with Retrievers]
The most common way to use a Document Store in Haystack is to fetch documents using a Retriever. A Document Store will often have a corresponding Retriever to get the most out of specific technologies. See more information in our [Retriever](../pipeline-components/retrievers.mdx) documentation.
:::
:::note[How to choose a Document Store?]
To learn about different types of Document Stores and their strengths and disadvantages, head to the [Choosing a Document Store](document-store/choosing-a-document-store.mdx) page.
:::
### DocumentStore Protocol
Document Stores in Haystack are designed to use the following methods as part of their protocol:
- `count_documents` returns the number of documents stored in the given store as an integer.
- `filter_documents` returns a list of documents that match the provided filters.
- `write_documents` writes or overwrites documents into the given store and returns the number of documents that were written as an integer.
- `delete_documents` deletes all documents with given `document_ids` from the Document Store.
### Initialization
To use a Document Store in a pipeline, you must initialize it first.
See the installation and initialization details for each Document Store in the "Document Stores" section in the navigation panel on your left.
### Work with Documents
Convert your data into `Document` objects before writing them into a Document Store along with its metadata and document ID.
The ID field is mandatory, so if you don’t choose a specific ID yourself, Haystack will do its best to come up with a unique ID based on the document’s information and assign it automatically. However, since Haystack uses the document’s contents to create an ID, two identical documents might have identical IDs. Keep it in mind as you update your documents, as the ID will not be updated automatically.
```python
document_store = ChromaDocumentStore()
documents = [
Document(
meta={"name": DOCUMENT_NAME, ...}, id="document_unique_id", content="this is content"
),
...
]
document_store.write_documents(documents)
```
To write documents into the `InMemoryDocumentStore`, simply call the `.write_documents()` function:
```python
document_store.write_documents(
[
Document(content="My name is Jean and I live in Paris."),
Document(content="My name is Mark and I live in Berlin."),
Document(content="My name is Giorgio and I live in Rome."),
],
)
```
:::note[`DocumentWriter`]
See `DocumentWriter` component [docs](../pipeline-components/writers/documentwriter.mdx) to write your documents into a Document Store in a pipeline.
:::
### DuplicatePolicy
The `DuplicatePolicy` is a class that defines the different options for handling documents with the same ID in a `DocumentStore`. It has three possible values:
- **OVERWRITE**: Indicates that if a document with the same ID already exists in the `DocumentStore`, it should be overwritten with the new document.
- **SKIP**: If a document with the same ID already exists, the new document will be skipped and not added to the `DocumentStore`.
- **FAIL**: Raises an error if a document with the same ID already exists in the `DocumentStore`. It prevents duplicate documents from being added.
Here is an example of how you could apply the policy to skip the existing document:
```python
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.writers import DocumentWriter
from haystack.document_stores.types import DuplicatePolicy
document_store = InMemoryDocumentStore()
document_writer = DocumentWriter(
document_store=document_store,
policy=DuplicatePolicy.SKIP,
)
```
### Custom Document Store
All custom document stores must implement the [protocol](https://github.com/deepset-ai/haystack/blob/13804293b1bb79743e5a30e980b76a0561dcfaf8/haystack/document_stores/types/protocol.py) with four mandatory methods: `count_documents`,`filter_documents`, `write_documents`, and `delete_documents`.
The `init` function should indicate all the specifics for the chosen database or vector store.
We also recommend having a custom corresponding Retriever to get the most out of a specific Document Store.
See [Creating Custom Document Stores](document-store/creating-custom-document-stores.mdx) page for more details.
docs-website/versioned_docs/version-2.26/concepts/document-store/choosing-a-document-store.mdx
+109
View File
@@ -0,0 +1,109 @@
---
title: "Choosing a Document Store"
id: choosing-a-document-store
slug: "/choosing-a-document-store"
description: "This article goes through different types of Document Stores and explains their advantages and disadvantages."
---
import ClickableImage from "@site/src/components/ClickableImage";
# Choosing a Document Store
This article goes through different types of Document Stores and explains their advantages and disadvantages.
### Introduction
Whether you are developing a chatbot, a RAG system, or an image captioner, at some point, it’ll be likely for your AI application to compare the input it gets with the information it already knows. Most of the time, this comparison is performed through vector similarity search.
If you’re unfamiliar with vectors, think about them as a way to represent text, images, or audio/video in a numerical form called vector embeddings. Vector databases are specifically designed to store such vectors efficiently, providing all the functionalities an AI application needs to implement data retrieval and similarity search.
Document Stores are special objects in Haystack that abstract all the different vector databases into a common interface that can be easily integrated into a pipeline, most commonly through a Retriever component. Normally, you will find specialized Document Store and Retriever objects for each vector database Haystack supports.
### Types of vector databases
But why are vector databases so different, and which one should you use in your Haystack pipeline?
We can group vector databases into five categories, from more specialized to general purpose:
- Vector libraries
- Pure vector databases
- Vector-capable SQL databases
- Vector-capable NoSQL databases
- Full-text search databases
We are working on supporting all these types in Haystack.
In the meantime, here’s the most recent overview of available integrations:
<ClickableImage src="/img/2c188e9-2.0_Document_Stores_6.png" alt="Document store categories diagram showing four types: pure vector databases (Chroma, Milvus, Pinecone, Weaviate, Qdrant), full-text search databases (Elasticsearch, OpenSearch), vector-capable SQL databases (Pgvector for PostgreSQL), and vector-capable NoSQL databases (DataStax Astra, MongoDB, neo4j)" className="img-light-bg" />
#### Summary
Here is a quick summary of different Document Stores available in Haystack.
Continue further down the article for a more complex explanation of the strengths and disadvantages of each type.
<div className="key-value-table">
| | |
| --- | --- |
| Type | Best for |
| Vector libraries | Managing hardware resources effectively. |
| Pure vector DBs | Managing lots of high-dimensional data. |
| Vector-capable SQL DBs | Lower maintenance costs with focus on structured data and less on vectors. |
| Vector-capable NoSQL DBs | Combining vectors with structured data without the limitations of the traditional relational model. |
| Full-text search DBs | Superior full-text search, reliable for production. |
| In-memory | Fast, minimal prototypes on small datasets. |
</div>
#### Vector libraries
Vector libraries are often included in the “vector database” category improperly, as they are limited to handling only vectors, are designed to work in-memory, and normally don’t have a clean way to store data on disk. Still, they are the way to go every time performance and speed are the top requirements for your AI application, as these libraries can use hardware resources very effectively.
:::warning[In progress]
We are currently developing the support for vector libraries in Haystack.
:::
#### Pure vector databases
Pure vector databases, also known as just “vector databases”, offer efficient similarity search capabilities through advanced indexing techniques. Most of them support metadata, and despite a recent trend to add more text-search features on top of it, you should consider pure vector databases closer to vector libraries than a regular database. Pick a pure vector database when your application needs to manage huge amounts of high-dimensional data effectively: they are designed to be highly scalable and highly available. Most are open source, but companies usually provide them “as a service” through paid subscriptions.
- [Chroma](../../document-stores/chromadocumentstore.mdx)
- [Pinecone](../../document-stores/pinecone-document-store.mdx)
- [Qdrant](../../document-stores/qdrant-document-store.mdx)
- [Weaviate](../../document-stores/weaviatedocumentstore.mdx)
- [Milvus](https://haystack.deepset.ai/integrations/milvus-document-store) (external integration)
#### Vector-capable SQL databases
This category is relatively small but growing fast and includes well-known relational databases where vector capabilities were added through plugins or extensions. They are not as performant as the previous categories, but the main advantage of these databases is the opportunity to easily combine vectors with structured data, having a one-stop data shop for your application. You should pick a vector-capable SQL database when the performance trade-off is paid off by the lower cost of maintaining a single database instance for your application or when the structured data plays a more fundamental role in your business logic, with vectors being more of a nice-to-have.
- [Pgvector](../../document-stores/pgvectordocumentstore.mdx)
#### Vector-capable NoSQL databases
Historically, the killer features of NoSQL databases were the ability to scale horizontally and the adoption of a flexible data model to overcome certain limitations of the traditional relational model. This stays true for databases in this category, where the vector capabilities are added on top of the existing features. Similarly to the previous category, vector support might not be as good as pure vector databases, but once again, there is a tradeoff that might be convenient to bear depending on the use case. For example, if a certain NoSQL database is already part of the stack of your application and a lower performance is not a show-stopper, you might give it a shot.
- [Astra](../../document-stores/astradocumentstore.mdx)
- [MongoDB](../../document-stores/mongodbatlasdocumentstore.mdx)
- [Neo4j](https://haystack.deepset.ai/integrations/neo4j-document-store) (external)
#### Full-text search databases
The main advantage of full-text search databases is they are already designed to work with text, so you can expect a high level of support for text data along with good performance and the opportunity to scale both horizontally and vertically. Initially, vector capabilities were subpar and provided through plugins or extensions, but this is rapidly changing. You can see how the market leaders in this category have recently added first-class support for vectors. Pick a full-text search database if text data plays a central role in your business logic so that you can easily and effectively implement techniques like hybrid search with a good level of support for similarity search and state-of-the-art support for full-text search.
- [Elasticsearch](../../document-stores/elasticsearch-document-store.mdx)
- [OpenSearch](../../document-stores/opensearch-document-store.mdx)
#### The in-memory Document Store
Haystack ships with an ephemeral document store that relies on pure Python data structures stored in memory, so it doesn’t fall into any of the vector database categories above. This special Document Store is ideal for creating quick prototypes with small datasets. It doesn’t require any special setup, and it can be used right away without installing additional dependencies.
- [InMemory](../../document-stores/inmemorydocumentstore.mdx)
### Final considerations
It can be very challenging to pick one vector database over another by only looking at pure performance, as even the slightest difference in the benchmark can produce a different leaderboard (for example, some benchmarks test the cloud services while others work on a reference machine). Thinking about including features like filtering or not can bring in a whole new set of complexities that make the comparison even harder.
What’s important for you to know is that the Document Store interface doesn’t add much to the costs, and the relative performance of one vector database over another should stay the same when used within Haystack pipelines.
docs-website/versioned_docs/version-2.26/concepts/document-store/creating-custom-document-stores.mdx
+174
View File
@@ -0,0 +1,174 @@
---
title: "Creating Custom Document Stores"
id: creating-custom-document-stores
slug: "/creating-custom-document-stores"
description: "Create your own Document Stores to manage your documents."
---
# Creating Custom Document Stores
Create your own Document Stores to manage your documents.
Custom Document Stores are resources that you can build and leverage in situations where a ready-made solution is not available in Haystack. For example:
- You’re working with a vector store that’s not yet supported in Haystack.
- You need a very specific retrieval strategy to search for your documents.
- You want to customize the way Haystack reads and writes documents.
Similar to [custom components](../components/custom-components.mdx), you can use a custom Document Store in a Haystack pipeline as long as you can import its code into your Python program. The best practice is distributing a custom Document Store as a standalone integration package.
## Recommendations
Before you start, there are a few recommendations we provide to ensure a custom Document Store behaves consistently with the rest of the Haystack ecosystem. At the end of the day, a Document Store is just Python code written in a way that Haystack can understand, but the way you name it, organize it, and distribute it can make a difference. None of these recommendations are mandatory, but we encourage you to follow as many as you can.
### Naming Convention
We recommend naming your Document Store following the format `<TECHNOLOGY>-haystack`, for example, `chroma-haystack`. This makes it consistent with the others, lowering the cognitive load for your users and easing discoverability.
This naming convention applies to the name of the git repository (`https://github.com/your-org/example-haystack`) and the name of the Python package (`example-haystack`).
### Structure
More often than not, a Document Store can be fairly complex, and setting up a dedicated Git repository can be handy and future-proof. To ease this step, we prepared a [GitHub template](https://github.com/deepset-ai/custom-component) that provides the structure you need to host a custom Document Store in a dedicated repository. It includes the boilerplate for packaging, testing, and distributing your custom Document Store as a standalone Python package.
See the instructions in the [template repository](https://github.com/deepset-ai/custom-component) to get started.
### Packaging
As with any other [Haystack integration](../integrations.mdx), a Document Store can be added to your Haystack applications by installing an additional Python package, for example, with `pip`. Once you have a Git repository hosting your Document Store and a `pyproject.toml` file to create an `example-haystack` package (using our [GitHub template](https://github.com/deepset-ai/custom-component)), it will be possible to `pip install` it directly from sources, for example:
```shell
pip install git+https://github.com/your-org/example-haystack.git
```
Though very practical to quickly deliver prototypes, if you want others to use your custom Document Store, we recommend you publish a package on PyPI so that it will be versioned and installable with simply:
```shell
pip install example-haystack
```
:::tip
👍
Our [GitHub template](https://github.com/deepset-ai/custom-component) ships a GitHub workflow that will automatically publish the Document Store package on PyPI.
:::
### Documentation
We recommend thoroughly documenting your custom Document Store with a detailed README file and possibly generating API documentation using a static generator.
For inspiration, see the [neo4j-haystack](https://github.com/prosto/neo4j-haystack) repository and its [documentation](https://prosto.github.io/neo4j-haystack/) pages.
## Implementation
### DocumentStore Protocol
You can use any Python class as a Document Store, provided that it implements all the methods of the `DocumentStore` Python protocol defined in Haystack:
```python
class DocumentStore(Protocol):
def to_dict(self) -> Dict[str, Any]:
"""
Serializes this store to a dictionary.
"""
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "DocumentStore":
"""
Deserializes the store from a dictionary.
"""
def count_documents(self) -> int:
"""
Returns the number of documents stored.
"""
def filter_documents(
self,
filters: Optional[Dict[str, Any]] = None,
) -> List[Document]:
"""
Returns the documents that match the filters provided.
"""
def write_documents(
self,
documents: List[Document],
policy: DuplicatePolicy = DuplicatePolicy.FAIL,
) -> int:
"""
Writes (or overwrites) documents into the DocumentStore, return the number of documents that was written.
"""
def delete_documents(self, document_ids: List[str]) -> None:
"""
Deletes all documents with a matching document_ids from the DocumentStore.
"""
```
The `DocumentStore` interface supports the basic CRUD operations you would normally perform on a database or a storage system, and mostly generic components like [`DocumentWriter`](../../pipeline-components/writers/documentwriter.mdx) use it.
### Additional Methods
Usually, a Document Store comes with additional methods that can provide advanced search functionalities. These methods are not part of the `DocumentStore` protocol and don’t follow any particular convention. We designed it like this to provide maximum flexibility to the Document Store when using any specific features of the underlying database.
Some additional methods that are not part of the `DocumentStore` protocol, but are implemented by most Document Stores in Haystack, include:
```python
def delete_all_documents(recreate_index: bool = False)
def update_by_filter(filters: dict[str, Any], meta: dict[str, Any], refresh: bool = False) -> int:
def delete_by_filter(filters: dict[str, Any]) -> int:
```
These methods are not part of the Protocol but highly recommended to implement in your custom Document Store, as users often expect them to be available.
For example, Haystack wouldn’t get in the way when your Document Store defines a specific `search` method that takes a long list of parameters that only make sense in the context of a particular vector database. Normally, a [Retriever](../../pipeline-components/retrievers.mdx) component would then use this additional search method.
### Retrievers
To get the most out of your custom Document Store, in most cases, you would need to create one or more accompanying Retrievers that use the additional search methods mentioned above. Before proceeding and implementing your custom Retriever, it might be helpful to learn more about [Retrievers](../../pipeline-components/retrievers.mdx) in general through the Haystack documentation.
From the implementation perspective, Retrievers in Haystack are like any other custom component. For more details, refer to the [creating custom components](../components/custom-components.mdx) documentation page.
Although not mandatory, we encourage you to follow more specific [naming conventions](../../pipeline-components/retrievers.mdx#naming-conventions) for your custom Retriever.
### Serialization
Haystack requires every component to be representable by a Python dictionary for correct serialization implementation. Some components, such as Retrievers and Writers, maintain a reference to a Document Store instance. Therefore, `DocumentStore` classes should implement the `from_dict` and `to_dict` methods. This allows to rebuild an instance after reading a pipeline from a file.
For a practical example of what to serialize in a custom Document Store, consider a database client you created using an IP address and a database name. When constructing the dictionary to return in `to_dict`, you would store the IP address and the database name, not the database client instance.
### Secrets Management
There's a likelihood that users will need to provide sensitive data, such as passwords, API keys, or private URLs, to create a Document Store instance. This sensitive data could potentially be leaked if it's passed around in plain text.
Haystack has a specific way to wrap sensitive data into special objects called Secrets. This prevents the data from being leaked during serialization roundtrips. We strongly recommend using this feature extensively for data security (better safe than sorry!).
You can read more about Secret Management in Haystack [documentation](../secret-management.mdx).
### Testing
Haystack comes with some testing functionalities you can use in a custom Document Store. In particular, an empty class inheriting from `DocumentStoreBaseTests` would already run the standard tests that any Document Store is expected to pass in order to work properly.
### Implementation Tips
- The best way to learn how to write a custom Document Store is to look at the existing ones: the `InMemoryDocumentStore`, which is part of Haystack, or the [`ElasticsearchDocumentStore`](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch), which is a Core Integration, are good places to start.
- When starting from scratch, it might be easier to create the four CRUD methods of the `DocumentStore` protocol one at a time and test them one at a time as well. For example:
1. Implement the logic for `count_documents`.
2. In your `test_document_store.py` module, define the test class `TestDocumentStore(CountDocumentsTest)`. Note how we only inherit from the specific testing mix-in `CountDocumentsTest`.
3. Make the tests pass.
4. Implement the logic for `write_documents`.
5. Change `test_document_store.py` so that your class now also derives from the `WriteDocumentsTest` mix-in: `TestDocumentStore(CountDocumentsTest, WriteDocumentsTest)`.
6. Keep iterating with the remaining methods.
- Having a notebook where users can try out your Document Store in a full pipeline can really help adoption, and it’s a great source of documentation. Our [haystack-cookbook](https://github.com/deepset-ai/haystack-cookbook) repository has good visibility, and we encourage contributors to create a PR and add their own.
Verifying that the implementation meets all `DocumentStoreBaseTests` [tests](https://github.com/deepset-ai/haystack/blob/main/haystack/testing/document_store.py) is the minimum requirement for a custom Document Store to be consistent with the rest of the Haystack ecosystem.
But, ideally making it compatible with the ``DocumentStoreBaseExtendedTests`` tests is a good way to ensure that your Document Store meets all the common used functionalities that users expect from a Document Store, such as `delete_all_documents` or `update_by_filter`.
If the technology you are using for your Document Store supports asynchronous operations, we recommend implementing `async` versions of the methods in the `DocumentStore` protocol as well. This allows users to take advantage of async features in their applications and pipelines, improving performance and scalability.
## Get Featured on the Integrations Page
The [Integrations web page](https://haystack.deepset.ai/integrations) makes Haystack integrations visible to the community, and it’s a great opportunity to showcase your work. Once your Document Store is usable and properly packaged, you can open a pull request in the [haystack-integrations](https://github.com/deepset-ai/haystack-integrations) GitHub repository to add an integration tile.
See the [integrations documentation page](../integrations.mdx#how-do-i-showcase-my-integration) for more details.
docs-website/versioned_docs/version-2.26/concepts/experimental-package.mdx
+67
View File
@@ -0,0 +1,67 @@
---
title: "Experimental Package"
id: experimental-package
slug: "/experimental-package"
description: "Try out new experimental features with Haystack."
---
# Experimental Package
Try out new experimental features with Haystack.
The `haystack-experimental` package allows you to test new experimental features without committing to their official release. Its main goal is to gather user feedback and iterate on new features quickly.
Check out the `haystack-experimental` [GitHub repository](https://github.com/deepset-ai/haystack-experimental) for the latest catalog of available features, or take a look at our [Experiments API Reference](/reference).
### Installation
For simplicity, every release of `haystack-experimental` includes all the available experiments at that time. To install the latest features, run:
```shell
pip install -U haystack-experimental
```
:::info
The latest version of the experimental package is only tested against the latest version of Haystack. Compatibility with older versions of Haystack is not guaranteed.
:::
### Lifecycle
Each experimental feature has a default lifespan of 3 months starting from the date of the first non-pre-release build that includes it. Once it reaches the end of its lifespan, we will remove it from `haystack-experimental` and either:
- Merge the feature into Haystack and publish it with the next minor release,
- Release the feature as an integration, or
- Drop the feature.
### Usage
You can import the experimental new features like any other Haystack integration package:
```python
from haystack.dataclasses import ChatMessage
from haystack_experimental.components.generators import FoobarGenerator
c = FoobarGenerator()
c.run([ChatMessage.from_user("What's an experiment? Be brief.")])
```
Experiments can also override existing Haystack features. For example, you can opt into an experimental type of `Pipeline` by changing the usual import:
```python
## from haystack import Pipeline
from haystack_experimental import Pipeline
pipe = Pipeline()
## ...
pipe.run(...)
```
## Additional References
🧑‍🍳 Cookbooks:
- [Improving Retrieval with Auto-Merging and Hierarchical Document Retrieval](https://haystack.deepset.ai/cookbook/auto_merging_retriever)
- [Invoking APIs with OpenAPITool](https://haystack.deepset.ai/cookbook/openapitool)
- [Conversational RAG using Memory](https://haystack.deepset.ai/cookbook/conversational_rag_using_memory)
- [Define & Run Tools](https://haystack.deepset.ai/cookbook/tools_support)
- [Newsletter Sending Agent with Experimental Haystack Tools](https://haystack.deepset.ai/cookbook/newsletter-agent)
docs-website/versioned_docs/version-2.26/concepts/integrations.mdx
+58
View File
@@ -0,0 +1,58 @@
---
title: "Introduction to Integrations"
id: integrations
slug: "/integrations"
description: "The Haystack ecosystem integrates with many other technologies, such as vector databases, model providers and even custom components made by the community. Here you can explore our integrations, which may be maintined by deepset, or submitted by others."
---
# Introduction to Integrations
The Haystack ecosystem integrates with many other technologies, such as vector databases, model providers and even custom components made by the community. Here you can explore our integrations, which may be maintined by deepset, or submitted by others.
Haystack integrates with a number of other technologies and tools. For example, you can use a number of different model providers or databases with Haystack.
There are two main types of integrations:
- **Maintained by deepset:** All of the integrations we maintain are hosted in the [haystack-core-integrations](https://github.com/deepset-ai/haystack-core-integrations) repository.
- **Maintained by our partners or community:** These are integrations that you, our partners, or anyone else can build and maintain themselves. Given they comply with some of our requirements, we will also showcase these on our website.
## What are integrations?
An integration is any type of external technology that can be used to extend the capabilities of the Haystack framework. Some integration examples are those providing access to model providers like OpenAI or Cohere, to databases like Weaviate and Qdrant, or even to monitoring tools such as Traceloop. They can be components, Document Stores, or any other feature that can be used with Haystack.
We maintain a list of available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page, where you can see which integrations we maintain or which have been contributed by the community.
An integrations page focuses on explaining how Haystack integrates with that technology. For example, the OpenAI integration page will provide a summary of the various ways Haystack and OpenAI can work together.
Here are the integration types you can currently choose from:
- **Model Provider**: You can see how we integrate with different model providers and the available components through these integrations
- **Document Store**: These are the databases and vector stores you can use with your Haystack pipelines.
- **Evaluation Framework**: Evaluation frameworks that are supported by Haystack that you can use to evaluate Haystack pipelines.
- **Monitoring Tool**: These are tools like Chainlit and Traceloop that integrate with Haystack and provide monitoring and observability capabilities.
- **Data Ingestion**: These are the integrations that allow you to ingest and use data from different resources, such as Notion, Mastodon, and others.
- **Custom Component**: Some integrations that cover very unique use cases are often contributed and maintained by our community members. We list these integrations under the _Custom Component_ tag.
## How do I use an integration?
Each page dedicated to an integration contains installation instructions and basic usage instructions. For example, the OpenAI integration page gives you an overview of the different ways in which you can interact with OpenAI.
## How can I create an integration?
The most common types of integrations are custom components and Document Stores. Integrations such as model providers might even include multiple custom components. Have a look at these documentation pages that will guide you through the requirements for each integration type:
- [Creating Custom Components](components/custom-components.mdx)
- [Creating Custom Document Stores](document-store/creating-custom-document-stores.mdx)
## How do I showcase my integration?
To make your integration visible to the Haystack community, contribute it to our [haystack-integrations](https://github.com/deepset-ai/haystack-integrations) GitHub repository. There are several requirements you have to follow:
- Make sure your contribution is [packaged](https://packaging.python.org/en/latest/), installable, and runnable. We suggest using [hatch](https://hatch.pypa.io/latest/) for this purpose.
- Provide the GitHub repo and issue link.
- Create a Pull Request in the [haystack-integrations](https://github.com/deepset-ai/haystack-integrations) repo by following the [draft-integration.md](https://github.com/deepset-ai/haystack-integrations/blob/main/draft-integration.md) and include a clear explanation of what your integration is. This page should include:
- Installation instructions
- A list of the components the integration includes
- Examples of how to use it with clear/runnable code
- Licensing information
- (Optionally) Documentation and/or API docs that you’ve generated for your repository
docs-website/versioned_docs/version-2.26/concepts/jinja-templates.mdx
+58
View File
@@ -0,0 +1,58 @@
---
title: "Jinja Templates"
id: jinja-templates
slug: "/jinja-templates"
description: "Learn how Jinja templates work with Haystack components."
---
# Jinja Templates
Learn how Jinja templates work with Haystack components.
Jinja templates are text structures that contain placeholders for generating dynamic content. These placeholders are filled in when the template is rendered. You can check out the full list of Jinja2 features in the [original documentation](https://jinja.palletsprojects.com/en/3.0.x/templates/).
You can use these templates in Haystack [Builders](../pipeline-components/builders.mdx), [OutputAdapter](../pipeline-components/converters/outputadapter.mdx), and [ConditionalRouter](../pipeline-components/routers/conditionalrouter.mdx) components.
Here is an example of `OutputAdapter` using a short Jinja template to output only the content field of the first document in the arrays of documents:
```python
from haystack import Document
from haystack.components.converters import OutputAdapter
adapter = OutputAdapter(template="{{ documents[0].content }}", output_type=str)
input_data = {"documents": [Document(content="Test content")]}
expected_output = {"output": "Test content"}
assert adapter.run(**input_data) == expected_output
```
### Using Python f‑strings with Jinja
When you embed Jinja placeholders inside a Python f‑string, you must escape Jinja’s `{` and `}` by doubling them (so `{{ var }}` becomes `{{{{ var }}}}`). Otherwise, Python will consume the braces and the Jinja variable won’t be found.
Preferred template:
```python
template = """
Language: {{ language }}
Question: {{ question }}
"""
## pass both variables when rendering
```
It you need to use an f‑string (escape braces):
```python
language = "en"
template = f"""
Language: {language}
Question: {{{{ question }}}}
"""
```
## Safety Features
Due to how we use Jinja in some Components, there are some security considerations to take into account. Jinja works by executing embedded in templates, so it’s _imperative_ that they stem from a trusted source. If the template is allowed to be customized by the end user, it can potentially lead to remote code execution.
To mitigate this risk, Jinja templates are executed and rendered in a [sandbox environment](https://jinja.palletsprojects.com/en/3.1.x/sandbox/). While this approach is safer, it's also less flexible and limits the expressiveness of the template. If you need the more advanced functionality of Jinja templates, components that use them provide an `unsafe` init parameter - setting it to `False` will disable the sandbox environment and enable unsafe template rendering.
With unsafe template rendering, the [OutputAdapter](../pipeline-components/converters/outputadapter.mdx) and [ConditionalRouter](../pipeline-components/routers/conditionalrouter.mdx) components allow their `output_type` to be set to one of the [Haystack data classes](data-classes.mdx) such as `ChatMessage`, `Document`, or `Answer`.
docs-website/versioned_docs/version-2.26/concepts/metadata-filtering.mdx
+155
View File
@@ -0,0 +1,155 @@
---
title: "Metadata Filtering"
id: metadata-filtering
slug: "/metadata-filtering"
description: "This page provides a detailed explanation of how to apply metadata filters at query time."
---
# Metadata Filtering
This page provides a detailed explanation of how to apply metadata filters at query time.
When you index documents into your Document Store, you can attach metadata to them. One example is the `DocumentLanguageClassifier`, which adds the language of the document's content to its metadata. Components like `MetadataRouter` can then route documents based on their metadata.
You can then use the metadata to filter your search queries, allowing you to narrow down the results by focusing on specific criteria. This ensures your Retriever fetches answers from the most relevant subset of your data.
To illustrate how metadata filters work, imagine you have a set of annual reports from various companies. You may want to perform a search on just a specific year and just on a small selection of companies. This can reduce the workload of the Retriever and also ensure that you get more relevant results.
## Filtering Types
Filters are defined as a dictionary or nested dictionaries that can be of two types: Comparison or Logic.
### Comparison
Comparison operators help search your metadata fields according the specified conditions.
Comparison dictionaries must contain the following keys:
\-`field`: the name of one of the meta fields of a document, such as `meta.years`.
\-`operator`: must be one of the following:
```
- `==`
- `!=`
- `>`
- `>=`
- `<`
- `<=`
- `in`
- `not in`
```
:::info
The available comparison operators may vary depending on the specific Document Store integration. For example, the `ChromaDocumentStore` supports two additional operators: `contains` and `not contains`. Find the details about the supported filters in the specific integration’s API reference.
:::
\-`value`: takes a single value or (in the case of "in" and “not in”) a list of values.
#### Example
Here is an example of a simple filter in the form of a dictionary. The filter selects documents classified as “article” in the `type` meta field of the document:
```python
filters = {"field": "meta.type", "operator": "==", "value": "article"}
```
### Logic
Logical operators can be used to create a nested dictionary, allowing you to apply multiple `fields` as filter conditions. Logic dictionaries must contain the following keys:
\-`operator`: usually one of the following:
```
- `NOT`
- `OR`
- `AND`
```
:::info
The available logic operators may vary depending on the specific Document Store integration. For example, the `ChromaDocumentStore` doesn’t support the `NOT` operator. Find the details about the supported filters in the specific integration’s API reference.
:::
\-`conditions`: must be a list of dictionaries, either of type Comparison or Logic.
#### Nested Filter Example
Here is a more complex filter that uses both Comparison and Logic to find documents where:
- Meta field `type` is "article",
- Meta field `date` is between 1420066800 and 1609455600 (a specific date range),
- Meta field `rating` is greater than or equal to 3,
- Documents are either classified as `genre`  ["economy", "politics"] `OR` the meta field `publisher` is "nytimes".
```python
filters = {
"operator": "AND",
"conditions": [
{"field": "meta.type", "operator": "==", "value": "article"},
{"field": "meta.date", "operator": ">=", "value": 1420066800},
{"field": "meta.date", "operator": "<", "value": 1609455600},
{"field": "meta.rating", "operator": ">=", "value": 3},
{
"operator": "OR",
"conditions": [
{
"field": "meta.genre",
"operator": "in",
"value": ["economy", "politics"],
},
{"field": "meta.publisher", "operator": "==", "value": "nytimes"},
],
},
],
}
```
## Filters Usage
Filters can be applied either through the `Retriever` class or directly within Document Stores.
In the `Retriever` class, filters are passed through the `filters` argument. When working with a pipeline, filters can be provided to `Pipeline.run()`, which will automatically route them to the `Retriever` class (refer to the [pipelines documentation](pipelines.mdx) for more information on working with pipelines).
The example below shows how filters can be passed to Retrievers within a pipeline:
```python
pipeline.run(
data={
"retriever": {
"query": "Why did the revenue increase?",
"filters": {
"operator": "AND",
"conditions": [
{"field": "meta.years", "operator": "==", "value": "2019"},
{
"field": "meta.companies",
"operator": "in",
"value": ["BMW", "Mercedes"],
},
],
},
},
},
)
```
In Document Stores, the `filter_documents` method is used to apply filters to stored documents, if the specific integration supports filtering.
The example below shows how filters can be passed to the `QdrantDocumentStore`:
```python
filters = {
"operator": "AND",
"conditions": [
{"field": "meta.type", "operator": "==", "value": "article"},
{"field": "meta.genre", "operator": "in", "value": ["economy", "politics"]},
],
}
results = QdrantDocumentStore.filter_documents(filters=filters)
```
## Additional References
:notebook: Tutorial: [Filtering Documents with Metadata](https://haystack.deepset.ai/tutorials/31_metadata_filtering)
🧑‍🍳 Cookbook: [Extracting Metadata Filters from a Query](https://haystack.deepset.ai/cookbook/extracting_metadata_filters_from_a_user_query)
docs-website/versioned_docs/version-2.26/concepts/pipelines.mdx
+116
View File
@@ -0,0 +1,116 @@
---
title: "Pipelines"
id: pipelines
slug: "/pipelines"
description: "To build modern search pipelines with LLMs, you need two things: powerful components and an easy way to put them together. The Haystack pipeline is built for this purpose and enables you to design and scale your interactions with LLMs."
---
import ClickableImage from "@site/src/components/ClickableImage"
import YoutubeEmbed from "@site/src/components/YoutubeEmbed";
# Pipelines
To build modern search pipelines with LLMs, you need two things: powerful components and an easy way to put them together. The Haystack pipeline is built for this purpose and enables you to design and scale your interactions with LLMs.
The pipelines in Haystack are directed multigraphs of different Haystack components and integrations. They give you the freedom to connect these components in various ways. This means that the pipeline doesn't need to be a continuous stream of information. With the flexibility of Haystack pipelines, you can have simultaneous flows, standalone components, loops, and other types of connections.
## Flexibility
Haystack pipelines are much more than just query and indexing pipelines. What a pipeline does, whether that be indexing, querying, fetching from an API, preprocessing or more, completely depends on how you design your pipeline and what components you use. While you can still create single-function pipelines, like indexing pipelines using ready-made components to clean up, split, and write the documents into a Document Store, or query pipelines that just take a query and return an answer, Haystack allows you to combine multiple use cases into one pipeline with decision components (like the `ConditionalRouter`) as well.
### Agentic Pipelines
Haystack loops and branches enable the creation of complex applications like agents. Here are a few examples on how to create them:
- [Tutorial: Building a Chat Agent with Function Calling](https://haystack.deepset.ai/tutorials/40_building_chat_application_with_function_calling)
- [Tutorial: Building an Agentic RAG with Fallback to Websearch](https://haystack.deepset.ai/tutorials/36_building_fallbacks_with_conditional_routing)
- [Tutorial: Generating Structured Output with Loop-Based Auto-Correction](https://haystack.deepset.ai/tutorials/28_structured_output_with_loop)
- [Cookbook: Define & Run Tools](https://haystack.deepset.ai/cookbook/tools_support)
- [Cookbook: Conversational RAG using Memory](https://haystack.deepset.ai/cookbook/conversational_rag_using_memory)
- [Cookbook: Newsletter Sending Agent with Experimental Haystack Tools](https://haystack.deepset.ai/cookbook/newsletter-agent)
### Branching
A pipeline can have multiple branches that process data concurrently. For example, to process different file types, you can have a pipeline with a bunch of converters, each handling a specific file type. You then feed all your files to the pipeline and it smartly divides and routes them to appropriate converters all at once, saving you the effort of sending your files one by one for processing.
<ClickableImage src="/img/83f686b-Pipeline_Illustrations_1_1.png" alt="Pipeline architecture diagram showing components arranged in parallel branches that converge into a single pipeline flow" size="large" />
### Loops
Components in a pipeline can work in iterative loops, which you can cap at a desired number. This can be handy for scenarios like self-correcting loops, where you have a generator producing some output and then a validator component to check if the output is correct. If the generator's output has errors, the validator component can loop back to the generator for a corrected output. The loop goes on until the output passes the validation and can be sent further down the pipeline.
See [Pipeline Loops](pipelines/pipeline-loops.mdx) for a deeper explanation of how loops are executed, how they terminate, and how to use them safely.
<ClickableImage src="/img/2390eea-Pipeline_Illustrations_1_2.png" alt="Pipeline architecture diagram illustrating a feedback loop where output from later components loops back to earlier components" size="large" />
### Async Pipelines
The AsyncPipeline enables parallel execution of Haystack components when their dependencies allow it. This improves performance in complex pipelines with independent operations. For example, it can run multiple Retrievers or LLM calls simultaneously, execute independent pipeline branches in parallel, and efficiently handle I/O-bound operations that would otherwise cause delays. Through concurrent execution, the AsyncPipeline significantly reduces total processing time compared to sequential execution.
Find out more in our [AsyncPipeline](pipelines/asyncpipeline.mdx) documentation.
## SuperComponents
To simplify your code, we have introduced [SuperComponents](components/supercomponents.mdx) that allow you to wrap complete pipelines and reuse them as a single component. Check out their documentation page for the details and examples.
## Data Flow
While the data (the initial query) flows through the entire pipeline, individual values are only passed from one component to another when they are connected. Therefore, not all components have access to all the data. This approach offers the benefits of speed and ease of debugging.
To connect components and integrations in a pipeline, you must know the names of their inputs and outputs. The output of one component must be accepted as input by the following component. When you connect components in a pipeline with `Pipeline.connect()`, it validates if the input and output types match.
### Smart Pipeline Connections
Pipelines support smarter connection semantics that simplify how components are wired together.
Compatible outputs can be implicitly combined when connected to a single input.
Pipelines also perform implicit type adaptation at connection time for some selected types.
These behaviors reduce the need for glue components like `Joiners` and `OutputAdapters`, keeping pipelines concise and easier to read.
See [Smart Pipeline Connections](pipelines/smart-pipeline-connections.mdx) for details and examples.
<YoutubeEmbed videoId="SxAwyeCkguc" title="Introduction to Haystack Pipelines" />
## Steps to Create a Pipeline Explained
Once all your components are created and ready to be combined in a pipeline, there are four steps to make it work:
1. Create the pipeline with `Pipeline()`.
This creates the Pipeline object.
2. Add components to the pipeline, one by one, with `.add_component(name, component)`.
This just adds components to the pipeline without connecting them yet. It's especially useful for loops as it allows the smooth connection of the components in the next step because they all already exist in the pipeline.
3. Connect components with `.connect("producer_component.output_name", "consumer_component.input_name")`.
At this step, you explicitly connect one of the outputs of a component to one of the inputs of the next component. This is also when the pipeline validates the connection without running the components. It makes the validation fast.
4. Run the pipeline with `.run({"component_1": {"mandatory_inputs": value}})`.
Finally, you run the Pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components, for example: `.run({"component_1": {"mandatory_inputs": value}, "component_2": {"inputs": value}})`.
The full pipeline [example](pipelines/creating-pipelines.mdx#example) in [Creating Pipelines](pipelines/creating-pipelines.mdx) shows how all the elements come together to create a working RAG pipeline.
Once you create your pipeline, you can [visualize it in a graph](pipelines/visualizing-pipelines.mdx) to understand how the components are connected and make sure that's how you want them. You can use Mermaid graphs to do that.
## Validation
Validation happens when you connect pipeline components with `.connect()`, but before running the components to make it faster. The pipeline validates that:
- The components exist in the pipeline.
- The components' outputs and inputs match and are explicitly indicated. For example, if a component produces two outputs, when connecting it to another component, you must indicate which output connects to which input.
- The components' types match.
- For input types other than `Variadic`, checks if the input is already occupied by another connection.
All of these checks produce detailed errors to help you quickly fix any issues identified.
## Serialization
Thanks to serialization, you can save and then load your pipelines. Serialization is converting a Haystack pipeline into a format you can store on disk or send over the wire. It's particularly useful for:
- Editing, storing, and sharing pipelines.
- Modifying existing pipelines in a format different than Python.
Haystack pipelines delegate the serialization to its components, so serializing a pipeline simply means serializing each component in the pipeline one after the other, along with their connections. The pipeline is serialized into a dictionary format, which acts as an intermediate format that you can then convert into the final format you want.
:::info[Serialization formats]
Haystack only supports YAML format at this time. We'll be rolling out more formats gradually.
:::
For serialization to be possible, components must support conversion from and to Python dictionaries. All Haystack components have two methods that make them serializable: `from_dict` and `to_dict`. The `Pipeline` class, in turn, has its own `from_dict` and `to_dict` methods that take care of serializing components and connections.
docs-website/versioned_docs/version-2.26/concepts/pipelines/asyncpipeline.mdx
+158
View File
@@ -0,0 +1,158 @@
---
title: "AsyncPipeline"
id: asyncpipeline
slug: "/asyncpipeline"
description: "Use AsyncPipeline to run multiple Haystack components at the same time for faster processing."
---
# AsyncPipeline
Use AsyncPipeline to run multiple Haystack components at the same time for faster processing.
The `AsyncPipeline` in Haystack introduces asynchronous execution capabilities, enabling concurrent component execution when dependencies allow. This optimizes performance, particularly in complex pipelines where multiple independent components can run in parallel.
The `AsyncPipeline` provides significant performance improvements in scenarios such as:
- Hybrid retrieval pipelines, where multiple Retrievers can run in parallel,
- Multiple LLM calls that can be executed concurrently,
- Complex pipelines with independent branches of execution,
- I/O-bound operations that benefit from asynchronous execution.
## Key Features
### Concurrent Execution
The `AsyncPipeline` schedules components based on input readiness and dependency resolution, ensuring efficient parallel execution when possible. For example, in a hybrid retrieval scenario, multiple Retrievers can run simultaneously if they do not depend on each other.
### Execution Methods
The `AsyncPipeline` offers three ways to run your pipeline:
#### Synchronous Run (`run`)
Executes the pipeline synchronously with the provided input data. This method is blocking, making it suitable for environments where asynchronous execution is not possible or desired. Although components execute concurrently internally, the method blocks until the pipeline completes.
#### Asynchronous Run (`run_async`)
Executes the pipeline in an asynchronous manner, allowing non-blocking execution. This method is ideal when integrating the pipeline into an async workflow, enabling smooth operation within larger async applications or services.
#### Asynchronous Generator (`run_async_generator`)
Allows step-by-step execution by yielding partial outputs as components complete their tasks. This is particularly useful for monitoring progress, debugging, and handling outputs incrementally. It differs from `run_async`, which executes the pipeline in a single async call.
In an `AsyncPipeline`, components such as A and B will run in parallel _only if they have no shared dependencies_ and can process inputs independently.
### Concurrency Control
You can control the maximum number of components that run simultaneously using the `concurrency_limit` parameter to ensure controlled resource usage.
You can find more details in our [API Reference](/reference/pipeline-api#asyncpipeline), or directly in the pipeline's [GitHub code](https://github.com/deepset-ai/haystack/blob/main/haystack/core/pipeline/async_pipeline.py).
## Example
```python
import asyncio
from haystack import AsyncPipeline, Document
from haystack.components.builders import ChatPromptBuilder
from haystack.components.embedders import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.joiners import DocumentJoiner
from haystack.components.retrievers import (
InMemoryBM25Retriever,
InMemoryEmbeddingRetriever,
)
from haystack.dataclasses import ChatMessage
from haystack.document_stores.in_memory import InMemoryDocumentStore
documents = [
Document(content="Khufu is the largest pyramid."),
Document(content="Khafre is the middle pyramid."),
Document(content="Menkaure is the smallest pyramid."),
]
docs_embedder = SentenceTransformersDocumentEmbedder()
docs_embedder.warm_up()
document_store = InMemoryDocumentStore()
document_store.write_documents(docs_embedder.run(documents=documents)["documents"])
prompt_template = [
ChatMessage.from_system(
"""
You are a precise, factual QA assistant.
According to the following documents:
{% for document in documents %}
{{document.content}}
{% endfor %}
If an answer cannot be deduced from the documents, say "I don't know based on these documents".
When answering:
- be concise
- list the documents that support your answer
Answer the given question.
""",
),
ChatMessage.from_user("{{query}}"),
ChatMessage.from_system("Answer:"),
]
hybrid_rag_retrieval = AsyncPipeline()
hybrid_rag_retrieval.add_component("text_embedder", SentenceTransformersTextEmbedder())
hybrid_rag_retrieval.add_component(
"embedding_retriever",
InMemoryEmbeddingRetriever(document_store=document_store, top_k=3),
)
hybrid_rag_retrieval.add_component(
"bm25_retriever",
InMemoryBM25Retriever(document_store=document_store, top_k=3),
)
hybrid_rag_retrieval.add_component("document_joiner", DocumentJoiner())
hybrid_rag_retrieval.add_component(
"prompt_builder",
ChatPromptBuilder(template=prompt_template),
)
hybrid_rag_retrieval.add_component("llm", OpenAIChatGenerator())
hybrid_rag_retrieval.connect(
"text_embedder.embedding",
"embedding_retriever.query_embedding",
)
hybrid_rag_retrieval.connect("bm25_retriever.documents", "document_joiner.documents")
hybrid_rag_retrieval.connect(
"embedding_retriever.documents",
"document_joiner.documents",
)
hybrid_rag_retrieval.connect("document_joiner.documents", "prompt_builder.documents")
hybrid_rag_retrieval.connect("prompt_builder.prompt", "llm.messages")
question = "Which pyramid is neither the smallest nor the biggest?"
data = {
"prompt_builder": {"query": question},
"text_embedder": {"text": question},
"bm25_retriever": {"query": question},
}
async def process_results():
async for partial_output in hybrid_rag_retrieval.run_async_generator(
data=data,
include_outputs_from={"document_joiner", "llm"},
):
if "document_joiner" in partial_output:
print(
"Retrieved documents:",
len(partial_output["document_joiner"]["documents"]),
)
if "llm" in partial_output:
print("Generated answer:", partial_output["llm"]["replies"][0])
asyncio.run(process_results())
```
docs-website/versioned_docs/version-2.26/concepts/pipelines/creating-pipelines.mdx
+250
View File
@@ -0,0 +1,250 @@
---
title: "Creating Pipelines"
id: creating-pipelines
slug: "/creating-pipelines"
description: "Learn the general principles of creating a pipeline."
---
import ClickableImage from "@site/src/components/ClickableImage";
# Creating Pipelines
Learn the general principles of creating a pipeline.
You can use these instructions to create both indexing and query pipelines.
This task uses an example of a semantic document search pipeline.
## Prerequisites
For each component you want to use in your pipeline, you must know the names of its input and output. You can check them on the documentation page for a specific component or in the component's `run()` method. For more information, see [Components: Input and Output](../components.mdx#input-and-output).
## Steps to Create a Pipeline
### 1\. Import dependencies
Import all the dependencies, like pipeline, documents, Document Store, and all the components you want to use in your pipeline.
For example, to create a semantic document search pipelines, you need the `Document` object, the pipeline, the Document Store, Embedders, and a Retriever:
```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
```
### 2\. Initialize components
Initialize the components, passing any parameters you want to configure:
```python
document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
text_embedder = SentenceTransformersTextEmbedder()
retriever = InMemoryEmbeddingRetriever(document_store=document_store)
```
### 3\. Create the pipeline
```python
query_pipeline = Pipeline()
```
### 4\. Add components
Add components to the pipeline one by one. The order in which you do this doesn't matter:
```python
query_pipeline.add_component("component_name", component_type)
## Here is an example of how you'd add the components initialized in step 2 above:
query_pipeline.add_component("text_embedder", text_embedder)
query_pipeline.add_component("retriever", retriever)
## You could also add components without initializing them before:
query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
query_pipeline.add_component(
"retriever",
InMemoryEmbeddingRetriever(document_store=document_store),
)
```
### 5\. Connect components
Connect the components by indicating which output of a component should be connected to the input of the next component. If a component has only one input or output and the connection is obvious, you can just pass the component name without specifying the input or output.
To understand what inputs are expected to run your pipeline, use an `.inputs()` pipeline function. See a detailed examples in the [Pipeline Inputs](#pipeline-inputs) section below.
Here's a more visual explanation within the code:
```python
## This is the syntax to connect components. Here you're connecting output1 of component1 to input1 of component2:
pipeline.connect("component1.output1", "component2.input1")
## If both components have only one output and input, you can just pass their names:
pipeline.connect("component1", "component2")
## If one of the components has only one output but the other has multiple inputs,
## you can pass just the name of the component with a single output, but for the component with
## multiple inputs, you must specify which input you want to connect
## Here, component1 has only one output, but component2 has multiple inputs:
pipeline.connect("component1", "component2.input1")
## And here's how it should look like for the semantic document search pipeline we're using as an example:
pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
## Because the InMemoryEmbeddingRetriever only has one input, this is also correct:
pipeline.connect("text_embedder.embedding", "retriever")
```
You need to link all the components together, connecting them gradually in pairs. Here's an explicit example for the pipeline we're assembling:
```python
## Imagine this pipeline has four components: text_embedder, retriever, prompt_builder and llm.
## Here's how you would connect them into a pipeline:
query_pipeline.connect("text_embedder.embedding", "retriever")
query_pipeline.connect("retriever", "prompt_builder.documents")
query_pipeline.connect("prompt_builder", "llm")
```
### 6\. Run the pipeline
Wait for the pipeline to validate the components and connections. If everything is OK, you can now run the pipeline. `Pipeline.run()` can be called in two ways, either passing a dictionary of the component names and their inputs, or by directly passing just the inputs. When passed directly, the pipeline resolves inputs to the correct components.
```python
## Here's one way of calling the run() method
results = pipeline.run({"component1": {"input1_value": value1, "input2_value": value2}})
## The inputs can also be passed directly without specifying component names
results = pipeline.run({"input1_value": value1, "input2_value": value2})
## This is how you'd run the semantic document search pipeline we're using as an example:
query = "Here comes the query text"
results = query_pipeline.run({"text_embedder": {"text": query}})
```
## Pipeline Inputs
If you need to understand what component inputs are expected to run your pipeline, Haystack features a useful pipeline function `.inputs()` that lists all the required inputs for the components.
This is how it works:
```python
## A short pipeline example that converts webpages into documents
from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.components.writers import DocumentWriter
document_store = InMemoryDocumentStore()
fetcher = LinkContentFetcher()
converter = HTMLToDocument()
writer = DocumentWriter(document_store=document_store)
pipeline = Pipeline()
pipeline.add_component(instance=fetcher, name="fetcher")
pipeline.add_component(instance=converter, name="converter")
pipeline.add_component(instance=writer, name="writer")
pipeline.connect("fetcher.streams", "converter.sources")
pipeline.connect("converter.documents", "writer.documents")
## Requesting a list of required inputs
pipeline.inputs()
## {'fetcher': {'urls': {'type': typing.List[str], 'is_mandatory': True}},
## 'converter': {'meta': {'type': typing.Union[typing.Dict[str, typing.Any], typing.List[typing.Dict[str, typing.Any]], NoneType],
## 'is_mandatory': False,
## 'default_value': None},
## 'extraction_kwargs': {'type': typing.Optional[typing.Dict[str, typing.Any]],
## 'is_mandatory': False,
## 'default_value': None}},
## 'writer': {'policy': {'type': typing.Optional[haystack.document_stores.types.policy.DuplicatePolicy],
## 'is_mandatory': False,
## 'default_value': None}}}
```
From the above response, you can see that the `urls` input is mandatory for `LinkContentFetcher`. This is how you would then run this pipeline:
```python
pipeline.run(
data={"fetcher": {"urls": ["https://docs.haystack.deepset.ai/docs/pipelines"]}},
)
```
## Example
The following example walks you through creating a RAG pipeline.
```python
# import necessary dependencies
from haystack import Pipeline, Document
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.retrievers import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.builders import ChatPromptBuilder
from haystack.utils import Secret
from haystack.dataclasses import ChatMessage
# create a document store and write documents to it
document_store = InMemoryDocumentStore()
document_store.write_documents(
[
Document(content="My name is Jean and I live in Paris."),
Document(content="My name is Mark and I live in Berlin."),
Document(content="My name is Giorgio and I live in Rome."),
],
)
# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer.
prompt_template = [
ChatMessage.from_system(
"""
Given these documents, answer the question.
Documents:
{% for doc in documents %}
{{ doc.content }}
{% endfor %}
Question:
""",
),
ChatMessage.from_user("{{question}}"),
ChatMessage.from_system("Answer:"),
]
# create the components adding the necessary parameters
retriever = InMemoryBM25Retriever(document_store=document_store)
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
llm = OpenAIChatGenerator(
api_key=Secret.from_env_var("OPENAI_API_KEY"),
model="gpt-4o-mini",
)
# Create the pipeline and add the components to it. The order doesn't matter.
# At this stage, the Pipeline validates the components without running them yet.
rag_pipeline = Pipeline()
rag_pipeline.add_component("retriever", retriever)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)
# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name").
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm")
# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components.
question = "Who lives in paris?"
results = rag_pipeline.run(
{
"retriever": {"query": question},
"prompt_builder": {"question": question},
},
)
print(results["llm"]["replies"])
```
Here's what a [visualized Mermaid graph](visualizing-pipelines.mdx) of this pipeline would look like:
<br />
<ClickableImage src="/img/vizualised-rag-pipeline.png" alt="RAG pipeline diagram with three connected components: InMemoryBM25Retriever receives a query string and outputs documents, ChatPromptBuilder combines the documents with a question input to create prompt messages, and OpenAIChatGenerator processes the messages to produce replies. Each component box displays its class name and optional input parameters." size="large" />
docs-website/versioned_docs/version-2.26/concepts/pipelines/debugging-pipelines.mdx
+125
View File
@@ -0,0 +1,125 @@
---
title: "Debugging Pipelines"
id: debugging-pipelines
slug: "/debugging-pipelines"
description: "Learn how to debug and troubleshoot your Haystack pipelines."
---
import ClickableImage from "@site/src/components/ClickableImage";
# Debugging Pipelines
Learn how to debug and troubleshoot your Haystack pipelines.
There are several options available to you to debug your pipelines:
- [Inspect your components' outputs](#inspecting-component-outputs)
- [Adjust logging](#logging)
- [Set up tracing](#tracing)
- [Try one of the monitoring tool integrations](#monitoring-tools)
## Inspecting Component Outputs
To view outputs from specific pipeline components, add the `include_outputs_from` parameter when executing your pipeline. Place it after the input dictionary and set it to the name of the component whose output you want included in the result.
For example, here’s how you can print the output of `PromptBuilder` in this pipeline:
```python
from haystack import Pipeline, Document
from haystack.utils import Secret
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
## Documents
documents = [
Document(content="Joe lives in Berlin"),
Document(content="Joe is a software engineer"),
]
## Define prompt template
prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given these documents, answer the question.\nDocuments:\n"
"{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
"Question: {{query}}\nAnswer:",
),
]
## Define pipeline
p = Pipeline()
p.add_component(
instance=ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
),
name="prompt_builder",
)
p.add_component(
instance=OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
name="llm",
)
p.connect("prompt_builder", "llm.messages")
## Define question
question = "Where does Joe live?"
## Execute pipeline
result = p.run(
{"prompt_builder": {"documents": documents, "query": question}},
include_outputs_from="prompt_builder",
)
## Print result
print(result)
```
## Logging
Adjust the logging format according to your debugging needs. See our [Logging](../../development/logging.mdx) documentation for details.
## Real-Time Pipeline Logging
Use Haystack's [`LoggingTracer`](https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py) logs to inspect the data that's flowing through your pipeline in real-time.
This feature is particularly helpful during experimentation and prototyping, as you don’t need to set up any tracing backend beforehand.
Here’s how you can enable this tracer. In this example, we are adding color tags (this is optional) to highlight the components' names and inputs:
```python
import logging
from haystack import tracing
from haystack.tracing.logging_tracer import LoggingTracer
logging.basicConfig(
format="%(levelname)s - %(name)s - %(message)s",
level=logging.WARNING,
)
logging.getLogger("haystack").setLevel(logging.DEBUG)
tracing.tracer.is_content_tracing_enabled = (
True # to enable tracing/logging content (inputs/outputs)
)
tracing.enable_tracing(
LoggingTracer(
tags_color_strings={
"haystack.component.input": "\x1b[1;31m",
"haystack.component.name": "\x1b[1;34m",
},
),
)
```
Here’s what the resulting log would look like when a pipeline is run:
<ClickableImage src="/img/55c3d5c84282d726c95fb3350ec36be49a354edca8a6164f5dffdab7121cec58-image_2.png" alt="Console output showing Haystack pipeline execution with DEBUG level tracing logs including component names, types, and input/output specifications" />
## Tracing
To get a bigger picture of the pipeline’s performance, try tracing it with [Langfuse](../../development/tracing.mdx#langfuse).
Our [Tracing](../../development/tracing.mdx) page has more about other tracing solutions for Haystack.
## Monitoring Tools
Take a look at available tracing and monitoring [integrations](https://haystack.deepset.ai/integrations?type=Monitoring+Tool&version=2.0) for Haystack pipelines, such as Arize AI or Arize Phoenix.
docs-website/versioned_docs/version-2.26/concepts/pipelines/pipeline-breakpoints.mdx
+214
View File
@@ -0,0 +1,214 @@
---
title: "Pipeline Breakpoints"
id: pipeline-breakpoints
slug: "/pipeline-breakpoints"
description: "Learn how to pause and resume Haystack pipeline or Agent execution using breakpoints to debug, inspect, and continue workflows from saved snapshots."
---
# Pipeline Breakpoints
Learn how to pause and resume Haystack pipeline or Agent execution using breakpoints to debug, inspect, and continue workflows from saved snapshots.
## Introduction
Haystack pipelines support breakpoints for debugging complex execution flows. A `Breakpoint` allows you to pause the execution at specific components, inspect the pipeline state, and resume execution from saved snapshots. This feature works for any regular component as well as an `Agent` component.
You can set a `Breakpoint` on any component in a pipeline with a specific visit count. When triggered, the system stops the execution of the `Pipeline` and captures a snapshot of the current pipeline state. The state can be saved to a JSON file when snapshot file saving is enabled, see [Snapshot file saving](#snapshot-file-saving) below. You can inspect and modify the snapshot and use it to resume execution from the exact point where it stopped.
You can also set breakpoints on an Agent, specifically on the `ChatGenerator` component or on any of the `Tool` specified in the `ToolInvoker` component .
## Setting a `Breakpoint` on a Regular Component
Create a `Breakpoint` by specifying the component name and the visit count at which to trigger it. This is useful for pipelines with loops. The default `visit_count` value is 0.
```python
from haystack.dataclasses.breakpoints import Breakpoint
from haystack.core.errors import BreakpointException
## Create a breakpoint that triggers on the first visit to the "llm" component
break_point = Breakpoint(
component_name="llm",
visit_count=0, # 0 = first visit, 1 = second visit, etc.
snapshot_file_path="/path/to/snapshots", # Optional: save snapshot to file
)
## Run pipeline with breakpoint
try:
result = pipeline.run(data=input_data, break_point=break_point)
except BreakpointException as e:
print(f"Breakpoint triggered at component: {e.component}")
print(f"Component inputs: {e.inputs}")
print(f"Pipeline results so far: {e.results}")
```
A `BreakpointException` is raised containing the component inputs and the outputs of the pipeline up until the moment where the execution was interrupted, such as just before the execution of component associated with the breakpoint – the `llm` in the example above.
If a `snapshot_file_path` is specified in the `Breakpoint` and snapshot file saving is enabled, the system saves a JSON snapshot with the same information as in the `BreakpointException`. Snapshot file saving to disk is disabled by default; see [Snapshot file saving](#snapshot-file-saving) below.
To access the pipeline state during the breakpoint we can both catch the exception raised by the breakpoint as well as specify where the JSON file should be saved, note that file saving is enabled must be enabled.
## Using a custom snapshot callback
You can pass a `snapshot_callback` to `Pipeline.run()` or `Agent.run()` to handle snapshots yourself instead of saving to a file. When a breakpoint is triggered or a snapshot is created on error, the callback is invoked with the `PipelineSnapshot` object. This is useful for saving snapshots to a database, sending them to a remote service, or custom logging.
```python
from haystack.core.errors import BreakpointException
from haystack.dataclasses.breakpoints import Breakpoint, PipelineSnapshot
def my_snapshot_callback(snapshot: PipelineSnapshot) -> None:
# Custom handling: e.g. save to DB, send to API, or log
print(f"Snapshot at component: {snapshot.break_point}")
break_point = Breakpoint(component_name="llm", visit_count=0)
try:
result = pipeline.run(
data=input_data,
break_point=break_point,
snapshot_callback=my_snapshot_callback,
)
except BreakpointException as e:
print(f"Breakpoint triggered: {e.component}")
```
When `snapshot_callback` is provided, file-saving is skipped and the callback is responsible for handling the snapshot. The same parameter is available on `Agent.run()` for agent breakpoints.
## Snapshot file saving
Snapshot file saving to disk is **disabled by default**. To save snapshots as JSON files when a breakpoint is triggered or on pipeline failure, set the environment variable `HAYSTACK_PIPELINE_SNAPSHOT_SAVE_ENABLED` to `"true"` or `"1"` (case-insensitive). When enabled, snapshots are written to the path given by `snapshot_file_path` on the breakpoint, or to the default directory in [Error Recovery with Snapshots](#error-recovery-with-snapshots) when a run fails.
Custom `snapshot_callback` functions are always invoked when provided, regardless of this setting.
```python
import os
# Enable saving snapshot files to disk
os.environ["HAYSTACK_PIPELINE_SNAPSHOT_SAVE_ENABLED"] = "true"
break_point = Breakpoint(
component_name="llm",
visit_count=0,
snapshot_file_path="/path/to/snapshots",
)
# When the breakpoint triggers, a JSON file will be written to /path/to/snapshots/
```
## Resuming a Pipeline Execution from a Breakpoint
To resume the execution of a pipeline from the breakpoint, pass the path to the generated JSON file at the run time of the pipeline, using the `pipeline_snapshot`.
Use the `load_pipeline_snapshot()` to first load the JSON and then pass it to the pipeline.
```python
from haystack.core.pipeline.breakpoint import load_pipeline_snapshot
## Load the snapshot
snapshot = load_pipeline_snapshot("llm_2025_05_03_11_23_23.json")
## Resume execution from the snapshot
result = pipeline.run(data={}, pipeline_snapshot=snapshot)
print(result["llm"]["replies"])
```
## Setting a Breakpoint on an Agent
You can also set breakpoints in an Agent component. An Agent supports two types of breakpoints:
1. **Chat Generator Breakpoint**: Pauses before LLM calls.
2. **Tool Invoker Breakpoint**: Pauses before any tool execution.
A `ChatGenerator` breakpoint is defined as shown below. You need to define a `Breakpoint` as for a pipeline breakpoint and then an `AgentBreakpoint` where you pass the breakpoint defined before and the name of Agent component.
```python
from haystack.dataclasses.breakpoints import AgentBreakpoint, Breakpoint, ToolBreakpoint
## Break at chat generator (LLM calls)
chat_bp = Breakpoint(component_name="chat_generator", visit_count=0)
agent_breakpoint = AgentBreakpoint(break_point=chat_bp, agent_name="my_agent")
```
To set a breakpoint on a Tool in an Agent, do the following:
First, define a `ToolBreakpoint` specifying the `ToolInvoker` component whose name is `tool_invoker` and then the tool associated with the breakpoint, in this case – a `weather_tool` .
Then, define an `AgentBreakpoint` passing the `ToolBreakpoint` defined before as the breakpoint.
```python
from haystack.dataclasses.breakpoints import AgentBreakpoint, Breakpoint, ToolBreakpoint
## Break at tool invoker (tool calls)
tool_bp = ToolBreakpoint(
component_name="tool_invoker",
visit_count=0,
tool_name="weather_tool", # Specific tool, or None for any tool
)
agent_breakpoint = AgentBreakpoint(break_point=tool_bp, agent_name="my_agent")
```
### Resuming Agent Execution
When an Agent breakpoint is triggered, you can resume execution using the saved snapshot. Similar to the regular component in a pipeline, pass the JSON file with the snapshot to the `run()` method of the pipeline.
```python
from haystack.core.pipeline.breakpoint import load_pipeline_snapshot
## Load the snapshot
snapshot_file = "./agent_debug/agent_chat_generator_2025_07_11_23_23.json"
snapshot = load_pipeline_snapshot(snapshot_file)
## Resume pipeline execution
result = pipeline.run(data={}, pipeline_snapshot=snapshot)
print("Pipeline resumed successfully")
print(f"Final result: {result}")
```
## Error Recovery with Snapshots
Pipelines automatically create a snapshot of the last valid state if a run fails. The snapshot contains inputs, visit counts, and intermediate outputs up to the failure. You can inspect it, fix the issue, and resume execution from that checkpoint instead of restarting the whole run.
### Access the Snapshot on Failure
Wrap `pipeline.run()` in a `try`/`except` block and retrieve the snapshot from the raised `PipelineRuntimeError`:
```python
from haystack.core.errors import PipelineRuntimeError
try:
pipeline.run(data=input_data)
except PipelineRuntimeError as e:
snapshot = e.pipeline_snapshot
if snapshot is not None:
intermediate_outputs = snapshot.pipeline_state.pipeline_outputs
# Inspect intermediate_outputs to diagnose the failure
```
When snapshot file saving is enabled (see [Snapshot file saving](#snapshot-file-saving)), Haystack also saves the same snapshot as a JSON file on disk.
The directory is chosen automatically in this order:
- `~/.haystack/pipeline_snapshot`
- `/tmp/haystack/pipeline_snapshot`
- `./.haystack/pipeline_snapshot`
Filenames will have the following pattern: `{component_name}_{visit_nr}_{YYYY_MM_DD_HH_MM_SS}.json`.
### Resume from a Snapshot
You can resume directly from the in-memory snapshot or load it from disk.
Resume from memory:
```python
result = pipeline.run(data={}, pipeline_snapshot=snapshot)
```
Resume from disk:
```python
from haystack.core.pipeline.breakpoint import load_pipeline_snapshot
snapshot = load_pipeline_snapshot(
"/path/to/.haystack/pipeline_snapshot/reader_0_2025_09_20_12_33_10.json",
)
result = pipeline.run(data={}, pipeline_snapshot=snapshot)
```
docs-website/versioned_docs/version-2.26/concepts/pipelines/pipeline-loops.mdx
+253
View File
@@ -0,0 +1,253 @@
---
title: "Pipeline Loops"
id: pipeline-loops
slug: "/pipeline-loops"
description: "Understand how loops work in Haystack pipelines, how they terminate, and how to use them safely for feedback and self-correction."
---
# Pipeline Loops
Learn how loops work in Haystack pipelines, how they terminate, and how to use them for feedback and self-correction.
Haystack pipelines support **loops**: cycles in the component graph where the output of a later component is fed back into an earlier one.
This enables feedback flows such as self-correction, validation, or iterative refinement, as well as more advanced [agentic behavior](../pipelines.mdx#agentic-pipelines).
At runtime, the pipeline re-runs a component whenever all of its required inputs are ready again.
You control when loops stop either by designing your graph and routing logic carefully or by using built-in [safety limits](#loop-termination-and-safety-limits).
## Multiple Runs of the Same Component
If a component participates in a loop, it can be run multiple times within a single `Pipeline.run()` call.
The pipeline keeps an internal visit counter for each component:
- Each time the component runs, its visit count increases by 1.
- You can use this visit count in debugging tools like [breakpoints](./pipeline-breakpoints.mdx) to inspect specific iterations of a loop.
In the final pipeline result:
- For each component that ran, the pipeline returns **only the last-produced output**.
- To capture outputs from intermediate components (for example, a validator or a router) in the final result dictionary, use the `include_outputs_from` argument of `Pipeline.run()`.
## Loop Termination and Safety Limits
Loops must eventually stop so that a pipeline run can complete.
There are two main ways a loop ends:
1. **Natural completion**: No more components are runnable
The pipeline finishes when the work queue is empty and no component can run again (for example, the router stops feeding inputs back into the loop).
2. **Reaching the maximum run count**
Every pipeline has a per-component run limit, controlled by the `max_runs_per_component` parameter of the `Pipeline` (or `AsyncPipeline`) constructor, which is `100` by default. If any component exceeds this limit, Haystack raises a `PipelineMaxComponentRuns` error.
You can set this limit to a lower value:
```python
from haystack import Pipeline
pipe = Pipeline(max_runs_per_component=5)
```
The limit is checked before each execution, so a component with a limit of 3 will complete 3 runs successfully before the error is raised on the 4th attempt.
This safeguard is especially important when experimenting with new loops or complex routing logic.
If your loop condition is wrong or never satisfied, the error prevents the pipeline from running indefinitely.
## Example: Feedback Loop for Self-Correction
The following example shows a simple feedback loop where:
- A `ChatPromptBuilder` creates a prompt that includes previous incorrect replies.
- An `OpenAIChatGenerator` produces an answer.
- A `ConditionalRouter` checks if the answer is correct:
- If correct, it sends the answer to `final_answer` and the loop ends.
- If incorrect, it sends the answer back to the `ChatPromptBuilder`, which triggers another iteration.
```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.routers import ConditionalRouter
from haystack.dataclasses import ChatMessage
template = [
ChatMessage.from_system(
"Answer the following question concisely with just the answer, no punctuation.",
),
ChatMessage.from_user(
"{% if previous_replies %}"
"Previously you replied incorrectly: {{ previous_replies[0].text }}\n"
"{% endif %}"
"Question: {{ query }}",
),
]
prompt_builder = ChatPromptBuilder(template=template, required_variables=["query"])
generator = OpenAIChatGenerator()
router = ConditionalRouter(
routes=[
{
# End the loop when the answer is correct
"condition": "{{ 'Rome' in replies[0].text }}",
"output": "{{ replies }}",
"output_name": "final_answer",
"output_type": list[ChatMessage],
},
{
# Loop back when the answer is incorrect
"condition": "{{ 'Rome' not in replies[0].text }}",
"output": "{{ replies }}",
"output_name": "previous_replies",
"output_type": list[ChatMessage],
},
],
unsafe=True, # Required to handle ChatMessage objects
)
pipe = Pipeline(max_runs_per_component=3)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("generator", generator)
pipe.add_component("router", router)
pipe.connect("prompt_builder.prompt", "generator.messages")
pipe.connect("generator.replies", "router.replies")
pipe.connect("router.previous_replies", "prompt_builder.previous_replies")
result = pipe.run(
{
"prompt_builder": {
"query": "What is the capital of Italy? If the statement 'Previously you replied incorrectly:' is missing "
"above then answer with Milan.",
},
},
include_outputs_from={"router", "prompt_builder"},
)
print(result["prompt_builder"]["prompt"][1].text) # Shows the last prompt used
print(result["router"]["final_answer"][0].text) # Rome
```
### What Happens During This Loop
1. **First iteration**
- `prompt_builder` runs with `query="What is the capital of Italy?"` and no previous replies.
- `generator` returns a `ChatMessage` with the LLM's answer.
- The router evaluates its conditions and checks if `"Rome"` is in the reply.
- If the answer is incorrect, `previous_replies` is fed back into `prompt_builder.previous_replies`.
2. **Subsequent iterations** (if needed)
- `prompt_builder` runs again, now including the previous incorrect reply in the user message.
- `generator` produces a new answer with the additional context.
- The router checks again whether the answer contains `"Rome"`.
3. **Termination**
- When the router routes to `final_answer`, no more inputs are fed back into the loop.
- The queue empties and the pipeline run finishes successfully.
Because we used `max_runs_per_component=3`, any unexpected behavior that causes the loop to continue would raise a `PipelineMaxComponentRuns` error instead of looping forever.
## Components for Building Loops
Two components are particularly useful for building loops:
- **[`ConditionalRouter`](../../pipeline-components/routers/conditionalrouter.mdx)**: Routes data to different outputs based on conditions. Use it to decide whether to exit the loop or continue iterating. The example above uses this pattern.
- **[`BranchJoiner`](../../pipeline-components/joiners/branchjoiner.mdx)**: Merges inputs from multiple sources into a single output. Use it when a component inside the loop needs to receive both the initial input (on the first iteration) and looped-back values (on subsequent iterations). For example, you might use `BranchJoiner` to feed both user input and validation errors into the same Generator. See the [BranchJoiner documentation](../../pipeline-components/joiners/branchjoiner.mdx#enabling-loops) for a complete loop example.
## Greedy vs. Lazy Variadic Sockets in Loops
Some components support variadic inputs that can receive multiple values on a single socket.
In loops, variadic behavior controls how inputs are consumed across iterations.
- **Greedy variadic sockets**
Consume exactly one value at a time and remove it after the component runs.
This includes user-provided inputs, which prevents them from retriggering the component indefinitely.
Most variadic sockets are greedy by default.
- **Lazy variadic sockets**
Accumulate all values received from predecessors across iterations.
Useful when you need to collect multiple partial results over time (for example, gathering outputs from several loop iterations before proceeding).
For most loop scenarios it's sufficient to just connect components as usual and use `max_runs_per_component` to protect against mistakes.
## Troubleshooting Loops
If your pipeline seems stuck or runs longer than expected, here are common causes and how to debug them.
### Common Causes of Infinite Loops
1. **Condition never satisfied**: Your exit condition (for example, `"Rome" in reply`) might never be true due to LLM behavior or data issues. Always set a reasonable `max_runs_per_component` as a safety net.
2. **Relying on optional outputs**: When a component has multiple output sockets but only returns some of them, the unreturned outputs don't trigger their downstream connections. This can cause confusion in loops.
For example, this pattern can be problematic:
```python
@component
class Validator:
@component.output_types(valid=str, invalid=Optional[str])
def run(self, text: str):
if is_valid(text):
return {"valid": text} # "invalid" is never returned
else:
return {"invalid": text}
```
If you connect `invalid` back to an upstream component for retry, but also have other connections that keep the loop alive, you might get unexpected behavior.
Instead, use a `ConditionalRouter` with explicit, mutually exclusive conditions:
```python
router = ConditionalRouter(
routes=[
{"condition": "{{ is_valid }}", "output": "{{ text }}", "output_name": "valid", ...},
{"condition": "{{ not is_valid }}", "output": "{{ text }}", "output_name": "invalid", ...},
]
)
```
3. **User inputs retriggering the loop**: If a user-provided input is connected to a socket inside the loop, it might cause the loop to restart unexpectedly.
```python
# Problematic: user input goes directly to a component inside the loop
result = pipe.run({
"generator": {"prompt": query}, # This input persists and may retrigger the loop
})
# Better: use an entry-point component outside the loop
result = pipe.run({
"prompt_builder": {"query": query}, # Entry point feeds into the loop once
})
```
See [Greedy vs. Lazy Variadic Sockets](#greedy-vs-lazy-variadic-sockets-in-loops) for details on how inputs are consumed.
4. **Multiple paths feeding the same component**: If a component inside the loop receives inputs from multiple sources, it runs whenever *any* path provides input.
```python
# Component receives from two sources – runs when either provides input
pipe.connect("source_a.output", "processor.input")
pipe.connect("source_b.output", "processor.input") # Variadic input
```
Ensure you understand when each path produces output, or use `BranchJoiner` to explicitly control the merge point.
### Debugging Tips
1. **Start with a low limit**: When developing loops, set `max_runs_per_component=3` or similar. This helps you catch issues early with a clear error instead of waiting for a timeout.
2. **Use `include_outputs_from`**: Add intermediate components (like your router) to see what's happening at each step:
```python
result = pipe.run(data, include_outputs_from={"router", "validator"})
```
3. **Enable tracing**: Use tracing to see every component execution, including inputs and outputs. This makes it easy to follow each iteration of the loop. For quick debugging, use `LoggingTracer` ([setup instructions](./debugging-pipelines.mdx#real-time-pipeline-logging)). For deeper analysis, integrate with tools like Langfuse or other [tracing backends](../../development/tracing.mdx).
4. **Visualize the pipeline**: Use `pipe.draw()` or `pipe.show()` to see the graph structure and verify your connections are correct. See the [Pipeline Visualization](./visualizing-pipelines.mdx) documentation for details.
5. **Use breakpoints**: Set a `Breakpoint` on a specific component and visit count to inspect the state at that iteration. See [Pipeline Breakpoints](./pipeline-breakpoints.mdx) for details.
6. **Check for blocked pipelines**: If you see a `PipelineComponentsBlockedError`, it means no components can run. This typically indicates a missing connection or a circular dependency. Check that all required inputs are provided.
By combining careful graph design, per-component run limits, and these debugging tools, you can build robust feedback loops in your Haystack pipelines.
docs-website/versioned_docs/version-2.26/concepts/pipelines/serialization.mdx
+233
View File
@@ -0,0 +1,233 @@
---
title: "Serializing Pipelines"
id: serialization
slug: "/serialization"
description: "Save your pipelines into a custom format and explore the serialization options."
---
# Serializing Pipelines
Save your pipelines into a custom format and explore the serialization options.
Serialization means converting a pipeline to a format that you can save on your disk and load later.
Haystack supports YAML format for pipeline serialization.
## Converting a Pipeline to YAML
Use the `dumps()` method to convert a Pipeline object to YAML:
```python
from haystack import Pipeline
pipe = Pipeline()
print(pipe.dumps())
## Prints:
##
## components: {}
## connections: []
## max_runs_per_component: 100
## metadata: {}
```
You can also use `dump()` method to save the YAML representation of a pipeline in a file:
```python
with open("/content/test.yml", "w") as file:
pipe.dump(file)
```
## Converting a Pipeline Back to Python
You can convert a YAML pipeline back into Python. Use the `loads()` method to convert a string representation of a pipeline (`str`, `bytes` or `bytearray`) or the `load()` method to convert a pipeline represented in a file-like object into a corresponding Python object.
Both loading methods support callbacks that let you modify components during the deserialization process.
Here is an example script:
```python
from haystack import Pipeline
from haystack.core.serialization import DeserializationCallbacks
from typing import Type, Dict, Any
## This is the YAML you want to convert to Python:
pipeline_yaml = """
components:
cleaner:
init_parameters:
remove_empty_lines: true
remove_extra_whitespaces: true
remove_regex: null
remove_repeated_substrings: false
remove_substrings: null
type: haystack.components.preprocessors.document_cleaner.DocumentCleaner
converter:
init_parameters:
encoding: utf-8
type: haystack.components.converters.txt.TextFileToDocument
connections:
- receiver: cleaner.documents
sender: converter.documents
max_runs_per_component: 100
metadata: {}
"""
def component_pre_init_callback(
component_name: str,
component_cls: Type,
init_params: Dict[str, Any],
):
# This function gets called every time a component is deserialized.
if component_name == "cleaner":
assert "DocumentCleaner" in component_cls.__name__
# Modify the init parameters. The modified parameters are passed to
# the init method of the component during deserialization.
init_params["remove_empty_lines"] = False
print("Modified 'remove_empty_lines' to False in 'cleaner' component")
else:
print(f"Not modifying component {component_name} of class {component_cls}")
pipe = Pipeline.loads(
pipeline_yaml,
callbacks=DeserializationCallbacks(component_pre_init_callback),
)
```
## Default Serialization Behavior
The serialization system uses `default_to_dict` and `default_from_dict` to handle many object types automatically. You typically do **not** need to implement custom `to_dict`/`from_dict` for:
- **Secrets**: serialized and deserialized automatically so that sensitive values aren't stored in plain text.
- **ComponentDevice**: device configuration is detected and restored automatically.
- **Objects with their own `to_dict`/`from_dict`**: any init parameter whose type defines `to_dict()` is serialized by calling it; any dict in `init_parameters` with a `type` key pointing to a class with `from_dict()` is deserialized automatically.
To serialize or deserialize a single component, you can use `component_to_dict` and `component_from_dict` from `haystack.core.serialization`. They use the default behavior above as a fallback when the component doesn't define custom `to_dict`/`from_dict`:
```python
from haystack import component
from haystack.core.serialization import component_from_dict, component_to_dict
@component
class Greeter:
def __init__(self, message: str = "Hello"):
self.message = message
@component.output_types(greeting=str)
def run(self, name: str):
return {"greeting": f"{self.message}, {name}!"}
# Serialize a component instance to a dictionary
greeter = Greeter(message="Hi")
data = component_to_dict(greeter, "my_greeter")
# Deserialize back to a component instance
restored = component_from_dict(Greeter, data, "my_greeter")
assert restored.message == greeter.message
```
:::caution[Init parameters must be stored as instance attributes]
Default serialization only works when there is a **1:1 mapping** between init parameter names and instance attributes. For every argument in `__init__`, the component must assign it to an attribute with the same name. For example, if you have `def __init__(self, prompt: str)`, you must have `self.prompt = prompt` in the class. Otherwise the serialization logic can't find the value to serialize and raises an error or uses the default value if the parameter has one.
:::
## Performing Custom Serialization
Pipelines and components in Haystack can serialize simple components, including custom ones, out of the box. Code like this just works:
```python
from haystack import component
@component
class RepeatWordComponent:
def __init__(self, times: int):
self.times = times
@component.output_types(result=str)
def run(self, word: str):
return word * self.times
```
On the other hand, this code doesn't work if the final format is JSON, as the `set` type is not JSON-serializable:
```python
from haystack import component
@component
class SetIntersector:
def __init__(self, intersect_with: set):
self.intersect_with = intersect_with
@component.output_types(result=set)
def run(self, data: set):
return data.intersection(self.intersect_with)
```
In such cases, you can provide your own implementation `from_dict` and `to_dict` to components:
```python
from haystack import component, default_from_dict, default_to_dict
class SetIntersector:
def __init__(self, intersect_with: set):
self.intersect_with = intersect_with
@component.output_types(result=set)
def run(self, data: set):
return data.intersect(self.intersect_with)
def to_dict(self):
return default_to_dict(self, intersect_with=list(self.intersect_with))
@classmethod
def from_dict(cls, data):
# convert the set into a list for the dict representation,
# so it can be converted to JSON
data["intersect_with"] = set(data["intersect_with"])
return default_from_dict(cls, data)
```
## Saving a Pipeline to a Custom Format
Once a pipeline is available in its dictionary format, the last step of serialization is to convert that dictionary into a format you can store or send over the wire. Haystack supports YAML out of the box, but if you need a different format, you can write a custom Marshaller.
A `Marshaller` is a Python class responsible for converting text to a dictionary and a dictionary to text according to a certain format. Marshallers must respect the `Marshaller` [protocol](https://github.com/deepset-ai/haystack/blob/main/haystack/marshal/protocol.py), providing the methods `marshal` and `unmarshal`.
This is the code for a custom TOML marshaller that relies on the `rtoml` library:
```python
## This code requires a `pip install rtoml`
from typing import Dict, Any, Union
import rtoml
class TomlMarshaller:
def marshal(self, dict_: Dict[str, Any]) -> str:
return rtoml.dumps(dict_)
def unmarshal(self, data_: Union[str, bytes]) -> Dict[str, Any]:
return dict(rtoml.loads(data_))
```
You can then pass a Marshaller instance to the methods `dump`, `dumps`, `load`, and `loads`:
```python
from haystack import Pipeline
from my_custom_marshallers import TomlMarshaller
pipe = Pipeline()
pipe.dumps(TomlMarshaller())
## prints:
## 'max_runs_per_component = 100\nconnections = []\n\n[metadata]\n\n[components]\n'
```
## Additional References
:notebook: Tutorial: [Serializing LLM Pipelines](https://haystack.deepset.ai/tutorials/29_serializing_pipelines)
docs-website/versioned_docs/version-2.26/concepts/pipelines/smart-pipeline-connections.mdx
+148
View File
@@ -0,0 +1,148 @@
---
title: "Smart Pipeline Connections"
id: smart-pipeline-connections
slug: "/smart-pipeline-connections"
description: "Learn how Haystack pipelines simplify connections through implicit joining and flexible type adaptation, reducing the need for glue components."
---
# Smart Pipeline Connections
Haystack pipelines support smarter connection semantics that reduce boilerplate and make pipeline definitions easier to read and maintain.
These features focus on simplifying how components are connected, without changing component behavior.
Smart connections help eliminate common glue components such as `Joiners` and `OutputAdapters` in many pipelines.
## Implicit List Joining
Pipelines natively support connecting multiple component outputs directly to a single component input, without requiring an explicit `Joiner` component.
This works when:
* The target input is typed as `list`, `list | None`, or a union of list types (e.g. `list[int] | list[str]`).
* All connected outputs are compatible list types.
When multiple outputs are connected to the same input, the pipeline implicitly concatenates the lists from the outputs into a single list for the input.
### Example
Multiple converters can write directly into a single `DocumentWriter` without using a `DocumentJoiner`:
<details>
<summary>Expand to see the pipeline graph</summary>
<ClickableImage src="/img/pipeline-illustration-auto-joiner.png" alt="Pipeline architecture diagram showing a DocumentWriter receiving inputs from multiple converters without a Joiner" size="large" />
</details>
```python
from haystack import Pipeline
from haystack.components.converters import HTMLToDocument, TextFileToDocument
from haystack.components.routers import FileTypeRouter
from haystack.components.writers import DocumentWriter
from haystack.dataclasses import ByteStream
from haystack.document_stores.in_memory import InMemoryDocumentStore
sources = [
ByteStream.from_string(text="Text file content", mime_type="text/plain"),
ByteStream.from_string(
text="<html><body>Some content</body></html>",
mime_type="text/html",
),
]
doc_store = InMemoryDocumentStore()
pipe = Pipeline()
pipe.add_component("router", FileTypeRouter(mime_types=["text/plain", "text/html"]))
pipe.add_component("txt_converter", TextFileToDocument())
pipe.add_component("html_converter", HTMLToDocument())
pipe.add_component("writer", DocumentWriter(doc_store))
pipe.connect("router.text/plain", "txt_converter.sources")
pipe.connect("router.text/html", "html_converter.sources")
pipe.connect("txt_converter.documents", "writer.documents")
pipe.connect("html_converter.documents", "writer.documents")
result = pipe.run({"router": {"sources": sources}})
```
This pattern is especially useful when routing files, documents, or results across multiple parallel branches.
## Flexible Type Connections
To further streamline pipeline definitions, Haystack pipelines support limited implicit type adaptation at connection time.
This makes pipeline connections more flexible and reduces the need for `OutputAdapter` components.
**Supported adaptations**
| Source Type | Target Type | Behavior |
|--------------------------|--------------------|---------------------------------------------------------------|
| `str` | `ChatMessage` | Wrapped into a `ChatMessage` with user role. |
| `ChatMessage` | `str` | Extracts `ChatMessage.text`; raises `PipelineRuntimeError` if `None`. |
| `T` | `list[T]` | Wraps the item into a single-element list. |
| `list[str] or list[ChatMessage]`| `str` or `ChatMessage` | Extracts the first item; raises `PipelineRuntimeError` if the list is empty. |
All adaptations are checked at connection time to ensure type safety, but applied at runtime during pipeline execution.
When multiple connections are possible, strict type matching is prioritized over implicit conversion.
This preserves backward compatibility with earlier versions of Haystack, where flexible type connections were not supported.
### Example
Pipeline connecting the Chat Generator `messages` output (`list[ChatMessage]`) to the retriever `query` input (`str`)
without using an `OutputAdapter`:
```python
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.dataclasses import Document
from haystack.components.retrievers import InMemoryBM25Retriever
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
document_store = InMemoryDocumentStore()
documents = [
Document(content="Bob lives in Paris."),
Document(content="Alice lives in London."),
Document(content="Ivy lives in Melbourne."),
Document(content="Kate lives in Brisbane."),
Document(content="Liam lives in Adelaide."),
]
document_store.write_documents(documents)
template = """{% message role="user" %}
Rewrite the following query to be used for keyword search.
{{ query }}
{% endmessage %}
"""
p = Pipeline()
p.add_component("prompt_builder", ChatPromptBuilder(template=template))
p.add_component("llm", OpenAIChatGenerator(model="gpt-4.1-mini"))
p.add_component(
"retriever",
InMemoryBM25Retriever(document_store=document_store, top_k=3),
)
p.connect("prompt_builder", "llm")
# implicitly converts list[ChatMessage] -> str
p.connect("llm", "retriever")
query = """Someday I'd love to visit Brisbane, but for now I just want
to know the names of the people who live there."""
result = p.run(data={"prompt_builder": {"query": query}})
```
## When You Still Need `Joiners` or `OutputAdapters`
Explicit `Joiners` or `OutputAdapters` are still useful when you need:
- Custom aggregation logic beyond simple list concatenation
- Type conversions not covered by implicit adaptation
- Explicit control over formatting or ordering
Smart connections reduce the need for glue components, but they do not remove them entirely.
When in doubt, explicit components provide clarity and more control.
docs-website/versioned_docs/version-2.26/concepts/pipelines/visualizing-pipelines.mdx
+88
View File
@@ -0,0 +1,88 @@
---
title: "Visualizing Haystack Pipelines"
id: visualizing-pipelines
slug: "/visualizing-pipelines"
description: "You can visualize your Haystack AI pipelines as graphs to better understand how the components are connected."
---
import ClickableImage from "@site/src/components/ClickableImage";
# Visualizing Haystack Pipelines
You can visualize your pipelines as graphs to better understand how the components are connected.
Haystack pipelines have `draw()` and `show()` methods that enable you to visualize the pipeline as a graph using Mermaid graphs.
:::note[Data Privacy Notice]
Exercise caution with sensitive data when using pipeline visualization.
This feature is based on Mermaid graphs web service that doesn't have clear terms of data retention or privacy policy.
:::
## Prerequisites
To use Mermaid graphs, you must have an internet connection to reach the Mermaid graph renderer at https://mermaid.ink.
## Displaying a Graph
Use the pipeline's `show()` method to display the diagram in Jupyter notebooks.
```python
my_pipeline.show()
```
## Saving a Graph
Use the pipeline's `draw()` method passing the path where you want to save the diagram and the diagram format. Possible formats are: `mermaid-text` and `mermaid-image` (default).
```python
my_pipeline.draw(path=local_path)
```
## Visualizing SuperComponents
To show the internal structure of [SuperComponents](../components/supercomponents.mdx) in your digram instead of a black box component, set the `super_component_expansion` parameter to `True`:
```python
my_pipeline.show(super_component_expansion=True)
## or
my_pipeline.draw(path=local_path, super_component_expansion=True)
```
## Visualizing Locally
If you don't have an internet connection or don't want to send your pipeline data to the remote https://mermaid.ink, you can install a local mermaid.ink server and use it to render your pipeline.
Let's run a local mermaid.ink server using their official Docker images from https://github.com/jihchi/mermaid.ink/pkgs/container/mermaid.ink.
In this case, let's install one for a system running a MacOS M3 chip and expose it on port 3000:
```dockerfile
docker run --platform linux/amd64 --publish 3000:3000 --cap-add=SYS_ADMIN ghcr.io/jihchi/mermaid.ink
```
Check that the local mermaid.ink server is running by going to http://localhost:3000/.
You should see a local server running, and now you can simply render the image using your local mermaid.ink server by specifying the URL when calling the`show()`or `draw()` method:
```python
my_pipeline.show(server_url="http://localhost:3000")
## or
my_pipeline.draw("my_pipeline.png", server_url="http://localhost:3000")
```
## Example
This is an example of what a pipeline graph may look like:
<ClickableImage src="/img/46a8989-Untitled.png" alt="RAG pipeline flowchart showing the data flow from query through retriever, prompt builder, language model, and answer builder components" size="large" />
<br />
## Importing a Pipeline to Haystack Enterprise Platform
You can import your Haystack pipeline into Haystack Enterprise Platform and continue visually building your pipeline
To do that, follow the steps described in Haystack Enterprise Platform [documentation](https://docs.cloud.deepset.ai/docs/import-a-pipeline#import-your-pipeline).
docs-website/versioned_docs/version-2.26/concepts/secret-management.mdx
+195
View File
@@ -0,0 +1,195 @@
---
title: "Secret Management"
id: secret-management
slug: "/secret-management"
description: "This page emphasizes secret management in Haystack components and introduces the `Secret` type for structured secret handling. It explains the drawbacks of hard-coding secrets in code and suggests using environment variables instead."
---
# Secret Management
This page emphasizes secret management in Haystack components and introduces the `Secret` type for structured secret handling. It explains the drawbacks of hard-coding secrets in code and suggests using environment variables instead.
Many Haystack components interact with third-party frameworks and service providers such as Azure, Google Vertex AI, and OpenAI. Their libraries often require the user to authenticate themselves to ensure they receive access to the underlying product. The authentication process usually works with a secret value that acts as an opaque identifier to the third-party backend.
This page describes the two main types of secrets: token-based and environment variable-based, and how to handle them when using Haystack.
You can find additional details for the `Secret` class in our [API reference](/reference/utils-api).
<details>
<summary>Example Use Case - Problem Statement</summary>
### Problem Statement
Let’s consider an example RAG pipeline that embeds a query, uses a Retriever component to locate documents relevant to the query, and then leverages an LLM to generate an answer based on the retrieved documents.
The `OpenAIGenerator` component used in the pipeline below expects an API key to authenticate with OpenAI’s servers and perform the generation. Let’s assume that the component accepts a `str` value for it:
```python
generator = OpenAIGenerator(model="gpt-4", api_key="sk-xxxxxxxxxxxxxxxxxx")
pipeline.add_component("generator", generator)
```
This works in a pinch, but this is bad practice - we shouldn’t hard-code such secrets in the codebase. An alternative would be to store the key in an environment variable externally, read from it in Python, and pass that to the component:
```python
import os
api_key = os.environ.get("OPENAI_API_KEY")
generator = OpenAIGenerator(model="gpt-4", api_key=api_key)
pipeline.add_component("generator", generator)
```
This is better – the pipeline works as intended, and we aren’t hard-coding any secrets in the code.
Remember that pipelines are serializable. Since the API key is a secret, we should definitely avoid saving it to disk. Let’s modify the component’s `to_dict` method to exclude the key:
```python
def to_dict(self) -> Dict[str, Any]:
# Do not pass the `api_key` init parameter.
return default_to_dict(self, model=self.model)
```
But what happens when the pipeline is loaded from disk? In the best-case scenario, the component’s backend will automatically try to read the key from a hard-coded environment variable, and that key is the same as the one that was passed to the component before it was serialized. But in a worse case, the backend doesn’t look up the key in a hard-coded environment variable and fails when it gets called inside a `pipeline.run()` invocation.
</details>
### Import
To use Haystack secrets within the code, first import with:
```python
from haystack.utils import Secret
```
### Token-Based Secrets
You can paste tokens directly as a string using the `from_token` method:
```python
llm = OpenAIGenerator(api_key=Secret.from_token("sk-randomAPIkeyasdsa32ekasd32e"))
```
Note that this type of code cannot be serialized, meaning you can't convert the above component to a dictionary or save a pipeline containing it to a YAML file. This is a security feature to prevent accidental exposure of sensitive data.
### Environment Variable-Based Secrets
Environment variable-based secrets are more flexible. They allow you to specify one or more environment variables that may contain your secret.
Existing Haystack components that require an API Key (like OpenAIGenerator) have a default value for `Secret.from_env_var` (in this case, `OPENAI_API_KEY`). This means that the `OpenAIGenerator` will look for the value of the environment variable `OPENAI_API_KEY` (if it exists) and use it for authentication. And when pipelines are serialized to YAML, only the name of the environment variable is save to the YAML file. In doing so, this method ensures that there are no security leaks and is therefore strongly recommended.
```bash
## First, export an environment variable name `OPENAI_API_KEY` with its value
export OPENAI_API_KEY=sk-randomAPIkeyasdsa32ekasd32e
## or alternatively, using Python
## import os
## os.environ[”OPENAI_API_KEY”]=sk-randomAPIkeyasdsa32ekasd32e
```
```python
llm_generator = (
OpenAIGenerator()
) # Uses the default value from the env var for the component
```
Alternatively, in components where a Secret is expected, you can customize the name of the environment variable from which the API Key is to be read.
```python
## Export an environment variable with custom name and its value
llm_generator = OpenAIGenerator(api_key=Secret.from_env_var("YOUR_ENV_VAR"))
```
When `OpenAIGenerator` is serialized within a pipeline, this is what the YAML code will look like, using the custom variable name:
```yaml
components:
llm:
init_parameters:
api_base_url: null
api_key:
env_vars:
- YOUR_ENV_VAR
strict: true
type: env_var
generation_kwargs: {}
model: gpt-4o-mini
organization: null
streaming_callback: null
system_prompt: null
type: haystack.components.generators.openai.OpenAIGenerator
...
```
### Serialization
While token-based secrets cannot be serialized, environment variable-based secrets can be converted to and from dictionaries:
```python
## Convert to dictionary
env_secret_dict = env_secret.to_dict()
## Create from dictionary
new_env_secret = Secret.from_dict(env_secret_dict)
```
### Resolving Secrets
Both types of secrets can be resolved to their actual values using the `resolve_value` method. This method returns the token or the value of the environment variable.
```python
## Resolve the token-based secret
token_value = api_key_secret.resolve_value()
## Resolve the environment variable-based secret
env_value = env_secret.resolve_value()
```
### Custom Component Example
Here is a complete example that shows how to create a component that uses the `Secret` class in Haystack, highlighting the differences between token-based and environment variable-based authentication, and showing that token-based secrets cannot be serialized:
```python
from haystack.utils import Secret, deserialize_secrets_inplace
@component
class MyComponent:
def __init__(self, api_key: Optional[Secret] = None, **kwargs):
self.api_key = api_key
self.backend = None
def warm_up(self):
# Call resolve_value to yield a single result. The semantics of the result is policy-dependent.
# Currently, all supported policies will return a single string token.
self.backend = SomeBackend(
api_key=self.api_key.resolve_value() if self.api_key else None, ...
)
def to_dict(self):
# Serialize the policy like any other (custom) data. If the policy is token-based, it will
# raise an error.
return default_to_dict(
self, api_key=self.api_key.to_dict() if self.api_key else None, ...
)
@classmethod
def from_dict(cls, data):
# Deserialize the policy data before passing it to the generic from_dict function.
api_key_data = data["init_parameters"]["api_key"]
api_key = Secret.from_dict(api_key_data) if api_key_data is not None else None
data["init_parameters"]["api_key"] = api_key
# Alternatively, use the helper function.
# deserialize_secrets_inplace(data["init_parameters"], keys=["api_key"])
return default_from_dict(cls, data)
## No authentication.
component = MyComponent(api_key=None)
## Token based authentication
component = MyComponent(api_key=Secret.from_token("sk-randomAPIkeyasdsa32ekasd32e"))
component.to_dict() # Error! Can't serialize authentication tokens
## Environment variable based authentication
component = MyComponent(api_key=Secret.from_env_var("OPENAI_API_KEY"))
component.to_dict() # This is fine
```