chore: import upstream snapshot with attribution

docs-website/docs/_templates/component-template.mdx

@@ -0,0 +1,44 @@
+---
+title: "Component Name"
+id: "component-name"
+description: "A short description of the component"
+slug: "/component-name"
+---
+
+# Component Name
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** |  |
+| **Mandatory init variables** |  |
+| **Mandatory run variables** |  |
+| **Output variables** |  |
+| **API reference** |  |
+| **GitHub link** |  |
+| **Package name** |  |
+
+</div>
+
+## Overview
+
+*What does it do in general? For example,..?*
+
+*How does it work more specifically? Are there any pitfalls to pay attention to?*
+
+*(if applicable) How is it different from this other very similar component? Which one do you choose?*
+
+## Usage
+
+*Any mandatory imports?*
+
+### On its own
+
+*Code snippet on how to run a component*
+
+### In a pipeline
+
+*Code snippet of a component being introduced in a pipeline*
+
+*There can be more than one example. Add examples of pipelines where this component would be most useful, for example RAG, doc retrieval, etc.*

docs-website/docs/_templates/document-store-template.mdx

@@ -0,0 +1,30 @@
+---
+title: "Document Store Name"
+id: "document-store-name"
+description: "A short description of the document store"
+slug: "/document-store-name"
+---
+
+# Document Store Name
+
+## Description
+
+*What are this Document Store features? When would a user select it, and when not?*
+
+*Are there any limitations?*
+
+*Users are often curious to know if a document store supports metadata filtering and sparse vectors.*
+
+## Initialization
+
+*Describe how to get this Document Store to work, with code samples.*
+
+## Supported Retrievers
+
+*Name of the supported Retriever(s).*
+
+*If several – describe how to choose an appropriate one for user’s goals (perhaps, one is faster and the other is more accurate).*
+
+## Link to GitHub
+
+*for example [https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/gradient](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/gradient)*

docs-website/docs/concepts/agents.mdx

@@ -0,0 +1,127 @@
+---
+title: "Agents"
+id: agents
+slug: "/agents"
+description: "This page explains how to create an AI agent in Haystack capable of retrieving information, generating responses, and taking actions using various Haystack components."
+---
+
+# Agents
+
+This page explains how to create an AI agent in Haystack capable of retrieving information, generating responses, and taking actions using various Haystack components.
+
+## What’s an AI Agent?
+
+An AI agent is a system that can:
+
+- Understand user input (text, image, audio, and other queries),
+- Retrieve relevant information (documents or structured data),
+- Generate intelligent responses (using LLMs like OpenAI or Hugging Face models),
+- Perform actions (calling APIs, fetching live data, executing functions).
+
+AI agents are autonomous systems that use large language models (LLMs) to make decisions and solve complex tasks.
+They interact with their environment using tools, memory, and reasoning.
+An AI agent is more than a chatbot — it actively plans, chooses the right tools, and executes tasks to achieve a goal.
+Unlike traditional software, it adapts to new information and refines its process as needed.
+
+1. **LLM as the Brain**: The agent’s core is an LLM, which understands context, processes natural language and serves as the central intelligence system.
+2. **Tools for Interaction**: Agents connect to external tools, APIs, and databases to gather information and take action.
+3. **Memory for Context**: Short-term memory helps track conversations, while long-term memory stores knowledge for future interactions.
+4. **Reasoning and Planning**: Agents break down complex problems, come up with step-by-step action plans, and adapt based on new data and feedback.
+
+An AI agent starts with a prompt that defines its role and objectives.
+It decides when to use tools, gathers data, and refines its approach through loops of reasoning and action.
+For example, a customer service agent answers queries using a database — if it lacks an answer, it fetches real-time data, summarizes it, and responds.
+A coding assistant understands project requirements, suggests solutions, and writes code.
+
+## Key Components
+
+### Agent Component
+
+Haystack has a built-in [Agent](../pipeline-components/agents-1/agent.mdx) component that manages the full tool-calling loop — it calls the LLM, invokes tools, updates state, and continues until a stopping condition is met.
+Key capabilities include:
+
+- **State management**: Share typed data between tools, accumulate results across iterations, and surface them in the result dict using `state_schema`. See [State](../pipeline-components/agents-1/state.mdx).
+- **Streaming**: Stream token-by-token output with a `streaming_callback`.
+- **Human-in-the-loop**: Intercept tool calls for human review before execution. See [Human in the Loop](../pipeline-components/agents-1/human-in-the-loop.mdx).
+- **Multi-agent systems**: Wrap an `Agent` as a `ComponentTool` to build coordinator/specialist architectures. See [Multi-Agent Systems](./agents/multi-agent-systems.mdx).
+- **MCP server exposure**: Expose your agent as an MCP server using [Hayhooks](../development/hayhooks.mdx), making it callable from any MCP-compatible client such as Claude Desktop or Cursor.
+- **Multimodal inputs**: Pass images alongside text using `ImageContent` in `ChatMessage` content parts, or return `ImageContent` from tools for dynamic image analysis. Requires a vision-capable model such as `gpt-5` or `gemini-2.5-flash`. See [Multimodal Inputs](../pipeline-components/agents-1/agent.mdx#multimodal-inputs).
+
+Check out the [Agent](../pipeline-components/agents-1/agent.mdx) documentation, or the [example](#tool-calling-agent) below to get started.
+
+### State
+
+[`State`](../pipeline-components/agents-1/state.mdx) is Haystack's built-in mechanism for sharing data between tools and accumulating results across multiple tool calls.
+You define a `state_schema` on the `Agent`, and any keys declared there are returned alongside `messages` and `last_message` in the agent's result dict.
+
+### Tools
+
+Haystack provides several ways to create and manage tools:
+
+- [`Tool`](../tools/tool.mdx) class / [`@tool`](../tools/tool.mdx#tool-decorator) decorator – Define a tool from a Python function. The `@tool` decorator automatically uses the function's name and docstring; the `Tool` class gives full control over the name, description, and schema.
+- [`ComponentTool`](../tools/componenttool.mdx) – Wraps any Haystack component as a callable tool.
+- [`PipelineTool`](../tools/pipelinetool.mdx) – Wraps a full Haystack pipeline as a callable tool.
+- [`MCPTool`](../tools/mcptool.mdx) / [`MCPToolset`](../tools/mcptoolset.mdx) – Connects to Model Context Protocol (MCP) servers to load external tools.
+- [`Toolset`](../tools/toolset.mdx) – Groups multiple tools into a single unit to pass to an Agent or Generator.
+- [`SearchableToolset`](../tools/searchabletoolset.mdx) – Enables keyword-based tool discovery for large catalogs, so the LLM only sees relevant tools at each step.
+
+## Example
+
+### Tool-Calling Agent
+
+Create a tool-calling agent with the `Agent` component. This example requires `OPENAI_API_KEY` and `SERPERDEV_API_KEY` to be set as environment variables:
+
+```shell
+export OPENAI_API_KEY=<your-openai-key>
+export SERPERDEV_API_KEY=<your-serperdev-key>
+```
+
+The examples on this page use SerperDev web search component that have moved to the `serperdev-haystack` package. Install it to run the examples:
+
+```shell
+pip install serperdev-haystack
+```
+
+```python
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
+from haystack.dataclasses import ChatMessage
+from haystack.tools import ComponentTool
+
+# Wrap the web search component as a tool
+web_tool = ComponentTool(
+    component=SerperDevWebSearch(top_k=3),
+    name="web_search",
+    description="Search the web for current information like weather, news, or facts.",
+)
+
+tool_calling_agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    system_prompt=(
+        "You're a helpful agent. When asked about current information like weather, news, or facts, "
+        "use the web_search tool to find the information and then summarize the findings."
+    ),
+    tools=[web_tool],
+    streaming_callback=print_streaming_chunk,
+)
+
+result = tool_calling_agent.run(
+    messages=[ChatMessage.from_user("How is the weather in Berlin?")],
+)
+print(result["last_message"].text)
+```
+
+Resulting in:
+
+```python
+>>> The current weather in Berlin is approximately 60°F. The forecast for today includes clouds in the morning with some sunshine later. The high temperature is expected to be around 65°F, and the low tonight will drop to 40°F.
+
+- **Morning**: 49°F
+- **Afternoon**: 57°F
+- **Evening**: 47°F
+- **Overnight**: 39°F
+
+For more details, you can check the full forecasts on [AccuWeather](https://www.accuweather.com/en/de/berlin/10178/current-weather/178087) or [Weather.com](https://weather.com/weather/today/l/5ca23443513a0fdc1d37ae2ffaf5586162c6fe592a66acc9320a0d0536be1bb9).
+```

docs-website/docs/concepts/agents/multi-agent-systems.mdx

@@ -0,0 +1,346 @@
+---
+title: "Multi-Agent Systems"
+id: multi-agent-systems
+slug: "/multi-agent-systems"
+description: "Learn how to build multi-agent systems in Haystack by spawning agents as tools. Use the @tool decorator or ComponentTool to connect specialist agents to a coordinator."
+---
+
+# Multi-Agent Systems
+
+Multi-agent systems let you compose multiple `Agent` instances into larger architectures where a **coordinator** agent delegates to **specialist** agents.
+Each specialist focuses on a specific task with its own tools and system prompt - the coordinator plans and routes work without needing to know how each task gets done.
+
+Spawning agents as tools is useful when:
+
+- A task is too broad for a single agent to handle reliably,
+- You want to isolate different capabilities into focused, reusable agents,
+- You need to keep the coordinator's context lean for better decisions and lower token usage.
+
+In Haystack, you spawn a specialist agent as a tool using either the `@tool` decorator (recommended) or `ComponentTool`.
+
+## Converting an Agent to a Tool
+
+### `@tool` Decorator (Recommended)
+
+Wrapping an agent inside a `@tool` function gives you full control over what the coordinator LLM sees:
+
+- **Simplified parameters**: define explicit `Annotated` arguments instead of exposing `agent.run()`'s full interface
+- **Formatted output**: extract and return only what the coordinator needs, rather than the full result dict
+- **Error handling**: catch exceptions and return a clean message so the coordinator can recover
+
+This approach works better with smaller LLMs because the tool has a clean, minimal signature.
+The coordinator only needs to provide a query string - all the `ChatMessage` construction and result unpacking is hidden inside the function.
+
+The examples on this page use SerperDev web search component that have moved to the `serperdev-haystack` package. Install it to run the examples:
+
+```shell
+pip install serperdev-haystack
+```
+
+```python
+from typing import Annotated
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.dataclasses import ChatMessage
+from haystack.tools import ComponentTool, tool
+from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
+from haystack.utils import Secret
+
+
+research_agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[
+        ComponentTool(
+            component=SerperDevWebSearch(
+                api_key=Secret.from_env_var("SERPERDEV_API_KEY"),
+                top_k=3,
+            ),
+            name="web_search",
+            description="Search the web for current information on any topic",
+        ),
+    ],
+    system_prompt="You are a research specialist. Search the web to find information.",
+)
+
+
+@tool
+def research(query: Annotated[str, "The research question to investigate"]) -> str:
+    """Research a topic and return a summary of findings."""
+    try:
+        result = research_agent.run(messages=[ChatMessage.from_user(query)])
+        return result["last_message"].text
+    except Exception as e:
+        return f"Research failed: {e}"
+
+
+coordinator = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[research],
+    system_prompt="You are a coordinator. Delegate research tasks to the research tool.",
+    streaming_callback=print_streaming_chunk,
+)
+
+result = coordinator.run(
+    messages=[
+        ChatMessage.from_user("What are the latest developments in Haystack AI?"),
+    ],
+)
+```
+
+### `ComponentTool`
+
+`ComponentTool` wraps an agent directly without a wrapper function.
+Choose it when you want **declarative configuration**: the full specialist setup (model, tools, system prompt) lives in one serializable object alongside the coordinator.
+
+Use `outputs_to_string={"source": "last_message"}` to surface only the specialist's final reply to the coordinator rather than the full result dict.
+
+```python
+from haystack.tools import ComponentTool
+
+research_tool = ComponentTool(
+    component=research_agent,
+    name="research_specialist",
+    description="A specialist that researches topics on the web",
+    outputs_to_string={"source": "last_message"},
+)
+
+coordinator = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[research_tool],
+    system_prompt="You are a coordinator. Delegate research tasks to the research specialist.",
+    streaming_callback=print_streaming_chunk,
+)
+
+result = coordinator.run(
+    messages=[
+        ChatMessage.from_user("What are the latest developments in Haystack AI?"),
+    ],
+)
+```
+
+The full specialist configuration is captured inline when serialized.
+Wrap the coordinator in a `Pipeline` and call `pipeline.dumps()` to get the YAML, which can be loaded back with `Pipeline.loads()`.
+
+<details>
+<summary>View YAML</summary>
+
+```yaml
+components:
+  coordinator:
+    init_parameters:
+      chat_generator:
+        init_parameters:
+          api_base_url: null
+          api_key:
+            env_vars:
+            - OPENAI_API_KEY
+            strict: true
+            type: env_var
+          generation_kwargs: {}
+          http_client_kwargs: null
+          max_retries: null
+          model: gpt-5.4-nano
+          organization: null
+          streaming_callback: null
+          timeout: null
+          tools: null
+          tools_strict: false
+        type: haystack.components.generators.chat.openai.OpenAIChatGenerator
+      exit_conditions:
+      - text
+      hooks: null
+      max_agent_steps: 100
+      raise_on_tool_invocation_failure: false
+      required_variables: null
+      state_schema: {}
+      streaming_callback: null
+      system_prompt: You are a coordinator. Delegate research tasks to the research
+        specialist. Keep your final answer concise.
+      tool_concurrency_limit: 4
+      tool_streaming_callback_passthrough: false
+      tools:
+      - data:
+          component:
+            init_parameters:
+              chat_generator:
+                init_parameters:
+                  api_base_url: null
+                  api_key:
+                    env_vars:
+                    - OPENAI_API_KEY
+                    strict: true
+                    type: env_var
+                  generation_kwargs: {}
+                  http_client_kwargs: null
+                  max_retries: null
+                  model: gpt-5.4-nano
+                  organization: null
+                  streaming_callback: null
+                  timeout: null
+                  tools: null
+                  tools_strict: false
+                type: haystack.components.generators.chat.openai.OpenAIChatGenerator
+              exit_conditions:
+              - text
+              hooks: null
+              max_agent_steps: 100
+              raise_on_tool_invocation_failure: false
+              required_variables: null
+              state_schema: {}
+              streaming_callback: null
+              system_prompt: You are a research specialist. Search the web to find
+                information. Return a concise summary of your findings in 3-5 sentences.
+              tool_concurrency_limit: 4
+              tool_streaming_callback_passthrough: false
+              tools:
+              - data:
+                  component:
+                    init_parameters:
+                      allowed_domains: null
+                      api_key:
+                        env_vars:
+                        - SERPERDEV_API_KEY
+                        strict: true
+                        type: env_var
+                      exclude_subdomains: false
+                      search_params: {}
+                      top_k: 3
+                    type: haystack_integrations.components.websearch.serperdev.websearch.SerperDevWebSearch
+                  description: Search the web for current information on any topic
+                  inputs_from_state: null
+                  name: web_search
+                  outputs_to_state: null
+                  outputs_to_string: null
+                  parameters: null
+                type: haystack.tools.component_tool.ComponentTool
+              user_prompt: null
+            type: haystack.components.agents.agent.Agent
+          description: A specialist that researches topics on the web
+          inputs_from_state: null
+          name: research_specialist
+          outputs_to_state: null
+          outputs_to_string:
+            source: last_message
+          parameters: null
+        type: haystack.tools.component_tool.ComponentTool
+      user_prompt: null
+    type: haystack.components.agents.agent.Agent
+connection_type_validation: true
+connections: []
+max_runs_per_component: 100
+metadata: {}
+```
+
+</details>
+
+## Coordinator / Specialist Pattern
+
+The coordinator/specialist pattern cleanly splits responsibilities: the coordinator handles planning and delegation, while each specialist owns a focused toolset and a targeted system prompt.
+
+This is also a form of **context engineering**: deliberately controlling what each agent sees.
+A specialist accumulates its own tool call trace, but the coordinator only needs the final answer.
+Returning just `result["last_message"].text` (with `@tool`) or using `outputs_to_string` (with `ComponentTool`) surfaces only the specialist's final reply, keeping the coordinator's context lean.
+
+When covering multiple topics, the coordinator can call the same specialist tool several times in a single response.
+All tool calls from one LLM response are executed concurrently using a thread pool.
+Control the level of parallelism with the `tool_concurrency_limit` init parameter (default: `4`).
+
+The example below asks the coordinator about two topics: it calls `research` twice and both specialists run in parallel.
+
+`HTMLToDocument` uses [Trafilatura](https://trafilatura.readthedocs.io) to extract clean text from HTML pages.
+Install it before running:
+
+```shell
+pip install trafilatura
+```
+
+```python
+from typing import Annotated
+from haystack.components.agents import Agent
+from haystack.components.converters import HTMLToDocument
+from haystack.components.fetchers.link_content import LinkContentFetcher
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
+from haystack.dataclasses import ChatMessage
+from haystack.tools import ComponentTool, tool
+from haystack.utils import Secret
+
+
+search_tool = ComponentTool(
+    component=SerperDevWebSearch(
+        api_key=Secret.from_env_var("SERPERDEV_API_KEY"),
+        top_k=3,
+    ),
+    name="web_search",
+    description="Search the web for current information on any topic",
+)
+
+
+@tool
+def fetch_page(url: Annotated[str, "The URL of the web page to fetch"]) -> str:
+    """Fetch the content of a web page given its URL."""
+    try:
+        streams = LinkContentFetcher().run(urls=[url])["streams"]
+        if not streams:
+            return "No content found."
+        documents = HTMLToDocument().run(sources=streams)["documents"]
+        return documents[0].content if documents else "No content extracted."
+    except Exception as e:
+        return f"Failed to fetch page: {e}"
+
+
+research_agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[search_tool, fetch_page],
+    system_prompt=(
+        "You are a research specialist. Search the web to find relevant pages, "
+        "then fetch their full content for detailed information. "
+        "Return a concise summary of your findings in 3-5 sentences."
+    ),
+)
+
+
+@tool
+def research(query: Annotated[str, "The research question to investigate"]) -> str:
+    """Research a topic and return a summary of findings."""
+    try:
+        result = research_agent.run(messages=[ChatMessage.from_user(query)])
+        return result["last_message"].text
+    except Exception as e:
+        return f"Research failed: {e}"
+
+
+coordinator = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[research],
+    system_prompt=(
+        "You are a coordinator. Delegate research tasks to the research tool. "
+        "For questions covering multiple topics, research each one independently. "
+        "Keep your final answer concise."
+    ),
+    streaming_callback=print_streaming_chunk,
+    tool_concurrency_limit=4,  # run up to 4 specialist calls in parallel
+)
+
+result = coordinator.run(
+    messages=[
+        ChatMessage.from_user(
+            "What are the latest developments in large language models and retrieval-augmented generation?",
+        ),
+    ],
+)
+```
+
+## Additional References
+
+📖 Related docs:
+
+- [Agent](../../pipeline-components/agents-1/agent.mdx)
+- [State](../../pipeline-components/agents-1/state.mdx)
+- [ComponentTool](../../tools/componenttool.mdx)
+
+📚 Tutorials:
+
+- [Creating a Multi-Agent System](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system)

docs-website/docs/concepts/components.mdx

@@ -0,0 +1,67 @@
+---
+title: "Components"
+id: components
+slug: "/components"
+description: "Components are the building blocks of a pipeline. They perform tasks such as preprocessing, retrieving, or summarizing text while routing queries through different branches of a pipeline. This page is a summary of all component types available in Haystack."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Components
+
+Components are the building blocks of a pipeline. They perform tasks such as preprocessing, retrieving, or summarizing text while routing queries through different branches of a pipeline. This page is a summary of all component types available in Haystack.
+
+Components are connected to each other using a [pipeline](pipelines.mdx), and they function like building blocks that can be easily switched out for each other. A component can take the selected outputs of other components as input. You can also provide input to a component when you call `pipeline.run()`.
+
+## Stand-Alone or In a Pipeline
+
+You can integrate components in a pipeline to perform a specific task. But you can also use some of them stand-alone, outside of a pipeline. For example, you can run `DocumentWriter` on its own, to write documents into a Document Store. To check how to use a component and if it's usable outside of a pipeline, check the _Usage_ section on the component's documentation page.
+
+Each component has a `run()` method. When you connect components in a pipeline, and you run the pipeline by calling `Pipeline.run()`, it invokes the `run()` method for each component sequentially.
+
+## Input and Output
+
+To connect components in a pipeline, you need to know the names of the inputs and outputs they accept. The output of one component must be compatible with the input the subsequent component accepts. For example, to connect Retriever and Ranker in a pipeline, you must know that the Retriever outputs `documents` and the Ranker accepts `documents` as input.
+
+The mandatory inputs and outputs are listed in a table at the top of each component's documentation page so that you can quickly check them:
+<ClickableImage src="/img/3a53f3e-inputs_and_outputs.png" alt="DocumentWriter component specification table showing Name, Folder Path, Position in Pipeline, Inputs (documents list), and Outputs (documents_written integer)" />
+
+You can also look them up in the code in the component`run()` method. Here's an example of the inputs and outputs of `TransformerSimilarityRanker`:
+
+```python
+@component.output_types(documents=List[Document]) # "documents" is the output name you need when connecting components in a pipeline
+def run(self, query: str, documents: List[Document], top_k: Optional[int] = None):# "query" and "documents" are the mandatory inputs, additionally you can also specify the optional top_k parameter
+"""
+Returns a list of Documents ranked by their similarity to the given query.
+
+:param query: Query string.
+:param documents: List of Documents.
+:param top_k: The maximum number of Documents you want the Ranker to return.
+:return: List of Documents sorted by their similarity to the query with the most similar Documents appearing first.
+"""
+```
+
+## Warming Up Components
+
+Components that use heavy resources, like LLMs or embedding models, have a `warm_up()` method that loads the necessary resources (such as models) into memory. This method is automatically called the first time the component runs, so you can use components directly without explicitly calling `warm_up()`:
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+```python
+from haystack import Document
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersDocumentEmbedder,
+)
+
+doc = Document(content="I love pizza!")
+doc_embedder = SentenceTransformersDocumentEmbedder()
+
+result = doc_embedder.run([doc])  # warm_up() is called automatically on first run
+print(result["documents"][0].embedding)
+```
+
+You can still call `warm_up()` explicitly if you want to control when resources are loaded.

docs-website/docs/concepts/components/custom-components.mdx

@@ -0,0 +1,182 @@
+---
+title: "Creating Custom Components"
+id: custom-components
+slug: "/custom-components"
+description: "Create your own components and use them standalone or in pipelines."
+---
+
+# Creating Custom Components
+
+Create your own components and use them standalone or in pipelines.
+
+With Haystack, you can easily create any custom components for various tasks, from filtering results to integrating with external software. You can then insert, reuse, and share these components within Haystack or even with an external audience by packaging them and submitting them to [Haystack Integrations](../integrations.mdx)!
+
+## Requirements
+
+Here are the requirements for all custom components:
+
+- `@component`: This decorator marks a class as a component, allowing it to be used in a pipeline.
+- `run()`: This is a required method in every component. It accepts input arguments and returns a `dict`. The inputs can either come from the pipeline when it’s executed, or from the output of another component when connected using `connect()`. The `run()` method should be compatible with the input/output definitions declared for the component. See an [Extended Example](#extended-example) below to check how it works.
+
+
+:::note[Avoid in-place input mutation]
+
+When building custom components, do not change the component's inputs directly. Instead, work on a copy or a new version of the input, modify that, and return it. The reason for this is that the original input values might be reused by other components or by later pipeline steps. Mutating the input directly can lead to unintended side effects and bugs in the pipeline, as other components might rely on the original input values.
+
+When only one or a few fields of the input need to be changed (for example, `meta` on a `Document`), use `dataclasses.replace()` to create a new instance with the updated fields. This is simpler and more efficient than deep-copying the whole object:
+
+```python
+from dataclasses import replace
+
+
+def run(self, documents):
+    updated = [replace(doc, meta={**doc.meta, "processed": True}) for doc in documents]
+    return {"documents": updated}
+```
+
+When you need to modify nested mutable structures, for example `list` or `dict` attributes, or update many fields of the dataclass instance, use a full deep copy instead:
+
+```python
+import copy
+
+
+def run(self, documents):
+    documents_copy = copy.deepcopy(documents)
+    # mutate documents_copy safely here
+    return {"documents": documents_copy}
+```
+
+:::
+
+
+### Inputs and Outputs
+
+Next, define the inputs and outputs for your component.
+
+#### Inputs
+
+You can choose between three input options:
+
+- `set_input_type`: This method defines or updates a single input socket for a component instance. It’s ideal for adding or modifying a specific input at runtime without affecting others. Use this when you need to dynamically set or modify a single input based on specific conditions.
+- `set_input_types`: This method allows you to define multiple input sockets at once, replacing any existing inputs. It’s useful when you know all the inputs the component will need and want to configure them in bulk. Use this when you want to define multiple inputs during initialization.
+- Declaring arguments directly in the `run()` method. Use this method when the component’s inputs are static and known at the time of class definition.
+
+#### Outputs
+
+You can choose between two output options:
+
+- `@component.output_types`: This decorator defines the output types and names at the time of class definition. The output names and types must match the `dict` returned by the `run()` method. Use this when the output types are static and known in advance. This decorator is cleaner and more readable for static components.
+- `set_output_types`: This method defines or updates multiple output sockets for a component instance at runtime. It’s useful when you need flexibility in configuring outputs dynamically. Use this when the output types need to be set at runtime for greater flexibility.
+
+## Short Example
+
+Here is an example of a simple minimal component setup:
+
+```python
+from haystack import component
+
+
+@component
+class WelcomeTextGenerator:
+    """
+    A component generating personal welcome message and making it upper case
+    """
+
+    @component.output_types(welcome_text=str, note=str)
+    def run(self, name: str):
+        return {
+            "welcome_text": f"Hello {name}, welcome to Haystack!".upper(),
+            "note": "welcome message is ready",
+        }
+```
+
+Here, the custom component `WelcomeTextGenerator` accepts one input: `name` string and returns two outputs: `welcome_text` and `note`.
+
+## Extended Example
+
+Check out an example below on how to create two custom components and connect them in a Haystack pipeline.
+
+```python
+# import necessary dependencies
+from haystack import component, Pipeline
+
+
+# Create two custom components. Note the mandatory @component decorator and @component.output_types, as well as the mandatory run method.
+@component
+class WelcomeTextGenerator:
+    """
+    A component generating personal welcome message and making it upper case
+    """
+
+    @component.output_types(welcome_text=str, note=str)
+    def run(self, name: str):
+        return {
+            "welcome_text": (
+                "Hello {name}, welcome to Haystack!".format(name=name)
+            ).upper(),
+            "note": "welcome message is ready",
+        }
+
+
+@component
+class WhitespaceSplitter:
+    """
+    A component for splitting the text by whitespace
+    """
+
+    @component.output_types(split_text=list[str])
+    def run(self, text: str):
+        return {"split_text": text.split()}
+
+
+# create a pipeline and add the custom components to it
+text_pipeline = Pipeline()
+text_pipeline.add_component(
+    name="welcome_text_generator",
+    instance=WelcomeTextGenerator(),
+)
+text_pipeline.add_component(name="splitter", instance=WhitespaceSplitter())
+
+# connect the components
+text_pipeline.connect(
+    sender="welcome_text_generator.welcome_text",
+    receiver="splitter.text",
+)
+
+# define the result and run the pipeline
+result = text_pipeline.run({"welcome_text_generator": {"name": "Bilge"}})
+
+print(result["splitter"]["split_text"])
+```
+
+## Extending the Existing Components
+
+To extend already existing components in Haystack, subclass an existing component and use the `@component` decorator to mark it. Override or extend the `run()` method to process inputs and outputs. Call `super()` with the derived class name from the init of the derived class to avoid initialization issues:
+
+```python
+class DerivedComponent(BaseComponent):
+    def __init__(self):
+        super(DerivedComponent, self).__init__()
+
+
+# ...
+
+dc = DerivedComponent()  # ok
+```
+
+An example of an extended component is Haystack's [FaithfulnessEvaluator](https://github.com/deepset-ai/haystack/blob/e5a80722c22c59eb99416bf0cd712f6de7cd581a/haystack/components/evaluators/faithfulness.py) derived from LLMEvaluator.
+
+## Project Template
+
+If you're building a custom component that you want to package and share, we provide a [GitHub template repository](https://github.com/deepset-ai/custom-component) that gives you a ready-made project structure. It includes the boilerplate for packaging, testing, and distributing your custom component as a standalone Python package. Use it to quickly scaffold a new integration or reusable component without setting up the project from scratch.
+
+Check out the [video walkthrough](https://www.youtube.com/watch?v=SWC0QecAMcI) for a step-by-step guide on how to use the template.
+
+## Additional References
+
+🧑‍🍳 Cookbooks:
+
+- [Build quizzes and adventures with Character Codex and llamafile](https://haystack.deepset.ai/cookbook/charactercodex_llamafile/)
+- [Run tasks concurrently within a custom component](https://haystack.deepset.ai/cookbook/concurrent_tasks/)
+- [Chat With Your SQL Database](https://haystack.deepset.ai/cookbook/chat_with_sql_3_ways/)
+- [Hacker News Summaries with Custom Components](https://haystack.deepset.ai/cookbook/hackernews-custom-component-rag/)

docs-website/docs/concepts/components/supercomponents.mdx

@@ -0,0 +1,195 @@
+---
+title: "SuperComponents"
+id: supercomponents
+slug: "/supercomponents"
+description: "`SuperComponent` lets you wrap a complete pipeline and use it like a single component. This is helpful when you want to simplify the interface of a complex pipeline, reuse it in different contexts, or expose only the necessary inputs and outputs."
+---
+
+# SuperComponents
+
+`SuperComponent` lets you wrap a complete pipeline and use it like a single component. This is helpful when you want to simplify the interface of a complex pipeline, reuse it in different contexts, or expose only the necessary inputs and outputs.
+
+## `@super_component` decorator (recommended)
+
+Haystack now provides a simple `@super_component` decorator for wrapping a pipeline as a component. All you need is to create a class with the decorator, and to include an `pipeline` attribute.
+
+With this decorator, the `to_dict` and `from_dict` serialization is optional, as is the input and output mapping.
+
+### Example
+
+The custom HybridRetriever example SuperComponent below turns your query into embeddings, then runs both a BM25 search and an embedding-based search at the same time. It finally merges those two result sets and returns the combined documents.
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+```python
+# pip install haystack-ai datasets sentence-transformers-haystack
+
+from haystack import Document, Pipeline, super_component
+from haystack.components.joiners import DocumentJoiner
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersTextEmbedder,
+)
+from haystack.components.retrievers import (
+    InMemoryBM25Retriever,
+    InMemoryEmbeddingRetriever,
+)
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+
+from datasets import load_dataset
+
+
+@super_component
+class HybridRetriever:
+    def __init__(
+        self,
+        document_store: InMemoryDocumentStore,
+        embedder_model: str = "BAAI/bge-small-en-v1.5",
+    ):
+        embedding_retriever = InMemoryEmbeddingRetriever(document_store)
+        bm25_retriever = InMemoryBM25Retriever(document_store)
+        text_embedder = SentenceTransformersTextEmbedder(embedder_model)
+        document_joiner = DocumentJoiner()
+
+        self.pipeline = Pipeline()
+        self.pipeline.add_component("text_embedder", text_embedder)
+        self.pipeline.add_component("embedding_retriever", embedding_retriever)
+        self.pipeline.add_component("bm25_retriever", bm25_retriever)
+        self.pipeline.add_component("document_joiner", document_joiner)
+
+        self.pipeline.connect("text_embedder", "embedding_retriever")
+        self.pipeline.connect("bm25_retriever", "document_joiner")
+        self.pipeline.connect("embedding_retriever", "document_joiner")
+
+
+dataset = load_dataset("HaystackBot/medrag-pubmed-chunk-with-embeddings", split="train")
+docs = [
+    Document(content=doc["contents"], embedding=doc["embedding"]) for doc in dataset
+]
+document_store = InMemoryDocumentStore()
+document_store.write_documents(docs)
+
+query = "What treatments are available for chronic bronchitis?"
+
+result = HybridRetriever(document_store).run(text=query, query=query)
+print(result)
+```
+
+### Input Mapping
+
+You can optionally map the input names of your SuperComponent to the actual sockets inside the pipeline.
+
+```python
+input_mapping = {"query": ["retriever.query", "prompt.query"]}
+```
+
+### Output Mapping
+
+You can also map the pipeline's output sockets that you want to expose to the SuperComponent's output names.
+
+```python
+output_mapping = {"llm.replies": "replies"}
+```
+
+If you don’t provide mappings, SuperComponent will try to auto-detect them. So, if multiple components have outputs with the same name, we recommend using `output_mapping` to avoid conflicts.
+
+## SuperComponent class
+
+Haystack also gives you an option to inherit from SuperComponent class. This option requires `to_dict` and `from_dict` serialization, as well as the input and output mapping described above.
+
+### Example
+
+Here is a simple example of initializing a `SuperComponent` with a pipeline:
+
+```python
+from haystack import Pipeline, SuperComponent
+
+with open("pipeline.yaml", "r") as file:
+    pipeline = Pipeline.load(file)
+
+super_component = SuperComponent(pipeline)
+```
+
+The example pipeline below retrieves relevant documents based on a user query, builds a custom prompt using those documents, then sends the prompt to an `OpenAIChatGenerator` to create an answer. The `SuperComponent` wraps the pipeline so it can be run with a simple input (`query`) and returns a clean output (`replies`).
+
+```python
+from haystack import Pipeline, SuperComponent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack.dataclasses.chat_message import ChatMessage
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.dataclasses import Document
+
+document_store = InMemoryDocumentStore()
+documents = [
+    Document(content="Paris is the capital of France."),
+    Document(content="London is the capital of England."),
+]
+document_store.write_documents(documents)
+
+prompt_template = [
+    ChatMessage.from_user(
+    '''
+    According to the following documents:
+    {% for document in documents %}
+    {{document.content}}
+    {% endfor %}
+    Answer the given question: {{query}}
+    Answer:
+    '''
+    )
+]
+
+prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
+
+pipeline = Pipeline()
+pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
+pipeline.add_component("prompt_builder", prompt_builder)
+pipeline.add_component("llm", OpenAIChatGenerator())
+pipeline.connect("retriever.documents", "prompt_builder.documents")
+pipeline.connect("prompt_builder.prompt", "llm.messages")
+
+# Create a super component with simplified input/output mapping
+wrapper = SuperComponent(
+    pipeline=pipeline,
+    input_mapping={
+        "query": ["retriever.query", "prompt_builder.query"],
+    },
+    output_mapping={
+        "llm.replies": "replies",
+        "retriever.documents": "documents"
+    }
+)
+
+# Run the pipeline with simplified interface
+result = wrapper.run(query="What is the capital of France?")
+print(result)
+{'replies': [ChatMessage(_role=<ChatRole.ASSISTANT: 'assistant'>,
+ _content=[TextContent(text='The capital of France is Paris.')],...)
+```
+
+## Type Checking and Static Code Analysis
+
+Creating SuperComponents using the @super_component decorator can induce type or linting errors. One way to avoid these issues is to add the exposed public methods to your SuperComponent. Here's an example:
+
+```python
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+
+    def run(self, *, documents: list[Document]) -> dict[str, list[Document]]: ...
+    def warm_up(self) -> None:  # noqa: D102
+        ...
+```
+
+## Ready-Made SuperComponents
+
+You can see two implementations of SuperComponents already integrated in Haystack:
+
+- [DocumentPreprocessor](../../pipeline-components/preprocessors/documentpreprocessor.mdx)
+- [MultiFileConverter](../../pipeline-components/converters/multifileconverter.mdx)
+- [OpenSearchHybridRetriever](../../pipeline-components/retrievers/opensearchhybridretriever.mdx)

docs-website/docs/concepts/concepts-overview.mdx

@@ -0,0 +1,56 @@
+---
+title: "Haystack Concepts Overview"
+id: concepts-overview
+slug: "/concepts-overview"
+description: "Haystack provides all the tools you need to build custom agents and RAG pipelines with LLMs that work for you. This includes everything from prototyping to deployment. This page discusses the most important concepts Haystack operates on."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Haystack Concepts Overview
+
+Haystack provides all the tools you need to build custom agents and RAG pipelines with LLMs that work for you. This includes everything from prototyping to deployment. This page discusses the most important concepts Haystack operates on.
+
+### Components
+
+Haystack offers various components, each performing different kinds of tasks. You can see the whole variety in the **PIPELINE COMPONENTS** section in the left-side navigation. These are often powered by the latest Large Language Models (LLMs) and transformer models. Code-wise, they are Python classes with methods you can directly call. Most commonly, all you need to do is initialize the component with the required parameters and then run it with a `run()` method.
+
+Working on this level with Haystack components is a hands-on approach. Components define the name and the type of all of their inputs and outputs. The Component API reduces complexity and makes it easier to [create custom components](components/custom-components.mdx), for example, for third-party APIs and databases. Haystack validates the connections between components before running the pipeline and, if needed, generates error messages with instructions on fixing the errors.
+
+#### Generators
+
+[Generators](../pipeline-components/generators.mdx) are responsible for generating text responses after you give them a prompt. They are specific for each LLM technology (OpenAI, Cohere, local models, and others). There are two types of Generators: chat and non-chat:
+
+- The chat ones enable chat completion and are designed for conversational contexts. It expects a list of messages to interact with the user.
+- The non-chat Generators use LLMs for simpler text generation (for example, translating or summarizing text).
+
+Read more about various Generators in our [guides](../pipeline-components/generators/guides-to-generators/choosing-the-right-generator.mdx).
+
+#### Retrievers
+
+[Retrievers](../pipeline-components/retrievers.mdx) go through all the documents in a Document Store, select the ones that match the user query, and pass it on to the next component. There are various Retrievers that are customized for specific Document Stores. This means that they can handle specific requirements for each database using customized parameters.
+
+For example, for Elasticsearch Document Store, you will find both the Document Store and Retriever packages in its GitHub [repo](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch).
+
+### Document Stores
+
+[Document Store](document-store.mdx) is an object that stores your documents in Haystack, like an interface to a storage database. It uses specific functions like `write_documents()` or `delete_documents()` to work with data. Various components have access to the Document Store and can interact with it by, for example, reading or writing Documents.
+
+If you are working with more complex pipelines in Haystack, you can use a [`DocumentWriter`](../pipeline-components/writers/documentwriter.mdx) component to write data into Document Stores for you
+
+### Data Classes
+
+You can use different [data classes](data-classes.mdx) in Haystack to carry the data through the system. The data classes are mostly likely to appear as inputs or outputs of your pipelines.
+
+`Document` class contains information to be carried through the pipeline. It can be text, metadata, tables, or binary data. Documents can be written into Document Stores but also written and read by other components.
+
+`Answer` class holds not only the answer generated in a pipeline but also the originating query and metadata.
+
+### Pipelines
+
+Finally, you can combine various components, Document Stores, and integrations into [pipelines](pipelines.mdx) to create powerful and customizable systems. It is a highly flexible system that allows you to have simultaneous flows, standalone components, loops, and other types of connections. You can have the preprocessing, indexing, and querying steps all in one pipeline, or you can split them up according to your needs.
+
+If you want to reuse pipelines, you can save them into a convenient format (YAML, TOML, and more) on a disk or share them around using the [serialization](pipelines/serialization.mdx) process.
+
+Here is a short Haystack pipeline, illustrated:
+<ClickableImage src="/img/00f5fe8-Pipeline_Illustrations_2.png" alt="RAG architecture overview showing query flow through retrieval and generation stages, with document stores providing context for the language model" />

docs-website/docs/concepts/data-classes.mdx

@@ -0,0 +1,315 @@
+---
+title: "Data Classes"
+id: data-classes
+slug: "/data-classes"
+description: "In Haystack, there are a handful of core classes that are regularly used in many different places. These are classes that carry data through the system and you are likely to interact with these as either the input or output of your pipeline."
+---
+
+# Data Classes
+
+In Haystack, there are a handful of core classes that are regularly used in many different places. These are classes that carry data through the system and you are likely to interact with these as either the input or output of your pipeline.
+
+Haystack uses data classes to help components communicate with each other in a simple and modular way. By doing this, data flows seamlessly through the Haystack pipelines. This page goes over the available data classes in Haystack: ByteStream, Answer (along with its variants ExtractedAnswer and GeneratedAnswer), ChatMessage, FileContent, ImageContent, Document, and StreamingChunk, explaining how they contribute to the Haystack ecosystem.
+
+You can check out the detailed parameters in our [Data Classes](/reference/data-classes-api) API reference.
+
+### Answer
+
+#### Overview
+
+The `Answer` class serves as the base for responses generated within Haystack, containing the answer's data, the originating query, and additional metadata.
+
+#### Key Features
+
+- Adaptable data handling, accommodating any data type (`data`).
+- Query tracking for contextual relevance (`query`).
+- Extensive metadata support for detailed answer description.
+
+#### Attributes
+
+```python
+@dataclass
+class Answer:
+    data: Any
+    query: str
+    meta: Dict[str, Any]
+```
+
+### ExtractedAnswer
+
+#### Overview
+
+`ExtractedAnswer` is a subclass of `Answer` that deals explicitly with answers derived from Documents, offering more detailed attributes.
+
+#### Key Features
+
+- Includes reference to the originating `Document`.
+- Score attribute to quantify the answer's confidence level.
+- Optional start and end indices for pinpointing answer location within the source.
+
+#### Attributes
+
+```python
+@dataclass
+class ExtractedAnswer:
+    query: str
+    score: float
+    data: Optional[str] = None
+    document: Optional[Document] = None
+    context: Optional[str] = None
+    document_offset: Optional["Span"] = None
+    context_offset: Optional["Span"] = None
+    meta: Dict[str, Any] = field(default_factory=dict)
+```
+
+### GeneratedAnswer
+
+#### Overview
+
+`GeneratedAnswer` extends the `Answer` class to accommodate answers generated from multiple Documents.
+
+#### Key Features
+
+- Handles string-type data.
+- Links to a list of `Document` objects, enhancing answer traceability.
+
+#### Attributes
+
+```python
+@dataclass
+class GeneratedAnswer:
+    data: str
+    query: str
+    documents: List[Document]
+    meta: Dict[str, Any] = field(default_factory=dict)
+```
+
+### ByteStream
+
+#### Overview
+
+`ByteStream` represents binary object abstraction in the Haystack framework and is crucial for handling various binary data formats.
+
+#### Key Features
+
+- Holds binary data and associated metadata.
+- Optional MIME type specification for flexibility.
+- File interaction methods (`to_file`, `from_file_path`, `from_string`) for easy data manipulation.
+
+#### Attributes
+
+```python
+@dataclass(repr=False)
+class ByteStream:
+    data: bytes
+    meta: Dict[str, Any] = field(default_factory=dict, hash=False)
+    mime_type: Optional[str] = field(default=None)
+```
+
+#### Example
+
+```python
+from haystack.dataclasses.byte_stream import ByteStream
+
+image = ByteStream.from_file_path("dog.jpg")
+```
+
+### ChatMessage
+
+`ChatMessage` is the central abstraction to represent a message for a LLM. It contains role, metadata and several types of content, including text, tool calls and tool calls results.
+
+Read the detailed documentation for the `ChatMessage` data class on a dedicated [ChatMessage](data-classes/chatmessage.mdx) page.
+
+### FileContent
+
+`FileContent` represents a file payload that can be attached to a `ChatMessage`, including base64 data, MIME type, filename, and provider-specific metadata.
+
+Read the detailed documentation for the `FileContent` data class on a dedicated [FileContent](data-classes/filecontent.mdx) page.
+
+### ImageContent
+
+`ImageContent` represents image-based content used in multimodal chat messages and vision-language pipelines.
+
+Read the detailed documentation for the `ImageContent` data class on a dedicated [ImageContent](data-classes/imagecontent.mdx) page.
+
+### Document
+
+#### Overview
+
+`Document` represents a central data abstraction in Haystack, capable of holding text, tables, and binary data.
+
+#### Key Features
+
+- Unique ID for each document.
+- Multiple content types are supported: text, binary (`blob`).
+- Custom metadata and scoring for advanced document management.
+- Optional embedding for AI-based applications.
+
+#### Attributes
+
+```python
+@dataclass
+class Document(metaclass=_BackwardCompatible):
+    id: str = field(default="")
+    content: Optional[str] = field(default=None)
+    blob: Optional[ByteStream] = field(default=None)
+    meta: Dict[str, Any] = field(default_factory=dict)
+    score: Optional[float] = field(default=None)
+    embedding: Optional[List[float]] = field(default=None)
+    sparse_embedding: Optional[SparseEmbedding] = field(default=None)
+```
+
+#### Example
+
+```python
+from haystack import Document
+
+documents = Document(
+    content="Here are the contents of your document",
+    embedding=[0.1] * 768,
+)
+```
+
+### StreamingChunk
+
+#### Overview
+
+`StreamingChunk` represents a partially streamed LLM response, enabling real-time LLM response processing. It encapsulates a segment of streamed content along with associated metadata and provides comprehensive information about the streaming state.
+
+#### Key Features
+
+- String-based content representation for text chunks
+- Support for tool calls and tool call results
+- Component tracking and metadata management
+- Streaming state indicators (start, finish reason)
+- Content block indexing for multi-part responses
+
+#### Attributes
+
+```python
+@dataclass
+class StreamingChunk:
+    content: str
+    meta: dict[str, Any] = field(default_factory=dict, hash=False)
+    component_info: Optional[ComponentInfo] = field(default=None)
+    index: Optional[int] = field(default=None)
+    tool_calls: Optional[list[ToolCallDelta]] = field(default=None)
+    tool_call_result: Optional[ToolCallResult] = field(default=None)
+    start: bool = field(default=False)
+    finish_reason: Optional[FinishReason] = field(default=None)
+    reasoning: Optional[ReasoningContent] = field(default=None)
+```
+
+#### Example
+
+```python
+from haystack.dataclasses import StreamingChunk, ToolCallDelta, ReasoningContent
+
+# Basic text chunk
+chunk = StreamingChunk(
+    content="Hello world",
+    start=True,
+    meta={"model": "gpt-5-mini"},
+)
+
+# Tool call chunk
+tool_chunk = StreamingChunk(
+    content="",
+    tool_calls=[
+        ToolCallDelta(
+            index=0,
+            tool_name="calculator",
+            arguments='{"operation": "add", "a": 2, "b": 3}',
+        ),
+    ],
+    index=0,
+    start=False,
+    finish_reason="tool_calls",
+)
+
+# Reasoning chunk
+reasoning_chunk = StreamingChunk(
+    content="",
+    reasoning=ReasoningContent(
+        reasoning_text="Thinking step by step about the answer.",
+    ),
+    index=0,
+    start=True,
+    meta={"model": "gpt-4.1-mini"},
+)
+```
+
+### ToolCallDelta
+
+#### Overview
+
+`ToolCallDelta` represents a tool call prepared by the model, usually contained in an assistant message during streaming.
+
+#### Attributes
+
+```python
+@dataclass
+class ToolCallDelta:
+    index: int
+    tool_name: Optional[str] = field(default=None)
+    arguments: Optional[str] = field(default=None)
+    id: Optional[str] = field(default=None)
+    extra: Optional[Dict[str, Any]] = field(default=None)
+```
+
+### ComponentInfo
+
+#### Overview
+
+The `ComponentInfo` class represents information about a component within a Haystack pipeline. It is used to track the type and name of components that generate or process data, aiding in debugging, tracing, and metadata management throughout the pipeline.
+
+#### Key Features
+
+- Stores the type of the component (including module and class name).
+- Optionally stores the name assigned to the component in the pipeline.
+- Provides a convenient class method to create a `ComponentInfo` instance from a `Component` object.
+
+#### Attributes
+
+```python
+@dataclass
+class ComponentInfo:
+    type: str
+    name: Optional[str] = field(default=None)
+
+    @classmethod
+    def from_component(cls, component: Component) -> "ComponentInfo": ...
+```
+
+#### Example
+
+```python
+from haystack.dataclasses.streaming_chunk import ComponentInfo
+from haystack.core.component import Component
+
+
+class MyComponent(Component): ...
+
+
+component = MyComponent()
+info = ComponentInfo.from_component(component)
+print(info.type)  # e.g., 'my_module.MyComponent'
+print(info.name)  # Name assigned in the pipeline, if any
+```
+
+### SparseEmbedding
+
+#### Overview
+
+The `SparseEmbedding` class represents a sparse embedding: a vector where most values are zeros.
+
+#### Attributes
+
+- `indices`: List of indices of non-zero elements in the embedding.
+- `values`: List of values of non-zero elements in the embedding.
+
+### Tool
+
+`Tool` is a data class representing a tool that Language Models can prepare a call for.
+
+Read the detailed documentation for the `Tool` data class on a dedicated [Tool](../tools/tool.mdx) page.

docs-website/docs/concepts/data-classes/chatmessage.mdx

@@ -0,0 +1,413 @@
+---
+title: "ChatMessage"
+id: chatmessage
+slug: "/chatmessage"
+description: "`ChatMessage` is the central abstraction to represent a message for a LLM. It contains role, metadata and several types of content, including text, images, tool calls, tool call results, and reasoning content."
+---
+
+# ChatMessage
+
+`ChatMessage` is the central abstraction to represent a message for a LLM. It contains role, metadata and several types of content, including text, images, tool calls, tool call results, and reasoning content.
+
+To create a `ChatMessage` instance, use `from_user`, `from_system`, `from_assistant`, and `from_tool` class methods.
+
+The [content](#types-of-content) of the `ChatMessage` can then be inspected using the `text`, `texts`, `image`, `images`, `file`, `files`, `tool_call`, `tool_calls`, `tool_call_result`, `tool_call_results`, `reasoning`, and `reasonings` properties.
+
+If you are looking for the details of this data class methods and parameters, head over to our [API documentation](/reference/data-classes-api#chatmessage).
+
+## Types of Content
+
+`ChatMessage` currently supports `TextContent`, `ImageContent`, `FileContent`, `ToolCall`, `ToolCallResult`, and `ReasoningContent` types of content:
+
+```python
+@dataclass
+class TextContent:
+    """
+    The textual content of a chat message.
+
+    :param text: The text content of the message.
+    """
+
+    text: str
+
+
+@dataclass
+class ToolCall:
+    """
+    Represents a Tool call prepared by the model, usually contained in an assistant message.
+
+    :param tool_name: The name of the Tool to call.
+    :param arguments: The arguments to call the Tool with.
+    :param id: The ID of the Tool call.
+    :param extra: Dictionary of extra information about the Tool call. Use to store provider-specific
+        information. To avoid serialization issues, values should be JSON serializable.
+    """
+
+    tool_name: str
+    arguments: Dict[str, Any]
+    id: Optional[str] = None  # noqa: A003
+    extra: Optional[Dict[str, Any]] = None
+
+
+@dataclass
+class ToolCallResult:
+    """
+    Represents the result of a Tool invocation.
+
+    :param result: The result of the Tool invocation.
+    :param origin: The Tool call that produced this result.
+    :param error: Whether the Tool invocation resulted in an error.
+    """
+
+    result: str | Sequence[TextContent | ImageContent]
+    origin: ToolCall
+    error: bool
+
+
+@dataclass
+class ImageContent:
+    """
+    The image content of a chat message.
+
+    :param base64_image: A base64 string representing the image.
+    :param mime_type: The MIME type of the image (e.g. "image/png", "image/jpeg").
+        Providing this value is recommended, as most LLM providers require it.
+        If not provided, the MIME type is guessed from the base64 string, which can be slow and not always reliable.
+    :param detail: Optional detail level of the image (only supported by OpenAI). One of "auto", "high", or "low".
+    :param meta: Optional metadata for the image.
+    :param validation: If True (default), a validation process is performed:
+        - Check whether the base64 string is valid;
+        - Guess the MIME type if not provided;
+        - Check if the MIME type is a valid image MIME type.
+        Set to False to skip validation and speed up initialization.
+    """
+
+    base64_image: str
+    mime_type: Optional[str] = None
+    detail: Optional[Literal["auto", "high", "low"]] = None
+    meta: Dict[str, Any] = field(default_factory=dict)
+    validation: bool = True
+
+
+@dataclass
+class FileContent:
+    """
+    The file content of a chat message.
+
+    :param base64_data: A base64 string representing the file.
+    :param mime_type: The MIME type of the file (e.g. "application/pdf").
+        Providing this value is recommended, as most LLM providers require it.
+        If not provided, the MIME type is guessed from the base64 string, which can be slow and not always reliable.
+    :param filename: Optional filename of the file. Some LLM providers use this information.
+    :param extra: Dictionary of extra information about the file. Can be used to store provider-specific information.
+        To avoid serialization issues, values should be JSON serializable.
+    :param validation: If True (default), a validation process is performed:
+        - Check whether the base64 string is valid;
+        - Guess the MIME type if not provided.
+        Set to False to skip validation and speed up initialization.
+    """
+
+    base64_data: str
+    mime_type: str | None = None
+    filename: str | None = None
+    extra: dict[str, Any] = field(default_factory=dict)
+    validation: bool = True
+
+
+@dataclass
+class ReasoningContent:
+    """
+    Represents the optional reasoning content prepared by the model, usually contained in an assistant message.
+
+    :param reasoning_text: The reasoning text produced by the model.
+    :param extra: Dictionary of extra information about the reasoning content. Use to store provider-specific
+        information. To avoid serialization issues, values should be JSON serializable.
+    """
+
+    reasoning_text: str
+    extra: Dict[str, Any] = field(default_factory=dict)
+```
+
+The `ImageContent` and `FileContent` dataclasses also provide two convenience class methods: `from_file_path` and `from_url`.
+For more details, refer to our [API documentation](/reference/data-classes-api).
+
+## Working with a ChatMessage
+
+The following examples demonstrate how to create a `ChatMessage` and inspect its properties.
+
+### from_user with TextContent
+
+```python
+from haystack.dataclasses import ChatMessage
+
+user_message = ChatMessage.from_user("What is the capital of Australia?")
+
+print(user_message)
+>>> ChatMessage(
+>>>    _role=<ChatRole.USER: 'user'>,
+>>>    _content=[TextContent(text='What is the capital of Australia?')],
+>>>    _name=None,
+>>>    _meta={}
+>>>)
+
+print(user_message.text)
+>>> What is the capital of Australia?
+
+print(user_message.texts)
+>>> ['What is the capital of Australia?']
+```
+
+### from_user with TextContent and ImageContent
+
+```python
+from haystack.dataclasses import ChatMessage, ImageContent
+
+lion_image_url = (
+    "https://images.unsplash.com/photo-1546182990-dffeafbe841d?"
+	"ixlib=rb-4.0&q=80&w=1080&fit=max"
+)
+
+image_content = ImageContent.from_url(lion_image_url, detail="low")
+
+user_message = ChatMessage.from_user(
+	content_parts=[
+		"What does the image show?",
+		image_content
+		])
+
+print(user_message)
+>>> ChatMessage(
+>>>     _role=<ChatRole.USER: 'user'>,
+>>>     _content=[
+>>>         TextContent(text='What does the image show?'),
+>>>         ImageContent(
+>>>             base64_image='/9j/4...',
+>>>             mime_type='image/jpeg',
+>>>             detail='low',
+>>>             meta={
+>>>                 'content_type': 'image/jpeg',
+>>>                 'url': '...'
+>>>             }
+>>>         )
+>>>     ],
+>>>     _name=None,
+>>>     _meta={}
+>>> )
+
+print(user_message.text)
+>>> What does the image show?
+
+print(user_message.texts)
+>>> ['What does the image show?']
+
+print(user_message.image)
+>>> ImageContent(
+>>>     base64_image='/9j/4...',
+>>>     mime_type='image/jpeg',
+>>>     detail='low',
+>>>     meta={
+>>>         'content_type': 'image/jpeg',
+>>>         'url': '...'
+>>>     }
+>>> )
+```
+
+### from_user with TextContent and FileContent
+
+```python
+from haystack.dataclasses import ChatMessage, FileContent
+
+paper_url = "https://arxiv.org/pdf/2309.08632"
+
+file_content = FileContent.from_url(paper_url)
+
+user_message = ChatMessage.from_user(
+	content_parts=[
+        file_content,
+        "Summarize this paper in 100 words."
+		])
+
+print(user_message)
+>>> ChatMessage(
+>>>     _role=<ChatRole.USER: 'user'>,
+>>>     _content=[
+>>>         FileContent(
+>>>             base64_data='JVBERi0...',
+>>>             mime_type='application/pdf',
+>>>             filename='2309.08632',
+>>>             extra={}
+>>>         ),
+>>>         TextContent(text='Summarize this paper in 100 words.')
+>>>     ],
+>>>     _name=None,
+>>>     _meta={}
+>>> )
+
+print(user_message.text)
+>>> Summarize this paper in 100 words.
+
+print(user_message.texts)
+>>> ['Summarize this paper in 100 words.']
+
+print(user_message.file)
+>>> FileContent(
+>>>     base64_data='JVBERi0...',
+>>>     mime_type='application/pdf',
+>>>     filename='2309.08632',
+>>>     extra={}
+>>> )
+```
+
+### from_assistant with TextContent
+
+```python
+from haystack.dataclasses import ChatMessage
+
+assistant_message = ChatMessage.from_assistant("How can I assist you today?")
+
+print(assistant_message)
+>>> ChatMessage(
+>>>    _role=<ChatRole.ASSISTANT: 'assistant'>,
+>>>    _content=[TextContent(text='How can I assist you today?')],
+>>>    _name=None,
+>>>    _meta={}
+>>>)
+
+print(assistant_message.text)
+>>> How can I assist you today?
+
+print(assistant_message.texts)
+>>> ['How can I assist you today?']
+```
+
+### from_assistant with ToolCall
+
+```python
+from haystack.dataclasses import ChatMessage, ToolCall
+
+tool_call = ToolCall(tool_name="weather_tool", arguments={"location": "Rome"})
+
+assistant_message_w_tool_call = ChatMessage.from_assistant(tool_calls=[tool_call])
+
+print(assistant_message_w_tool_call)
+>>> ChatMessage(
+>>>    _role=<ChatRole.ASSISTANT: 'assistant'>,
+>>>    _content=[ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None)],
+>>>    _name=None,
+>>>    _meta={}
+>>>)
+
+print(assistant_message_w_tool_call.text)
+>>> None
+
+print(assistant_message_w_tool_call.texts)
+>>> []
+
+print(assistant_message_w_tool_call.tool_call)
+>>> ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None)
+
+print(assistant_message_w_tool_call.tool_calls)
+>>> [ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None)]
+
+print(assistant_message_w_tool_call.tool_call_result)
+>>> None
+
+print(assistant_message_w_tool_call.tool_call_results)
+>>> []
+```
+
+### from_tool
+
+```python
+from haystack.dataclasses import ChatMessage
+
+tool_message = ChatMessage.from_tool(tool_result="temperature: 25°C", origin=tool_call, error=False)
+
+print(tool_message)
+>>> ChatMessage(
+>>>    _role=<ChatRole.TOOL: 'tool'>,
+>>>    _content=[ToolCallResult(
+>>>							   result='temperature: 25°C',
+>>>                origin=ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None),
+>>>                error=False
+>>>                )],
+>>>    _name=None,
+>>>    _meta={}
+>>>)
+
+print(tool_message.text)
+>>> None
+
+print(tool_message.texts)
+>>> []
+
+print(tool_message.tool_call)
+>>> None
+
+print(tool_message.tool_calls)
+>>> []
+
+print(tool_message.tool_call_result)
+>>> ToolCallResult(
+>>>     result='temperature: 25°C',
+>>>     origin=ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None),
+>>>     error=False
+>>> )
+
+print(tool_message.tool_call_results)
+>>> [
+>>>     ToolCallResult(
+>>>         result='temperature: 25°C',
+>>>         origin=ToolCall(tool_name='weather_tool', arguments={'location': 'Rome'}, id=None),
+>>>         error=False
+>>>     )
+>>> ]
+```
+
+## Migrating from Legacy ChatMessage (before v2.9)
+
+In Haystack 2.9, we updated the `ChatMessage` data class for greater flexibility and support for multiple content types: text, tool calls, and tool call results.
+
+There are some breaking changes involved, so we recommend reviewing this guide to migrate smoothly.
+
+### Creating a ChatMessage
+
+You can no longer directly initialize `ChatMessage` using `role`, `content`, and `meta`.
+
+- Use the following class methods instead: `from_assistant`, `from_user`, `from_system`, and `from_tool`.
+- Replace the `content` parameter with `text`.
+
+```python
+from haystack.dataclasses import ChatMessage
+
+# LEGACY - DOES NOT WORK IN 2.9.0
+message = ChatMessage(role=ChatRole.USER, content="Hello!")
+
+# Use the class method instead
+message = ChatMessage.from_user("Hello!")
+```
+
+### Accessing ChatMessage Attributes
+
+- The legacy `content` attribute is now internal (`_content`).
+- Inspect `ChatMessage` attributes using the following properties:
+  - `role`
+  - `meta`
+  - `name`
+  - `text` and `texts`
+  - `image` and `images`
+  - `tool_call` and `tool_calls`
+  - `tool_call_result` and `tool_call_results`
+  - `reasoning` and `reasonings`
+
+```python
+from haystack.dataclasses import ChatMessage
+
+message = ChatMessage.from_user("Hello!")
+
+# LEGACY - DOES NOT WORK IN 2.9.0
+print(message.content)
+
+# Use the appropriate property instead
+print(message.text)
+```

docs-website/docs/concepts/data-classes/filecontent.mdx

@@ -0,0 +1,119 @@
+---
+title: "FileContent"
+id: filecontent
+slug: "/filecontent"
+description: "`FileContent` represents file payloads in chat messages, including base64 data, MIME type, filename, and provider-specific metadata."
+---
+
+# FileContent
+
+`FileContent` represents a file payload that can be attached to a [`ChatMessage`](chatmessage.mdx). Use it when a chat model accepts file inputs, such as PDFs or other documents, together with the user's text prompt.
+
+If you need the full list of parameters and methods, see the [`FileContent` API reference](/reference/data-classes-api#filecontent).
+
+## Attributes
+
+```python
+@dataclass
+class FileContent:
+    base64_data: str
+    mime_type: str | None = None
+    filename: str | None = None
+    extra: dict[str, Any] = field(default_factory=dict)
+    validation: bool = True
+```
+
+- `base64_data` stores the file content as a base64-encoded string.
+- `mime_type` identifies the file type, for example `application/pdf`. Providing it explicitly is recommended because many model providers require it.
+- `filename` is optional, but some providers use it when processing uploaded files.
+- `extra` can store provider-specific metadata. Values should be JSON serializable.
+- `validation` checks that `base64_data` is valid and tries to infer the MIME type when one is not provided.
+
+## Create from a file path
+
+Use `from_file_path` to read a local file, base64-encode it, infer the MIME type from the path, and populate the filename.
+
+```python
+from haystack.dataclasses import ChatMessage, FileContent
+
+file_content = FileContent.from_file_path("data/attention-is-all-you-need.pdf")
+
+message = ChatMessage.from_user(
+    content_parts=[
+        file_content,
+        "Summarize the key ideas in this paper.",
+    ]
+)
+```
+
+Pass `filename` or `extra` when a provider expects a specific filename or provider-specific options:
+
+```python
+file_content = FileContent.from_file_path(
+    "data/report.pdf",
+    filename="quarterly-report.pdf",
+    extra={"source": "finance"},
+)
+```
+
+## Create from a URL
+
+Use `from_url` to download a file and convert it into a `FileContent` instance.
+
+```python
+from haystack.dataclasses import FileContent
+
+file_content = FileContent.from_url(
+    "https://example.com/reports/quarterly-report.pdf",
+    timeout=30,
+)
+```
+
+If no filename is provided, Haystack uses the final path segment of the URL.
+
+## Create from base64 data
+
+If you already have file bytes, encode them and pass the MIME type explicitly.
+
+```python
+import base64
+from pathlib import Path
+
+from haystack.dataclasses import FileContent
+
+data = Path("data/manual.pdf").read_bytes()
+file_content = FileContent(
+    base64_data=base64.b64encode(data).decode("utf-8"),
+    mime_type="application/pdf",
+    filename="manual.pdf",
+)
+```
+
+Set `validation=False` only when the base64 data and MIME type are already trusted and you want to skip validation.
+
+## Inspect files in a ChatMessage
+
+After adding `FileContent` to a `ChatMessage`, use the `file` and `files` properties to access file payloads.
+
+```python
+from haystack.dataclasses import ChatMessage, FileContent
+
+file_content = FileContent.from_file_path("data/invoice.pdf")
+message = ChatMessage.from_user(content_parts=[file_content, "Extract the invoice total."])
+
+print(message.file)
+print(message.files)
+```
+
+`message.file` returns the first file payload, or `None` if there are no files. `message.files` returns all file payloads.
+
+## Serialization
+
+Use `to_dict` and `from_dict` to serialize and restore file content.
+
+```python
+payload = file_content.to_dict()
+restored = FileContent.from_dict(payload)
+```
+
+For tracing, Haystack replaces the full base64 payload with a placeholder so large files are not sent to the tracing backend.

docs-website/docs/concepts/data-classes/imagecontent.mdx

@@ -0,0 +1,247 @@
+---
+title: "ImageContent"
+id: imagecontent
+slug: "/imagecontent"
+description: "`ImageContent` represents image-based content in Haystack chat messages and multimodal pipelines."
+---
+
+# ImageContent
+
+`ImageContent` is a Haystack data class used to represent image-based content in chat messages and multimodal AI pipelines.
+
+It is commonly used with:
+
+* multimodal LLMs
+* vision-language models
+* image-aware chat applications
+* document/image processing workflows
+
+`ImageContent` stores images as base64-encoded strings together with metadata such as MIME type and image detail level.
+
+If you are looking for the full API reference, see the [API documentation](/reference/data-classes-api#imagecontent).
+
+---
+
+# Creating ImageContent
+
+You can create an `ImageContent` object directly from a base64 string:
+
+```python
+from haystack.dataclasses import ImageContent
+
+image = ImageContent(base64_image="your_base64_encoded_image", mime_type="image/png")
+
+print(image)
+```
+
+---
+
+# Loading Images from a File Path
+
+The `from_file_path()` class method provides a convenient way to load local image files.
+
+```python
+from haystack.dataclasses import ImageContent
+
+image = ImageContent.from_file_path("sample.png", detail="low")
+
+print(image)
+```
+
+The optional `detail` parameter is currently supported by OpenAI vision models and accepts:
+
+* `"auto"`
+* `"high"`
+* `"low"`
+
+You can also resize images while loading:
+
+```python
+image = ImageContent.from_file_path("sample.png", size=(512, 512))
+```
+
+This helps reduce:
+
+* memory usage
+* processing time
+* payload size
+
+when working with multimodal LLM APIs.
+
+---
+
+# Loading Images from a URL
+
+You can also create an `ImageContent` object directly from an image URL:
+
+```python
+from haystack.dataclasses import ImageContent
+
+image = ImageContent.from_url(
+    "https://images.unsplash.com/photo-1546182990-dffeafbe841d",
+    detail="low",
+)
+
+print(image)
+```
+
+Internally, Haystack downloads the image and converts it into a base64 representation.
+
+---
+
+# Producing ImageContent with Converters
+
+In a pipeline, you usually don't create `ImageContent` objects by hand. Instead, you use converter components that read files and produce `ImageContent` for you:
+
+* [`ImageFileToImageContent`](../../pipeline-components/converters/imagefiletoimagecontent.mdx) converts local image files (such as PNG or JPEG) into `ImageContent` objects.
+* [`PDFToImageContent`](../../pipeline-components/converters/pdftoimagecontent.mdx) renders the pages of PDF files into `ImageContent` objects.
+
+```python
+from haystack.components.converters.image import (
+    ImageFileToImageContent,
+    PDFToImageContent,
+)
+
+image_converter = ImageFileToImageContent()
+image_contents = image_converter.run(sources=["image.jpg", "another_image.png"])[
+    "image_contents"
+]
+
+pdf_converter = PDFToImageContent()
+pdf_image_contents = pdf_converter.run(sources=["file.pdf"])["image_contents"]
+```
+
+Both converters accept the optional `detail` and `size` parameters, which are forwarded to the `ImageContent` objects they create.
+
+---
+
+# Using ImageContent with ChatMessage
+
+`ImageContent` is commonly used together with [`ChatMessage`](chatmessage.mdx) for multimodal conversations.
+
+```python
+from haystack.dataclasses import ChatMessage, ImageContent
+
+image = ImageContent.from_url(
+    "https://images.unsplash.com/photo-1546182990-dffeafbe841d",
+    detail="low",
+)
+
+message = ChatMessage.from_user(content_parts=["What does this image show?", image])
+
+print(message)
+```
+
+This allows multimodal LLMs to process both:
+
+* textual prompts
+* image inputs
+
+within the same message.
+
+For more dynamic prompts, you can build multimodal messages with [`ChatPromptBuilder`](../../pipeline-components/builders/chatpromptbuilder.mdx) using Jinja2 string templates. The `| templatize_part` filter inserts an `ImageContent` object as a structured content part instead of plain text:
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage, ImageContent
+
+template = """
+{% message role="user" %}
+Hello! I am {{user_name}}. What's the difference between the following images?
+{% for image in images %}
+{{ image | templatize_part }}
+{% endfor %}
+{% endmessage %}
+"""
+
+builder = ChatPromptBuilder(template=template)
+images = [
+    ImageContent.from_file_path("apple.jpg"),
+    ImageContent.from_file_path("kiwi.jpg"),
+]
+result = builder.run(user_name="John", images=images)
+
+print(result["prompt"])
+```
+
+---
+
+# Metadata
+
+The optional `meta` parameter allows you to attach custom metadata to the image.
+
+```python
+image = ImageContent.from_url(
+    "https://images.unsplash.com/photo-1546182990-dffeafbe841d",
+    meta={"source": "example-dataset"},
+)
+```
+
+This can be useful for:
+
+* tracing
+* dataset tracking
+* workflow metadata
+* custom application logic
+
+---
+
+# Validation
+
+By default, `ImageContent` validates:
+
+* base64 encoding
+* MIME type correctness
+* image MIME compatibility
+
+Validation can be disabled to improve performance:
+
+```python
+image = ImageContent(
+    base64_image="your_base64_encoded_image",
+    mime_type="image/png",
+    validation=False,
+)
+```
+
+---
+
+# Serialization
+
+`ImageContent` supports dictionary serialization.
+
+```python
+image_dict = image.to_dict()
+
+restored_image = ImageContent.from_dict(image_dict)
+```
+
+---
+
+# Displaying Images
+
+The `show()` method can display images directly in:
+
+* Jupyter notebooks
+* local desktop environments
+
+```python
+image.show()
+```
+
+This requires the `Pillow` package:
+
+```bash
+pip install pillow
+```
+
+---
+
+# Related Components
+
+`ImageContent` is frequently used with:
+
+* [`ChatMessage`](chatmessage.mdx) — to build multimodal messages
+* [`ChatPromptBuilder`](../../pipeline-components/builders/chatpromptbuilder.mdx) — to template multimodal prompts
+* [`ImageFileToImageContent`](../../pipeline-components/converters/imagefiletoimagecontent.mdx) — to convert image files into `ImageContent`
+* [`PDFToImageContent`](../../pipeline-components/converters/pdftoimagecontent.mdx) — to convert PDF pages into `ImageContent`

docs-website/docs/concepts/device-management.mdx

@@ -0,0 +1,145 @@
+---
+title: "Device Management"
+id: device-management
+slug: "/device-management"
+description: "This page discusses the concept of device management in the context of Haystack."
+---
+
+# Device Management
+
+This page discusses the concept of device management in the context of Haystack.
+
+Many Haystack components, such as `TransformersChatGenerator`, `AzureOpenAIChatGenerator`, and others, allow users the ability to pick and choose which language model is to be queried and executed. For components that interface with cloud-based services, the service provider automatically takes care of the details of provisioning the requisite hardware (like GPUs). However, if you wish to use models on your local machine, you’ll need to figure out how to deploy them on your hardware. Further complicating things, different ML libraries have different APIs to launch models on specific devices.
+
+To make the process of running inference on local models as straightforward as possible, Haystack uses a framework-agnostic device management implementation. Exposing devices through this interface means you no longer need to worry about library-specific invocations and device representations.
+
+## Concepts
+
+Haystack’s device management is built on the following abstractions:
+
+- `DeviceType`  - An enumeration that lists all the different types of supported devices.
+- `Device`  - A generic representation of a device composed of a `DeviceType` and a unique identifier. Together, it represents a single device in the group of all available devices.
+- `DeviceMap` - A mapping of strings to `Device` instances. The strings represent model-specific identifiers, usually model parameters. This allows us to map specific parts of a model to specific devices.
+- `ComponentDevice` - A tagged union of a single `Device` or a `DeviceMap` instance. Components that support local inference will expose an optional `device` parameter of this type in their constructor.
+
+With the above abstractions, Haystack can fully address any supported device that’s part of your local machine and can support the usage of multiple devices at the same time. Every component that supports local inference will internally handle the conversion of these generic representations to their backend-specific representations.
+
+:::info[Source Code]
+
+Find the full code for the abstractions above in the Haystack GitHub [repo](https://github.com/deepset-ai/haystack/blob/6a776e672fb69cc4ee42df9039066200f1baf24e/haystack/utils/device.py).
+:::
+
+## Usage
+
+:::info
+The examples below use the [`TransformersChatGenerator`](../pipeline-components/generators/transformerschatgenerator.mdx), which is part of the `transformers-haystack` integration. Install it with:
+
+```bash
+pip install transformers-haystack
+```
+:::
+
+To use a single device for inference, use either the `ComponentDevice.from_single` or `ComponentDevice.from_str` class method:
+
+```python
+from haystack.utils import ComponentDevice, Device
+from haystack_integrations.components.generators.transformers import (
+    TransformersChatGenerator,
+)
+
+device = ComponentDevice.from_single(Device.gpu(id=1))
+# Alternatively, use a PyTorch device string
+device = ComponentDevice.from_str("cuda:1")
+generator = TransformersChatGenerator(model="Qwen/Qwen3-0.6B", device=device)
+```
+
+To use multiple devices, use the `ComponentDevice.from_multiple` class method:
+
+```python
+from haystack.utils import ComponentDevice, Device, DeviceMap
+from haystack_integrations.components.generators.transformers import (
+    TransformersChatGenerator,
+)
+
+device_map = DeviceMap(
+    {
+        "encoder.layer1": Device.gpu(id=0),
+        "decoder.layer2": Device.gpu(id=1),
+        "self_attention": Device.disk(),
+        "lm_head": Device.cpu(),
+    },
+)
+device = ComponentDevice.from_multiple(device_map)
+generator = TransformersChatGenerator(model="Qwen/Qwen3-0.6B", device=device)
+```
+
+### Integrating Devices in Custom Components
+
+Components should expose an optional `device` parameter of type `ComponentDevice`.  Once exposed, they can determine what to do with it:
+
+- If `device=None`, the component can pass that to the backend. In this case, the backend decides which device the model will be placed on.
+- Alternatively, the component can attempt to automatically pick an available device before passing it to the backend using the `ComponentDevice.resolve_device` class method.
+
+Once the device has been resolved, the component can use the `ComponentDevice.to_*` methods to get the backend-specific representation of the underlying device, which is then passed to the backend.
+
+The `ComponentDevice` instance should be serialized in the component’s `to_dict` and `from_dict` methods.
+
+```python
+from haystack.utils import ComponentDevice, Device, DeviceMap
+
+class MyComponent(Component):
+    def __init__(self, device: Optional[ComponentDevice] = None):
+        # If device is None, automatically select a device.
+        self.device = ComponentDevice.resolve_device(device)
+
+    def warm_up(self):
+        # Call the framework-specific conversion method.
+        self.model = AutoModel.from_pretrained(
+            "deepset/bert-base-cased-squad2", device=self.device.to_hf()
+            )
+
+    def to_dict(self):
+        # Serialize the policy like any other (custom) data.
+        return default_to_dict(
+            self, device=self.device.to_dict() if self.device else None, ...
+        )
+
+    @classmethod
+    def from_dict(cls, data):
+        # Deserialize the device data inplace before passing
+        # it to the generic from_dict function.
+	    init_params = data["init_parameters"]
+        init_params["device"] = ComponentDevice.from_dict(init_params["device"])
+        return default_from_dict(cls, data)
+
+# Automatically selects a device.
+c = MyComponent(device=None)
+
+# Uses the first GPU available.
+c = MyComponent(device=ComponentDevice.from_str("cuda:0"))
+
+# Uses the CPU.
+c = MyComponent(device=ComponentDevice.from_single(Device.cpu()))
+
+# Allow the component to use multiple devices using a device map.
+c = MyComponent(device=ComponentDevice.from_multiple(DeviceMap({
+    "layer1": Device.cpu(),
+    "layer2": Device.gpu(1),
+    "layer3": Device.disk()
+})))
+```
+
+If the component’s backend provides a more specialized API to manage devices, it could add an additional init parameter that acts as a conduit. For instance, `TransformersChatGenerator` exposes a `huggingface_pipeline_kwargs` parameter through which Hugging Face-specific `device_map`  arguments can be passed:
+
+```python
+from haystack_integrations.components.generators.transformers import (
+    TransformersChatGenerator,
+)
+
+generator = TransformersChatGenerator(
+    model="Qwen/Qwen3-0.6B",
+    huggingface_pipeline_kwargs={"device_map": "balanced"},
+)
+```
+
+In such cases, ensure that the parameter precedence and selection behavior is clearly documented. In the case of `TransformersChatGenerator`, the device map passed through the `huggingface_pipeline_kwargs` parameter overrides the explicit `device` parameter and is documented as such.

docs-website/docs/concepts/document-store.mdx

@@ -0,0 +1,103 @@
+---
+title: "Document Store"
+id: document-store
+slug: "/document-store"
+description: "You can think of the Document Store as a database that stores your data and provides them to the Retriever at query time. Learn how to use Document Store in a pipeline or how to create your own."
+---
+
+# Document Store
+
+You can think of the Document Store as a database that stores your data and provides them to the Retriever at query time. Learn how to use Document Store in a pipeline or how to create your own.
+
+Document Store is an object that stores your documents. In Haystack, a Document Store is different from a component, as it doesn't have the `run()` method. You can think of it as an interface to your database – you put the information there, or you can look through it. This means that a Document Store is not a piece of a pipeline but rather a tool that the components of a pipeline have access to and can interact with.
+
+:::tip[Work with Retrievers]
+
+The most common way to use a Document Store in Haystack is to fetch documents using a Retriever. A Document Store will often have a corresponding Retriever to get the most out of specific technologies. See more information in our [Retriever](../pipeline-components/retrievers.mdx) documentation.
+:::
+
+:::note[How to choose a Document Store?]
+
+To learn about different types of Document Stores and their strengths and disadvantages, head to the [Choosing a Document Store](document-store/choosing-a-document-store.mdx) page.
+:::
+
+### DocumentStore Protocol
+
+Document Stores in Haystack are designed to use the following methods as part of their protocol:
+
+- `count_documents` returns the number of documents stored in the given store as an integer.
+- `filter_documents` returns a list of documents that match the provided filters.
+- `write_documents` writes or overwrites documents into the given store and returns the number of documents that were written as an integer.
+- `delete_documents` deletes all documents with given `document_ids` from the Document Store.
+
+### Initialization
+
+To use a Document Store in a pipeline, you must initialize it first.
+
+See the installation and initialization details for each Document Store in the "Document Stores" section in the navigation panel on your left.
+
+### Work with Documents
+
+Convert your data into `Document` objects before writing them into a Document Store along with its metadata and document ID.
+
+The ID field is mandatory, so if you don’t choose a specific ID yourself, Haystack will do its best to come up with a unique ID based on the document’s information and assign it automatically. However, since Haystack uses the document’s contents to create an ID, two identical documents might have identical IDs. Keep it in mind as you update your documents, as the ID will not be updated automatically.
+
+```python
+document_store = ChromaDocumentStore()
+documents = [
+    Document(
+        meta={"name": DOCUMENT_NAME, ...}, id="document_unique_id", content="this is content"
+    ),
+  	...
+]
+document_store.write_documents(documents)
+```
+
+To write documents into the `InMemoryDocumentStore`, simply call the `.write_documents()` function:
+
+```python
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+```
+
+:::note[`DocumentWriter`]
+
+See `DocumentWriter` component [docs](../pipeline-components/writers/documentwriter.mdx) to write your documents into a Document Store in a pipeline.
+:::
+
+### DuplicatePolicy
+
+The `DuplicatePolicy` is a class that defines the different options for handling documents with the same ID in a `DocumentStore`. It has three possible values:
+
+- **OVERWRITE**: Indicates that if a document with the same ID already exists in the `DocumentStore`, it should be overwritten with the new document.
+- **SKIP**: If a document with the same ID already exists, the new document will be skipped and not added to the `DocumentStore`.
+- **FAIL**: Raises an error if a document with the same ID already exists in the `DocumentStore`. It prevents duplicate documents from being added.
+
+Here is an example of how you could apply the policy to skip the existing document:
+
+```python
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.writers import DocumentWriter
+from haystack.document_stores.types import DuplicatePolicy
+
+document_store = InMemoryDocumentStore()
+document_writer = DocumentWriter(
+    document_store=document_store,
+    policy=DuplicatePolicy.SKIP,
+)
+```
+
+### Custom Document Store
+
+All custom document stores must implement the [protocol](https://github.com/deepset-ai/haystack/blob/13804293b1bb79743e5a30e980b76a0561dcfaf8/haystack/document_stores/types/protocol.py) with four mandatory methods: `count_documents`,`filter_documents`, `write_documents`, and `delete_documents`.
+
+The `init` function should indicate all the specifics for the chosen database or vector store.
+
+We also recommend having a custom corresponding Retriever to get the most out of a specific Document Store.
+
+See [Creating Custom Document Stores](document-store/creating-custom-document-stores.mdx) page for more details.

docs-website/docs/concepts/document-store/choosing-a-document-store.mdx

@@ -0,0 +1,183 @@
+---
+title: "Choosing a Document Store"
+id: choosing-a-document-store
+slug: "/choosing-a-document-store"
+description: "This article goes through different types of Document Stores and explains their advantages and disadvantages."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Choosing a Document Store
+
+Whether you are developing a chatbot, a RAG system, or an image captioner, at some point, it's likely for your AI
+application to compare the input it gets with the information it already knows.
+
+Haystack currently has integrations with seven categories of Document Stores:
+
+- **Vector Databases** — purpose-built for embedding search and semantic retrieval
+- **Search Engines** — full-text search engines extended with vector (kNN) capabilities
+- **Relational Databases** — SQL databases with vector search via plugins or extensions
+- **Document / NoSQL Databases** — flexible document stores with vector search added on top
+- **In-memory Key-Value Stores** — ultra-low-latency stores with HNSW vector search
+- **Vector Index Libraries** — lightweight in-process vector similarity search, no external service
+- **Multi-model Databases** — single engine supporting graph, document, and vector data models
+
+Here is an overview of all the integrations currently available, grouped by category:
+
+<ClickableImage src="/img/document-stores-overview.svg" alt="Overview of DocumentStore integrations in Haystack grouped by category: Vector Databases, Search Engines, Relational Databases, Document/NoSQL Databases, In-memory Key-Value Stores, Vector Index Libraries, Multi-model Databases" className="img-light-bg img-full-width" />
+
+## DocumentStore Integrations Available in Haystack
+
+Haystack integrations come in two tiers. **Core integrations** are built and maintained by the Haystack team — they are tested against every release, follow the same API conventions, and come with full documentation and support. **External integrations** are contributed and maintained by the community; they extend Haystack's reach but are not covered by the core release cycle.
+
+The tables below list every available integration alongside the key properties you need to choose the right one for your use case.
+
+#### Core integrations
+
+| Integration | Category | Engine Type | Open Source | Async Support | Retrievers |
+| --- | --- | --- | --- | --- | --- |
+| ArcadeDB | Multi-model Database | Multi-model database (graph, document, key-value) with HNSW vector search via HTTP/JSON API | Yes | No | Embedding |
+| AlloyDB | Relational Database | Managed PostgreSQL-compatible database (Google Cloud) with pgvector extension | No | Yes | Embedding, Keyword |
+| ArangoDB | Multi-model Database | Multi-model database (graph, document, key-value) with AQL vector search (requires v3.12+) | Yes (BUSL) | No | Embedding |
+| Astra | Document / NoSQL Database | Cloud-native managed NoSQL (Apache Cassandra-based) with vector search via DataStax JSON API | No | No | Embedding |
+| Azure AI Search | Search Engine | Managed cloud search service (Microsoft Azure AI Search) with HNSW vector search | No | No | BM25, Embedding, Hybrid |
+| Chroma | Vector Database | Purpose-built vector database | Yes | Yes | Embedding |
+| Elasticsearch | Search Engine | Distributed search & analytics engine with BM25 + vector (kNN) search | Partial | Yes | BM25, Embedding, SQL |
+| FAISS | Vector Index Library | In-memory vector similarity search library (Meta/Facebook) with JSON file for metadata | Yes | No | Embedding |
+| FalkorDB | Graph Database | OpenCypher graph database with ANN vector search | Yes (SSPL) | No | Embedding, Cypher |
+| MongoDB Atlas | Document / NoSQL Database | Cloud document database with Atlas Vector Search and full-text search | No | Yes | Embedding, Full-text |
+| Oracle | Relational Database | Oracle with native AI Vector Search, HNSW vector index and DBMS_SEARCH full-text keyword index | No | Yes | Embedding, Keyword |
+| OpenSearch | Search Engine | Distributed search engine (AWS fork of Elasticsearch) with BM25 + kNN vector search | Yes | Yes | BM25, Embedding, Hybrid, Metadata, SQL |
+| PGVector | Relational Database | Relational database (PostgreSQL) with the `pgvector` extension for vector similarity search | Yes | Yes | Embedding, Keyword |
+| Pinecone | Vector Database | Managed cloud vector database | No | Yes | Embedding |
+| Qdrant | Vector Database | Purpose-built vector database with dense + sparse embedding support | Yes | Yes | Embedding, Sparse Embedding, Hybrid |
+| Supabase | Relational Database | Managed cloud Supabase — a wrapper over PgvectorDocumentStore with Supabase-specific defaults | Yes | Yes | Embedding, Keyword |
+| Valkey | In-memory Key-Value Store | In-memory key-value store (Redis fork) with HNSW vector search via `glide` client | Yes | Yes | Embedding |
+| Vespa | Search Engine | Distributed search & serving engine with BM25 lexical + HNSW vector (ANN) search | Yes | No | BM25, Embedding |
+| Weaviate | Vector Database | Purpose-built vector database with hybrid search support | Yes | Yes | BM25, Embedding, Hybrid |
+
+#### External integrations
+
+| Integration | Category | Engine Type | Open Source | Async Support | Retrievers |
+| --- | --- | --- | --- | --- | --- |
+| Couchbase | Document / NoSQL Database | Distributed NoSQL document database with vector search via Search Service | Partial | Yes | Embedding, Full-text |
+| LanceDB | Vector Database | Embedded vector database built on the Lance columnar format, optimized for multimodal data | Yes | Yes | Embedding, Full-text, Hybrid |
+| Milvus | Vector Database | Open-source vector database built for scalable similarity search | Yes | No | Embedding |
+| Needle | Search Engine | Managed RAG-as-a-service platform with built-in document storage and vector search | No | Yes | Embedding, Sparse Embedding, Hybrid |
+| Neo4j | Multi-model Database | Graph database with native vector index support for combined graph traversal and similarity search | Partial | No | Embedding |
+| SingleStore | Relational Database | Distributed SQL database with native vector search and full-text search support | No | Yes | Embedding, Full-text, Keyword |
+
+## Vector Databases
+
+- Purpose-built for vector and embedding search
+- Advanced indexing techniques for efficient similarity search
+- Designed for high scalability and availability with large volumes of high-dimensional data
+- Most support metadata filtering alongside vector search
+- Increasingly adding hybrid (vector + keyword) search support
+- Mostly open source, widely available as managed cloud services
+
+**Best for** semantic search over large document corpora — e.g. a knowledge base where users search by meaning rather than exact keywords.
+
+- [Chroma](../../document-stores/chromadocumentstore.mdx)
+- [Pinecone](../../document-stores/pinecone-document-store.mdx)
+- [Qdrant](../../document-stores/qdrant-document-store.mdx)
+- [Weaviate](../../document-stores/weaviatedocumentstore.mdx)
+- [LanceDB](https://haystack.deepset.ai/integrations/lancedb) (external integration)
+- [Milvus](https://haystack.deepset.ai/integrations/milvus-document-store) (external integration)
+
+## Search Engines
+
+- Originally built for full-text (BM25) search, with vector (kNN) capabilities added later
+- Excellent support for text data, tokenisation, and language-aware querying
+- Scale both horizontally and vertically in production environments
+- Strong foundation for hybrid search combining keyword and semantic retrieval
+- Battle-tested in enterprise environments with mature tooling and observability
+
+**Best for** enterprise search or log analytics where both full-text (BM25) and vector search are needed — e.g. an e-commerce product search with filters.
+
+- Azure AI Search ([AzureAISearchDocumentStore](../../document-stores/azureaisearchdocumentstore.mdx))
+- [Elasticsearch](../../document-stores/elasticsearch-document-store.mdx)
+- [OpenSearch](../../document-stores/opensearch-document-store.mdx)
+- [Needle](https://haystack.deepset.ai/integrations/needle) (external integration)
+- [Vespa](../../document-stores/vespadocumentstore.mdx)
+
+## Relational Databases
+
+- Standard SQL databases extended with vector search via plugins or extensions
+- Vectors live alongside relational data, enabling combined vector + SQL queries in a single store
+- Lower operational overhead when PostgreSQL is already part of the stack
+- Vector search performance is lower than purpose-built databases, but sufficient for many use cases
+- Familiar tooling, transactions, and data integrity guarantees of a relational database
+
+**Best for** use cases where documents live alongside structured relational data — e.g. a product catalogue where vector search and SQL JOINs are both needed.
+
+- [AlloyDB](../../document-stores/alloydbdocumentstore.mdx)
+- [Oracle](../../document-stores/oracledocumentstore.mdx)
+- [PGVector](../../document-stores/pgvectordocumentstore.mdx)
+- [Supabase](../../document-stores/supabasedocumentstore.mdx)
+- [SingleStore](https://haystack.deepset.ai/integrations/singlestore) (external integration)
+
+## Document / NoSQL Databases
+
+- General-purpose document stores with vector search added on top
+- Flexible, schema-less data model suited for heterogeneous document collections
+- Horizontal scaling and high availability inherited from the underlying NoSQL engine
+- Good choice when the database is already in use and adding a separate vector store is undesirable
+- Vector search performance may trail behind purpose-built databases
+
+**Best for** applications already that want to add RAG capabilities without introducing a new infrastructure component.
+
+- Astra ([AstraDocumentStore](../../document-stores/astradocumentstore.mdx))
+- [MongoDB Atlas](../../document-stores/mongodbatlasdocumentstore.mdx)
+- [Couchbase](https://haystack.deepset.ai/integrations/couchbase-document-store) (external integration)
+
+## In-memory Key-Value Stores
+
+- In-memory architecture delivers extremely low read/write latency
+- Vector search (HNSW) layered on top of an existing caching infrastructure
+- Ideal when the stack already includes Valkey as a cache or session store
+- Data is ephemeral by default; persistence requires explicit configuration
+- Less suited for large corpora where memory cost becomes significant
+
+**Best for** low-latency, real-time retrieval — e.g. a chatbot that needs sub-millisecond response times.
+
+- [Valkey](../../document-stores/valkeydocumentstore.mdx)
+
+## Vector Index Libraries
+
+- Low-level, in-process vector similarity search — not a full database
+- No network overhead; runs entirely within the application process
+- Very efficient use of hardware resources (CPU/GPU)
+- Limited to vectors only; metadata must be managed separately (e.g. via a JSON file)
+- No built-in persistence, replication, or multi-client access
+
+**Best for** local prototyping, research, or small-scale applications where a lightweight in-process solution is preferred over running an external database server.
+
+- [FAISS](../../document-stores/faissdocumentstore.mdx)
+
+## Multi-model Databases
+
+- Single engine supporting multiple data models: graph, document, key-value, and vector
+- Eliminates the need to maintain separate databases for different data representations
+- Suited for knowledge graphs or applications with complex entity relationships
+- Vector search (HNSW) available alongside graph traversal and document queries
+- Smaller community and ecosystem compared to more established categories
+
+**Best for** applications requiring multiple data models in a single engine — e.g. a knowledge graph where entities are connected by relationships and also need vector similarity search.
+
+- [ArangoDB](../../document-stores/arangodocumentstore.mdx)
+- [ArcadeDB](../../document-stores/arcadedbdocumentstore.mdx)
+- [FalkorDB](../../document-stores/falkordbdocumentstore.mdx)
+- [Neo4j](https://haystack.deepset.ai/integrations/neo4j-document-store) (external integration)
+
+## The In-memory Document Store
+
+Haystack ships with an ephemeral document store that relies on pure Python data structures stored in memory, so it doesn't fall into any of the vector database categories above. This special Document Store is ideal for creating quick prototypes with small datasets. It doesn't require any special setup, and it can be used right away without installing additional dependencies.
+
+- [InMemoryDocumentStore](../../document-stores/inmemorydocumentstore.mdx)
+
+## Final Considerations
+
+It can be very challenging to pick one Document Store over another by only looking at pure performance, as even the slightest difference in the benchmark can produce a different leaderboard (for example, some benchmarks test the cloud services while others work on a reference machine). Thinking about including features like filtering or not can bring in a whole new set of complexities that make the comparison even harder.
+
+What's important for you to know is that the Document Store interface doesn't add much to the costs, and the relative performance of one vector database over another should stay the same when used within Haystack pipelines.

docs-website/docs/concepts/document-store/creating-custom-document-stores.mdx

@@ -0,0 +1,174 @@
+---
+title: "Creating Custom Document Stores"
+id: creating-custom-document-stores
+slug: "/creating-custom-document-stores"
+description: "Create your own Document Stores to manage your documents."
+---
+
+# Creating Custom Document Stores
+
+Create your own Document Stores to manage your documents.
+
+Custom Document Stores are resources that you can build and leverage in situations where a ready-made solution is not available in Haystack. For example:
+
+- You’re working with a vector store that’s not yet supported in Haystack.
+- You need a very specific retrieval strategy to search for your documents.
+- You want to customize the way Haystack reads and writes documents.
+
+Similar to [custom components](../components/custom-components.mdx), you can use a custom Document Store in a Haystack pipeline as long as you can import its code into your Python program. The best practice is distributing a custom Document Store as a standalone integration package.
+
+## Recommendations
+
+Before you start, there are a few recommendations we provide to ensure a custom Document Store behaves consistently with the rest of the Haystack ecosystem. At the end of the day, a Document Store is just Python code written in a way that Haystack can understand, but the way you name it, organize it, and distribute it can make a difference. None of these recommendations are mandatory, but we encourage you to follow as many as you can.
+
+### Naming Convention
+
+We recommend naming your Document Store following the format `<TECHNOLOGY>-haystack`, for example, `chroma-haystack`. This makes it consistent with the others, lowering the cognitive load for your users and easing discoverability.
+
+This naming convention applies to the name of the git repository (`https://github.com/your-org/example-haystack`) and the name of the Python package (`example-haystack`).
+
+### Structure
+
+More often than not, a Document Store can be fairly complex, and setting up a dedicated Git repository can be handy and future-proof. To ease this step, we prepared a [GitHub template](https://github.com/deepset-ai/custom-component) that provides the structure you need to host a custom Document Store in a dedicated repository. It includes the boilerplate for packaging, testing, and distributing your custom Document Store as a standalone Python package.
+
+See the instructions in the [template repository](https://github.com/deepset-ai/custom-component) to get started. You can also watch the [video walkthrough](https://www.youtube.com/watch?v=SWC0QecAMcI) for a step-by-step guide.
+
+### Packaging
+
+As with any other [Haystack integration](../integrations.mdx), a Document Store can be added to your Haystack applications by installing an additional Python package, for example, with `pip`. Once you have a Git repository hosting your Document Store and a `pyproject.toml` file to create an `example-haystack` package (using our [GitHub template](https://github.com/deepset-ai/custom-component)), it will be possible to `pip install` it directly from sources, for example:
+
+```shell
+pip install git+https://github.com/your-org/example-haystack.git
+```
+
+Though very practical to quickly deliver prototypes, if you want others to use your custom Document Store, we recommend you publish a package on PyPI so that it will be versioned and installable with simply:
+
+```shell
+pip install example-haystack
+```
+
+:::tip
+👍
+
+Our [GitHub template](https://github.com/deepset-ai/custom-component) ships a GitHub workflow that will automatically publish the Document Store package on PyPI.
+:::
+
+### Documentation
+
+We recommend thoroughly documenting your custom Document Store with a detailed README file and possibly generating API documentation using a static generator.
+
+For inspiration, see the [neo4j-haystack](https://github.com/prosto/neo4j-haystack) repository and its [documentation](https://prosto.github.io/neo4j-haystack/) pages.
+
+## Implementation
+
+### DocumentStore Protocol
+
+You can use any Python class as a Document Store, provided that it implements all the methods of the `DocumentStore` Python protocol defined in Haystack:
+
+```python
+class DocumentStore(Protocol):
+    def to_dict(self) -> Dict[str, Any]:
+        """
+        Serializes this store to a dictionary.
+        """
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "DocumentStore":
+        """
+        Deserializes the store from a dictionary.
+        """
+
+    def count_documents(self) -> int:
+        """
+        Returns the number of documents stored.
+        """
+
+    def filter_documents(
+        self,
+        filters: Optional[Dict[str, Any]] = None,
+    ) -> List[Document]:
+        """
+        Returns the documents that match the filters provided.
+        """
+
+    def write_documents(
+        self,
+        documents: List[Document],
+        policy: DuplicatePolicy = DuplicatePolicy.FAIL,
+    ) -> int:
+        """
+        Writes (or overwrites) documents into the DocumentStore, return the number of documents that was written.
+        """
+
+    def delete_documents(self, document_ids: List[str]) -> None:
+        """
+        Deletes all documents with a matching document_ids from the DocumentStore.
+        """
+```
+
+The `DocumentStore` interface supports the basic CRUD operations you would normally perform on a database or a storage system, and mostly generic components like [`DocumentWriter`](../../pipeline-components/writers/documentwriter.mdx) use it.
+
+### Additional Methods
+
+Usually, a Document Store comes with additional methods that can provide advanced search functionalities. These methods are not part of the `DocumentStore` protocol and don’t follow any particular convention. We designed it like this to provide maximum flexibility to the Document Store when using any specific features of the underlying database.
+
+Some additional methods that are not part of the `DocumentStore` protocol, but are implemented by most Document Stores in Haystack, include:
+
+```python
+def delete_all_documents(recreate_index: bool = False)
+def update_by_filter(filters: dict[str, Any], meta: dict[str, Any], refresh: bool = False) -> int:
+def delete_by_filter(filters: dict[str, Any]) -> int:
+```
+These methods are not part of the Protocol but highly recommended to implement in your custom Document Store, as users often expect them to be available.
+
+For example, Haystack wouldn’t get in the way when your Document Store defines a specific `search` method that takes a long list of parameters that only make sense in the context of a particular vector database. Normally, a [Retriever](../../pipeline-components/retrievers.mdx) component would then use this additional search method.
+
+### Retrievers
+
+To get the most out of your custom Document Store, in most cases, you would need to create one or more accompanying Retrievers that use the additional search methods mentioned above. Before proceeding and implementing your custom Retriever, it might be helpful to learn more about  [Retrievers](../../pipeline-components/retrievers.mdx) in general through the Haystack documentation.
+
+From the implementation perspective, Retrievers in Haystack are like any other custom component. For more details, refer to the [creating custom components](../components/custom-components.mdx) documentation page.
+
+Although not mandatory, we encourage you to follow more specific [naming conventions](../../pipeline-components/retrievers.mdx#naming-conventions) for your custom Retriever.
+
+### Serialization
+
+Haystack requires every component to be representable by a Python dictionary for correct serialization implementation. Some components, such as Retrievers and Writers, maintain a reference to a Document Store instance. Therefore, `DocumentStore` classes should implement the `from_dict` and `to_dict` methods. This allows to rebuild an instance after reading a pipeline from a file.
+
+For a practical example of what to serialize in a custom Document Store, consider a database client you created using an IP address and a database name. When constructing the dictionary to return in `to_dict`, you would store the IP address and the database name, not the database client instance.
+
+### Secrets Management
+
+There's a likelihood that users will need to provide sensitive data, such as passwords, API keys, or private URLs, to create a Document Store instance. This sensitive data could potentially be leaked if it's passed around in plain text.
+
+Haystack has a specific way to wrap sensitive data into special objects called Secrets. This prevents the data from being leaked during serialization roundtrips. We strongly recommend using this feature extensively for data security (better safe than sorry!).
+
+You can read more about Secret Management in Haystack [documentation](../secret-management.mdx).
+
+### Testing
+
+Haystack comes with some testing functionalities you can use in a custom Document Store. In particular, an empty class inheriting from `DocumentStoreBaseTests` would already run the standard tests that any Document Store is expected to pass in order to work properly.
+
+### Implementation Tips
+
+- The best way to learn how to write a custom Document Store is to look at the existing ones: the `InMemoryDocumentStore`, which is part of Haystack, or the [`ElasticsearchDocumentStore`](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch), which is a Core Integration, are good places to start.
+- When starting from scratch, it might be easier to create the four CRUD methods of the `DocumentStore` protocol one at a time and test them one at a time as well. For example:
+  1. Implement the logic for `count_documents`.
+  2. In your `test_document_store.py` module, define the test class `TestDocumentStore(CountDocumentsTest)`. Note how we only inherit from the specific testing mix-in `CountDocumentsTest`.
+  3. Make the tests pass.
+  4. Implement the logic for `write_documents`.
+  5. Change `test_document_store.py` so that your class now also derives from the `WriteDocumentsTest` mix-in: `TestDocumentStore(CountDocumentsTest, WriteDocumentsTest)`.
+  6. Keep iterating with the remaining methods.
+- Having a notebook where users can try out your Document Store in a full pipeline can really help adoption, and it’s a great source of documentation. Our [haystack-cookbook](https://github.com/deepset-ai/haystack-cookbook) repository has good visibility, and we encourage contributors to create a PR and add their own.
+
+Verifying that the implementation meets all `DocumentStoreBaseTests` [tests](https://github.com/deepset-ai/haystack/blob/main/haystack/testing/document_store.py) is the minimum requirement for a custom Document Store to be consistent with the rest of the Haystack ecosystem.
+
+But, ideally making it compatible with the ``DocumentStoreBaseExtendedTests`` tests is a good way to ensure that your Document Store meets all the common used functionalities that users expect from a Document Store, such as `delete_all_documents` or `update_by_filter`.
+
+If the technology you are using for your Document Store supports asynchronous operations, we recommend implementing `async` versions of the methods in the `DocumentStore` protocol as well. This allows users to take advantage of async features in their applications and pipelines, improving performance and scalability.
+
+## Get Featured on the Integrations Page
+
+The [Integrations web page](https://haystack.deepset.ai/integrations) makes Haystack integrations visible to the community, and it’s a great opportunity to showcase your work. Once your Document Store is usable and properly packaged, you can open a pull request in the [haystack-integrations](https://github.com/deepset-ai/haystack-integrations) GitHub repository to add an integration tile.
+
+See the [integrations documentation page](../integrations.mdx#how-do-i-showcase-my-integration) for more details.

docs-website/docs/concepts/experimental-package.mdx

@@ -0,0 +1,67 @@
+---
+title: "Experimental Package"
+id: experimental-package
+slug: "/experimental-package"
+description: "Try out new experimental features with Haystack."
+---
+
+# Experimental Package
+
+Try out new experimental features with Haystack.
+
+The `haystack-experimental` package allows you to test new experimental features without committing to their official release. Its main goal is to gather user feedback and iterate on new features quickly.
+
+Check out the `haystack-experimental` [GitHub repository](https://github.com/deepset-ai/haystack-experimental) for the latest catalog of available features, or take a look at our [Experiments API Reference](/reference).
+
+### Installation
+
+For simplicity, every release of `haystack-experimental` includes all the available experiments at that time. To install the latest features, run:
+
+```shell
+pip install -U haystack-experimental
+```
+
+:::info
+The latest version of the experimental package is only tested against the latest version of Haystack. Compatibility with older versions of Haystack is not guaranteed.
+:::
+
+### Lifecycle
+
+Each experimental feature has a default lifespan of 3 months starting from the date of the first non-pre-release build that includes it. Once it reaches the end of its lifespan, we will remove it from `haystack-experimental` and either:
+
+- Merge the feature into Haystack and publish it with the next minor release,
+- Release the feature as an integration, or
+- Drop the feature.
+
+### Usage
+
+You can import the experimental new features like any other Haystack integration package:
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_experimental.components.generators import FoobarGenerator
+
+c = FoobarGenerator()
+c.run([ChatMessage.from_user("What's an experiment? Be brief.")])
+```
+
+Experiments can also override existing Haystack features. For example, you can opt into an experimental type of `Pipeline` by changing the usual import:
+
+```python
+# from haystack import Pipeline
+from haystack_experimental import Pipeline
+
+pipe = Pipeline()
+# ...
+pipe.run(...)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbooks:
+
+- [Improving Retrieval with Auto-Merging and Hierarchical Document Retrieval](https://haystack.deepset.ai/cookbook/auto_merging_retriever)
+- [Invoking APIs with OpenAPITool](https://haystack.deepset.ai/cookbook/openapitool)
+- [Conversational RAG using Memory](https://haystack.deepset.ai/cookbook/conversational_rag_using_memory)
+- [Define & Run Tools](https://haystack.deepset.ai/cookbook/tools_support)
+- [Newsletter Sending Agent with Experimental Haystack Tools](https://haystack.deepset.ai/cookbook/newsletter-agent)

docs-website/docs/concepts/integrations.mdx

@@ -0,0 +1,60 @@
+---
+title: "Introduction to Integrations"
+id: integrations
+slug: "/integrations"
+description: "The Haystack ecosystem integrates with many other technologies, such as vector databases, model providers and even custom components made by the community.  Here you can explore our integrations, which may be maintined by deepset, or submitted by others."
+---
+
+# Introduction to Integrations
+
+The Haystack ecosystem integrates with many other technologies, such as vector databases, model providers and even custom components made by the community.  Here you can explore our integrations, which may be maintined by deepset, or submitted by others.
+
+Haystack integrates with a number of other technologies and tools. For example, you can use a number of different model providers or databases with Haystack. 
+
+There are two main types of integrations:
+
+- **Maintained by deepset:** All of the integrations we maintain are hosted in the [haystack-core-integrations](https://github.com/deepset-ai/haystack-core-integrations) repository.
+- **Maintained by our partners or community:** These are integrations that you, our partners, or anyone else can build and maintain themselves. Given they comply with some of our requirements, we will also showcase these on our website.
+
+## What are integrations?
+
+An integration is any type of external technology that can be used to extend the capabilities of the Haystack framework. Some integration examples are those providing access to model providers like OpenAI or Cohere, to databases like Weaviate and Qdrant, or even to monitoring tools such as Traceloop. They can be components, Document Stores, or any other feature that can be used with Haystack.
+
+We maintain a list of available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page, where you can see which integrations we maintain or which have been contributed by the community.
+
+An integrations page focuses on explaining how Haystack integrates with that technology. For example, the OpenAI integration page will provide a summary of the various ways Haystack and OpenAI can work together.
+
+Here are the integration types you can currently choose from:
+
+- **Model Provider**: You can see how we integrate with different model providers and the available components through these integrations
+- **Document Store**: These are the databases and vector stores you can use with your Haystack pipelines.
+- **Evaluation Framework**: Evaluation frameworks that are supported by Haystack that you can use to evaluate Haystack pipelines.
+- **Monitoring Tool**: These are tools like Chainlit and Traceloop that integrate with Haystack and provide monitoring and observability capabilities.
+- **Data Ingestion**: These are the integrations that allow you to ingest and use data from different resources, such as Notion, Mastodon, and others.
+- **Custom Component**: Some integrations that cover very unique use cases are often contributed and maintained by our community members. We list these integrations under the _Custom Component_ tag.
+
+## How do I use an integration?
+
+Each page dedicated to an integration contains installation instructions and basic usage instructions. For example, the OpenAI integration page gives you an overview of the different ways in which you can interact with OpenAI.
+
+## How can I create an integration?
+
+The most common types of integrations are custom components and Document Stores. Integrations such as model providers might even include multiple custom components. Have a look at these documentation pages that will guide you through the requirements for each integration type:
+
+- [Creating Custom Components](components/custom-components.mdx)
+- [Creating Custom Document Stores](document-store/creating-custom-document-stores.mdx)
+
+Check out the [video walkthrough](https://www.youtube.com/watch?v=SWC0QecAMcI) for a step-by-step guide on how to use the [custom-component template](https://github.com/deepset-ai/custom-component) to create a Haystack integration.
+
+## How do I showcase my integration?
+
+To make your integration visible to the Haystack community, contribute it to our [haystack-integrations](https://github.com/deepset-ai/haystack-integrations) GitHub repository. There are several requirements you have to follow:
+
+- Make sure your contribution is [packaged](https://packaging.python.org/en/latest/), installable, and runnable. We suggest using [hatch](https://hatch.pypa.io/latest/) for this purpose.
+- Provide the GitHub repo and issue link.
+- Create a Pull Request in the [haystack-integrations](https://github.com/deepset-ai/haystack-integrations) repo by following the [draft-integration.md](https://github.com/deepset-ai/haystack-integrations/blob/main/draft-integration.md) and include a clear explanation of what your integration is. This page should include:
+  - Installation instructions
+  - A list of the components the integration includes
+  - Examples of how to use it with clear/runnable code
+  - Licensing information
+  - (Optionally) Documentation and/or API docs that you’ve generated for your repository

docs-website/docs/concepts/jinja-templates.mdx

@@ -0,0 +1,58 @@
+---
+title: "Jinja Templates"
+id: jinja-templates
+slug: "/jinja-templates"
+description: "Learn how Jinja templates work with Haystack components."
+---
+
+# Jinja Templates
+
+Learn how Jinja templates work with Haystack components.
+
+Jinja templates are text structures that contain placeholders for generating dynamic content. These placeholders are filled in when the template is rendered. You can check out the full list of Jinja2 features in the [original documentation](https://jinja.palletsprojects.com/en/3.0.x/templates/).
+
+You can use these templates in Haystack [Builders](../pipeline-components/builders.mdx), [OutputAdapter](../pipeline-components/converters/outputadapter.mdx), and [ConditionalRouter](../pipeline-components/routers/conditionalrouter.mdx) components.
+
+Here is an example of `OutputAdapter` using a short Jinja template to output only the content field of the first document in the arrays of documents:
+
+```python
+from haystack import Document
+from haystack.components.converters import OutputAdapter
+
+adapter = OutputAdapter(template="{{ documents[0].content }}", output_type=str)
+input_data = {"documents": [Document(content="Test content")]}
+expected_output = {"output": "Test content"}
+assert adapter.run(**input_data) == expected_output
+```
+
+### Using Python f‑strings with Jinja
+
+When you embed Jinja placeholders inside a Python f‑string, you must escape Jinja’s `{` and `}` by doubling them (so `{{ var }}` becomes `{{{{ var }}}}`). Otherwise, Python will consume the braces and the Jinja variable won’t be found.
+
+Preferred template:
+
+```python
+template = """
+Language: {{ language }}
+Question: {{ question }}
+"""
+# pass both variables when rendering
+```
+
+It you need to use an f‑string (escape braces):
+
+```python
+language = "en"
+template = f"""
+Language: {language}
+Question: {{{{ question }}}}
+"""
+```
+
+## Safety Features
+
+Due to how we use Jinja in some Components, there are some security considerations to take into account. Jinja works by executing embedded in templates, so it’s _imperative_ that they stem from a trusted source. If the template is allowed to be customized by the end user, it can potentially lead to remote code execution.
+
+To mitigate this risk, Jinja templates are executed and rendered in a [sandbox environment](https://jinja.palletsprojects.com/en/3.1.x/sandbox/). While this approach is safer, it's also less flexible and limits the expressiveness of the template. If you need the more advanced functionality of Jinja templates, components that use them provide an `unsafe` init parameter - setting it to `False` will disable the sandbox environment and enable unsafe template rendering.
+
+With unsafe template rendering, the [OutputAdapter](../pipeline-components/converters/outputadapter.mdx) and [ConditionalRouter](../pipeline-components/routers/conditionalrouter.mdx) components allow their `output_type` to be set to one of the [Haystack data classes](data-classes.mdx) such as `ChatMessage`, `Document`, or `Answer`.

docs-website/docs/concepts/metadata-filtering.mdx

@@ -0,0 +1,155 @@
+---
+title: "Metadata Filtering"
+id: metadata-filtering
+slug: "/metadata-filtering"
+description: "This page provides a detailed explanation of how to apply metadata filters at query time."
+---
+
+# Metadata Filtering
+
+This page provides a detailed explanation of how to apply metadata filters at query time.
+
+When you index documents into your Document Store, you can attach metadata to them. One example is the `DocumentLanguageClassifier`, which adds the language of the document's content to its metadata. Components like `MetadataRouter` can then route documents based on their metadata.
+
+You can then use the metadata to filter your search queries, allowing you to narrow down the results by focusing on specific criteria. This ensures your Retriever fetches answers from the most relevant subset of your data.
+
+To illustrate how metadata filters work, imagine you have a set of annual reports from various companies. You may want to perform a search on just a specific year and just on a small selection of companies. This can reduce the workload of the Retriever and also ensure that you get more relevant results.
+
+## Filtering Types
+
+Filters are defined as a dictionary or nested dictionaries that can be of two types: Comparison or Logic.
+
+### Comparison
+
+Comparison operators help search your metadata fields according the specified conditions.
+
+Comparison dictionaries must contain the following keys:
+
+\-`field`: the name of one of the meta fields of a document, such as `meta.years`.
+
+\-`operator`: must be one of the following:
+
+```
+    - `==`
+    - `!=`
+    - `>`
+    - `>=`
+    - `<`
+    - `<=`
+    - `in`
+    - `not in`
+```
+
+:::info
+The available comparison operators may vary depending on the specific Document Store integration. For example, the `ChromaDocumentStore` supports two additional operators: `contains` and `not contains`. Find the details about the supported filters in the specific integration’s API reference.
+:::
+
+\-`value`: takes a single value or (in the case of "in" and “not in”) a list of values.
+
+#### Example
+
+Here is an example of a simple filter in the form of a dictionary. The filter selects documents classified as “article” in the `type` meta field of the document:
+
+```python
+filters = {"field": "meta.type", "operator": "==", "value": "article"}
+```
+
+### Logic
+
+Logical operators can be used to create a nested dictionary, allowing you to apply multiple `fields` as filter conditions. Logic dictionaries must contain the following keys:
+
+\-`operator`: usually one of the following:
+
+```
+    - `NOT`
+    - `OR`
+    - `AND`
+```
+
+:::info
+The available logic operators may vary depending on the specific Document Store integration. For example, the `ChromaDocumentStore` doesn’t support the `NOT` operator. Find the details about the supported filters in the specific integration’s API reference.
+:::
+
+\-`conditions`: must be a list of dictionaries, either of type Comparison or Logic.
+
+#### Nested Filter Example
+
+Here is a more complex filter that uses both Comparison and Logic to find documents where:
+
+- Meta field `type` is "article",
+- Meta field `date` is between 1420066800 and 1609455600 (a specific date range),
+- Meta field `rating` is greater than or equal to 3,
+- Documents are either classified as `genre`  ["economy", "politics"] `OR` the meta field `publisher` is "nytimes".
+
+```python
+filters = {
+    "operator": "AND",
+    "conditions": [
+        {"field": "meta.type", "operator": "==", "value": "article"},
+        {"field": "meta.date", "operator": ">=", "value": 1420066800},
+        {"field": "meta.date", "operator": "<", "value": 1609455600},
+        {"field": "meta.rating", "operator": ">=", "value": 3},
+        {
+            "operator": "OR",
+            "conditions": [
+                {
+                    "field": "meta.genre",
+                    "operator": "in",
+                    "value": ["economy", "politics"],
+                },
+                {"field": "meta.publisher", "operator": "==", "value": "nytimes"},
+            ],
+        },
+    ],
+}
+```
+
+## Filters Usage
+
+Filters can be applied either through the `Retriever` class or directly within Document Stores.
+
+In the `Retriever` class, filters are passed through the `filters` argument. When working with a pipeline, filters can be provided to `Pipeline.run()`, which will automatically route them to the `Retriever` class (refer to the [pipelines documentation](pipelines.mdx) for more information on working with pipelines).
+
+The example below shows how filters can be passed to Retrievers within a pipeline:
+
+```python
+pipeline.run(
+    data={
+        "retriever": {
+            "query": "Why did the revenue increase?",
+            "filters": {
+                "operator": "AND",
+                "conditions": [
+                    {"field": "meta.years", "operator": "==", "value": "2019"},
+                    {
+                        "field": "meta.companies",
+                        "operator": "in",
+                        "value": ["BMW", "Mercedes"],
+                    },
+                ],
+            },
+        },
+    },
+)
+```
+
+In Document Stores, the `filter_documents` method is used to apply filters to stored documents, if the specific integration supports filtering.
+
+The example below shows how filters can be passed to the `QdrantDocumentStore`:
+
+```python
+filters = {
+    "operator": "AND",
+    "conditions": [
+        {"field": "meta.type", "operator": "==", "value": "article"},
+        {"field": "meta.genre", "operator": "in", "value": ["economy", "politics"]},
+    ],
+}
+results = QdrantDocumentStore.filter_documents(filters=filters)
+```
+
+## Additional References
+
+:notebook: Tutorial: [Filtering Documents with Metadata](https://haystack.deepset.ai/tutorials/31_metadata_filtering)
+
+🧑‍🍳 Cookbook: [Extracting Metadata Filters from a Query](https://haystack.deepset.ai/cookbook/extracting_metadata_filters_from_a_user_query)

docs-website/docs/concepts/pipelines.mdx

@@ -0,0 +1,157 @@
+---
+title: "Pipelines"
+id: pipelines
+slug: "/pipelines"
+description: "To build modern search pipelines with LLMs, you need two things: powerful components and an easy way to put them together. The Haystack pipeline is built for this purpose and enables you to design and scale your interactions with LLMs."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+import YoutubeEmbed from "@site/src/components/YoutubeEmbed";
+
+# Pipelines
+
+To build modern search pipelines with LLMs, you need two things: powerful components and an easy way to put them together. The Haystack pipeline is built for this purpose and enables you to design and scale your interactions with LLMs.
+
+The pipelines in Haystack are directed multigraphs of different Haystack components and integrations. They give you the freedom to connect these components in various ways. This means that the pipeline doesn't need to be a continuous stream of information. With the flexibility of Haystack pipelines, you can have simultaneous flows, standalone components, loops, and other types of connections.
+
+## Flexibility
+
+Haystack pipelines are much more than just query and indexing pipelines. What a pipeline does, whether that be indexing, querying, fetching from an API, preprocessing or more, completely depends on how you design your pipeline and what components you use. While you can still create single-function pipelines, like indexing pipelines using ready-made components to clean up, split, and write the documents into a Document Store, or query pipelines that just take a query and return an answer, Haystack allows you to combine multiple use cases into one pipeline with decision components (like the `ConditionalRouter`) as well.
+
+### Agentic Pipelines
+
+Haystack loops and branches enable the creation of complex applications like agents. Here are a few examples on how to create them:
+
+- [Tutorial: Building a Chat Agent with Function Calling](https://haystack.deepset.ai/tutorials/40_building_chat_application_with_function_calling)
+- [Tutorial: Building an Agentic RAG with Fallback to Websearch](https://haystack.deepset.ai/tutorials/36_building_fallbacks_with_conditional_routing)
+- [Tutorial: Generating Structured Output with Loop-Based Auto-Correction](https://haystack.deepset.ai/tutorials/28_structured_output_with_loop)
+- [Cookbook: Define & Run Tools](https://haystack.deepset.ai/cookbook/tools_support)
+- [Cookbook: Conversational RAG using Memory](https://haystack.deepset.ai/cookbook/conversational_rag_using_memory)
+- [Cookbook: Newsletter Sending Agent with Experimental Haystack Tools](https://haystack.deepset.ai/cookbook/newsletter-agent)
+
+### Branching
+
+A pipeline can have multiple branches that process data concurrently. For example, to process different file types, you can have a pipeline with a bunch of converters, each handling a specific file type. You then feed all your files to the pipeline and it smartly divides and routes them to appropriate converters all at once, saving you the effort of sending your files one by one for processing.
+<ClickableImage src="/img/83f686b-Pipeline_Illustrations_1_1.png" alt="Pipeline architecture diagram showing components arranged in parallel branches that converge into a single pipeline flow" size="large" />
+
+### Loops
+
+Components in a pipeline can work in iterative loops, which you can cap at a desired number. This can be handy for scenarios like self-correcting loops, where you have a generator producing some output and then a validator component to check if the output is correct. If the generator's output has errors, the validator component can loop back to the generator for a corrected output. The loop goes on until the output passes the validation and can be sent further down the pipeline.
+
+See [Pipeline Loops](pipelines/pipeline-loops.mdx) for a deeper explanation of how loops are executed, how they terminate, and how to use them safely.
+
+<ClickableImage src="/img/2390eea-Pipeline_Illustrations_1_2.png" alt="Pipeline architecture diagram illustrating a feedback loop where output from later components loops back to earlier components" size="large" />
+
+### Async Execution and Streaming
+
+Pipelines execute components in parallel when their dependencies allow it. This improves performance in complex pipelines with independent operations. For example, a pipeline can run multiple Retrievers or LLM calls simultaneously, execute independent pipeline branches in parallel, and efficiently handle I/O-bound operations that would otherwise cause delays. You can cap the number of components running at the same time with the `concurrency_limit` init parameter.
+
+Besides the blocking `run` method, every pipeline offers three ways to run asynchronously:
+
+- `run_async`: Executes the pipeline in a single non-blocking call, ideal for integrating a pipeline into a larger async application or service.
+- `run_async_generator`: Yields partial outputs as components complete their tasks, which is useful for monitoring progress, debugging, and handling outputs incrementally.
+- `stream`: Runs the pipeline and returns a handle that streams [`StreamingChunk`](/reference/data-classes-api#streamingchunk) objects as they are produced — a convenient way to stream LLM output from an async application, such as an API endpoint. Iterate the handle with `async for` to consume the chunks; after iteration ends, `handle.result` holds the final pipeline output (the same dictionary returned by `run_async`).
+
+```python
+import asyncio
+
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+pipe = Pipeline()
+pipe.add_component(
+    "prompt_builder",
+    ChatPromptBuilder(template=[ChatMessage.from_user("Tell me about {{topic}}")]),
+)
+pipe.add_component("llm", OpenAIChatGenerator())
+pipe.connect("prompt_builder.prompt", "llm.messages")
+
+
+async def main():
+    handle = pipe.stream(data={"prompt_builder": {"topic": "Italy"}})
+    async for chunk in handle:
+        print(chunk.content, end="", flush=True)
+    return handle.result
+
+
+result = asyncio.run(main())
+```
+
+By default, chunks from every streaming-capable component are forwarded; pass `streaming_components` with a list of component names to stream only specific components. If the consumer abandons iteration, the underlying pipeline run is cancelled automatically; pass `cancel_on_abandon=False` to let it run to completion instead.
+
+If a `streaming_callback` is set on a component (at init or at runtime through `data`), it is still invoked for each chunk in addition to the chunks being pushed to the handle. When streaming, components accept a sync `streaming_callback` in `run_async` too — see the [Choosing the Right Generator guide](../pipeline-components/generators/guides-to-generators/choosing-the-right-generator.mdx#sync-and-async-callbacks) for details.
+
+#### Error Handling and Task Cancellation
+
+If a component raises an error while sibling components are still running concurrently, the pipeline cancels and drains those in-flight tasks before re-raising the original error, so no tasks keep running in the background. The same cleanup applies when you stop iterating `run_async_generator` early (for example, by breaking out of the loop and closing the generator) or when the run itself is cancelled.
+
+Note that cancellation only interrupts components that run natively async. Sync components are offloaded to a worker thread, which cannot be interrupted and runs to completion in the background. Their outputs are discarded, so the pipeline state stays consistent, but the component's side effects still complete.
+
+## SuperComponents
+
+To simplify your code, we have introduced [SuperComponents](components/supercomponents.mdx) that allow you to wrap complete pipelines and reuse them as a single component. Check out their documentation page for the details and examples.
+
+## Data Flow
+
+While the data (the initial query) flows through the entire pipeline, individual values are only passed from one component to another when they are connected. Therefore, not all components have access to all the data. This approach offers the benefits of speed and ease of debugging.
+
+To connect components and integrations in a pipeline, you must know the names of their inputs and outputs. The output of one component must be accepted as input by the following component. When you connect components in a pipeline with `Pipeline.connect()`, it validates if the input and output types match.
+
+### Smart Pipeline Connections
+
+Pipelines support smarter connection semantics that simplify how components are wired together.
+
+Compatible outputs can be implicitly combined when connected to a single input.
+Pipelines also perform implicit type adaptation at connection time for some selected types.
+
+These behaviors reduce the need for glue components like `Joiners` and `OutputAdapters`, keeping pipelines concise and easier to read.
+
+See [Smart Pipeline Connections](pipelines/smart-pipeline-connections.mdx) for details and examples.
+
+<YoutubeEmbed videoId="SxAwyeCkguc" title="Introduction to Haystack Pipelines" />
+
+## Steps to Create a Pipeline Explained
+
+Once all your components are created and ready to be combined in a pipeline, there are four steps to make it work:
+
+1. Create the pipeline with `Pipeline()`.
+   This creates the Pipeline object.
+2. Add components to the pipeline, one by one, with `.add_component(name, component)`.
+   This just adds components to the pipeline without connecting them yet. It's especially useful for loops as it allows the smooth connection of the components in the next step because they all already exist in the pipeline.
+3. Connect components with `.connect("producer_component.output_name", "consumer_component.input_name")`.
+   At this step, you explicitly connect one of the outputs of a component to one of the inputs of the next component. This is also when the pipeline validates the connection without running the components. It makes the validation fast.
+4. Run the pipeline with `.run({"component_1": {"mandatory_inputs": value}})`.
+   Finally, you run the Pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components, for example: `.run({"component_1": {"mandatory_inputs": value}, "component_2": {"inputs": value}})`.
+
+The full pipeline [example](pipelines/creating-pipelines.mdx#example) in [Creating Pipelines](pipelines/creating-pipelines.mdx) shows how all the elements come together to create a working RAG pipeline.
+
+Once you create your pipeline, you can [visualize it in a graph](pipelines/visualizing-pipelines.mdx) to understand how the components are connected and make sure that's how you want them. You can use Mermaid graphs to do that.
+
+## Validation
+
+Validation happens when you connect pipeline components with `.connect()`, but before running the components to make it faster. The pipeline validates that:
+
+- The components exist in the pipeline.
+- The components' outputs and inputs match and are explicitly indicated. For example, if a component produces two outputs, when connecting it to another component, you must indicate which output connects to which input.
+- The components' types match.
+- For input types other than `Variadic`, checks if the input is already occupied by another connection.
+
+All of these checks produce detailed errors to help you quickly fix any issues identified.
+
+## Serialization
+
+Thanks to serialization, you can save and then load your pipelines. Serialization is converting a Haystack pipeline into a format you can store on disk or send over the wire. It's particularly useful for:
+
+- Editing, storing, and sharing pipelines.
+- Modifying existing pipelines in a format different than Python.
+
+Haystack pipelines delegate the serialization to its components, so serializing a pipeline simply means serializing each component in the pipeline one after the other, along with their connections. The pipeline is serialized into a dictionary format, which acts as an intermediate format that you can then convert into the final format you want.
+
+:::info[Serialization formats]
+
+Haystack only supports YAML format at this time. We'll be rolling out more formats gradually.
+:::
+
+For serialization to be possible, components must support conversion from and to Python dictionaries. All Haystack components have two methods that make them serializable: `from_dict` and `to_dict`. The `Pipeline` class, in turn, has its own `from_dict` and `to_dict` methods that take care of serializing components and connections.

docs-website/docs/concepts/pipelines/creating-pipelines.mdx

@@ -0,0 +1,258 @@
+---
+title: "Creating Pipelines"
+id: creating-pipelines
+slug: "/creating-pipelines"
+description: "Learn the general principles of creating a pipeline."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Creating Pipelines
+
+Learn the general principles of creating a pipeline.
+
+You can use these instructions to create both indexing and query pipelines.
+
+This task uses an example of a semantic document search pipeline.
+
+## Prerequisites
+
+For each component you want to use in your pipeline, you must know the names of its input and output. You can check them on the documentation page for a specific component or in the component's `run()` method. For more information, see [Components: Input and Output](../components.mdx#input-and-output).
+
+## Steps to Create a Pipeline
+
+### 1\. Import dependencies
+
+Import all the dependencies, like pipeline, documents, Document Store, and all the components you want to use in your pipeline.
+For example, to create a semantic document search pipelines, you need the `Document` object, the pipeline, the Document Store, Embedders, and a Retriever:
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+```python
+from haystack import Document, Pipeline
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersTextEmbedder,
+)
+from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
+```
+
+### 2\. Initialize components
+
+Initialize the components, passing any parameters you want to configure:
+
+```python
+document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
+text_embedder = SentenceTransformersTextEmbedder()
+retriever = InMemoryEmbeddingRetriever(document_store=document_store)
+```
+
+### 3\. Create the pipeline
+
+```python
+query_pipeline = Pipeline()
+```
+
+### 4\. Add components
+
+Add components to the pipeline one by one. The order in which you do this doesn't matter:
+
+```python
+query_pipeline.add_component("component_name", component_type)
+
+# Here is an example of how you'd add the components initialized in step 2 above:
+query_pipeline.add_component("text_embedder", text_embedder)
+query_pipeline.add_component("retriever", retriever)
+
+# You could also add components without initializing them before:
+query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
+query_pipeline.add_component(
+    "retriever",
+    InMemoryEmbeddingRetriever(document_store=document_store),
+)
+```
+
+### 5\. Connect components
+
+Connect the components by indicating which output of a component should be connected to the input of the next component. If a component has only one input or output and the connection is obvious, you can just pass the component name without specifying the input or output.
+
+To understand what inputs are expected to run your pipeline, use an `.inputs()` pipeline function. See a detailed examples in the [Pipeline Inputs](#pipeline-inputs) section below.
+
+Here's a more visual explanation within the code:
+
+```python
+# This is the syntax to connect components. Here you're connecting output1 of component1 to input1 of component2:
+pipeline.connect("component1.output1", "component2.input1")
+
+# If both components have only one output and input, you can just pass their names:
+pipeline.connect("component1", "component2")
+
+# If one of the components has only one output but the other has multiple inputs,
+# you can pass just the name of the component with a single output, but for the component with
+# multiple inputs, you must specify which input you want to connect
+
+# Here, component1 has only one output, but component2 has multiple inputs:
+pipeline.connect("component1", "component2.input1")
+
+# And here's how it should look like for the semantic document search pipeline we're using as an example:
+pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
+# Because the InMemoryEmbeddingRetriever only has one input, this is also correct:
+pipeline.connect("text_embedder.embedding", "retriever")
+```
+
+You need to link all the components together, connecting them gradually in pairs. Here's an explicit example for the pipeline we're assembling:
+
+```python
+# Imagine this pipeline has four components: text_embedder, retriever, prompt_builder and llm.
+# Here's how you would connect them into a pipeline:
+
+query_pipeline.connect("text_embedder.embedding", "retriever")
+query_pipeline.connect("retriever", "prompt_builder.documents")
+query_pipeline.connect("prompt_builder", "llm")
+```
+
+### 6\. Run the pipeline
+
+Wait for the pipeline to validate the components and connections. If everything is OK, you can now run the pipeline. `Pipeline.run()` can be called in two ways, either passing a dictionary of the component names and their inputs, or by directly passing just the inputs. When passed directly, the pipeline resolves inputs to the correct components.
+
+```python
+# Here's one way of calling the run() method
+results = pipeline.run({"component1": {"input1_value": value1, "input2_value": value2}})
+
+# The inputs can also be passed directly without specifying component names
+results = pipeline.run({"input1_value": value1, "input2_value": value2})
+
+# This is how you'd run the semantic document search pipeline we're using as an example:
+query = "Here comes the query text"
+results = query_pipeline.run({"text_embedder": {"text": query}})
+```
+
+## Pipeline Inputs
+
+If you need to understand what component inputs are expected to run your pipeline, Haystack features a useful pipeline function `.inputs()` that lists all the required inputs for the components.
+
+This is how it works:
+
+```python
+# A short pipeline example that converts webpages into documents
+from haystack import Pipeline
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.fetchers import LinkContentFetcher
+from haystack.components.converters import HTMLToDocument
+from haystack.components.writers import DocumentWriter
+
+document_store = InMemoryDocumentStore()
+fetcher = LinkContentFetcher()
+converter = HTMLToDocument()
+writer = DocumentWriter(document_store=document_store)
+
+pipeline = Pipeline()
+pipeline.add_component(instance=fetcher, name="fetcher")
+pipeline.add_component(instance=converter, name="converter")
+pipeline.add_component(instance=writer, name="writer")
+
+pipeline.connect("fetcher.streams", "converter.sources")
+pipeline.connect("converter.documents", "writer.documents")
+
+# Requesting a list of required inputs
+pipeline.inputs()
+
+# {'fetcher': {'urls': {'type': typing.List[str], 'is_mandatory': True}},
+# 'converter': {'meta': {'type': typing.Union[typing.Dict[str, typing.Any], typing.List[typing.Dict[str, typing.Any]], NoneType],
+# 'is_mandatory': False,
+# 'default_value': None},
+# 'extraction_kwargs': {'type': typing.Optional[typing.Dict[str, typing.Any]],
+# 'is_mandatory': False,
+# 'default_value': None}},
+# 'writer': {'policy': {'type': typing.Optional[haystack.document_stores.types.policy.DuplicatePolicy],
+# 'is_mandatory': False,
+# 'default_value': None}}}
+```
+
+From the above response, you can see that the `urls` input is mandatory for `LinkContentFetcher`. This is how you would then run this pipeline:
+
+```python
+pipeline.run(
+    data={"fetcher": {"urls": ["https://docs.haystack.deepset.ai/docs/pipelines"]}},
+)
+```
+
+## Example
+
+The following example walks you through creating a RAG pipeline.
+
+```python
+# import necessary dependencies
+from haystack import Pipeline, Document
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.builders import ChatPromptBuilder
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+
+# create a document store and write documents to it
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+
+# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer.
+prompt_template = [
+    ChatMessage.from_system(
+        """
+        Given these documents, answer the question.
+        Documents:
+        {% for doc in documents %}
+            {{ doc.content }}
+        {% endfor %}
+        Question:
+        """,
+    ),
+    ChatMessage.from_user("{{question}}"),
+    ChatMessage.from_system("Answer:"),
+]
+
+# create the components adding the necessary parameters
+retriever = InMemoryBM25Retriever(document_store=document_store)
+prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
+llm = OpenAIChatGenerator(
+    api_key=Secret.from_env_var("OPENAI_API_KEY"),
+    model="gpt-4o-mini",
+)
+
+# Create the pipeline and add the components to it. The order doesn't matter.
+# At this stage, the Pipeline validates the components without running them yet.
+rag_pipeline = Pipeline()
+rag_pipeline.add_component("retriever", retriever)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+
+# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name").
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+
+# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components.
+question = "Who lives in paris?"
+results = rag_pipeline.run(
+    {
+        "retriever": {"query": question},
+        "prompt_builder": {"question": question},
+    },
+)
+
+print(results["llm"]["replies"])
+```
+
+Here's what a [visualized Mermaid graph](visualizing-pipelines.mdx) of this pipeline would look like:
+
+<br />
+<ClickableImage src="/img/vizualised-rag-pipeline.png" alt="RAG pipeline diagram with three connected components: InMemoryBM25Retriever receives a query string and outputs documents, ChatPromptBuilder combines the documents with a question input to create prompt messages, and OpenAIChatGenerator processes the messages to produce replies. Each component box displays its class name and optional input parameters." size="large" />

docs-website/docs/concepts/pipelines/debugging-pipelines.mdx

@@ -0,0 +1,125 @@
+---
+title: "Debugging Pipelines"
+id: debugging-pipelines
+slug: "/debugging-pipelines"
+description: "Learn how to debug and troubleshoot your Haystack pipelines."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Debugging Pipelines
+
+Learn how to debug and troubleshoot your Haystack pipelines.
+
+There are several options available to you to debug your pipelines:
+
+- [Inspect your components' outputs](#inspecting-component-outputs)
+- [Adjust logging](#logging)
+- [Set up tracing](#tracing)
+- [Try one of the monitoring tool integrations](#monitoring-tools)
+
+## Inspecting Component Outputs
+
+To view outputs from specific pipeline components, add the `include_outputs_from` parameter when executing your pipeline. Place it after the input dictionary and set it to the name of the component whose output you want included in the result.
+
+For example, here’s how you can print the output of `PromptBuilder` in this pipeline:
+
+```python
+from haystack import Pipeline, Document
+from haystack.utils import Secret
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+# Documents
+documents = [
+    Document(content="Joe lives in Berlin"),
+    Document(content="Joe is a software engineer"),
+]
+
+# Define prompt template
+prompt_template = [
+    ChatMessage.from_system("You are a helpful assistant."),
+    ChatMessage.from_user(
+        "Given these documents, answer the question.\nDocuments:\n"
+        "{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
+        "Question: {{query}}\nAnswer:",
+    ),
+]
+
+# Define pipeline
+p = Pipeline()
+p.add_component(
+    instance=ChatPromptBuilder(
+        template=prompt_template,
+        required_variables={"query", "documents"},
+    ),
+    name="prompt_builder",
+)
+p.add_component(
+    instance=OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
+    name="llm",
+)
+p.connect("prompt_builder", "llm.messages")
+
+# Define question
+question = "Where does Joe live?"
+
+# Execute pipeline
+result = p.run(
+    {"prompt_builder": {"documents": documents, "query": question}},
+    include_outputs_from="prompt_builder",
+)
+
+# Print result
+print(result)
+```
+
+## Logging
+
+Adjust the logging format according to your debugging needs. See our [Logging](../../development/logging.mdx) documentation for details.
+
+## Real-Time Pipeline Logging
+
+Use Haystack's [`LoggingTracer`](https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py) logs to inspect the data that's flowing through your pipeline in real-time.
+
+This feature is particularly helpful during experimentation and prototyping, as you don’t need to set up any tracing backend beforehand.
+
+Here’s how you can enable this tracer. In this example, we are adding color tags (this is optional) to highlight the components' names and inputs:
+
+```python
+import logging
+from haystack import tracing
+from haystack.tracing.logging_tracer import LoggingTracer
+
+logging.basicConfig(
+    format="%(levelname)s - %(name)s -  %(message)s",
+    level=logging.WARNING,
+)
+logging.getLogger("haystack").setLevel(logging.DEBUG)
+
+tracing.tracer.is_content_tracing_enabled = (
+    True  # to enable tracing/logging content (inputs/outputs)
+)
+tracing.enable_tracing(
+    LoggingTracer(
+        tags_color_strings={
+            "haystack.component.input": "\x1b[1;31m",
+            "haystack.component.name": "\x1b[1;34m",
+        },
+    ),
+)
+```
+
+Here’s what the resulting log would look like when a pipeline is run:
+<ClickableImage src="/img/55c3d5c84282d726c95fb3350ec36be49a354edca8a6164f5dffdab7121cec58-image_2.png" alt="Console output showing Haystack pipeline execution with DEBUG level tracing logs including component names, types, and input/output specifications" />
+
+## Tracing
+
+To get a bigger picture of the pipeline’s performance, try tracing it with [Langfuse](../../development/tracing/langfuse.mdx).
+
+Our [Tracing](../../development/tracing.mdx) page has more about other tracing solutions for Haystack.
+
+## Monitoring Tools
+
+Take a look at available tracing and monitoring [integrations](https://haystack.deepset.ai/integrations?type=Monitoring+Tool&version=2.0) for Haystack pipelines, such as Arize AI or Arize Phoenix.

docs-website/docs/concepts/pipelines/pipeline-breakpoints.mdx

@@ -0,0 +1,160 @@
+---
+title: "Pipeline Breakpoints"
+id: pipeline-breakpoints
+slug: "/pipeline-breakpoints"
+description: "Learn how to pause and resume Haystack pipeline execution using breakpoints to debug, inspect, and continue workflows from saved snapshots."
+---
+
+# Pipeline Breakpoints
+
+Learn how to pause and resume Haystack pipeline execution using breakpoints to debug, inspect, and continue workflows from saved snapshots.
+
+## Introduction
+
+Haystack pipelines support breakpoints for debugging complex execution flows. A `Breakpoint` allows you to pause the execution at specific components, inspect the pipeline state, and resume execution from saved snapshots. This feature works for any regular component as well as an `Agent` component.
+
+You can set a `Breakpoint` on any component in a pipeline with a specific visit count. When triggered, the system stops the execution of the `Pipeline` and captures a snapshot of the current pipeline state. The state can be saved to a JSON file when snapshot file saving is enabled, see [Snapshot file saving](#snapshot-file-saving) below. You can inspect and modify the snapshot and use it to resume execution from the exact point where it stopped.
+
+## Setting a `Breakpoint` on a Regular Component
+
+Create a `Breakpoint` by specifying the component name and the visit count at which to trigger it. This is useful for pipelines with loops. The default `visit_count` value is 0.
+
+```python
+from haystack.dataclasses.breakpoints import Breakpoint
+from haystack.core.errors import BreakpointException
+
+# Create a breakpoint that triggers on the first visit to the "llm" component
+break_point = Breakpoint(
+    component_name="llm",
+    visit_count=0,  # 0 = first visit, 1 = second visit, etc.
+    snapshot_file_path="/path/to/snapshots",  # Optional: save snapshot to file
+)
+
+# Run pipeline with breakpoint
+try:
+    result = pipeline.run(data=input_data, break_point=break_point)
+except BreakpointException as e:
+    print(f"Breakpoint triggered at component: {e.component}")
+    print(f"Component inputs: {e.inputs}")
+    print(f"Pipeline results so far: {e.results}")
+```
+
+A `BreakpointException` is raised containing the component inputs and the outputs of the pipeline up until the moment where the execution was interrupted, such as just before the execution of component associated with the breakpoint – the `llm` in the example above.
+
+If a `snapshot_file_path` is specified in the `Breakpoint` and snapshot file saving is enabled, the system saves a JSON snapshot with the same information as in the `BreakpointException`. Snapshot file saving to disk is disabled by default; see [Snapshot file saving](#snapshot-file-saving) below.
+
+To access the pipeline state during the breakpoint we can both catch the exception raised by the breakpoint as well as specify where the JSON file should be saved, note that file saving is enabled must be enabled.
+
+## Using a custom snapshot callback
+
+You can pass a `snapshot_callback` to `Pipeline.run()` to handle snapshots yourself instead of saving to a file. When a breakpoint is triggered or a snapshot is created on error, the callback is invoked with the `PipelineSnapshot` object. This is useful for saving snapshots to a database, sending them to a remote service, or custom logging.
+
+```python
+from haystack.core.errors import BreakpointException
+from haystack.dataclasses.breakpoints import Breakpoint, PipelineSnapshot
+
+
+def my_snapshot_callback(snapshot: PipelineSnapshot) -> None:
+    # Custom handling: e.g. save to DB, send to API, or log
+    print(f"Snapshot at component: {snapshot.break_point}")
+
+
+break_point = Breakpoint(component_name="llm", visit_count=0)
+try:
+    result = pipeline.run(
+        data=input_data,
+        break_point=break_point,
+        snapshot_callback=my_snapshot_callback,
+    )
+except BreakpointException as e:
+    print(f"Breakpoint triggered: {e.component}")
+```
+
+When `snapshot_callback` is provided, file-saving is skipped and the callback is responsible for handling the snapshot.
+
+## Snapshot file saving
+
+Snapshot file saving to disk is **disabled by default**. To save snapshots as JSON files when a breakpoint is triggered or on pipeline failure, set the environment variable `HAYSTACK_PIPELINE_SNAPSHOT_SAVE_ENABLED` to `"true"` or `"1"` (case-insensitive). When enabled, snapshots are written to the path given by `snapshot_file_path` on the breakpoint, or to the default directory in [Error Recovery with Snapshots](#error-recovery-with-snapshots) when a run fails.
+
+Custom `snapshot_callback` functions are always invoked when provided, regardless of this setting.
+
+```python
+import os
+
+# Enable saving snapshot files to disk
+os.environ["HAYSTACK_PIPELINE_SNAPSHOT_SAVE_ENABLED"] = "true"
+
+break_point = Breakpoint(
+    component_name="llm",
+    visit_count=0,
+    snapshot_file_path="/path/to/snapshots",
+)
+# When the breakpoint triggers, a JSON file will be written to /path/to/snapshots/
+```
+
+## Resuming a Pipeline Execution from a Breakpoint
+
+To resume the execution of a pipeline from the breakpoint, pass the path to the generated JSON file at the run time of the pipeline, using the `pipeline_snapshot`.
+
+Use the `load_pipeline_snapshot()` to first load the JSON and then pass it to the pipeline.
+
+```python
+from haystack.core.pipeline.breakpoint import load_pipeline_snapshot
+
+# Load the snapshot
+snapshot = load_pipeline_snapshot("llm_2025_05_03_11_23_23.json")
+
+# Resume execution from the snapshot
+result = pipeline.run(data={}, pipeline_snapshot=snapshot)
+print(result["llm"]["replies"])
+```
+
+## Error Recovery with Snapshots
+
+Pipelines automatically create a snapshot of the last valid state if a run fails. The snapshot contains inputs, visit counts, and intermediate outputs up to the failure. You can inspect it, fix the issue, and resume execution from that checkpoint instead of restarting the whole run.
+
+### Access the Snapshot on Failure
+
+Wrap `pipeline.run()` in a `try`/`except` block and retrieve the snapshot from the raised `PipelineRuntimeError`:
+
+```python
+from haystack.core.errors import PipelineRuntimeError
+
+try:
+    pipeline.run(data=input_data)
+except PipelineRuntimeError as e:
+    snapshot = e.pipeline_snapshot
+    if snapshot is not None:
+        intermediate_outputs = snapshot.pipeline_state.pipeline_outputs
+        # Inspect intermediate_outputs to diagnose the failure
+```
+
+When snapshot file saving is enabled (see [Snapshot file saving](#snapshot-file-saving)), Haystack also saves the same snapshot as a JSON file on disk.
+The directory is chosen automatically in this order:
+
+- `~/.haystack/pipeline_snapshot`
+- `/tmp/haystack/pipeline_snapshot`
+- `./.haystack/pipeline_snapshot`
+
+Filenames will have the following pattern: `{component_name}_{visit_nr}_{YYYY_MM_DD_HH_MM_SS}.json`.
+
+### Resume from a Snapshot
+
+You can resume directly from the in-memory snapshot or load it from disk.
+
+Resume from memory:
+
+```python
+result = pipeline.run(data={}, pipeline_snapshot=snapshot)
+```
+
+Resume from disk:
+
+```python
+from haystack.core.pipeline.breakpoint import load_pipeline_snapshot
+
+snapshot = load_pipeline_snapshot(
+    "/path/to/.haystack/pipeline_snapshot/reader_0_2025_09_20_12_33_10.json",
+)
+result = pipeline.run(data={}, pipeline_snapshot=snapshot)
+```

docs-website/docs/concepts/pipelines/pipeline-loops.mdx

@@ -0,0 +1,253 @@
+---
+title: "Pipeline Loops"
+id: pipeline-loops
+slug: "/pipeline-loops"
+description: "Understand how loops work in Haystack pipelines, how they terminate, and how to use them safely for feedback and self-correction."
+---
+
+# Pipeline Loops
+
+Learn how loops work in Haystack pipelines, how they terminate, and how to use them for feedback and self-correction.
+
+Haystack pipelines support **loops**: cycles in the component graph where the output of a later component is fed back into an earlier one.
+This enables feedback flows such as self-correction, validation, or iterative refinement, as well as more advanced [agentic behavior](../pipelines.mdx#agentic-pipelines).
+
+At runtime, the pipeline re-runs a component whenever all of its required inputs are ready again.
+You control when loops stop either by designing your graph and routing logic carefully or by using built-in [safety limits](#loop-termination-and-safety-limits).
+
+## Multiple Runs of the Same Component
+
+If a component participates in a loop, it can be run multiple times within a single `Pipeline.run()` call.
+The pipeline keeps an internal visit counter for each component:
+
+- Each time the component runs, its visit count increases by 1.
+- You can use this visit count in debugging tools like [breakpoints](./pipeline-breakpoints.mdx) to inspect specific iterations of a loop.
+
+In the final pipeline result:
+
+- For each component that ran, the pipeline returns **only the last-produced output**.
+- To capture outputs from intermediate components (for example, a validator or a router) in the final result dictionary, use the `include_outputs_from` argument of `Pipeline.run()`.
+
+## Loop Termination and Safety Limits
+
+Loops must eventually stop so that a pipeline run can complete.
+There are two main ways a loop ends:
+
+1. **Natural completion**: No more components are runnable  
+   The pipeline finishes when the work queue is empty and no component can run again (for example, the router stops feeding inputs back into the loop).
+
+2. **Reaching the maximum run count**  
+   Every pipeline has a per-component run limit, controlled by the `max_runs_per_component` parameter of the `Pipeline` constructor, which is `100` by default. If any component exceeds this limit, Haystack raises a `PipelineMaxComponentRuns` error.
+
+   You can set this limit to a lower value:
+
+   ```python
+   from haystack import Pipeline
+
+   pipe = Pipeline(max_runs_per_component=5)
+   ```
+
+   The limit is checked before each execution, so a component with a limit of 3 will complete 3 runs successfully before the error is raised on the 4th attempt.
+
+   This safeguard is especially important when experimenting with new loops or complex routing logic.
+   If your loop condition is wrong or never satisfied, the error prevents the pipeline from running indefinitely.
+
+## Example: Feedback Loop for Self-Correction
+
+The following example shows a simple feedback loop where:
+
+- A `ChatPromptBuilder` creates a prompt that includes previous incorrect replies.
+- An `OpenAIChatGenerator` produces an answer.
+- A `ConditionalRouter` checks if the answer is correct:
+  - If correct, it sends the answer to `final_answer` and the loop ends.
+  - If incorrect, it sends the answer back to the `ChatPromptBuilder`, which triggers another iteration.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.routers import ConditionalRouter
+from haystack.dataclasses import ChatMessage
+
+template = [
+    ChatMessage.from_system(
+        "Answer the following question concisely with just the answer, no punctuation.",
+    ),
+    ChatMessage.from_user(
+        "{% if previous_replies %}"
+        "Previously you replied incorrectly: {{ previous_replies[0].text }}\n"
+        "{% endif %}"
+        "Question: {{ query }}",
+    ),
+]
+
+prompt_builder = ChatPromptBuilder(template=template, required_variables=["query"])
+generator = OpenAIChatGenerator()
+
+router = ConditionalRouter(
+    routes=[
+        {
+            # End the loop when the answer is correct
+            "condition": "{{ 'Rome' in replies[0].text }}",
+            "output": "{{ replies }}",
+            "output_name": "final_answer",
+            "output_type": list[ChatMessage],
+        },
+        {
+            # Loop back when the answer is incorrect
+            "condition": "{{ 'Rome' not in replies[0].text }}",
+            "output": "{{ replies }}",
+            "output_name": "previous_replies",
+            "output_type": list[ChatMessage],
+        },
+    ],
+    unsafe=True,  # Required to handle ChatMessage objects
+)
+
+pipe = Pipeline(max_runs_per_component=3)
+
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("generator", generator)
+pipe.add_component("router", router)
+
+pipe.connect("prompt_builder.prompt", "generator.messages")
+pipe.connect("generator.replies", "router.replies")
+pipe.connect("router.previous_replies", "prompt_builder.previous_replies")
+
+result = pipe.run(
+    {
+        "prompt_builder": {
+            "query": "What is the capital of Italy? If the statement 'Previously you replied incorrectly:' is missing "
+            "above then answer with Milan.",
+        },
+    },
+    include_outputs_from={"router", "prompt_builder"},
+)
+
+print(result["prompt_builder"]["prompt"][1].text)  # Shows the last prompt used
+print(result["router"]["final_answer"][0].text)  # Rome
+```
+
+### What Happens During This Loop
+
+1. **First iteration**
+   - `prompt_builder` runs with `query="What is the capital of Italy?"` and no previous replies.
+   - `generator` returns a `ChatMessage` with the LLM's answer.
+   - The router evaluates its conditions and checks if `"Rome"` is in the reply.
+   - If the answer is incorrect, `previous_replies` is fed back into `prompt_builder.previous_replies`.
+
+2. **Subsequent iterations** (if needed)
+   - `prompt_builder` runs again, now including the previous incorrect reply in the user message.
+   - `generator` produces a new answer with the additional context.
+   - The router checks again whether the answer contains `"Rome"`.
+
+3. **Termination**
+   - When the router routes to `final_answer`, no more inputs are fed back into the loop.
+   - The queue empties and the pipeline run finishes successfully.
+
+Because we used `max_runs_per_component=3`, any unexpected behavior that causes the loop to continue would raise a `PipelineMaxComponentRuns` error instead of looping forever.
+
+## Components for Building Loops
+
+Two components are particularly useful for building loops:
+
+- **[`ConditionalRouter`](../../pipeline-components/routers/conditionalrouter.mdx)**: Routes data to different outputs based on conditions. Use it to decide whether to exit the loop or continue iterating. The example above uses this pattern.
+
+- **[`BranchJoiner`](../../pipeline-components/joiners/branchjoiner.mdx)**: Merges inputs from multiple sources into a single output. Use it when a component inside the loop needs to receive both the initial input (on the first iteration) and looped-back values (on subsequent iterations). For example, you might use `BranchJoiner` to feed both user input and validation errors into the same Generator. See the [BranchJoiner documentation](../../pipeline-components/joiners/branchjoiner.mdx#enabling-loops) for a complete loop example.
+
+## Greedy vs. Lazy Variadic Sockets in Loops
+
+Some components support variadic inputs that can receive multiple values on a single socket.
+In loops, variadic behavior controls how inputs are consumed across iterations.
+
+- **Greedy variadic sockets**  
+  Consume exactly one value at a time and remove it after the component runs.
+  This includes user-provided inputs, which prevents them from retriggering the component indefinitely.
+  Most variadic sockets are greedy by default.
+
+- **Lazy variadic sockets**  
+  Accumulate all values received from predecessors across iterations.
+  Useful when you need to collect multiple partial results over time (for example, gathering outputs from several loop iterations before proceeding).
+
+For most loop scenarios it's sufficient to just connect components as usual and use `max_runs_per_component` to protect against mistakes.
+
+## Troubleshooting Loops
+
+If your pipeline seems stuck or runs longer than expected, here are common causes and how to debug them.
+
+### Common Causes of Infinite Loops
+
+1. **Condition never satisfied**: Your exit condition (for example, `"Rome" in reply`) might never be true due to LLM behavior or data issues. Always set a reasonable `max_runs_per_component` as a safety net.
+
+2. **Relying on optional outputs**: When a component has multiple output sockets but only returns some of them, the unreturned outputs don't trigger their downstream connections. This can cause confusion in loops.
+
+   For example, this pattern can be problematic:
+
+   ```python
+   @component
+   class Validator:
+       @component.output_types(valid=str, invalid=Optional[str])
+       def run(self, text: str):
+           if is_valid(text):
+               return {"valid": text}  # "invalid" is never returned
+           else:
+               return {"invalid": text}
+   ```
+
+   If you connect `invalid` back to an upstream component for retry, but also have other connections that keep the loop alive, you might get unexpected behavior.
+
+   Instead, use a `ConditionalRouter` with explicit, mutually exclusive conditions:
+
+   ```python
+   router = ConditionalRouter(
+       routes=[
+           {"condition": "{{ is_valid }}", "output": "{{ text }}", "output_name": "valid", ...},
+           {"condition": "{{ not is_valid }}", "output": "{{ text }}", "output_name": "invalid", ...},
+       ]
+   )
+   ```
+
+3. **User inputs retriggering the loop**: If a user-provided input is connected to a socket inside the loop, it might cause the loop to restart unexpectedly.
+
+   ```python
+   # Problematic: user input goes directly to a component inside the loop
+   result = pipe.run({
+       "generator": {"prompt": query},  # This input persists and may retrigger the loop
+   })
+
+   # Better: use an entry-point component outside the loop
+   result = pipe.run({
+       "prompt_builder": {"query": query},  # Entry point feeds into the loop once
+   })
+   ```
+
+   See [Greedy vs. Lazy Variadic Sockets](#greedy-vs-lazy-variadic-sockets-in-loops) for details on how inputs are consumed.
+
+4. **Multiple paths feeding the same component**: If a component inside the loop receives inputs from multiple sources, it runs whenever *any* path provides input.
+
+   ```python
+   # Component receives from two sources – runs when either provides input
+   pipe.connect("source_a.output", "processor.input")
+   pipe.connect("source_b.output", "processor.input")  # Variadic input
+   ```
+
+   Ensure you understand when each path produces output, or use `BranchJoiner` to explicitly control the merge point.
+
+### Debugging Tips
+
+1. **Start with a low limit**: When developing loops, set `max_runs_per_component=3` or similar. This helps you catch issues early with a clear error instead of waiting for a timeout.
+
+2. **Use `include_outputs_from`**: Add intermediate components (like your router) to see what's happening at each step:
+   ```python
+   result = pipe.run(data, include_outputs_from={"router", "validator"})
+   ```
+
+3. **Enable tracing**: Use tracing to see every component execution, including inputs and outputs. This makes it easy to follow each iteration of the loop. For quick debugging, use `LoggingTracer` ([setup instructions](./debugging-pipelines.mdx#real-time-pipeline-logging)). For deeper analysis, integrate with tools like Langfuse or other [tracing backends](../../development/tracing.mdx).
+
+4. **Visualize the pipeline**: Use `pipe.draw()` or `pipe.show()` to see the graph structure and verify your connections are correct. See the [Pipeline Visualization](./visualizing-pipelines.mdx) documentation for details.
+
+5. **Use breakpoints**: Set a `Breakpoint` on a specific component and visit count to inspect the state at that iteration. See [Pipeline Breakpoints](./pipeline-breakpoints.mdx) for details.
+
+6. **Check for blocked pipelines**: If you see a `PipelineComponentsBlockedError`, it means no components can run. This typically indicates a missing connection or a circular dependency. Check that all required inputs are provided.
+
+By combining careful graph design, per-component run limits, and these debugging tools, you can build robust feedback loops in your Haystack pipelines.

docs-website/docs/concepts/pipelines/serialization.mdx

@@ -0,0 +1,276 @@
+---
+title: "Serializing Pipelines"
+id: serialization
+slug: "/serialization"
+description: "Save your pipelines into a custom format and explore the serialization options."
+---
+
+# Serializing Pipelines
+
+Save your pipelines into a custom format and explore the serialization options.
+
+Serialization means converting a pipeline to a format that you can save on your disk and load later.
+
+Haystack supports YAML format for pipeline serialization.
+
+## Converting a Pipeline to YAML
+
+Use the `dumps()` method to convert a Pipeline object to YAML:
+
+```python
+from haystack import Pipeline
+
+pipe = Pipeline()
+print(pipe.dumps())
+
+# Prints:
+#
+# components: {}
+# connections: []
+# max_runs_per_component: 100
+# metadata: {}
+```
+
+You can also use `dump()` method to save the YAML representation of a pipeline in a file:
+
+```python
+with open("/content/test.yml", "w") as file:
+    pipe.dump(file)
+```
+
+## Converting a Pipeline Back to Python
+
+You can convert a YAML pipeline back into Python. Use the `loads()` method to convert a string representation of a pipeline (`str`, `bytes` or `bytearray`)  or the `load()` method to convert a pipeline represented in a file-like object into a corresponding Python object.
+
+Both loading methods support callbacks that let you modify components during the deserialization process. Deserialization is gated by a trusted-module allowlist, so pipelines referencing classes outside of it fail to load until you extend the allowlist — see [Deserialization Security](#deserialization-security) below.
+
+Here is an example script:
+
+```python
+from haystack import Pipeline
+from haystack.core.serialization import DeserializationCallbacks
+from typing import Type, Dict, Any
+
+# This is the YAML you want to convert to Python:
+pipeline_yaml = """
+components:
+  cleaner:
+    init_parameters:
+      remove_empty_lines: true
+      remove_extra_whitespaces: true
+      remove_regex: null
+      remove_repeated_substrings: false
+      remove_substrings: null
+    type: haystack.components.preprocessors.document_cleaner.DocumentCleaner
+  converter:
+    init_parameters:
+      encoding: utf-8
+    type: haystack.components.converters.txt.TextFileToDocument
+connections:
+- receiver: cleaner.documents
+  sender: converter.documents
+max_runs_per_component: 100
+metadata: {}
+"""
+
+
+def component_pre_init_callback(
+    component_name: str,
+    component_cls: Type,
+    init_params: Dict[str, Any],
+):
+    # This function gets called every time a component is deserialized.
+    if component_name == "cleaner":
+        assert "DocumentCleaner" in component_cls.__name__
+        # Modify the init parameters. The modified parameters are passed to
+        # the init method of the component during deserialization.
+        init_params["remove_empty_lines"] = False
+        print("Modified 'remove_empty_lines' to False in 'cleaner' component")
+    else:
+        print(f"Not modifying component {component_name} of class {component_cls}")
+
+
+pipe = Pipeline.loads(
+    pipeline_yaml,
+    callbacks=DeserializationCallbacks(component_pre_init_callback),
+)
+```
+
+## Deserialization Security
+
+Loading a pipeline instantiates the classes referenced in the serialized data. To prevent a crafted YAML file from importing and instantiating arbitrary classes, `Pipeline.load`, `Pipeline.loads`, and `Pipeline.from_dict` refuse to import classes from modules outside a trusted-module allowlist and raise a `DeserializationError` instead.
+
+By default, the allowlist contains `haystack`, `haystack_integrations`, `haystack_experimental`, `builtins`, `typing`, and `collections`. Dangerous builtins such as `eval`, `exec`, `compile`, `__import__`, `open`, and `getattr` are blocked even though `builtins` is allowlisted.
+
+### Allowing Custom Modules
+
+Pipelines that reference custom components or callables in other packages fail to load until you add the modules to the allowlist. You can extend it in three ways:
+
+```python
+from haystack import Pipeline
+
+# 1. Per call: pass additional module patterns for this deserialization only
+pipe = Pipeline.load(open("pipeline.yaml"), allowed_modules=["mypkg.*"])
+
+# 2. Process-wide: extend the allowlist programmatically
+from haystack.core.serialization import allow_deserialization_module
+
+allow_deserialization_module("mypkg")
+```
+
+```shell
+# 3. Environment variable with comma-separated patterns, read on every deserialization call
+export HAYSTACK_DESERIALIZATION_ALLOWLIST="mypkg.*,otherpkg.*"
+```
+
+Patterns are matched as prefixes by default (`"mypkg"` matches `mypkg` and any of its submodules), or as `fnmatch` globs if they contain `*`, `?`, or `[` somewhere other than a trailing `.*`. A trailing `.*` is treated as a prefix match, so `"mypkg"` and `"mypkg.*"` behave identically.
+
+If the source of the serialized data is fully trusted, you can bypass the allowlist entirely with `unsafe=True`:
+
+```python
+pipe = Pipeline.load(open("pipeline.yaml"), unsafe=True)
+```
+
+Only use `unsafe=True` when you fully trust where the serialized pipeline comes from — it also lifts the block on dangerous builtins.
+
+### Nested Init Parameter Validation
+
+As an additional safeguard, deserialization validates the keys of `init_parameters` against the class's `__init__` signature before recursing into any nested `{"type": "...", "init_parameters": {...}}` dictionary. A nested dictionary whose key is not an accepted parameter name is rejected with a `DeserializationError` *before* the nested type is imported, which blocks attempts to smuggle untrusted classes into unused parameter slots. Classes whose constructor takes `**kwargs` are exempt, since their accepted parameter set cannot be statically determined.
+
+This validation may surface pre-existing bugs in YAML files — for example typos, leftovers from renamed or removed parameters, or stale snapshots from older Haystack versions. The fix is to update the YAML so each nested-component key matches a real `__init__` parameter of the parent class.
+
+## Default Serialization Behavior
+
+The serialization system uses `default_to_dict` and `default_from_dict` to handle many object types automatically. You typically do **not** need to implement custom `to_dict`/`from_dict` for:
+
+- **Secrets**: serialized and deserialized automatically so that sensitive values aren't stored in plain text.
+- **ComponentDevice**: device configuration is detected and restored automatically.
+- **Objects with their own `to_dict`/`from_dict`**: any init parameter whose type defines `to_dict()` is serialized by calling it; any dict in `init_parameters` with a `type` key pointing to a class with `from_dict()` is deserialized automatically.
+
+To serialize or deserialize a single component, you can use `component_to_dict` and `component_from_dict` from `haystack.core.serialization`. They use the default behavior above as a fallback when the component doesn't define custom `to_dict`/`from_dict`:
+
+```python
+from haystack import component
+from haystack.core.serialization import component_from_dict, component_to_dict
+
+
+@component
+class Greeter:
+    def __init__(self, message: str = "Hello"):
+        self.message = message
+
+    @component.output_types(greeting=str)
+    def run(self, name: str):
+        return {"greeting": f"{self.message}, {name}!"}
+
+
+# Serialize a component instance to a dictionary
+greeter = Greeter(message="Hi")
+data = component_to_dict(greeter, "my_greeter")
+
+# Deserialize back to a component instance
+restored = component_from_dict(Greeter, data, "my_greeter")
+assert restored.message == greeter.message
+```
+
+:::caution[Init parameters must be stored as instance attributes]
+
+Default serialization only works when there is a **1:1 mapping** between init parameter names and instance attributes. For every argument in `__init__`, the component must assign it to an attribute with the same name. For example, if you have `def __init__(self, prompt: str)`, you must have `self.prompt = prompt` in the class. Otherwise the serialization logic can't find the value to serialize and raises an error or uses the default value if the parameter has one.
+:::
+
+## Performing Custom Serialization
+
+Pipelines and components in Haystack can serialize simple components, including custom ones, out of the box. Code like this just works:
+
+```python
+from haystack import component
+
+
+@component
+class RepeatWordComponent:
+    def __init__(self, times: int):
+        self.times = times
+
+    @component.output_types(result=str)
+    def run(self, word: str):
+        return word * self.times
+```
+
+On the other hand, this code doesn't work if the final format is JSON, as the `set` type is not JSON-serializable:
+
+```python
+from haystack import component
+
+
+@component
+class SetIntersector:
+    def __init__(self, intersect_with: set):
+        self.intersect_with = intersect_with
+
+    @component.output_types(result=set)
+    def run(self, data: set):
+        return data.intersection(self.intersect_with)
+```
+
+In such cases, you can provide your own implementation  `from_dict` and `to_dict` to components:
+
+```python
+from haystack import component, default_from_dict, default_to_dict
+
+
+class SetIntersector:
+    def __init__(self, intersect_with: set):
+        self.intersect_with = intersect_with
+
+    @component.output_types(result=set)
+    def run(self, data: set):
+        return data.intersect(self.intersect_with)
+
+    def to_dict(self):
+        return default_to_dict(self, intersect_with=list(self.intersect_with))
+
+    @classmethod
+    def from_dict(cls, data):
+        # convert the set into a list for the dict representation,
+        # so it can be converted to JSON
+        data["intersect_with"] = set(data["intersect_with"])
+        return default_from_dict(cls, data)
+```
+
+## Saving a Pipeline to a Custom Format
+
+Once a pipeline is available in its dictionary format, the last step of serialization is to convert that dictionary into a format you can store or send over the wire. Haystack supports YAML out of the box, but if you need a different format, you can write a custom Marshaller.
+
+A `Marshaller` is a Python class responsible for converting text to a dictionary and a dictionary to text according to a certain format. Marshallers must respect the `Marshaller` [protocol](https://github.com/deepset-ai/haystack/blob/main/haystack/marshal/protocol.py), providing the methods `marshal` and `unmarshal`.
+
+This is the code for a custom TOML marshaller that relies on the `rtoml` library:
+
+```python
+# This code requires a `pip install rtoml`
+from typing import Dict, Any, Union
+import rtoml
+
+
+class TomlMarshaller:
+    def marshal(self, dict_: Dict[str, Any]) -> str:
+        return rtoml.dumps(dict_)
+
+    def unmarshal(self, data_: Union[str, bytes]) -> Dict[str, Any]:
+        return dict(rtoml.loads(data_))
+```
+
+You can then pass a Marshaller instance to the methods `dump`, `dumps`, `load`, and `loads`:
+
+```python
+from haystack import Pipeline
+from my_custom_marshallers import TomlMarshaller
+
+pipe = Pipeline()
+pipe.dumps(TomlMarshaller())
+# prints:
+# 'max_runs_per_component = 100\nconnections = []\n\n[metadata]\n\n[components]\n'
+```
+
+## Additional References
+
+:notebook: Tutorial: [Serializing LLM Pipelines](https://haystack.deepset.ai/tutorials/29_serializing_pipelines)

docs-website/docs/concepts/pipelines/smart-pipeline-connections.mdx

@@ -0,0 +1,148 @@
+---
+title: "Smart Pipeline Connections"
+id: smart-pipeline-connections
+slug: "/smart-pipeline-connections"
+description: "Learn how Haystack pipelines simplify connections through implicit joining and flexible type adaptation, reducing the need for glue components."
+---
+
+# Smart Pipeline Connections
+
+Haystack pipelines support smarter connection semantics that reduce boilerplate and make pipeline definitions easier to read and maintain.
+These features focus on simplifying how components are connected, without changing component behavior.
+
+Smart connections help eliminate common glue components such as `Joiners` and `OutputAdapters` in many pipelines.
+
+## Implicit List Joining
+
+Pipelines natively support connecting multiple component outputs directly to a single component input, without requiring an explicit `Joiner` component.
+
+This works when:
+
+* The target input is typed as `list`, `list | None`, or a union of list types (e.g. `list[int] | list[str]`).
+* All connected outputs are compatible list types.
+
+When multiple outputs are connected to the same input, the pipeline implicitly concatenates the lists from the outputs into a single list for the input.
+
+### Example
+
+Multiple converters can write directly into a single `DocumentWriter` without using a `DocumentJoiner`:
+
+<details>
+
+<summary>Expand to see the pipeline graph</summary>
+<ClickableImage src="/img/pipeline-illustration-auto-joiner.png" alt="Pipeline architecture diagram showing a DocumentWriter receiving inputs from multiple converters without a Joiner" size="large" />
+
+</details>
+
+```python
+from haystack import Pipeline
+from haystack.components.converters import HTMLToDocument, TextFileToDocument
+from haystack.components.routers import FileTypeRouter
+from haystack.components.writers import DocumentWriter
+from haystack.dataclasses import ByteStream
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+
+sources = [
+    ByteStream.from_string(text="Text file content", mime_type="text/plain"),
+    ByteStream.from_string(
+        text="<html><body>Some content</body></html>",
+        mime_type="text/html",
+    ),
+]
+
+doc_store = InMemoryDocumentStore()
+
+pipe = Pipeline()
+pipe.add_component("router", FileTypeRouter(mime_types=["text/plain", "text/html"]))
+pipe.add_component("txt_converter", TextFileToDocument())
+pipe.add_component("html_converter", HTMLToDocument())
+pipe.add_component("writer", DocumentWriter(doc_store))
+pipe.connect("router.text/plain", "txt_converter.sources")
+pipe.connect("router.text/html", "html_converter.sources")
+pipe.connect("txt_converter.documents", "writer.documents")
+pipe.connect("html_converter.documents", "writer.documents")
+
+result = pipe.run({"router": {"sources": sources}})
+```
+
+This pattern is especially useful when routing files, documents, or results across multiple parallel branches.
+
+## Flexible Type Connections
+
+To further streamline pipeline definitions, Haystack pipelines support limited implicit type adaptation at connection time.
+This makes pipeline connections more flexible and reduces the need for `OutputAdapter` components.
+
+**Supported adaptations**
+
+| Source Type              | Target Type          | Behavior                                                      |
+|--------------------------|--------------------|---------------------------------------------------------------|
+| `str`                    | `ChatMessage`       | Wrapped into a `ChatMessage` with user role.                 |
+| `ChatMessage`            | `str`               | Extracts `ChatMessage.text`; raises `PipelineRuntimeError` if `None`. |
+| `T`                      | `list[T]`           | Wraps the item into a single-element list.                   |
+| `list[str] or list[ChatMessage]`| `str` or `ChatMessage` | Extracts the first item; raises `PipelineRuntimeError` if the list is empty. |
+
+
+All adaptations are checked at connection time to ensure type safety, but applied at runtime during pipeline execution.
+
+When multiple connections are possible, strict type matching is prioritized over implicit conversion.
+This preserves backward compatibility with earlier versions of Haystack, where flexible type connections were not supported.
+
+### Example
+
+Pipeline connecting the Chat Generator `messages` output (`list[ChatMessage]`) to the retriever `query` input (`str`)
+without using an `OutputAdapter`:
+
+```python
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.dataclasses import Document
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+
+document_store = InMemoryDocumentStore()
+
+documents = [
+    Document(content="Bob lives in Paris."),
+    Document(content="Alice lives in London."),
+    Document(content="Ivy lives in Melbourne."),
+    Document(content="Kate lives in Brisbane."),
+    Document(content="Liam lives in Adelaide."),
+]
+
+document_store.write_documents(documents)
+
+template = """{% message role="user" %}
+Rewrite the following query to be used for keyword search.
+{{ query }}
+{% endmessage %}
+"""
+
+p = Pipeline()
+p.add_component("prompt_builder", ChatPromptBuilder(template=template))
+p.add_component("llm", OpenAIChatGenerator(model="gpt-4.1-mini"))
+p.add_component(
+    "retriever",
+    InMemoryBM25Retriever(document_store=document_store, top_k=3),
+)
+
+p.connect("prompt_builder", "llm")
+# implicitly converts list[ChatMessage] -> str
+p.connect("llm", "retriever")
+
+query = """Someday I'd love to visit Brisbane, but for now I just want
+to know the names of the people who live there."""
+
+result = p.run(data={"prompt_builder": {"query": query}})
+```
+
+## When You Still Need `Joiners` or `OutputAdapters`
+
+Explicit `Joiners` or `OutputAdapters` are still useful when you need:
+
+- Custom aggregation logic beyond simple list concatenation
+- Type conversions not covered by implicit adaptation
+- Explicit control over formatting or ordering
+
+Smart connections reduce the need for glue components, but they do not remove them entirely.
+When in doubt, explicit components provide clarity and more control.

docs-website/docs/concepts/pipelines/visualizing-pipelines.mdx

@@ -0,0 +1,88 @@
+---
+title: "Visualizing Haystack Pipelines"
+id: visualizing-pipelines
+slug: "/visualizing-pipelines"
+description: "You can visualize your Haystack AI pipelines as graphs to better understand how the components are connected."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Visualizing Haystack Pipelines
+
+You can visualize your pipelines as graphs to better understand how the components are connected.
+
+Haystack pipelines have  `draw()` and `show()` methods that enable you to visualize the pipeline as a graph using Mermaid graphs.
+
+:::note[Data Privacy Notice]
+
+Exercise caution with sensitive data when using pipeline visualization.
+
+This feature is based on Mermaid graphs web service that doesn't have clear terms of data retention or privacy policy.
+:::
+
+## Prerequisites
+
+To use Mermaid graphs, you must have an internet connection to reach the Mermaid graph renderer at https://mermaid.ink.
+
+## Displaying a Graph
+
+Use the pipeline's `show()` method to display the diagram in Jupyter notebooks.
+
+```python
+my_pipeline.show()
+```
+
+## Saving a Graph
+
+Use the pipeline's `draw()` method passing the path where you want to save the diagram and the diagram format. Possible formats are:  `mermaid-text` and `mermaid-image` (default).
+
+```python
+my_pipeline.draw(path=local_path)
+```
+
+## Visualizing SuperComponents
+
+To show the internal structure of [SuperComponents](../components/supercomponents.mdx) in your digram instead of a black box component, set the `super_component_expansion` parameter to `True`:
+
+```python
+my_pipeline.show(super_component_expansion=True)
+
+# or
+
+my_pipeline.draw(path=local_path, super_component_expansion=True)
+```
+
+## Visualizing Locally
+
+If you don't have an internet connection or don't want to send your pipeline data to the remote https://mermaid.ink, you can install a local mermaid.ink server and use it to render your pipeline.
+
+Let's run a local mermaid.ink server using their official Docker images from https://github.com/jihchi/mermaid.ink/pkgs/container/mermaid.ink.
+
+In this case, let's install one for a system running a MacOS M3 chip and expose it on port 3000:
+
+```dockerfile
+docker run --platform linux/amd64 --publish 3000:3000 --cap-add=SYS_ADMIN ghcr.io/jihchi/mermaid.ink
+```
+
+Check that the local mermaid.ink server is running by going to http://localhost:3000/.
+
+You should see a local server running, and now you can simply render the image using your local mermaid.ink server by specifying the URL when calling the`show()`or `draw()`  method:
+
+```python
+my_pipeline.show(server_url="http://localhost:3000")
+# or
+my_pipeline.draw("my_pipeline.png", server_url="http://localhost:3000")
+```
+
+## Example
+
+This is an example of what a pipeline graph may look like:
+<ClickableImage src="/img/46a8989-Untitled.png" alt="RAG pipeline flowchart showing the data flow from query through retriever, prompt builder, language model, and answer builder components" size="large" />
+
+<br />
+
+## Importing a Pipeline to Haystack Enterprise Platform
+
+You can import your Haystack pipeline into Haystack Enterprise Platform and continue visually building your pipeline
+
+To do that, follow the steps described in Haystack Enterprise Platform [documentation](https://docs.cloud.deepset.ai/docs/import-a-pipeline#import-your-pipeline).

docs-website/docs/concepts/secret-management.mdx

@@ -0,0 +1,202 @@
+---
+title: "Secret Management"
+id: secret-management
+slug: "/secret-management"
+description: "This page emphasizes secret management in Haystack components and introduces the `Secret` type for structured secret handling. It explains the drawbacks of hard-coding secrets in code and suggests using environment variables instead."
+---
+
+# Secret Management
+
+This page emphasizes secret management in Haystack components and introduces the `Secret` type for structured secret handling. It explains the drawbacks of hard-coding secrets in code and suggests using environment variables instead.
+
+Many Haystack components interact with third-party frameworks and service providers such as Azure, Google Vertex AI, and OpenAI. Their libraries often require the user to authenticate themselves to ensure they receive access to the underlying product. The authentication process usually works with a secret value that acts as an opaque identifier to the third-party backend.
+
+This page describes the two main types of secrets: token-based and environment variable-based, and how to handle them when using Haystack.
+
+You can find additional details for the `Secret` class in our [API reference](/reference/utils-api).
+
+<details>
+
+<summary>Example Use Case - Problem Statement</summary>
+
+### Problem Statement
+
+Let’s consider an example RAG pipeline that embeds a query, uses a Retriever component to locate documents relevant to the query, and then leverages an LLM to generate an answer based on the retrieved documents.
+
+The `OpenAIChatGenerator` component used in the pipeline below expects an API key to authenticate with OpenAI’s servers and perform the generation. Let’s assume that the component accepts a `str`  value for it:
+
+```python
+generator = OpenAIChatGenerator(api_key="sk-xxxxxxxxxxxxxxxxxx")
+pipeline.add_component("generator", generator)
+```
+
+This works in a pinch, but this is bad practice - we shouldn’t hard-code such secrets in the codebase. An alternative would be to store the key in an environment variable externally, read from it in Python, and pass that to the component:
+
+```python
+import os
+
+api_key = os.environ.get("OPENAI_API_KEY")
+generator = OpenAIChatGenerator(api_key=api_key)
+pipeline.add_component("generator", generator)
+```
+
+This is better – the pipeline works as intended, and we aren’t hard-coding any secrets in the code.
+
+Remember that pipelines are serializable. Since the API key is a secret, we should definitely avoid saving it to disk. Let’s modify the component’s `to_dict` method to exclude the key:
+
+```python
+def to_dict(self) -> Dict[str, Any]:
+    # Do not pass the `api_key` init parameter.
+    return default_to_dict(self, model=self.model)
+```
+
+But what happens when the pipeline is loaded from disk? In the best-case scenario, the component’s backend will automatically try to read the key from a hard-coded environment variable, and that key is the same as the one that was passed to the component before it was serialized. But in a worse case, the backend doesn’t look up the key in a hard-coded environment variable and fails when it gets called inside a `pipeline.run()`  invocation.
+
+</details>
+
+### Import
+
+To use Haystack secrets within the code, first import with:
+
+```python
+from haystack.utils import Secret
+```
+
+### Token-Based Secrets
+
+You can paste tokens directly as a string using the `from_token` method:
+
+```python
+llm = OpenAIChatGenerator(api_key=Secret.from_token("sk-randomAPIkeyasdsa32ekasd32e"))
+```
+
+Note that this type of code cannot be serialized, meaning you can't convert the above component to a dictionary or save a pipeline containing it to a YAML file. This is a security feature to prevent accidental exposure of sensitive data.
+
+### Environment Variable-Based Secrets
+
+Environment variable-based secrets are more flexible. They allow you to specify one or more environment variables that may contain your secret.
+
+Existing Haystack components that require an API Key (like OpenAIChatGenerator) have a default value for `Secret.from_env_var` (in this case, `OPENAI_API_KEY`). This means that the `OpenAIChatGenerator` will look for the value of the environment variable `OPENAI_API_KEY` (if it exists) and use it for authentication. And when pipelines are serialized to YAML, only the name of the environment variable is save to the YAML file. In doing so, this method ensures that there are no security leaks and is therefore strongly recommended.
+
+```bash
+## First, export an environment variable name `OPENAI_API_KEY` with its value
+export OPENAI_API_KEY=sk-randomAPIkeyasdsa32ekasd32e
+
+## or alternatively, using Python
+## import os
+## os.environ[”OPENAI_API_KEY”]=sk-randomAPIkeyasdsa32ekasd32e
+```
+
+```python
+llm_generator = (
+    OpenAIChatGenerator()
+)  # Uses the default value from the env var for the component
+```
+
+Alternatively, in components where a Secret is expected, you can customize the name of the environment variable from which the API Key is to be read.
+
+```python
+# Export an environment variable with custom name and its value
+llm_generator = OpenAIChatGenerator(api_key=Secret.from_env_var("YOUR_ENV_VAR"))
+```
+
+When `OpenAIChatGenerator` is serialized within a pipeline, this is what the YAML code will look like, using the custom variable name:
+
+```yaml
+components:
+  llm:
+    init_parameters:
+      api_base_url: null
+      api_key:
+        env_vars:
+        - YOUR_ENV_VAR
+        strict: true
+        type: env_var
+      generation_kwargs: {}
+      http_client_kwargs: null
+      max_retries: null
+      model: gpt-5-mini
+      organization: null
+      streaming_callback: null
+      timeout: null
+      tools: null
+      tools_strict: false
+    type: haystack.components.generators.chat.openai.OpenAIChatGenerator
+    ...
+```
+
+### Serialization
+
+While token-based secrets cannot be serialized, environment variable-based secrets can be converted to and from dictionaries:
+
+```python
+# Convert to dictionary
+env_secret_dict = env_secret.to_dict()
+
+# Create from dictionary
+new_env_secret = Secret.from_dict(env_secret_dict)
+```
+
+### Resolving Secrets
+
+Both types of secrets can be resolved to their actual values using the `resolve_value` method. This method returns the token or the value of the environment variable.
+
+```python
+# Resolve the token-based secret
+token_value = api_key_secret.resolve_value()
+
+# Resolve the environment variable-based secret
+env_value = env_secret.resolve_value()
+```
+
+### Custom Component Example
+
+Here is a complete example that shows how to create a component that uses the `Secret` class in Haystack, highlighting the differences between token-based and environment variable-based authentication, and showing that token-based secrets cannot be serialized:
+
+```python
+from haystack.utils import Secret, deserialize_secrets_inplace
+
+
+@component
+class MyComponent:
+    def __init__(self, api_key: Optional[Secret] = None, **kwargs):
+        self.api_key = api_key
+        self.backend = None
+
+    def warm_up(self):
+        # Call resolve_value to yield a single result. The semantics of the result is policy-dependent.
+        # Currently, all supported policies will return a single string token.
+        self.backend = SomeBackend(
+            api_key=self.api_key.resolve_value() if self.api_key else None,  # ...
+        )
+
+    def to_dict(self):
+        # Serialize the policy like any other (custom) data. If the policy is token-based, it will
+        # raise an error.
+        return default_to_dict(
+            self,
+            api_key=self.api_key.to_dict() if self.api_key else None,  # ...
+        )
+
+    @classmethod
+    def from_dict(cls, data):
+        # Deserialize the policy data before passing it to the generic from_dict function.
+        api_key_data = data["init_parameters"]["api_key"]
+        api_key = Secret.from_dict(api_key_data) if api_key_data is not None else None
+        data["init_parameters"]["api_key"] = api_key
+        # Alternatively, use the helper function.
+        # deserialize_secrets_inplace(data["init_parameters"], keys=["api_key"])
+        return default_from_dict(cls, data)
+
+
+# No authentication.
+component = MyComponent(api_key=None)
+
+# Token based authentication
+component = MyComponent(api_key=Secret.from_token("sk-randomAPIkeyasdsa32ekasd32e"))
+component.to_dict()  # Error! Can't serialize authentication tokens
+
+# Environment variable based authentication
+component = MyComponent(api_key=Secret.from_env_var("OPENAI_API_KEY"))
+component.to_dict()  # This is fine
+```

docs-website/docs/development/deployment.mdx

@@ -0,0 +1,36 @@
+---
+title: "Deployment"
+id: deployment
+slug: "/deployment"
+description: "Deploy your Haystack pipelines through various services such as Docker, Kubernetes, Ray, or a variety of Serverless options."
+---
+
+# Deployment
+
+Deploy your Haystack pipelines through various services such as Docker, Kubernetes, Ray, or a variety of Serverless options.
+
+As a framework, Haystack is typically integrated into a variety of applications and environments, and there is no single, specific deployment strategy to follow. However, it is very common to make Haystack pipelines accessible through a service that can be easily called from other software systems.
+
+These guides focus on tools and techniques that can be used to run Haystack pipelines in common scenarios. While these suggestions should not be considered the only way to do so, they should provide inspiration and the ability to customize them according to your needs.
+
+### Guides
+
+Here are the currently available guides on Haystack pipeline deployment:
+
+- [Deploying with Docker](deployment/docker.mdx)
+- [Deploying with Kubernetes](deployment/kubernetes.mdx)
+- [Deploying with OpenShift](deployment/openshift.mdx)
+
+### Hayhooks
+
+Haystack can be easily integrated into any HTTP application, but if you don’t have one, you can use Hayhooks, a ready-made application that serves Haystack pipelines as REST endpoints. We’ll be using Hayhooks throughout this guide to streamline the code examples. Refer to the Hayhooks [overview](hayhooks.mdx) for a quick start, or the [official Hayhooks documentation](https://deepset-ai.github.io/hayhooks/) for comprehensive guides and reference.
+
+:::note[Looking to scale with confidence?]
+
+If your team needs **enterprise-grade support, best practices, and deployment guidance** to run Haystack in production, check out **Haystack Enterprise Starter**.
+
+📜 [Learn more about Haystack Enterprise Starter](https://haystack.deepset.ai/blog/announcing-haystack-enterprise)  
+🤝 [Get in touch with our team](https://www.deepset.ai/products-and-services/haystack-enterprise-starter)
+
+👉 For platform tooling to **manage data, pipelines, testing, and governance at scale**, explore the [Haystack Enterprise Platform](https://www.deepset.ai/products-and-services/haystack-enterprise-platform).
+:::

docs-website/docs/development/deployment/docker.mdx

@@ -0,0 +1,117 @@
+---
+title: "Docker"
+id: docker
+slug: "/docker"
+description: "Learn how to deploy your Haystack pipelines through Docker starting from the basic Docker container to a complex application using Hayhooks."
+---
+
+# Docker
+
+Learn how to deploy your Haystack pipelines through Docker starting from the basic Docker container to a complex application using Hayhooks.
+
+## Running Haystack in Docker
+
+The most basic form of Haystack deployment happens through Docker containers. Becoming familiar with running and customizing Haystack Docker images is useful as they form the basis for more advanced deployment.
+
+Haystack releases are officially distributed through the [`deepset/haystack`](https://hub.docker.com/r/deepset/haystack) Docker image. Haystack images come in different flavors depending on the specific components they ship and the Haystack version.
+
+:::info
+At the moment, the only flavor available for Haystack is `base`, which ships exactly what you would get by installing Haystack locally with `pip install haystack-ai`.
+:::
+
+You can pull a specific Haystack flavor using Docker tags: for example, to pull the image containing Haystack `2.12.1`, you can run the command:
+
+```shell
+docker pull deepset/haystack:base-v2.12.1
+```
+
+Although the `base` flavor is meant to be customized, it can also be used to quickly run Haystack scripts locally without the need to set up a Python environment and its dependencies. For example, this is how you would print Haystack’s version running a Docker container:
+
+```shell
+docker run -it --rm deepset/haystack:base-v2.12.1 python -c"from haystack.version import __version__; print(__version__)"
+```
+
+## Customizing the Haystack Docker Image
+
+Chances are your application will be more complex than a simple script, and you’ll need to install additional dependencies inside the Docker image along with Haystack.
+
+For example, you might want to run a simple indexing pipeline using [Chroma](../../document-stores/chromadocumentstore.mdx) as your Document Store using a Docker container. The `base` image only contains a basic install of Haystack, but you need to install the Chroma integration (`chroma-haystack`) package additionally. The best approach would be to create a custom Docker image shipping the extra dependency.
+
+Assuming you have a `main.py` script in your current folder, the Dockerfile would look like this:
+
+```shell
+FROM deepset/haystack:base-v2.12.1
+
+RUN pip install chroma-haystack
+
+COPY ./main.py /usr/src/myapp/main.py
+
+ENTRYPOINT ["python", "/usr/src/myapp/main.py"]
+```
+
+Then you can create your custom Haystack image with:
+
+```shell
+docker build . -t my-haystack-image
+```
+
+## Complex Application with Docker Compose
+
+A Haystack application running in Docker can go pretty far: with an internet connection, the container can reach external services providing vector databases, inference endpoints, and observability features.
+
+Still, you might want to orchestrate additional services for your Haystack container locally, for example, to reduce costs or increase performance. When your application runtime depends on more than one Docker container, [Docker Compose](https://docs.docker.com/compose/) is a great tool to keep everything together.
+
+As an example, let’s say your application wraps two pipelines: one to _index_ documents into a Qdrant instance and the other to _query_ those documents at a later time. This setup would require two Docker containers: one to run the pipelines as REST APIs using [Hayhooks](../hayhooks.mdx) and a second to run a Qdrant instance. For more information on configuring Hayhooks using Docker Compose, see the [official Hayhooks documentation](https://deepset-ai.github.io/hayhooks/getting-started/quick-start-docker/).
+
+For building the Hayhooks image, we can easily customize the base image of one of the latest versions of Hayhooks, adding required dependencies required by [`QdrantDocumentStore`](../../document-stores/qdrant-document-store.mdx). The Dockerfile would look like this:
+
+```dockerfile Dockerfile
+FROM deepset/hayhooks:v1.16.0
+
+RUN pip install qdrant-haystack sentence-transformers
+
+CMD ["hayhooks", "run", "--host", "0.0.0.0"]
+
+```
+
+We wouldn’t need to customize Qdrant, so their official Docker image would work perfectly. The `docker-compose.yml` file would then look like this:
+
+```yaml
+services:
+  qdrant:
+    image: qdrant/qdrant:latest
+    restart: always
+    container_name: qdrant
+    ports:
+      - 6333:6333
+      - 6334:6334
+    expose:
+      - 6333
+      - 6334
+      - 6335
+    configs:
+      - source: qdrant_config
+        target: /qdrant/config/production.yaml
+    volumes:
+      - ./qdrant_data:/qdrant_data
+
+  hayhooks:
+    build: . # Build from local Dockerfile
+    container_name: hayhooks
+    ports:
+      - "1416:1416"
+    volumes:
+      - ./pipelines:/pipelines
+    environment:
+      - HAYHOOKS_PIPELINES_DIR=/pipelines
+      - LOG=DEBUG
+    depends_on:
+      - qdrant
+
+configs:
+  qdrant_config:
+    content: |
+      log_level: INFO
+```
+
+For a functional example of a Docker Compose deployment, check out the [“RAG indexing and querying with Elasticsearch”](https://github.com/deepset-ai/hayhooks/tree/main/examples/rag_indexing_query) example from GitHub.

docs-website/docs/development/deployment/kubernetes.mdx

@@ -0,0 +1,269 @@
+---
+title: "Kubernetes"
+id: kubernetes
+slug: "/kubernetes"
+description: "Learn how to deploy your Haystack pipelines through Kubernetes."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Kubernetes
+
+Learn how to deploy your Haystack pipelines through Kubernetes.
+
+The best way to get Haystack running as a workload in a container orchestrator like Kubernetes is to create a service to expose one or more [Hayhooks](../hayhooks.mdx) instances.
+
+## Create a Haystack Kubernetes Service using Hayhooks
+
+As a first step, we recommend to create a local [KinD](https://github.com/kubernetes-sigs/kind) or [Minikube](https://github.com/kubernetes/minikube) Kubernetes cluster. You can manage your cluster from CLI, but tools like [k9s](https://k9scli.io/) or [Lens](https://k8slens.dev/) can ease the process.
+
+When done, start with a very simple Kubernetes Service running a single Hayhooks Pod:
+
+```yaml
+kind: Pod
+apiVersion: v1
+metadata:
+  name: hayhooks
+  labels:
+    app: haystack
+spec:
+  containers:
+    - image: deepset/hayhooks:v1.16.0
+      name: hayhooks
+      imagePullPolicy: IfNotPresent
+      resources:
+        limits:
+          memory: "512Mi"
+          cpu: "500m"
+        requests:
+          memory: "256Mi"
+          cpu: "250m"
+
+---
+
+kind: Service
+apiVersion: v1
+metadata:
+  name: haystack-service
+spec:
+  selector:
+    app: haystack
+  type: ClusterIP
+  ports:
+    # Default port used by the Hayhooks Docker image
+    - port: 1416
+
+```
+
+After applying the above to an existing Kubernetes cluster, a `hayhooks` Pod will show up as a Service called `haystack-service`.
+<ClickableImage src="/img/6eb9fb0c7b00367bfbe8182ffc7c3746f3f3d03b720e963df045e28160362d7f-Screenshot_2025-04-15_at_16.15.28.png" alt="Kubernetes Lens interface showing the hayhooks Pod running in the default namespace with status Running" />
+
+Note that the `Service` defined above is of type `ClusterIP`. That means it's exposed only _inside_ the Kubernetes cluster. To expose the Hayhooks API to the _outside_ world as well, you need a `NodePort` or `Ingress` resource. As an alternative, it's also possible to use [Port Forwarding](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/) to access the `Service` locally.
+
+To do that, add port `30080` to Host-To-Node Mapping of our KinD cluster. In other words, make sure that the cluster is created with a node configuration similar to the following:
+
+```yaml
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+nodes:
+  - role: control-plane
+    # ...
+    extraPortMappings:
+      - containerPort: 30080
+        hostPort: 30080
+        protocol: TCP
+```
+
+Then, create a simple `NodePort`  to test if Hayhooks Pod is running correctly:
+
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+  name: haystack-nodeport
+spec:
+  selector:
+    app: haystack
+  type: NodePort
+  ports:
+  - port: 1416
+    targetPort: 1416
+    nodePort: 30080
+    name: http
+```
+
+After applying this, `hayhooks` Pod will be accessible on `localhost:30080`.
+
+From here, you should be able to manage pipelines. Remember that it's possible to deploy multiple different pipelines on a single Hayhooks instance. Check the [Hayhooks overview](../hayhooks.mdx) or the [official Hayhooks documentation](https://deepset-ai.github.io/hayhooks/) for more details.
+
+## Auto-Run Pipelines at Pod Start
+
+Hayhooks can load Haystack pipelines at startup, making them readily available when the server starts. You can leverage this mechanism to have your pods immediately serve one or more pipelines when they start.
+
+At startup, it will look for deployed pipelines on the path specified at `HAYHOOKS_PIPELINES_DIR`, then load them.
+
+A [deployed pipeline](https://github.com/deepset-ai/hayhooks?tab=readme-ov-file#deploy-a-pipeline) is essentially a directory which must contain a `pipeline_wrapper.py` file and possibly other files. To preload an [example pipeline](https://github.com/deepset-ai/hayhooks/tree/main/examples/pipeline_wrappers/chat_with_website), you need to mount a local folder inside the cluster node, then make it available on Hayhooks Pod as well.
+
+First, ensure that a local folder is mounted correctly on the KinD cluster node at `/data`:
+
+```yaml
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+nodes:
+  - role: control-plane
+    # ...
+    extraMounts:
+      - hostPath: /path/to/local/pipelines/folder
+        containerPath: /data
+```
+
+Next, make `/data` available as a volume and mount it on Hayhooks Pod. To do that, update your previous Pod configuration to the following:
+
+```yaml
+kind: Pod
+apiVersion: v1
+metadata:
+  name: hayhooks
+  labels:
+    app: haystack
+spec:
+  containers:
+    - image: deepset/hayhooks:v1.16.0
+      name: hayhooks
+      imagePullPolicy: IfNotPresent
+      command: ["/bin/sh", "-c"]
+      args:
+        - |
+          pip install trafilatura && \
+          hayhooks run --host 0.0.0.0
+      volumeMounts:
+        - name: local-data
+          mountPath: /mnt/data
+      env:
+        - name: HAYHOOKS_PIPELINES_DIR
+          value: /mnt/data
+        - name: OPENAI_API_KEY
+          valueFrom:
+            secretKeyRef:
+              name: openai-secret
+              key: api-key
+      resources:
+        limits:
+          memory: "512Mi"
+          cpu: "500m"
+        requests:
+          memory: "256Mi"
+          cpu: "250m"
+  volumes:
+    - name: local-data
+      hostPath:
+        path: /data
+        type: Directory
+
+```
+
+Note that:
+
+- We changed the Hayhooks container `command` to install the `trafilatura` dependency before startup, since it's needed for our [chat_with_website](https://github.com/deepset-ai/hayhooks/tree/main/examples/pipeline_wrappers/chat_with_website) example pipeline. For a real production environment, we recommend creating a custom Hayhooks image as described [here](docker.mdx#customizing-the-haystack-docker-image).
+- We make Hayhooks container read `OPENAI_API_KEY` from a Kubernetes Secret.
+
+Before applying this new configuration, create the `openai-secret`:
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openai-secret
+type: Opaque
+data:
+  # Replace the placeholder below with the base64 encoded value of your API key
+  # Generate it using: echo -n $OPENAI_API_KEY | base64
+  api-key: YOUR_BASE64_ENCODED_API_KEY_HERE
+```
+
+After applying this, check your Hayhooks Pod logs, and you'll see that the `chat_with_website` pipelines have already been deployed.
+<ClickableImage src="/img/2dbf42dd2db1cb355ee7222d7f8e96c45b611200d83ca289be3456264a854c38-Screenshot_2025-04-16_at_09.19.14.png" alt="Kubernetes Lens interface displaying pod logs with application startup messages and deployed pipeline confirmation" />
+
+## Roll Out Multiple Pods
+
+Haystack pipelines are usually stateless, which is a perfect use case for distributing the requests to multiple pods running the same set of pipelines. Let's convert the single-Pod configuration to an actual Kubernetes `Deployment`:
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: haystack-deployment
+spec:
+  replicas: 3
+  selector:
+    matchLabels:
+      app: haystack
+  template:
+    metadata:
+      labels:
+        app: haystack
+    spec:
+      initContainers:
+        - name: install-dependencies
+          image: python:3.12-slim
+          workingDir: /mnt/data
+          command: ["/bin/bash", "-c"]
+          args:
+            - |
+              echo "Installing dependencies..."
+              pip install trafilatura
+              echo "Dependencies installed successfully!"
+              touch /mnt/data/init-complete
+          volumeMounts:
+            - name: local-data
+              mountPath: /mnt/data
+          resources:
+            requests:
+              memory: "64Mi"
+              cpu: "100m"
+            limits:
+              memory: "128Mi"
+              cpu: "250m"
+      containers:
+        - image: deepset/hayhooks:v1.16.0
+          name: hayhooks
+          imagePullPolicy: IfNotPresent
+          command: ["/bin/sh", "-c"]
+          args:
+            - |
+              pip install trafilatura && \
+              hayhooks run --host 0.0.0.0
+          ports:
+            - containerPort: 1416
+              name: http
+          volumeMounts:
+            - name: local-data
+              mountPath: /mnt/data
+          env:
+            - name: HAYHOOKS_PIPELINES_DIR
+              value: /mnt/data
+            - name: OPENAI_API_KEY
+              valueFrom:
+                secretKeyRef:
+                  name: openai-secret
+                  key: api-key
+          resources:
+            requests:
+              memory: "256Mi"
+              cpu: "250m"
+            limits:
+              memory: "512Mi"
+              cpu: "500m"
+      volumes:
+        - name: local-data
+          hostPath:
+            path: /data
+            type: Directory
+
+```
+
+Implementing the above configuration will create three pods. Each pod will run a different instance of Hayhooks, all serving the same example pipeline provided by the mounted volume in the previous example.
+
+<ClickableImage src="/img/f3f0ac4b22a37039f0837c22b0cb8b640937bbb0db4acfcbdf7bd016b545d84a-Screenshot_2025-04-16_at_09.32.07.png" alt="Kubernetes Lens interface showing three haystack-deployment pods in Running status with their resource configurations" />
+
+Note that the `NodePort` you created before will now act as a load balancer and will distribute incoming requests to the three Hayhooks Pods.

docs-website/docs/development/deployment/openshift.mdx

@@ -0,0 +1,73 @@
+---
+title: "OpenShift"
+id: openshift
+slug: "/openshift"
+description: "Learn how to deploy your applications running Haystack pipelines using OpenShift."
+---
+
+# OpenShift
+
+Learn how to deploy your applications running Haystack pipelines using OpenShift.
+
+## Introduction
+
+OpenShift by Red Hat is a platform that helps create and manage applications built on top of Kubernetes. It can be used to build, update, launch, and oversee applications running Haystack pipelines. A [developer sandbox](https://developers.redhat.com/developer-sandbox) is available, ideal for getting familiar with the platform and building prototypes that can be smoothly moved to production using a public cloud, private network, hybrid cloud, or edge computing.
+
+## Prerequisites
+
+The fastest way to deploy a Haystack pipeline is to deploy an OpenShift application that runs Hayhooks. Before starting, make sure to have the following prerequisites:
+
+- Access to an OpenShift project. Follow RedHat's [instructions](https://developers.redhat.com/developer-sandbox) to create one and start experimenting immediately.
+- Hayhooks is installed. Run `pip install hayhooks` and make sure it works by running `hayhooks --version`. Read more about Hayhooks in our [overview](../hayhooks.mdx) or the [official Hayhooks documentation](https://deepset-ai.github.io/hayhooks/).
+- You can optionally install the OpenShift command-line utility `oc`. Follow the [installation instructions](https://docs.openshift.com/container-platform/4.15/cli_reference/openshift_cli/getting-started-cli.html) for your platform and make sure it works by running `oc—h`.
+
+## Creating a Hayhooks Application
+
+In this guide, we’ll be using the `oc` command line, but you can achieve the same by interacting with the user interface offered by the OpenShift console.
+
+1. The first step is to log into your OpenShift account using `oc`. From the top-right corner of your OpenShift console, click on your username and open the menu. Click **Copy login command** and follow the instructions.
+
+2. The console will show you the exact command to run in your terminal to log in. It’s something like the following:  
+   ```
+   oc login --token=<your-token> --server=https://<your-server-url>:6443
+   ```
+
+3. Assuming you already have a project (it’s the case for the developer sandbox), create an application running the Hayhooks Docker image available on Docker Hub:  
+   Note how you can pass environment variables that your application will use at runtime. In this case, we disable Haystack’s internal telemetry and set an OpenAI key that will be used by the pipelines we’ll eventually deploy in Hayhooks.
+   ```
+   oc new-app deepset/hayhooks:v1.16.0 -e HAYSTACK_TELEMETRY_ENABLED=false -e OPENAI_API_KEY=$OPENAI_API_KEY
+   ```
+
+4. To make sure you make the most out of OpenShift's ability to manage the lifecycle of the application, you can set a [liveness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/):  
+   ```
+   oc set probe deployment/hayhooks --liveness --get-url=http://:1416/status
+   ```
+
+5. Finally, you can expose our Hayhooks instance to the public Internet:  
+   ```
+   oc expose service/hayhooks
+   ```
+
+6. You can get the public address that was assigned to your application by running:  
+
+   ```
+   oc status
+   ```
+
+   In the output, look for something like this:
+
+   ```
+   In project <your-project-name> on server https://<your-server-url>:6443
+
+   http://hayhooks-XXX.openshiftapps.com to pod port 1416-tcp (svc/hayhooks)
+   ```
+
+7. `http://hayhooks-XXX.openshiftapps.com` will be the public URL serving your Hayhooks instance. At this point, you can query Hayhooks status by running:  
+   ```
+   HAYHOOKS_HOST=hayhooks-XXX.openshiftapps.com HAYHOOKS_PORT=80 hayhooks status
+   ```
+
+8. Lastly, deploy your pipeline as usual:  
+   ```
+   HAYHOOKS_HOST=hayhooks-XXX.openshiftapps.com HAYHOOKS_PORT=80 hayhooks pipeline deploy-files -n my_pipeline /path/to/my_pipeline_dir
+   ```

docs-website/docs/development/enabling-gpu-acceleration.mdx

@@ -0,0 +1,43 @@
+---
+title: "Enabling GPU Acceleration"
+id: enabling-gpu-acceleration
+slug: "/enabling-gpu-acceleration"
+description: "Speed up your Haystack application by engaging the GPU."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Enabling GPU Acceleration
+
+Speed up your Haystack application by engaging the GPU.
+
+The Transformer models used in Haystack are designed to be run on GPU-accelerated hardware. The steps for GPU acceleration setup depend on the environment that you're working in.
+
+Once you have GPU enabled on your machine, you can set the `device` on which a given model for a component is loaded.
+
+For example, to load a model for the `TransformersChatGenerator`, set `device=ComponentDevice.from_single(Device.gpu(id=0))` or `device = ComponentDevice.from_str("cuda:0")` when initializing.
+
+You can find more information on the [Device management](../concepts/device-management.mdx) page.
+
+### Enabling the GPU in Linux
+
+1. Ensure that you have a fitting version of NVIDIA CUDA installed. To learn how to install CUDA, see the [NVIDIA CUDA Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html).
+
+2. Run the `nvidia-smi`in the command line to check if the GPU is enabled. If the GPU is enabled, the output shows a list of available GPUs and their memory usage:
+   <ClickableImage src="/img/b44c7f4-gpu_enabled_cropped.png" alt="A screenshot of the command output with the name of the GPU device and its memory usage highlighted." />
+
+### Enabling the GPU in Colab
+
+1. In your Colab environment, select **Runtime>Change Runtime type**.
+<ClickableImage src="/img/85079c7-68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f646565707365742d61692f686179737461636b2f6d61696e2f646f63732f696d672f636f6c61625f6770755f72756e74696d652e6a7067.jpeg" alt="Google Colab Runtime menu with Change runtime type option highlighted for selecting GPU acceleration" size="large" />
+
+2. Choose **Hardware accelerator>GPU**.
+3. To check if the GPU is enabled, run:
+
+```python python
+%%bash
+
+nvidia-smi
+```
+
+The output should show the GPUs available and their usage.

docs-website/docs/development/external-integrations-development.mdx

@@ -0,0 +1,18 @@
+---
+title: "External Integrations"
+id: external-integrations-development
+slug: "/external-integrations-development"
+description: "External integrations that enable tracing, monitoring, and deploying your pipelines."
+---
+
+# External Integrations
+
+External integrations that enable tracing, monitoring, and deploying your pipelines.
+
+| Name | Description |
+| --- | --- |
+| [Arize Phoenix](https://haystack.deepset.ai/integrations/arize-phoenix) | Trace your pipelines with Arize Phoenix.                 |
+| [Arize AI](https://haystack.deepset.ai/integrations/arize)              | Trace and monitor your pipelines with Arize AI.          |
+| [Burr](https://haystack.deepset.ai/integrations/burr)                   | Build Burr agents using Haystack.                        |
+| [Context AI](https://haystack.deepset.ai/integrations/context-ai)       | Log conversations for analytics by Context.ai.           |
+| [Ray](https://haystack.deepset.ai/integrations/ray)                     | Run and scale your pipelines with in distributed manner. |

docs-website/docs/development/hayhooks.mdx

@@ -0,0 +1,196 @@
+---
+title: "Hayhooks"
+id: hayhooks
+slug: "/hayhooks"
+description: "Hayhooks is a web application you can use to serve Haystack pipelines through HTTP endpoints. This page provides an overview of the main features of Hayhooks."
+---
+
+# Hayhooks
+
+Hayhooks is a web application you can use to serve Haystack pipelines through HTTP endpoints. This page provides an overview of the main features of Hayhooks.
+
+:::info[Hayhooks Documentation]
+
+For comprehensive documentation, including detailed configuration reference, advanced features,
+and examples, see the [official Hayhooks documentation](https://deepset-ai.github.io/hayhooks/).
+
+The source code is available in the [Hayhooks GitHub repository](https://github.com/deepset-ai/hayhooks).
+:::
+
+## Overview
+
+Hayhooks simplifies the deployment of Haystack pipelines as REST APIs. It allows you to:
+
+- Expose Haystack pipelines as HTTP endpoints, including OpenAI-compatible chat endpoints,
+- Customize logic while keeping minimal boilerplate,
+- Deploy pipelines quickly and efficiently.
+
+### Installation
+
+Install Hayhooks using pip:
+
+```shell
+pip install hayhooks
+```
+
+The `hayhooks` package ships both the server and the client component, and the client is capable of starting the server. From a shell, start the server with:
+
+```shell
+$ hayhooks run
+INFO:     Started server process [44782]
+INFO:     Waiting for application startup.
+INFO:     Application startup complete.
+INFO:     Uvicorn running on http://localhost:1416 (Press CTRL+C to quit)
+```
+
+### Check Status
+
+From a different shell, you can query the status of the server with:
+
+```shell
+$ hayhooks status
+Hayhooks server is up and running.
+```
+
+## Configuration
+
+Hayhooks can be configured in three ways:
+
+1. Using an `.env` file in the project root.
+2. Passing environment variables when running the command.
+3. Using command-line arguments with `hayhooks run`.
+
+For a complete list of environment variables including server settings, CORS, SSL, logging, streaming, and Chainlit UI options, see the [Hayhooks environment variables reference](https://deepset-ai.github.io/hayhooks/reference/environment-variables/).
+
+## Running Hayhooks
+
+To start the server:
+
+```shell
+hayhooks run
+```
+
+This will launch Hayhooks at `HAYHOOKS_HOST:HAYHOOKS_PORT`.
+
+## Deploying a Pipeline
+
+### Steps
+
+1. Prepare a pipeline definition (`.yml` file) and a `pipeline_wrapper.py` file.
+2. Deploy the pipeline:
+
+   ```shell
+   hayhooks pipeline deploy-files -n my_pipeline my_pipeline_dir
+   ```
+3. Access the pipeline at `{pipeline_name}/run` endpoint.
+
+### Pipeline Wrapper
+
+A `PipelineWrapper` class is required to wrap the pipeline:
+
+```python
+from pathlib import Path
+from haystack import Pipeline
+from hayhooks import BasePipelineWrapper
+
+
+class PipelineWrapper(BasePipelineWrapper):
+    def setup(self) -> None:
+        pipeline_yaml = (Path(__file__).parent / "pipeline.yml").read_text()
+        self.pipeline = Pipeline.loads(pipeline_yaml)
+
+    def run_api(self, input_text: str) -> str:
+        result = self.pipeline.run({"input": {"text": input_text}})
+        return result["output"]["text"]
+```
+
+## File Uploads
+
+Hayhooks enables handling file uploads in your pipeline wrapper's `run_api` method by including `files: list[UploadFile] | None = None` as an argument.
+
+```python
+def run_api(self, files: list[UploadFile] | None = None) -> str:
+    if files and len(files) > 0:
+        filenames = [f.filename for f in files if f.filename is not None]
+        file_contents = [f.file.read() for f in files]
+        return f"Received files: {', '.join(filenames)}"
+    return "No files received"
+```
+
+Hayhooks automatically processes uploaded files and passes them to the `run_api` method when present. The HTTP request must be a `multipart/form-data` request. For more details on file uploads, including combining files with parameters, see the [official Hayhooks documentation](https://deepset-ai.github.io/hayhooks/features/file-upload-support/).
+
+## Running Pipelines from the CLI
+
+You can execute a pipeline through the command line using the `hayhooks pipeline run` command. Internally, this triggers the `run_api` method of the pipeline wrapper, passing parameters as a JSON payload.
+
+```shell
+hayhooks pipeline run <pipeline_name> --param 'question="Is this recipe vegan?"'
+```
+
+You can also upload files when running a pipeline:
+
+```shell
+hayhooks pipeline run <pipeline_name> --file file.pdf --param 'question="Is this recipe vegan?"'
+```
+
+For the full CLI reference, see the [Hayhooks CLI documentation](https://deepset-ai.github.io/hayhooks/features/cli-commands/).
+
+## MCP Support
+
+Hayhooks supports the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) and can act as an MCP Server. It automatically lists your deployed pipelines and agents as MCP Tools using Server-Sent Events (SSE) as the transport method. Agents are deployed using the same `PipelineWrapper` mechanism as pipelines.
+
+To start the Hayhooks MCP server, run:
+
+```shell
+hayhooks mcp run
+```
+
+For each deployed pipeline, Hayhooks uses the pipeline wrapper name as the MCP Tool name and generates the tool schema from the `run_api` method arguments. For details on configuring MCP tools, see the [Hayhooks MCP documentation](https://deepset-ai.github.io/hayhooks/features/mcp-support/).
+
+## OpenAI Compatibility
+
+Hayhooks supports OpenAI-compatible endpoints through the `run_chat_completion` method.
+
+```python
+from hayhooks import BasePipelineWrapper, get_last_user_message
+
+
+class PipelineWrapper(BasePipelineWrapper):
+    def run_chat_completion(self, model: str, messages: list, body: dict):
+        question = get_last_user_message(messages)
+        return self.pipeline.run({"query": question})
+```
+
+This makes Hayhooks pipelines compatible with any tool that supports the OpenAI chat completion API, including streaming responses. For details, see the [Hayhooks OpenAI compatibility documentation](https://deepset-ai.github.io/hayhooks/features/openai-compatibility/).
+
+## Running Programmatically
+
+Hayhooks can be embedded in a FastAPI application:
+
+```python
+import uvicorn
+from hayhooks.settings import settings
+from fastapi import Request
+from hayhooks import create_app
+
+# Create the Hayhooks app
+hayhooks = create_app()
+
+
+# Add a custom route
+@hayhooks.get("/custom")
+async def custom_route():
+    return {"message": "Hi, this is a custom route!"}
+
+
+# Add a custom middleware
+@hayhooks.middleware("http")
+async def custom_middleware(request: Request, call_next):
+    response = await call_next(request)
+    response.headers["X-Custom-Header"] = "custom-header-value"
+    return response
+
+
+if __name__ == "__main__":
+    uvicorn.run("app:hayhooks", host=settings.host, port=settings.port)
+```

docs-website/docs/development/logging.mdx

@@ -0,0 +1,105 @@
+---
+title: "Logging"
+id: logging
+slug: "/logging"
+description: "Logging is crucial for monitoring and debugging LLM applications during development as well as in production. Haystack provides different logging solutions out of the box to get you started quickly, depending on your use case."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Logging
+
+Logging is crucial for monitoring and debugging LLM applications during development as well as in production. Haystack provides different logging solutions out of the box to get you started quickly, depending on your use case.
+
+## Standard Library Logging (default)
+
+Haystack logs through Python’s standard library. This gives you full flexibility and customizability to adjust the log format according to your needs.
+
+### Changing the Log Level
+
+By default, Haystack's logging level is set to `WARNING`. To display more information, you can change it to `INFO`. This way, not only warnings but also information messages are displayed in the console output.
+
+To change the logging level to `INFO`, run:
+
+```python
+import logging
+
+logging.basicConfig(
+    format="%(levelname)s - %(name)s -  %(message)s",
+    level=logging.WARNING,
+)
+logging.getLogger("haystack").setLevel(logging.INFO)
+```
+
+#### Further Configuration
+
+See [Python’s documentation on logging](https://docs.python.org/3/howto/logging.html) for more advanced configuration.
+
+## Real-Time Pipeline Logging
+
+Use Haystack's [`LoggingTracer`](https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py) logs to inspect the data that's flowing through your pipeline in real-time.
+
+This feature is particularly helpful during experimentation and prototyping, as you don’t need to set up any tracing backend beforehand.
+
+Here’s how you can enable this tracer. In this example, we are adding color tags (this is optional) to highlight the components' names and inputs:
+
+```python
+import logging
+from haystack import tracing
+from haystack.tracing.logging_tracer import LoggingTracer
+
+logging.basicConfig(
+    format="%(levelname)s - %(name)s -  %(message)s",
+    level=logging.WARNING,
+)
+logging.getLogger("haystack").setLevel(logging.DEBUG)
+
+tracing.tracer.is_content_tracing_enabled = (
+    True  # to enable tracing/logging content (inputs/outputs)
+)
+tracing.enable_tracing(
+    LoggingTracer(
+        tags_color_strings={
+            "haystack.component.input": "\x1b[1;31m",
+            "haystack.component.name": "\x1b[1;34m",
+        },
+    ),
+)
+```
+
+Here’s what the resulting log would look like when a pipeline is run:
+<ClickableImage src="/img/55c3d5c84282d726c95fb3350ec36be49a354edca8a6164f5dffdab7121cec58-image_2.png" alt="Console output showing Haystack pipeline execution with DEBUG level tracing logs including component names, types, and input/output specifications" />
+
+## Structured Logging
+
+Haystack leverages the [structlog library](https://www.structlog.org/en/stable/) to provide structured key-value logs. This provides additional metadata with each log message and is especially useful if you archive your logs with tools like [ELK](https://www.elastic.co/de/elastic-stack), [Grafana](https://grafana.com/oss/agent/?plcmt=footer), or [Datadog](https://www.datadoghq.com/).
+
+If Haystack detects a [structlog installation](https://www.structlog.org/en/stable/) on your system, it will automatically switch to structlog for logging.
+
+### Console Rendering
+
+To make development a more pleasurable experience, Haystack uses [structlog’s `ConsoleRender`](https://www.structlog.org/en/stable/console-output.html) by default to render structured logs as a nicely aligned and colorful output:
+<ClickableImage src="/img/e49a1f2-Screenshot_2024-02-27_at_16.13.51.png" alt="Python code snippet demonstrating basic logging setup with getLogger and a warning level log message output" />
+
+:::tip[Rich Formatting]
+
+Install [_rich_](https://rich.readthedocs.io/en/stable/index.html) to beautify your logs even more!
+:::
+
+### JSON Rendering
+
+We recommend JSON logging when deploying Haystack to production. Haystack will automatically switch to JSON format if it detects no interactive terminal session. If you want to enforce JSON logging:
+
+- Run Haystack with the environment variable `HAYSTACK_LOGGING_USE_JSON` set to `true`.
+- Or, use Python to tell Haystack to log as JSON:
+
+  ```python
+  import haystack.logging
+
+  haystack.logging.configure_logging(use_json=True)
+  ```
+<ClickableImage src="/img/bff93d4-Screenshot_2024-02-27_at_16.15.35.png" alt="Python code snippet showing structured JSON logging configuration with example JSON formatted log output including event, level, and timestamp fields" />
+
+### Disabling Structured Logging
+
+To disable structured logging despite an existing installation of structlog, set the environment variable `HAYSTACK_LOGGING_IGNORE_STRUCTLOG_ENV_VAR` to `true` when running Haystack.

docs-website/docs/development/tracing.mdx

@@ -0,0 +1,56 @@
+---
+title: "Tracing"
+id: tracing
+slug: "/tracing"
+description: "This page explains how to use tracing in Haystack. It lists the tracing backends Haystack supports out of the box and explains how to enable, configure, and disable tracing."
+---
+
+# Tracing
+
+Traces document the flow of requests through your application and are vital for monitoring applications in production. This helps you understand the execution order of your pipeline components and analyze where your pipeline spends the most time.
+
+Instrumented applications typically send traces to a trace collector or a tracing backend. Haystack provides out-of-the-box support for several backends, and you can also quickly implement support for additional providers of your choosing.
+
+## Supported Tracers
+
+| Tracer | Description |
+| --- | --- |
+| [OpenTelemetry](tracing/opentelemetry.mdx) | Send traces to any [OpenTelemetry](https://opentelemetry.io/)-compatible backend using the `OpenTelemetryTracer` or the `OpenTelemetryConnector` component. Includes a Jaeger setup for local development. |
+| [MLflow](tracing/mlflow.mdx) | Capture traces with [MLflow](https://mlflow.org/)'s native Haystack tracing support. |
+| [Datadog](tracing/datadog.mdx) | Trace your pipelines with [Datadog](https://www.datadoghq.com/) using the `DatadogTracer` or the `DatadogConnector` component. |
+| [Langfuse](tracing/langfuse.mdx) | Trace your pipelines with the [Langfuse](https://langfuse.com/) UI using the `LangfuseTracer` or the `LangfuseConnector` component. |
+| [Weights & Biases Weave](tracing/weave.mdx) | Trace and visualize pipeline execution in [Weights & Biases](https://wandb.ai/site/) using the `WeaveTracer` or the `WeaveConnector` component. |
+| [LoggingTracer](tracing/logging-tracer.mdx) | Inspect the data flowing through your pipeline in real time through logs, with no backend setup. |
+| [Custom Tracer](tracing/custom-tracer.mdx) | Connect any tracing backend by implementing the `Tracer` interface. |
+
+## Enabling and Disabling Tracing
+
+Haystack never enables tracing automatically. To enable it, either call `haystack.tracing.enable_tracing(...)` with the tracer of your choice, or add a tracing connector component such as the [`OpenTelemetryConnector`](tracing/opentelemetry.mdx) or the [`DatadogConnector`](tracing/datadog.mdx) to your pipeline.
+
+To disable an enabled tracer:
+
+```python
+from haystack.tracing import disable_tracing
+
+disable_tracing()
+```
+
+## Content Tracing
+
+Haystack also allows you to trace your pipeline components' input and output values. This is useful for investigating your pipeline execution step by step.
+
+By default, this behavior is disabled to prevent sensitive user information from being sent to your tracing backend.
+
+To enable content tracing, there are two options:
+
+- Set the environment variable `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` when running your Haystack application
+
+— or —
+
+- Explicitly enable content tracing in Python:
+
+  ```python
+  from haystack import tracing
+
+  tracing.tracer.is_content_tracing_enabled = True
+  ```

docs-website/docs/development/tracing/custom-tracer.mdx

@@ -0,0 +1,82 @@
+---
+title: "Custom Tracer"
+id: custom-tracer
+slug: "/tracing-custom-tracer"
+description: "Learn how to connect Haystack to a custom tracing backend by implementing the Tracer interface."
+---
+
+# Custom Tracer
+
+Learn how to connect Haystack to a custom tracing backend by implementing the `Tracer` interface.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Base classes** | `Tracer` and `Span` |
+| **How to enable** | Implement the `Tracer` interface, then `tracing.enable_tracing(your_tracer)` |
+| **Content tracing** | Optional. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
+| **Package** | Built into Haystack |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/tracer.py |
+
+</div>
+
+## Overview
+
+If your tracing backend isn't supported out of the box, you can connect it to Haystack by implementing the `Tracer` interface. This gives you full control over how spans are created and how tags are recorded.
+
+## Usage
+
+1. Implement the `Tracer` interface. The following code snippet provides an example using the OpenTelemetry package:
+
+   ```python
+   import contextlib
+   from typing import Optional, Dict, Any, Iterator
+
+   from opentelemetry import trace
+   from opentelemetry.trace import NonRecordingSpan
+
+   from haystack.tracing import Tracer, Span
+   from haystack.tracing import utils as tracing_utils
+   import opentelemetry.trace
+
+   class OpenTelemetrySpan(Span):
+      def __init__(self, span: opentelemetry.trace.Span) -> None:
+          self._span = span
+
+      def set_tag(self, key: str, value: Any) -> None:
+   			 # Tracing backends usually don't support any tag value
+   			 # `coerce_tag_value` forces the value to either be a Python
+   			 # primitive (int, float, boolean, str) or tries to dump it as string.
+          coerced_value = tracing_utils.coerce_tag_value(value)
+          self._span.set_attribute(key, coerced_value)
+
+   class OpenTelemetryTracer(Tracer):
+      def __init__(self, tracer: opentelemetry.trace.Tracer) -> None:
+          self._tracer = tracer
+
+      @contextlib.contextmanager
+      def trace(self, operation_name: str, tags: Optional[Dict[str, Any]] = None) -> Iterator[Span]:
+          with self._tracer.start_as_current_span(operation_name) as span:
+              span = OpenTelemetrySpan(span)
+              if tags:
+                  span.set_tags(tags)
+
+              yield span
+
+      def current_span(self) -> Optional[Span]:
+          current_span = trace.get_current_span()
+          if isinstance(current_span, NonRecordingSpan):
+              return None
+
+          return OpenTelemetrySpan(current_span)
+   ```
+
+2. Tell Haystack to use your custom tracer:
+
+   ```python
+   from haystack import tracing
+
+   haystack_tracer = OpenTelemetryTracer(tracer)
+   tracing.enable_tracing(haystack_tracer)
+   ```

docs-website/docs/development/tracing/datadog.mdx

@@ -0,0 +1,94 @@
+---
+title: "Datadog"
+id: datadog
+slug: "/tracing-datadog"
+description: "Learn how to trace your Haystack pipelines with Datadog."
+---
+
+# Datadog
+
+Learn how to trace your Haystack pipelines with Datadog.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Tracer class** | `DatadogTracer` |
+| **How to enable** | Enable the tracer with `tracing.enable_tracing(DatadogTracer(ddtrace.tracer))`, or add the `DatadogConnector` component to your pipeline |
+| **Content tracing** | Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
+| **Package** | `datadog-haystack` |
+| **API reference** | [datadog](/reference/integrations-datadog) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/datadog |
+
+</div>
+
+## Overview
+
+Trace your Haystack pipelines with [Datadog](https://www.datadoghq.com/) through [Datadog's tracing library `ddtrace`](https://ddtrace.readthedocs.io/en/stable/). Haystack captures detailed information about pipeline runs, like API calls, context data, and prompts, so you can see the complete trace of your pipeline execution in Datadog.
+
+## Installation
+
+Install the `datadog-haystack` package:
+
+```shell
+pip install datadog-haystack
+```
+
+## Prerequisites
+
+1. A way to receive traces, such as a running [Datadog Agent](https://docs.datadoghq.com/agent/). `ddtrace` sends traces to the Datadog Agent at `localhost:8126` by default.
+2. Configure `ddtrace` through the standard mechanisms, for example the `DD_SERVICE`, `DD_ENV`, and `DD_VERSION` environment variables, or by running your application with the `ddtrace-run` command. See the [ddtrace documentation](https://ddtrace.readthedocs.io/en/stable/) for more details.
+
+## Usage
+
+Enable the `DatadogTracer` directly to trace any Haystack pipeline, without adding a component to it. Make sure to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable before importing any Haystack components.
+
+```python
+import os
+
+os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
+
+import ddtrace
+
+from haystack import Pipeline, tracing
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+from haystack_integrations.tracing.datadog import DatadogTracer
+
+# Enable the Datadog tracer
+tracing.enable_tracing(DatadogTracer(ddtrace.tracer))
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component("llm", OpenAIChatGenerator())
+pipe.connect("prompt_builder.prompt", "llm.messages")
+
+messages = [
+    ChatMessage.from_system(
+        "Always respond in German even if some input data is in other languages.",
+    ),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
+
+response = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": "Berlin"},
+            "template": messages,
+        },
+    },
+)
+print(response["llm"]["replies"][0])
+```
+
+Each pipeline run produces a trace that includes the entire execution context, including prompts, completions, and metadata. You can then view the traces in your Datadog dashboard.
+
+## Alternative: the DatadogConnector component
+
+If you prefer to manage tracing as part of your pipeline definition (for example, so it serializes to YAML), you can add the `DatadogConnector` component instead. It enables the same Datadog tracing as soon as it is initialized.
+
+:::info
+See the [`DatadogConnector` documentation page](../../pipeline-components/connectors/datadogconnector.mdx) for full usage examples, or check out the [integration page](https://haystack.deepset.ai/integrations/datadog).
+:::

docs-website/docs/development/tracing/langfuse.mdx

@@ -0,0 +1,110 @@
+---
+title: "Langfuse"
+id: langfuse
+slug: "/tracing-langfuse"
+description: "Learn how to trace your Haystack pipelines with Langfuse."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Langfuse
+
+Learn how to trace your Haystack pipelines with Langfuse.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Tracer class** | `LangfuseTracer` |
+| **How to enable** | Enable the tracer with `tracing.enable_tracing(LangfuseTracer(langfuse))`, or add the `LangfuseConnector` component to your pipeline |
+| **Content tracing** | Required. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` |
+| **Package** | `langfuse-haystack` |
+| **API reference** | [langfuse](/reference/integrations-langfuse) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langfuse |
+
+</div>
+
+## Overview
+
+Trace your Haystack pipelines with the [Langfuse](https://langfuse.com/) UI. Langfuse captures detailed information about pipeline runs, like API calls, context data, prompts, and more. Use it to monitor model performance such as token usage and cost, find areas for improvement, and create datasets from your pipeline executions.
+
+## Installation
+
+Install the `langfuse-haystack` package:
+
+```shell
+pip install langfuse-haystack
+```
+
+## Prerequisites
+
+1. An active Langfuse [account](https://cloud.langfuse.com/).
+2. Set the `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` environment variables with your Langfuse secret and public keys, found in your account profile.
+3. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` to enable tracing.
+
+:::info[Usage Notice]
+To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import. An even better practice is to set these environment variables in your shell before running the script.
+:::
+
+## Usage
+
+Enable the `LangfuseTracer` directly to trace any Haystack pipeline, without adding a component to it.
+
+```python
+import os
+
+os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com"
+os.environ["LANGFUSE_SECRET_KEY"] = "<your-secret-key>"
+os.environ["LANGFUSE_PUBLIC_KEY"] = "<your-public-key>"
+os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
+
+from langfuse import Langfuse
+
+from haystack import Pipeline, tracing
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+from haystack_integrations.tracing.langfuse import LangfuseTracer
+
+# Enable the Langfuse tracer. The client reads your keys from the environment.
+langfuse = Langfuse()
+langfuse_tracer = LangfuseTracer(langfuse, name="Chat example")
+tracing.enable_tracing(langfuse_tracer)
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component("llm", OpenAIChatGenerator())
+pipe.connect("prompt_builder.prompt", "llm.messages")
+
+messages = [
+    ChatMessage.from_system(
+        "Always respond in German even if some input data is in other languages.",
+    ),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
+
+response = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": "Berlin"},
+            "template": messages,
+        },
+    },
+)
+print(response["llm"]["replies"][0])
+
+# Flush any pending spans before the program exits
+langfuse_tracer.flush()
+```
+
+Each pipeline run produces one trace that includes the entire execution context, including prompts, completions, and metadata. You can then view the trace in the Langfuse UI.
+<ClickableImage src="/img/11cec4f-langfuse-generation-span.png" alt="Langfuse trace detail view showing generation span with input prompt, output, metadata, latency, and cost information for a language model call" />
+
+## Alternative: the LangfuseConnector component
+
+If you prefer to manage tracing as part of your pipeline definition, you can add the `LangfuseConnector` component instead. It enables the same Langfuse tracing, exposes the `trace_url` as an output, and supports a custom `SpanHandler` for advanced span processing.
+
+:::info
+See the [`LangfuseConnector` documentation page](../../pipeline-components/connectors/langfuseconnector.mdx) for full usage examples and advanced span customization, or read the [blog post](https://haystack.deepset.ai/blog/langfuse-integration) for a complete walkthrough.
+:::

docs-website/docs/development/tracing/logging-tracer.mdx

@@ -0,0 +1,61 @@
+---
+title: "LoggingTracer"
+id: logging-tracer
+slug: "/tracing-logging-tracer"
+description: "Learn how to inspect the data flowing through your Haystack pipelines in real time with the LoggingTracer."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# LoggingTracer
+
+Learn how to inspect the data flowing through your Haystack pipelines in real time with the `LoggingTracer`.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Tracer class** | `LoggingTracer` |
+| **How to enable** | `tracing.enable_tracing(LoggingTracer(...))` |
+| **Content tracing** | Required to log inputs and outputs. Set `tracing.tracer.is_content_tracing_enabled = True` |
+| **Package** | Built into Haystack |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py |
+
+</div>
+
+## Overview
+
+Use Haystack's [`LoggingTracer`](https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py) logs to inspect the data that's flowing through your pipeline in real time.
+
+This feature is particularly helpful during experimentation and prototyping, as you don’t need to set up any tracing backend beforehand.
+
+## Usage
+
+Here’s how you can enable this tracer. In this example, we are adding color tags (this is optional) to highlight the components' names and inputs:
+
+```python
+import logging
+from haystack import tracing
+from haystack.tracing.logging_tracer import LoggingTracer
+
+logging.basicConfig(
+    format="%(levelname)s - %(name)s -  %(message)s",
+    level=logging.WARNING,
+)
+logging.getLogger("haystack").setLevel(logging.DEBUG)
+
+tracing.tracer.is_content_tracing_enabled = (
+    True  # to enable tracing/logging content (inputs/outputs)
+)
+tracing.enable_tracing(
+    LoggingTracer(
+        tags_color_strings={
+            "haystack.component.input": "\x1b[1;31m",
+            "haystack.component.name": "\x1b[1;34m",
+        },
+    ),
+)
+```
+
+Here’s what the resulting log would look like when a pipeline is run:
+<ClickableImage src="/img/55c3d5c84282d726c95fb3350ec36be49a354edca8a6164f5dffdab7121cec58-image_2.png" alt="Console output showing Haystack pipeline execution with DEBUG level tracing logs including component names, types, and input/output specifications" />

docs-website/docs/development/tracing/mlflow.mdx

@@ -0,0 +1,51 @@
+---
+title: "MLflow"
+id: mlflow
+slug: "/tracing-mlflow"
+description: "Learn how to trace your Haystack pipelines with MLflow."
+---
+
+# MLflow
+
+Learn how to trace your Haystack pipelines with MLflow.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **How to enable** | `mlflow.haystack.autolog()` |
+| **Content tracing** | Captured automatically, including latencies, token usage, cost, and exceptions |
+| **Package** | `mlflow` |
+| **Integration guide** | https://haystack.deepset.ai/integrations/mlflow |
+
+</div>
+
+## Overview
+
+[MLflow](https://mlflow.org/) is an open-source platform for managing the end-to-end machine learning and AI lifecycle. MLflow provides native tracing support for Haystack, so you can capture traces from all your pipelines and components with a single line of code.
+
+## Installation
+
+Install MLflow:
+
+```shell
+pip install mlflow
+```
+
+## Usage
+
+Enable automatic tracing for all Haystack pipelines and components:
+
+```python
+import mlflow
+
+mlflow.haystack.autolog()
+# Optionally set an experiment name
+mlflow.set_experiment("Haystack")
+```
+
+This automatically captures traces from all Haystack pipelines and components, including latencies, token usage, cost, and any exceptions.
+
+:::info
+Check out the [MLflow Haystack integration guide](https://haystack.deepset.ai/integrations/mlflow) for a full walkthrough with examples.
+:::

docs-website/docs/development/tracing/opentelemetry.mdx

@@ -0,0 +1,153 @@
+---
+title: "OpenTelemetry"
+id: opentelemetry
+slug: "/tracing-opentelemetry"
+description: "Learn how to trace your Haystack pipelines with OpenTelemetry."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# OpenTelemetry
+
+Learn how to trace your Haystack pipelines with OpenTelemetry.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Tracer class** | `OpenTelemetryTracer` |
+| **How to enable** | Configure an OpenTelemetry `TracerProvider`, then enable the tracer with `tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))`, or add the `OpenTelemetryConnector` component to your pipeline |
+| **Content tracing** | Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
+| **Package** | `opentelemetry-haystack` |
+| **API reference** | [opentelemetry](/reference/integrations-opentelemetry) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opentelemetry |
+
+</div>
+
+## Overview
+
+[OpenTelemetry](https://opentelemetry.io/) is an open-source observability framework for collecting traces, metrics, and logs. Haystack integrates with OpenTelemetry, so you can send traces of your pipeline runs to any OpenTelemetry-compatible backend.
+
+:::info[Moving to an integration]
+`OpenTelemetryTracer` is deprecated in Haystack core and is moving to the `opentelemetry-haystack` package. Starting with Haystack 3.0, OpenTelemetry tracing is no longer auto-enabled when `opentelemetry-sdk` is installed. Install the integration and either enable the `OpenTelemetryTracer` directly or add the `OpenTelemetryConnector` component to your pipeline.
+:::
+
+## Installation
+
+Install the `opentelemetry-haystack` package:
+
+```shell
+pip install opentelemetry-haystack
+```
+
+To add traces to even deeper levels of your pipelines, we recommend you check out [OpenTelemetry integrations](https://opentelemetry.io/ecosystem/registry/?s=python), such as:
+
+- [`urllib3` instrumentation](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-urllib3) for tracing HTTP requests in your pipeline,
+- [OpenAI instrumentation](https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-openai) for tracing OpenAI requests.
+
+## Prerequisites
+
+A configured OpenTelemetry `TracerProvider` with an exporter, for example an OTLP exporter that sends traces to a collector or a backend. Set up the provider before enabling the tracer.
+
+## Usage
+
+Enable the `OpenTelemetryTracer` directly to trace any Haystack pipeline, without adding a component to it. Configure your `TracerProvider` and set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable before importing any Haystack components.
+
+```python
+import os
+
+os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
+
+from opentelemetry import trace
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+from opentelemetry.sdk.resources import Resource
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+from opentelemetry.semconv.resource import ResourceAttributes
+
+# Configure the OpenTelemetry SDK. A service name is required for most backends.
+resource = Resource(attributes={ResourceAttributes.SERVICE_NAME: "haystack"})
+tracer_provider = TracerProvider(resource=resource)
+tracer_provider.add_span_processor(
+    BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")),
+)
+trace.set_tracer_provider(tracer_provider)
+
+from haystack import tracing
+from haystack_integrations.tracing.opentelemetry import OpenTelemetryTracer
+
+# Enable the OpenTelemetry tracer
+tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))
+```
+
+Each pipeline run then produces a trace that includes the entire execution context, including prompts, completions, and metadata. You can view the traces in your OpenTelemetry-compatible backend.
+
+## Alternative: the OpenTelemetryConnector component
+
+If you prefer to manage tracing as part of your pipeline definition, you can add the `OpenTelemetryConnector` component instead. It enables the same OpenTelemetry tracing as soon as it is initialized.
+
+:::info
+See the [`OpenTelemetryConnector` documentation page](../../pipeline-components/connectors/opentelemetryconnector.mdx) for full usage examples, or check out the [integration page](https://haystack.deepset.ai/integrations/opentelemetry).
+:::
+
+## Visualizing Traces During Development
+
+Use [Jaeger](https://www.jaegertracing.io/docs/1.6/getting-started/) as a lightweight tracing backend for local pipeline development. This allows you to experiment with tracing without the need for a complex tracing backend.
+<ClickableImage src="/img/dd906d7-Screenshot_2024-02-22_at_16.51.01.png" alt="Jaeger UI trace timeline displaying haystack pipeline execution with component spans showing duration and nesting of operations" />
+
+1. Run the Jaeger container. This creates a tracing backend as well as a UI to visualize the traces:
+
+   ```shell
+   docker run --rm -d --name jaeger \
+     -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
+     -p 6831:6831/udp \
+     -p 6832:6832/udp \
+     -p 5778:5778 \
+     -p 16686:16686 \
+     -p 4317:4317 \
+     -p 4318:4318 \
+     -p 14250:14250 \
+     -p 14268:14268 \
+     -p 14269:14269 \
+     -p 9411:9411 \
+     jaegertracing/all-in-one:latest
+   ```
+2. Install the integration and the OTLP exporter:
+
+   ```shell
+   pip install opentelemetry-haystack
+   pip install opentelemetry-exporter-otlp
+   ```
+3. Configure `OpenTelemetry` to use the Jaeger backend and enable the tracer:
+
+   ```python
+   from opentelemetry import trace
+   from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+   from opentelemetry.sdk.resources import Resource
+   from opentelemetry.sdk.trace import TracerProvider
+   from opentelemetry.sdk.trace.export import BatchSpanProcessor
+   from opentelemetry.semconv.resource import ResourceAttributes
+
+   from haystack import tracing
+   from haystack_integrations.tracing.opentelemetry import OpenTelemetryTracer
+
+   # Service name is required for most backends
+   resource = Resource(attributes={
+       ResourceAttributes.SERVICE_NAME: "haystack"
+   })
+
+   tracer_provider = TracerProvider(resource=resource)
+   processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces"))
+   tracer_provider.add_span_processor(processor)
+   trace.set_tracer_provider(tracer_provider)
+
+   tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))
+   ```
+4. Run your pipeline:
+
+   ```python
+   ...
+   pipeline.run(...)
+   ...
+   ```
+5. Inspect the traces in the UI provided by Jaeger at [http://localhost:16686](http://localhost:16686/search).

docs-website/docs/development/tracing/weave.mdx

@@ -0,0 +1,93 @@
+---
+title: "Weights & Biases Weave"
+id: weave
+slug: "/tracing-weave"
+description: "Learn how to trace your Haystack pipelines with Weights & Biases Weave."
+---
+
+# Weights & Biases Weave
+
+Learn how to trace your Haystack pipelines with Weights & Biases Weave.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Tracer class** | `WeaveTracer` |
+| **How to enable** | Enable the tracer with `tracing.enable_tracing(WeaveTracer(project_name="..."))`, or add the `WeaveConnector` component to your pipeline |
+| **Content tracing** | Required. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` |
+| **Package** | `weave-haystack` |
+| **API reference** | [Weave](/reference/integrations-weave) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weave |
+
+</div>
+
+## Overview
+
+Trace and visualize your pipeline execution in [Weights & Biases](https://wandb.ai/site/). Information captured by the Haystack tracing tool, such as API calls, context data, and prompts, is sent to Weights & Biases, where you can see the complete trace of your pipeline execution.
+
+## Installation
+
+Install the `weave-haystack` package:
+
+```shell
+pip install weave-haystack
+```
+
+## Prerequisites
+
+1. A Weave account. You can sign up for free on the [Weights & Biases website](https://wandb.ai/site).
+2. Set the `WANDB_API_KEY` environment variable with your Weights & Biases API key. Once logged in, you can find your API key on [your home page](https://wandb.ai/home).
+3. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true`.
+
+## Usage
+
+Enable the `WeaveTracer` directly to trace any Haystack pipeline, without adding a component to it. The `project_name` is the name that will appear in your Weave project.
+
+```python
+import os
+
+os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
+
+from haystack import Pipeline, tracing
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+
+from haystack_integrations.tracing.weave import WeaveTracer
+
+# Enable the Weave tracer
+tracing.enable_tracing(WeaveTracer(project_name="test_pipeline"))
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component("llm", OpenAIChatGenerator())
+pipe.connect("prompt_builder.prompt", "llm.messages")
+
+messages = [
+    ChatMessage.from_system(
+        "Always respond in German even if some input data is in other languages.",
+    ),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
+
+response = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": "Berlin"},
+            "template": messages,
+        },
+    },
+)
+print(response["llm"]["replies"][0])
+```
+
+You can then see the complete trace for your pipeline at `https://wandb.ai/<user_name>/projects` under the project name you specified.
+
+## Alternative: the WeaveConnector component
+
+If you prefer to manage tracing as part of your pipeline definition, you can add the `WeaveConnector` component instead. It enables the same Weave tracing as soon as it runs.
+
+:::info
+See the [`WeaveConnector` documentation page](../../pipeline-components/connectors/weaveconnector.mdx) for full usage examples.
+:::

docs-website/docs/document-stores/alloydbdocumentstore.mdx

@@ -0,0 +1,100 @@
+---
+title: "AlloyDBDocumentStore"
+id: alloydbdocumentstore
+slug: "/alloydbdocumentstore"
+---
+
+# AlloyDBDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [AlloyDB](/reference/integrations-alloydb)                                              |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/alloydb |
+
+</div>
+
+[AlloyDB](https://cloud.google.com/alloydb) is a fully managed, PostgreSQL-compatible database service on Google Cloud. The `AlloyDBDocumentStore` uses the [pgvector extension](https://cloud.google.com/alloydb/docs/ai/work-with-embeddings) to perform vector similarity search.
+
+Connection is handled securely via the [AlloyDB Python Connector](https://github.com/GoogleCloudPlatform/alloydb-python-connector), which provides TLS encryption and IAM-based authorization without requiring manual SSL certificate management, firewall rules, or IP allowlisting.
+
+The `AlloyDBDocumentStore` supports embedding retrieval, keyword retrieval, and metadata filtering.
+
+## Installation
+
+Install the `alloydb-haystack` integration:
+
+```shell
+pip install alloydb-haystack
+```
+
+To set up an AlloyDB cluster and instance, follow the [AlloyDB quickstart](https://cloud.google.com/alloydb/docs/quickstart).
+
+## Usage
+
+### Authentication
+
+The `AlloyDBDocumentStore` uses [Secrets](../concepts/secret-management.mdx) and reads connection details from environment variables by default:
+
+- `ALLOYDB_INSTANCE_URI`: the AlloyDB instance URI in the format `projects/PROJECT/locations/REGION/clusters/CLUSTER/instances/INSTANCE`.
+- `ALLOYDB_USER`: the database user. When using IAM database authentication, use the service account email (omitting `.gserviceaccount.com`) or the full IAM user email.
+- `ALLOYDB_PASSWORD`: the database password. Not required when `enable_iam_auth=True`.
+
+```shell
+export ALLOYDB_INSTANCE_URI="projects/MY_PROJECT/locations/MY_REGION/clusters/MY_CLUSTER/instances/MY_INSTANCE"
+export ALLOYDB_USER="my-db-user"
+export ALLOYDB_PASSWORD="my-db-password"
+```
+
+To authenticate with IAM instead of a password, set `enable_iam_auth=True` and grant the IAM principal the AlloyDB Client role. See the [AlloyDB IAM authentication documentation](https://cloud.google.com/alloydb/docs/manage-iam-authn) for details.
+
+## Initialization
+
+Initialize an `AlloyDBDocumentStore` and write Documents to it. Connection to AlloyDB is established lazily on first use, and the table that stores Haystack Documents is created automatically if it doesn't exist:
+
+```python
+from haystack import Document
+from haystack_integrations.document_stores.alloydb import AlloyDBDocumentStore
+
+document_store = AlloyDBDocumentStore(
+    db="my-database",
+    embedding_dimension=768,
+    vector_function="cosine_similarity",
+    recreate_table=True,
+)
+
+document_store.write_documents(
+    [
+        Document(content="This is first", embedding=[0.1] * 768),
+        Document(content="This is second", embedding=[0.3] * 768),
+    ],
+)
+print(document_store.count_documents())
+```
+
+To learn more about the initialization parameters, see our [API docs](/reference/integrations-alloydb#alloydbdocumentstore).
+
+To compute embeddings for your Documents, you can use a Document Embedder, such as the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx).
+
+### Search Strategy
+
+The `AlloyDBDocumentStore` supports two search strategies for embedding retrieval:
+
+- `"exact_nearest_neighbor"` (default): provides perfect recall but can be slow on large numbers of documents.
+- `"hnsw"`: an approximate nearest neighbor search strategy that trades off some accuracy for speed. Recommended for large numbers of documents.
+
+When using `"hnsw"`, an index is created based on the `vector_function` you choose, so subsequent queries should keep using the same vector similarity function in order to take advantage of the index. You can tune index creation through `hnsw_index_creation_kwargs` (see the [pgvector documentation](https://github.com/pgvector/pgvector?tab=readme-ov-file#hnsw)).
+
+### Metadata Filtering
+
+The `AlloyDBDocumentStore` fully supports comparison operators (`==`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `not in`, `like`, `not like`) and the logical operators `AND` and `OR`. The `like` and `not like` operators are PostgreSQL-specific extensions to the standard Haystack filter syntax and map to the SQL `LIKE` / `NOT LIKE` pattern-matching operators.
+
+The `NOT` logical operator is **not** supported. Because every comparison operator already has a negated counterpart (`==`/`!=`, `in`/`not in`, `like`/`not like`), any filter expressible with `NOT` around a single condition can be rewritten by inverting the comparison operator instead. To negate a nested `AND`/`OR` group, apply De Morgan's laws — for example, `NOT (A AND B)` becomes `(NOT A) OR (NOT B)`, where each `NOT A` / `NOT B` is expressed via the inverted comparison.
+
+For more details on filter syntax, refer to [Metadata Filtering](../concepts/metadata-filtering.mdx).
+
+### Supported Retrievers
+
+- [`AlloyDBEmbeddingRetriever`](../pipeline-components/retrievers/alloydbembeddingretriever.mdx): An embedding-based Retriever that fetches Documents from the Document Store based on a query embedding.
+- [`AlloyDBKeywordRetriever`](../pipeline-components/retrievers/alloydbkeywordretriever.mdx): A keyword-based Retriever that fetches Documents matching a query using PostgreSQL full-text search.

docs-website/docs/document-stores/arangodocumentstore.mdx

@@ -0,0 +1,116 @@
+---
+title: "ArangoDocumentStore"
+id: arangodocumentstore
+slug: "/arangodocumentstore"
+description: "Use the ArangoDB multi-model database with Haystack for embedding retrieval and GraphRAG workloads."
+---
+
+# ArangoDocumentStore
+
+Use the ArangoDB multi-model database with Haystack for embedding retrieval and GraphRAG workloads.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [ArangoDB](/reference/integrations-arangodb) |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/arangodb |
+
+</div>
+
+ArangoDB is a multi-model database that combines documents, graphs, and key-value data in a single engine. The `ArangoDocumentStore` stores documents in an ArangoDB collection and runs vector similarity search using AQL (ArangoDB Query Language) vector functions. Because documents and their relationships live in the same database, ArangoDB is a good fit for GraphRAG pipelines that combine semantic search with graph traversal.
+
+Vector search requires **ArangoDB 3.12 or later** with the vector index feature enabled (the `--vector-index` startup flag).
+
+For more information, see the [ArangoDB documentation](https://docs.arangodb.com/).
+
+## Installation
+
+Run ArangoDB with Docker, enabling the vector index and setting a root password:
+
+```shell
+docker run -d -p 8529:8529 \
+  -e ARANGO_ROOT_PASSWORD=test-password \
+  arangodb:3.12 arangod --vector-index
+```
+
+Install the Haystack integration:
+
+```shell
+pip install arangodb-haystack
+```
+
+## Usage
+
+The store reads its credentials from the `ARANGO_USERNAME` and `ARANGO_PASSWORD` environment variables by default. `ARANGO_USERNAME` falls back to `root` if it is not set, so you typically only need to provide the password:
+
+```shell
+export ARANGO_PASSWORD=test-password
+```
+
+Initialize the document store and write documents:
+
+```python
+from haystack import Document
+from haystack_integrations.document_stores.arangodb import ArangoDocumentStore
+
+document_store = ArangoDocumentStore(
+    host="http://localhost:8529",
+    database="haystack",
+    collection_name="documents",
+    embedding_dimension=768,
+    recreate_collection=True,
+)
+
+document_store.write_documents(
+    [
+        Document(
+            content="There are over 7,000 languages spoken around the world today.",
+        ),
+        Document(
+            content="Elephants have been observed to recognize themselves in mirrors.",
+        ),
+    ],
+)
+print(document_store.count_documents())
+```
+
+To learn more about the initialization parameters, see the [API docs](/reference/integrations-arangodb#arangodocumentstore).
+
+To compute real embeddings for your documents, use a Document Embedder such as the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx). The embedding dimension produced by the embedder must match the `embedding_dimension` configured on the store.
+
+### Authentication
+
+Credentials are passed as Haystack [`Secret`](../concepts/secret-management.mdx) objects. By default they are read from environment variables, but you can also pass them explicitly:
+
+```python
+from haystack.utils import Secret
+from haystack_integrations.document_stores.arangodb import ArangoDocumentStore
+
+document_store = ArangoDocumentStore(
+    host="http://localhost:8529",
+    database="haystack",
+    username=Secret.from_env_var("ARANGO_USERNAME", strict=False),
+    password=Secret.from_env_var("ARANGO_PASSWORD"),
+)
+```
+
+### Similarity Functions
+
+`ArangoDocumentStore` supports three similarity functions for vector search, configured at initialization with the `similarity_function` parameter:
+
+- `"cosine"` (default): cosine similarity, best for normalized embeddings.
+- `"dot_product"`: dot product, useful when embedding magnitude carries meaning.
+- `"l2"`: Euclidean (L2) distance.
+
+```python
+document_store = ArangoDocumentStore(
+    host="http://localhost:8529",
+    embedding_dimension=768,
+    similarity_function="dot_product",
+)
+```
+
+### Supported Retrievers
+
+- [`ArangoEmbeddingRetriever`](../pipeline-components/retrievers/arangoembeddingretriever.mdx): Retrieves documents from the `ArangoDocumentStore` based on vector similarity using ArangoDB's AQL vector functions.

docs-website/docs/document-stores/arcadedbdocumentstore.mdx

@@ -0,0 +1,73 @@
+---
+title: "ArcadeDBDocumentStore"
+id: arcadedbdocumentstore
+slug: "/arcadedbdocumentstore"
+---
+
+# ArcadeDBDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [ArcadeDB](/reference/integrations-arcadedb) |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/arcadedb |
+
+</div>
+
+ArcadeDB is a multi-model database that supports vector search via its LSM_VECTOR (HNSW) index. The `ArcadeDBDocumentStore` uses ArcadeDB's HTTP/JSON API for all operations—no special drivers required. It supports dense embedding retrieval and SQL-based metadata filtering.
+
+For more information, see the [ArcadeDB documentation](https://docs.arcadedb.com/).
+
+## Installation
+
+Run ArcadeDB with Docker and update the password according to your setup:
+
+```shell
+docker run -d -p 2480:2480 \
+  -e JAVA_OPTS="-Darcadedb.server.rootPassword=arcadedb" \
+  arcadedata/arcadedb:latest
+```
+
+Install the Haystack integration:
+
+```shell
+pip install arcadedb-haystack
+```
+
+## Usage
+
+Set credentials via environment variables (recommended) or pass them explicitly:
+
+```shell
+export ARCADEDB_USERNAME=root
+export ARCADEDB_PASSWORD=arcadedb
+```
+
+Initialize the document store and write documents:
+
+```python
+from haystack import Document
+from haystack_integrations.document_stores.arcadedb import ArcadeDBDocumentStore
+
+document_store = ArcadeDBDocumentStore(
+    url="http://localhost:2480",
+    database="haystack",
+    embedding_dimension=768,
+    recreate_type=True,
+)
+
+document_store.write_documents([
+    Document(content="This is first", embedding=[0.0] * 768),
+    Document(content="This is second", embedding=[0.1, 0.2, 0.3] + [0.0] * 765),
+])
+print(document_store.count_documents())
+```
+
+To learn more about the initialization parameters, see the [API docs](/reference/integrations-arcadedb#arcadedbdocumentstore).
+
+Documents without embeddings or with a different dimension are stored with a zero-padded vector so they can be written and filtered; use an [Embedder](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx) for real embeddings.
+
+### Supported Retrievers
+
+- [ArcadeDBEmbeddingRetriever](../pipeline-components/retrievers/arcadedbembeddingretriever.mdx): An embedding-based Retriever that fetches documents from the Document Store by vector similarity (HNSW).

docs-website/docs/document-stores/astradocumentstore.mdx

@@ -0,0 +1,82 @@
+---
+title: "AstraDocumentStore"
+id: astradocumentstore
+slug: "/astradocumentstore"
+---
+
+# AstraDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Astra](/reference/integrations-astra)                                                         |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/astra |
+
+</div>
+
+DataStax Astra DB is a serverless vector database built on Apache Cassandra, and it supports vector-based search and auto-scaling. You can deploy it on AWS, GCP, or Azure and easily expand to one or more regions within those clouds for multi-region availability, low latency data access, data sovereignty, and to avoid cloud vendor lock-in. For more information, see the [DataStax documentation](https://docs.datastax.com/en/home/docs/index.html).
+
+### Initialization
+
+Once you have an AstraDB account and have created a database, install the `astra-haystack` integration:
+
+```shell
+pip install astra-haystack
+```
+
+From the configuration in AstraDB’s web UI, you need the database ID and a generated token.
+
+You will additionally need a collection name and a namespace. When you create the collection name, you also need to set the embedding dimensions and the similarity metric. The namespace organizes data in a database and is called a keyspace in Apache Cassandra.
+
+Then, in Haystack, initialize an `AstraDocumentStore` object that’s connected to the AstraDB instance, and write documents to it.
+
+We strongly encourage passing authentication data through environment variables: make sure to populate the environment variables `ASTRA_DB_API_ENDPOINT` and  `ASTRA_DB_APPLICATION_TOKEN`  before running the following example.
+
+```python
+from haystack import Document
+from haystack_integrations.document_stores.astra import AstraDocumentStore
+
+document_store = AstraDocumentStore()
+
+document_store.write_documents(
+    [Document(content="This is first"), Document(content="This is second")],
+)
+print(document_store.count_documents())
+```
+
+### Supported Retrievers
+
+[AstraEmbeddingRetriever](../pipeline-components/retrievers/astraretriever.mdx): An embedding-based Retriever that fetches documents from the Document Store based on a query embedding provided to the Retriever.
+
+### Indexing Warnings
+
+When you create an Astra DB Document Store, you might see one of these warnings:
+
+> Astra DB collection `...` is detected as having indexing turned on for all fields (either created manually or by older versions of this plugin). This implies stricter limitations on the amount of text each string in a document can store. Consider indexing anew on a fresh collection to be able to store longer texts.
+
+Or:
+
+> Astra DB collection `...` is detected as having the following indexing policy: `{...}`. This does not match the requested indexing policy for this object: `{...}`. In particular, there may be stricter limitations on the amount of text each string in a document can store. Consider indexing anew on a fresh collection to be able to store longer texts.
+
+#### Why You See This Warning
+
+The collection already exists and is configured to [index all fields for search](https://docs.datastax.com/en/astra-db-serverless/api-reference/collections.html#the-indexing-option), possibly because you created it earlier or an older plugin did. When Haystack tries to create the collection, it applies an indexing policy optimized for your intended use. This policy lets you store longer texts and avoids indexing fields you won’t filter on, which also reduces write overhead.
+
+#### Common Causes
+
+1. You created the collection outside Haystack (for example, in the Astra UI or with AstraPy’s `Database.create_collection()`).
+2. You created the collection with an older version of the plugin.
+
+#### Impact
+
+This is only a warning. Your application keeps running unless you try to store very long text fields. If you do, Astra DB returns an indexing error.
+
+#### Solutions
+
+- **Recommended:** _Drop and recreate the collection_ if you can repopulate it. Then rerun your Haystack application so it creates the collection with the optimized indexing policy.
+- _Ignore the warning_ if you’re sure you won’t store very long text fields.
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Using AstraDB as a data store in your Haystack pipelines](https://haystack.deepset.ai/cookbook/astradb_haystack_integration)

docs-website/docs/document-stores/azureaisearchdocumentstore.mdx

@@ -0,0 +1,70 @@
+---
+title: "AzureAISearchDocumentStore"
+id: azureaisearchdocumentstore
+slug: "/azureaisearchdocumentstore"
+description: "A Document Store for storing and retrieval from Azure AI Search Index."
+---
+
+# AzureAISearchDocumentStore
+
+A Document Store for storing and retrieval from Azure AI Search Index.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **API reference** | [Azure AI Search](/reference/integrations-azure_ai_search)                                               |
+| **GitHub link**   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/azure_ai_search |
+
+</div>
+
+[Azure AI Search](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search) is an enterprise-ready search and retrieval system to build RAG-based applications on Azure, with native LLM integrations.
+
+`AzureAISearchDocumentStore` supports semantic reranking and metadata/content filtering. The Document Store is useful for various tasks such as generating knowledge base insights (catalog or document search), information discovery (data exploration), RAG, and automation.
+
+### Initialization
+
+This integration requires you to have an active Azure subscription with a deployed [Azure AI Search](https://azure.microsoft.com/en-us/products/ai-services/ai-search) service.
+
+Once you have the subscription, install the `azure-ai-search-haystack` integration:
+
+```python
+pip install azure-ai-search-haystack
+```
+
+To use the `AzureAISearchDocumentStore`, you need to provide a search service endpoint as an `AZURE_AI_SEARCH_ENDPOINT` and an API key as `AZURE_AI_SEARCH_API_KEY` for authentication. If the API key is not provided, the `DefaultAzureCredential` will attempt to authenticate you through the browser.
+
+During initialization the Document Store will either retrieve the existing search index for the given `index_name` or create a new one if it doesn't already exist. Note that one of the limitations of `AzureAISearchDocumentStore` is that the fields of the Azure search index cannot be modified through the API after creation. Therefore, any additional fields beyond the default ones must be provided as `metadata_fields` during the Document Store's initialization. However, if needed, [Azure AI portal](https://azure.microsoft.com/) can be used to modify the fields without deleting the index.
+
+It is recommended to pass authentication data through `AZURE_AI_SEARCH_API_KEY` and `AZURE_AI_SEARCH_ENDPOINT` before running the following example.
+
+```python
+from haystack_integrations.document_stores.azure_ai_search import (
+    AzureAISearchDocumentStore,
+)
+from haystack import Document
+
+document_store = AzureAISearchDocumentStore(index_name="haystack-docs")
+document_store.write_documents(
+    [
+        Document(content="This is the first document."),
+        Document(content="This is the second document."),
+    ],
+)
+print(document_store.count_documents())
+```
+
+:::info[Latency Notice]
+
+Due to Azure search index latency, the document count returned in the example might be zero if executed immediately. To ensure accurate results, be mindful of this latency when retrieving documents from the search index.
+:::
+
+You can enable semantic reranking in `AzureAISearchDocumentStore` by providing [SemanticSearch](https://learn.microsoft.com/en-us/python/api/azure-search-documents/azure.search.documents.indexes.models.semanticsearch?view=azure-python) configuration in `index_creation_kwargs` during initialization and calling it from one of the Retrievers. For more information, refer to the [Azure AI tutorial](https://learn.microsoft.com/en-us/azure/search/search-get-started-semantic) on this feature.
+
+### Supported Retrievers
+
+The Haystack Azure AI Search integration includes three Retriever components. Each Retriever leverages the Azure AI Search API  and you can select the one that best suits your pipeline:
+
+- [`AzureAISearchEmbeddingRetriever`](../pipeline-components/retrievers/azureaisearchembeddingretriever.mdx): This Retriever accepts the embeddings of a single query as input and returns a list of matching documents. The query must be embedded beforehand, which can be done using an [Embedder](../pipeline-components/embedders.mdx) component.
+- [`AzureAISearchBM25Retriever`](../pipeline-components/retrievers/azureaisearchbm25retriever.mdx): A keyword-based Retriever that retrieves documents matching a query from the Azure AI Search index.
+- [`AzureAISearchHybridRetriever`](../pipeline-components/retrievers/azureaisearchhybridretriever.mdx): This Retriever combines embedding-based retrieval and keyword search to find matching documents in the search index to get more relevant results.

docs-website/docs/document-stores/chromadocumentstore.mdx

@@ -0,0 +1,97 @@
+---
+title: "ChromaDocumentStore"
+id: chromadocumentstore
+slug: "/chromadocumentstore"
+---
+
+# ChromaDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Chroma](/reference/integrations-chroma)                                                        |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/chroma |
+
+</div>
+
+[Chroma](https://docs.trychroma.com/) is an open source vector database capable of storing collections of documents along with their metadata, creating embeddings for documents and queries, and searching the collections filtering by document metadata or content. Additionally, Chroma supports multi-modal embedding functions.
+
+Chroma can be used in-memory, as an embedded database, or in a client-server fashion. When running in-memory, Chroma can still keep its contents on disk across different sessions. This allows users to quickly put together prototypes using the in-memory version and later move to production, where the client-server version is deployed.
+
+## Initialization
+
+First, install the Chroma integration, which will install Haystack and Chroma if they are not already present. The following command is all you need to start:
+
+```shell
+pip install chroma-haystack
+```
+
+To store data in Chroma, create a `ChromaDocumentStore` instance and write documents with:
+
+```python
+from haystack_integrations.document_stores.chroma import ChromaDocumentStore
+from haystack import Document
+
+document_store = ChromaDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="This is the first document."),
+        Document(content="This is the second document."),
+    ],
+)
+print(document_store.count_documents())
+```
+
+In this case, since we didn’t pass any embeddings along with our documents, Chroma will create them for us using its [default embedding function](https://docs.trychroma.com/embeddings#default-all-minilm-l6-v2).
+
+### Connection Options
+
+1. **In-Memory Mode (Local)**: Chroma can be set up as a local Document Store for fast and lightweight usage. You can use this option during development or small-scale experiments. Set up a local in-memory instance of `ChromaDocumentStore` like this:
+
+   ```python
+   from haystack_integrations.document_stores.chroma import ChromaDocumentStore
+
+   document_store = ChromaDocumentStore()
+   ```
+2. **Persistent Storage**: If you need to retain the documents between sessions, Chroma supports persistent storage by specifying a path to store data on disk:
+
+   ```python
+   from haystack_integrations.document_stores.chroma import ChromaDocumentStore
+
+   document_store = ChromaDocumentStore(persist_path="your_directory_path")
+   ```
+3. **Remote Connection**: You can connect to a remote Chroma database through HTTP. This is suitable for distributed setups where multiple clients might interact with the same remote Chroma instance.
+
+   Note that this option is incompatible with in-memory or persistent storage modes.
+
+   First, start a Chroma server:
+
+   ```shell
+   chroma run --path /db_path
+   ```
+
+   Or using docker:
+
+   ```shell
+   docker run -p 8000:8000 chromadb/chroma
+   ```
+
+   Then, initialize the Document Store with `host` and `port` parameters:
+
+   ```python
+   from haystack_integrations.document_stores.chroma import ChromaDocumentStore
+
+   document_store = ChromaDocumentStore(host="localhost", port="8000")
+   ```
+
+## Supported Retrievers
+
+The Haystack Chroma integration comes with three Retriever components. They all rely on the Chroma [query API](https://docs.trychroma.com/reference/Collection#query), but they have different inputs and outputs so that you can pick the one that best fits your pipeline:
+
+- [`ChromaQueryTextRetriever`](../pipeline-components/retrievers/chromaqueryretriever.mdx): This Retriever takes a plain-text query string in input and returns a list of matching documents. Chroma will create the embeddings for the query using its [default embedding function](https://docs.trychroma.com/embeddings#default-all-minilm-l6-v2).
+- [`ChromaEmbeddingRetriever`](../pipeline-components/retrievers/chromaembeddingretriever.mdx): This Retriever takes the embeddings of a single query in input and returns a list of matching documents. The query needs to be embedded before being passed to this component. For example, you can use an [embedder](../pipeline-components/embedders.mdx) component.
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Use Chroma for RAG and Indexing](https://haystack.deepset.ai/cookbook/chroma-indexing-and-rag-examples)

docs-website/docs/document-stores/elasticsearch-document-store.mdx

@@ -0,0 +1,67 @@
+---
+title: "ElasticsearchDocumentStore"
+id: elasticsearch-document-store
+slug: "/elasticsearch-document-store"
+description: "Use an Elasticsearch database with Haystack."
+---
+
+# ElasticsearchDocumentStore
+
+Use an Elasticsearch database with Haystack.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Elasticsearch](/reference/integrations-elasticsearch)                                                 |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch |
+
+</div>
+
+ElasticsearchDocumentStore is excellent if you want to evaluate the performance of different retrieval options (dense vs. sparse) and aim for a smooth transition from PoC to production.
+
+It features the approximate nearest neighbours (ANN) search.
+
+### Initialization
+
+[Install](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html) Elasticsearch and then [start](https://www.elastic.co/guide/en/elasticsearch/reference/current/starting-elasticsearch.html) an instance. Haystack supports Elasticsearch 8.
+
+If you have Docker set up, we recommend pulling the Docker image and running it.
+
+```shell
+docker pull docker.elastic.co/elasticsearch/elasticsearch:8.11.1
+docker run -p 9200:9200 -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" -e "xpack.security.enabled=false" elasticsearch:8.11.1
+```
+
+As an alternative, you can go to [Elasticsearch integration GitHub](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/elasticsearch) and start a Docker container running Elasticsearch using the provided `docker-compose.yml`:
+
+```shell
+docker compose up
+```
+
+Once you have a running Elasticsearch instance, install the `elasticsearch-haystack` integration:
+
+```shell
+pip install elasticsearch-haystack
+```
+
+Then, initialize an `ElasticsearchDocumentStore` object that’s connected to the Elasticsearch instance and writes documents to it:
+
+```python
+from haystack_integrations.document_stores.elasticsearch import (
+    ElasticsearchDocumentStore,
+)
+from haystack import Document
+
+document_store = ElasticsearchDocumentStore(hosts="http://localhost:9200")
+document_store.write_documents(
+    [Document(content="This is first"), Document(content="This is second")],
+)
+print(document_store.count_documents())
+```
+
+### Supported Retrievers
+
+[`ElasticsearchBM25Retriever`](../pipeline-components/retrievers/elasticsearchbm25retriever.mdx): A keyword-based Retriever that fetches documents matching a query from the Document Store.
+
+[`ElasticsearchEmbeddingRetriever`](../pipeline-components/retrievers/elasticsearchembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.

docs-website/docs/document-stores/faissdocumentstore.mdx

@@ -0,0 +1,152 @@
+---
+title: "FAISSDocumentStore"
+id: faissdocumentstore
+slug: "/faissdocumentstore"
+---
+
+# FAISSDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [FAISS](/reference/integrations-faiss)                                                      |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/faiss |
+
+</div>
+
+`FAISSDocumentStore` is a local Document Store backed by [FAISS](https://github.com/facebookresearch/faiss) for vector similarity search.
+It keeps vectors in a FAISS index and stores document data in memory, with optional persistence to disk.
+
+`FAISSDocumentStore` is a good fit for local development and small to medium-sized datasets where you want a lightweight setup without running an external database service.
+
+## Installation
+
+Install the FAISS integration:
+
+```shell
+pip install faiss-haystack
+```
+
+## Initialization
+
+Create a `FAISSDocumentStore` instance and write embedded documents:
+
+```python
+from haystack import Document
+from haystack.document_stores.types import DuplicatePolicy
+from haystack_integrations.document_stores.faiss import FAISSDocumentStore
+
+document_store = FAISSDocumentStore(
+    index_path="my_faiss_index",  # Optional: enables persistence on disk
+    index_string="Flat",
+    embedding_dim=768,
+)
+
+document_store.write_documents(
+    [
+        Document(content="This is first", embedding=[0.1] * 768),
+        Document(content="This is second", embedding=[0.2] * 768),
+    ],
+    policy=DuplicatePolicy.OVERWRITE,
+)
+
+print(document_store.count_documents())
+
+# Persist index and metadata files (`.faiss` and `.json`)
+document_store.save("my_faiss_index")
+```
+
+### Persistence
+
+If you provide `index_path` when initializing `FAISSDocumentStore`, it tries to load existing persisted files (`.faiss` and `.json`) from that path.
+You can also explicitly call:
+
+- `save(index_path)` to write index and metadata to disk.
+- `load(index_path)` to load them later.
+
+Example of loading from a previously saved folder/path:
+
+```python
+from haystack_integrations.document_stores.faiss import FAISSDocumentStore
+
+# This loads `my_faiss_index.faiss` and `my_faiss_index.json` if they exist
+document_store = FAISSDocumentStore(index_path="my_faiss_index")
+
+# Alternatively, initialize first and then load explicitly
+another_store = FAISSDocumentStore(embedding_dim=768)
+another_store.load("my_faiss_index")
+```
+
+## Supported Retrievers
+
+[`FAISSEmbeddingRetriever`](../pipeline-components/retrievers/faissembeddingretriever.mdx): Retrieves documents from `FAISSDocumentStore` based on query embeddings.
+
+
+### Fixing OpenMP Runtime Conflicts on macOS
+
+#### Symptoms
+
+You may encounter one or both of the following errors at runtime:
+
+```
+OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
+OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program.
+```
+
+```
+resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
+```
+
+If setting `OMP_NUM_THREADS=1` prevents the crash, the root cause is **multiple OpenMP runtimes loaded simultaneously**. Each runtime maintains its own thread pool and thread-local storage (TLS). When two runtimes spin up worker threads at the same time, they corrupt each other's memory — causing segfaults at `N > 1` threads.
+
+---
+
+#### Diagnosis
+
+First, find how many copies of `libomp.dylib` exist in your virtual environment:
+
+```bash
+find /path/to/your/.venv -name "libomp.dylib" 2>/dev/null
+```
+
+If you see more than one, e.g.:
+
+```
+.venv/lib/pythonX.Y/site-packages/torch/lib/libomp.dylib
+.venv/lib/pythonX.Y/site-packages/sklearn/.dylibs/libomp.dylib
+.venv/lib/pythonX.Y/site-packages/faiss/.dylibs/libomp.dylib
+```
+
+you need to consolidate them into a single runtime.
+
+---
+
+#### Fix
+
+The solution is to pick one canonical `libomp.dylib` (torch's is a good choice) and replace all other copies with symlinks pointing to it.
+
+For each duplicate, delete the copy and replace it with a symlink:
+
+```bash
+# Delete the duplicate
+rm /path/to/.venv/lib/pythonX.Y/site-packages/<package>/.dylibs/libomp.dylib
+
+# Replace with a symlink to the canonical copy
+ln -s /path/to/.venv/lib/pythonX.Y/site-packages/torch/lib/libomp.dylib \
+      /path/to/.venv/lib/pythonX.Y/site-packages/<package>/.dylibs/libomp.dylib
+```
+
+Repeat for every duplicate found. Because these packages use `@loader_path`-relative references to load `libomp.dylib`, the symlink will be transparently resolved to the single canonical runtime at load time.
+
+---
+
+#### Verify
+
+After applying the fix, confirm only one unique `libomp.dylib` is being referenced:
+
+```bash
+find /path/to/your/.venv -name "*.so" | xargs otool -L 2>/dev/null | grep libomp | sort -u
+```
+
+All entries should resolve to the same canonical path. You should now be able to run without `OMP_NUM_THREADS=1`.

docs-website/docs/document-stores/falkordbdocumentstore.mdx

@@ -0,0 +1,105 @@
+---
+title: "FalkorDBDocumentStore"
+id: falkordbdocumentstore
+slug: "/falkordbdocumentstore"
+description: "Use the FalkorDB graph database with Haystack for GraphRAG workloads."
+---
+
+# FalkorDBDocumentStore
+
+Use the FalkorDB graph database with Haystack for GraphRAG workloads.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [FalkorDB](/reference/integrations-falkordb)                                                        |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/falkordb |
+
+</div>
+
+FalkorDB is a high-performance graph database optimized for GraphRAG workloads. The `FalkorDBDocumentStore` stores documents as graph nodes and supports native vector search — no APOC is required. Documents and their `meta` fields are stored flat on each node, and all bulk writes use `UNWIND` + `MERGE` for safe OpenCypher upserts.
+
+For more information, see the [FalkorDB documentation](https://docs.falkordb.com/).
+
+## Installation
+
+Run FalkorDB with Docker:
+
+```shell
+docker run -d -p 6379:6379 falkordb/falkordb:latest
+```
+
+Install the Haystack integration:
+
+```shell
+pip install falkordb-haystack
+```
+
+## Usage
+
+Initialize the document store and write documents:
+
+```python
+from haystack import Document
+from haystack_integrations.document_stores.falkordb import FalkorDBDocumentStore
+
+document_store = FalkorDBDocumentStore(
+    host="localhost",
+    port=6379,
+    embedding_dim=768,
+    recreate_graph=True,
+)
+
+document_store.write_documents(
+    [
+        Document(
+            content="There are over 7,000 languages spoken around the world today.",
+        ),
+        Document(
+            content="Elephants have been observed to recognize themselves in mirrors.",
+        ),
+    ],
+)
+print(document_store.count_documents())
+```
+
+To learn more about the initialization parameters, see the [API docs](/reference/integrations-falkordb#falkordbdocumentstore).
+
+To compute real embeddings for your documents, use a Document Embedder such as the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx).
+
+### Authentication
+
+To connect to a password-protected FalkorDB instance, pass the password via `Secret`:
+
+```python
+from haystack.utils import Secret
+from haystack_integrations.document_stores.falkordb import FalkorDBDocumentStore
+
+document_store = FalkorDBDocumentStore(
+    host="localhost",
+    port=6379,
+    password=Secret.from_env_var("FALKORDB_PASSWORD"),
+)
+```
+
+### Similarity Functions
+
+`FalkorDBDocumentStore` supports two similarity functions for vector search:
+
+- `"cosine"` (default): cosine similarity, best for normalized embeddings.
+- `"euclidean"`: Euclidean distance, useful when embedding magnitude matters.
+
+```python
+document_store = FalkorDBDocumentStore(
+    host="localhost",
+    port=6379,
+    embedding_dim=768,
+    similarity="euclidean",
+)
+```
+
+### Supported Retrievers
+
+- [`FalkorDBEmbeddingRetriever`](../pipeline-components/retrievers/falkordbembeddingretriever.mdx): Retrieves documents from the `FalkorDBDocumentStore` based on vector similarity using FalkorDB's native vector index.
+- [`FalkorDBCypherRetriever`](../pipeline-components/retrievers/falkordbcypherretriever.mdx): Retrieves documents by executing arbitrary OpenCypher queries, enabling graph traversal and multi-hop queries for GraphRAG pipelines.

docs-website/docs/document-stores/inmemorydocumentstore.mdx

@@ -0,0 +1,27 @@
+---
+title: "InMemoryDocumentStore"
+id: inmemorydocumentstore
+slug: "/inmemorydocumentstore"
+---
+
+# InMemoryDocumentStore
+
+The `InMemoryDocumentStore` is a very simple document store with no extra services or dependencies.
+
+It is great for experimenting with Haystack, however we do not recommend using it for production.
+
+### Initialization
+
+`InMemoryDocumentStore` requires no external setup. Simply use this code:
+
+```python
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+
+document_store = InMemoryDocumentStore()
+```
+
+### Supported Retrievers
+
+[`InMemoryBM25Retriever`](../pipeline-components/retrievers/inmemorybm25retriever.mdx): A keyword-based Retriever that fetches documents matching a query from a temporary in-memory database.
+
+[`InMemoryEmbeddingRetriever`](../pipeline-components/retrievers/inmemoryembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.

docs-website/docs/document-stores/mongodbatlasdocumentstore.mdx

@@ -0,0 +1,59 @@
+---
+title: "MongoDBAtlasDocumentStore"
+id: mongodbatlasdocumentstore
+slug: "/mongodbatlasdocumentstore"
+---
+
+# MongoDBAtlasDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [MongoDB Atlas](/reference/integrations-mongodb-atlas)                                                 |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mongodb_atlas |
+
+</div>
+
+`MongoDBAtlasDocumentStore` can be used to manage documents using [MongoDB Atlas](https://www.mongodb.com/atlas),  a multi-cloud database service by the same people who build MongoDB. Atlas simplifies deploying and managing your databases while offering the versatility you need to build resilient and performant global applications on the cloud providers of your choice. You can use MongoDB Atlas on cloud providers such as AWS, Azure, or Google Cloud, all without leaving Atlas' web UI.
+
+MongoDB Atlas supports embeddings and can therefore be used for embedding retrieval.
+
+## Installation
+
+To use MongoDB Atlas with Haystack, install the integration first:
+
+```shell
+pip install mongodb-atlas-haystack
+```
+
+## Initialization
+
+To use MongoDB Atlas with Haystack, you will need to create your MongoDB Atlas account: check the [MongoDB Atlas documentation](https://www.mongodb.com/docs/atlas/getting-started/) for help. You also need to [create a vector search index](https://www.mongodb.com/docs/atlas/atlas-vector-search/create-index/#std-label-avs-create-index)  and [a full-text search index](https://www.mongodb.com/docs/atlas/atlas-search/manage-indexes/#create-an-atlas-search-index) for the collection you plan to use.
+
+Once you have your connection string, you should export it in an environment variable called `MONGO_CONNECTION_STRING`. It should look something like this:
+
+```python
+export MONGO_CONNECTION_STRING="mongodb+srv://<username>:<password>@<cluster_name>.gwkckbk.mongodb.net/?retryWrites=true&w=majority"
+```
+
+At this point, you’re ready to initialize the store:
+
+```python
+from haystack_integrations.document_stores.mongodb_atlas import (
+    MongoDBAtlasDocumentStore,
+)
+
+# Initialize the document store
+document_store = MongoDBAtlasDocumentStore(
+    database_name="haystack_test",
+    collection_name="test_collection",
+    vector_search_index="embedding_index",
+    full_text_search_index="search_index",
+)
+```
+
+## Supported Retrievers
+
+- [`MongoDBAtlasEmbeddingRetriever`](../pipeline-components/retrievers/mongodbatlasembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.
+- [`MongoDBAtlasFullTextRetriever`](../pipeline-components/retrievers/mongodbatlasfulltextretriever.mdx): A full-text search Retriever.

docs-website/docs/document-stores/opensearch-document-store.mdx

@@ -0,0 +1,82 @@
+---
+title: "OpenSearchDocumentStore"
+id: opensearch-document-store
+slug: "/opensearch-document-store"
+description: "A Document Store for storing and retrieval from OpenSearch."
+---
+
+# OpenSearchDocumentStore
+
+A Document Store for storing and retrieval from OpenSearch.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [OpenSearch](/reference/integrations-opensearch)                                                    |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch |
+
+</div>
+
+OpenSearch is a fully open source search and analytics engine for use cases such as log analytics, real-time application monitoring, and clickstream analysis. For more information, see the [OpenSearch documentation](https://opensearch.org/docs/).
+
+This Document Store is great if you want to evaluate the performance of different retrieval options (dense vs. sparse). It’s compatible with the Amazon OpenSearch Service.
+
+OpenSearch provides support for vector similarity comparisons and approximate nearest neighbors algorithms.
+
+### Initialization
+
+[Install](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/) and run an OpenSearch instance.
+
+If you have Docker set up, we recommend pulling the Docker image and running it.
+
+```shell
+docker pull opensearchproject/opensearch:3.5.0
+docker run \
+    -p 9200:9200 \
+    -p 9600:9600 \
+    -e "discovery.type=single-node" \
+    -e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" \
+    -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=SecureHaystack*2026" \
+    opensearchproject/opensearch:3.5.0
+```
+
+As an alternative, you can go to [OpenSearch integration GitHub](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opensearch) and start a Docker container running OpenSearch using the provided `docker-compose.yml`:
+
+```shell
+docker compose up
+```
+
+Once you have a running OpenSearch instance, install the `opensearch-haystack` integration:
+
+```shell
+pip install opensearch-haystack
+```
+
+Then, initialize an `OpenSearchDocumentStore` object that’s connected to the OpenSearch instance and writes documents to it:
+
+```python
+from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore
+from haystack import Document
+
+document_store = OpenSearchDocumentStore(
+    hosts="http://localhost:9200",
+    use_ssl=True,
+    verify_certs=False,
+    http_auth=("admin", "SecureHaystack*2026"),
+)
+document_store.write_documents(
+    [Document(content="This is first"), Document(content="This is second")],
+)
+print(document_store.count_documents())
+```
+
+### Supported Retrievers
+
+[`OpenSearchBM25Retriever`](../pipeline-components/retrievers/opensearchbm25retriever.mdx): A keyword-based Retriever that fetches documents matching a query from the Document Store.
+
+[`OpenSearchEmbeddingRetriever`](../pipeline-components/retrievers/opensearchembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.
+
+## Additional References
+
+🧑‍🍳 Cookbook: [PDF-Based Question Answering with Amazon Bedrock and Haystack](https://haystack.deepset.ai/cookbook/amazon_bedrock_for_documentation_qa)

docs-website/docs/document-stores/oracledocumentstore.mdx

@@ -0,0 +1,196 @@
+---
+title: "OracleDocumentStore"
+id: oracledocumentstore
+slug: "/oracledocumentstore"
+description: "Use Oracle AI Vector Search as a document store in Haystack, with vector similarity and keyword search powered by Oracle Database 23ai."
+---
+
+# OracleDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Oracle](/reference/integrations-oracle)                                                      |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/oracle |
+
+</div>
+
+`OracleDocumentStore` is a Document Store backed by [Oracle AI Vector Search](https://www.oracle.com/database/ai-vector-search/), available in Oracle Database 23ai and later.
+It stores documents alongside dense vector embeddings in a native `VECTOR` column, and supports both vector similarity search and keyword search via an automatically managed DBMS_SEARCH index.
+
+## Installation
+
+```shell
+pip install oracle-haystack
+```
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+## Connection
+
+`OracleDocumentStore` connects to Oracle using the `OracleConnectionConfig` dataclass, which supports two connection modes:
+
+- **Thin mode** (default): connects directly over TCP. No Oracle Instant Client required.
+- **Thick mode**: activated automatically when `wallet_location` is provided. Used for Oracle Autonomous Database (ADB-S) connections.
+
+Set the connection parameters as environment variables:
+
+```shell
+export ORACLE_USER="haystack"
+export ORACLE_PASSWORD="secret"
+export ORACLE_DSN="localhost:1521/freepdb1"
+```
+
+## Initialization
+
+```python
+from haystack.utils import Secret
+from haystack_integrations.document_stores.oracle import (
+    OracleDocumentStore,
+    OracleConnectionConfig,
+)
+
+document_store = OracleDocumentStore(
+    connection_config=OracleConnectionConfig(
+        user=Secret.from_env_var("ORACLE_USER"),
+        password=Secret.from_env_var("ORACLE_PASSWORD"),
+        dsn=Secret.from_env_var("ORACLE_DSN"),
+    ),
+    embedding_dim=768,
+)
+```
+
+To learn more about the initialization parameters, see the [API docs](/reference/integrations-oracle#oracledocumentstore).
+
+### Connecting to Oracle Autonomous Database
+
+For Oracle Autonomous Database (ADB-S), provide a wallet for authentication. The store automatically activates thick mode when `wallet_location` is set:
+
+```python
+document_store = OracleDocumentStore(
+    connection_config=OracleConnectionConfig(
+        user=Secret.from_env_var("ORACLE_USER"),
+        password=Secret.from_env_var("ORACLE_PASSWORD"),
+        dsn=Secret.from_env_var("ORACLE_DSN"),
+        wallet_location="/path/to/wallet",
+        wallet_password=Secret.from_env_var("WALLET_PASSWORD"),
+    ),
+    embedding_dim=1536,
+)
+```
+
+### HNSW Vector Index
+
+By default, the store performs exact vector search. To enable approximate nearest-neighbor search (faster on large datasets), create an HNSW index:
+
+```python
+document_store = OracleDocumentStore(
+    connection_config=OracleConnectionConfig(
+        user=Secret.from_env_var("ORACLE_USER"),
+        password=Secret.from_env_var("ORACLE_PASSWORD"),
+        dsn=Secret.from_env_var("ORACLE_DSN"),
+    ),
+    embedding_dim=768,
+    distance_metric="COSINE",
+    create_index=True,  # creates the HNSW index on startup
+    hnsw_neighbors=32,
+    hnsw_ef_construction=200,
+    hnsw_accuracy=95,
+)
+```
+
+## Supported Retrievers
+
+- [`OracleEmbeddingRetriever`](../pipeline-components/retrievers/oracleembeddingretriever.mdx): Retrieves documents from `OracleDocumentStore` based on vector similarity to a query embedding.
+- [`OracleKeywordRetriever`](../pipeline-components/retrievers/oraclekeywordretriever.mdx): Retrieves documents matching a keyword query using Oracle's DBMS_SEARCH full-text index.
+
+## Example: RAG pipeline
+
+```python
+from haystack import Document, Pipeline
+from haystack.document_stores.types import DuplicatePolicy
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersDocumentEmbedder,
+    SentenceTransformersTextEmbedder,
+)
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.utils import Secret
+
+from haystack_integrations.document_stores.oracle import (
+    OracleDocumentStore,
+    OracleConnectionConfig,
+)
+from haystack_integrations.components.retrievers.oracle import OracleEmbeddingRetriever
+
+document_store = OracleDocumentStore(
+    connection_config=OracleConnectionConfig(
+        user=Secret.from_env_var("ORACLE_USER"),
+        password=Secret.from_env_var("ORACLE_PASSWORD"),
+        dsn=Secret.from_env_var("ORACLE_DSN"),
+    ),
+    embedding_dim=768,
+)
+
+# Index documents
+documents = [
+    Document(content="There are over 7,000 languages spoken around the world today."),
+    Document(
+        content="Elephants have been observed to behave in a way that indicates a high level of self-awareness.",
+    ),
+    Document(
+        content="In certain places, you can witness the phenomenon of bioluminescent waves.",
+    ),
+]
+
+doc_embedder = SentenceTransformersDocumentEmbedder(
+    model="sentence-transformers/all-MiniLM-L6-v2",
+)
+embedded_docs = doc_embedder.run(documents)["documents"]
+document_store.write_documents(embedded_docs, policy=DuplicatePolicy.OVERWRITE)
+
+# Build a RAG pipeline
+template = [
+    ChatMessage.from_user(
+        """
+        Given the following context, answer the question.
+        Context: {% for doc in documents %}{{ doc.content }}{% endfor %}
+        Question: {{ query }}
+        """,
+    ),
+]
+
+pipeline = Pipeline()
+pipeline.add_component(
+    "embedder",
+    SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"),
+)
+pipeline.add_component(
+    "retriever",
+    OracleEmbeddingRetriever(document_store=document_store, top_k=3),
+)
+pipeline.add_component("prompt_builder", ChatPromptBuilder(template=template))
+pipeline.add_component(
+    "llm",
+    OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
+)
+
+pipeline.connect("embedder.embedding", "retriever.query_embedding")
+pipeline.connect("retriever.documents", "prompt_builder.documents")
+pipeline.connect("prompt_builder.prompt", "llm.messages")
+
+result = pipeline.run(
+    {
+        "embedder": {"text": "How many languages are there?"},
+        "prompt_builder": {"query": "How many languages are there?"},
+    },
+)
+
+print(result["llm"]["replies"][0].text)
+```

docs-website/docs/document-stores/pgvectordocumentstore.mdx

@@ -0,0 +1,109 @@
+---
+title: "PgvectorDocumentStore"
+id: pgvectordocumentstore
+slug: "/pgvectordocumentstore"
+---
+
+# PgvectorDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Pgvector](/reference/integrations-pgvector)                                                       |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/pgvector/ |
+
+</div>
+
+Pgvector is an extension for PostgreSQL that enhances its capabilities with vector similarity search. It builds upon the classic features of PostgreSQL, such as ACID compliance and point-in-time recovery, and introduces the ability to perform exact and approximate nearest neighbor search using vectors.
+
+For more information, see the [pgvector repository](https://github.com/pgvector/pgvector).
+
+Pgvector Document Store supports embedding retrieval and metadata filtering.
+
+## Installation
+
+To quickly set up a PostgreSQL database with pgvector, you can use Docker:
+
+```shell
+docker run -d -p 5432:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=postgres ankane/pgvector
+```
+
+For more information on installing pgvector, visit the [pgvector GitHub repository](https://github.com/pgvector/pgvector).
+
+To use pgvector with Haystack, install the `pgvector-haystack` integration:
+
+```shell
+pip install pgvector-haystack
+```
+
+## Usage
+
+### Connection String
+
+Define the connection string to your PostgreSQL database in the `PG_CONN_STR` environment variable. Two formats are supported:
+
+**URI format:**
+
+```shell
+export PG_CONN_STR="postgresql://USER:PASSWORD@HOST:PORT/DB_NAME"
+```
+
+**Keyword/value format:**
+
+```shell
+export PG_CONN_STR="host=HOST port=PORT dbname=DB_NAME user=USER password=PASSWORD"
+```
+
+:::caution[Special Characters in Connection URIs]
+
+When using the URI format, special characters in the password must be [percent-encoded](https://en.wikipedia.org/wiki/Percent-encoding). Otherwise, connection errors may occur. A password like `p=ssword` would cause the error `psycopg.OperationalError: [Errno -2] Name or service not known`.
+
+For example, if your password is `p=ssword`, the connection string should be:
+
+```shell
+export PG_CONN_STR="postgresql://postgres:p%3Dssword@localhost:5432/postgres"
+```
+
+Alternatively, use the keyword/value format, which does not require percent-encoding:
+
+```shell
+export PG_CONN_STR="host=localhost port=5432 dbname=postgres user=postgres password=p=ssword"
+```
+
+:::
+
+For more details, see the [PostgreSQL connection string documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING).
+
+## Initialization
+
+Initialize a `PgvectorDocumentStore` object that’s connected to the PostgreSQL database and writes documents to it:
+
+```python
+from haystack_integrations.document_stores.pgvector import PgvectorDocumentStore
+from haystack import Document
+
+document_store = PgvectorDocumentStore(
+    embedding_dimension=768,
+    vector_function="cosine_similarity",
+    recreate_table=True,
+    search_strategy="hnsw",
+)
+
+document_store.write_documents(
+    [
+        Document(content="This is first", embedding=[0.1] * 768),
+        Document(content="This is second", embedding=[0.3] * 768),
+    ],
+)
+print(document_store.count_documents())
+```
+
+To learn more about the initialization parameters, see our [API docs](/reference/integrations-pgvector#pgvectordocumentstore).
+
+To properly compute embeddings for your documents, you can use a Document Embedder (for instance, the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx)).
+
+### Supported Retrievers
+
+- [`PgvectorEmbeddingRetriever`](../pipeline-components/retrievers/pgvectorembeddingretriever.mdx): An embedding-based Retriever that fetches documents from the Document Store based on a query embedding provided to the Retriever.
+- [`PgvectorKeywordRetriever`](../pipeline-components/retrievers/pgvectorembeddingretriever.mdx): A keyword-based Retriever that fetches documents matching a query from the Pgvector Document Store.

docs-website/docs/document-stores/pinecone-document-store.mdx

@@ -0,0 +1,67 @@
+---
+title: "PineconeDocumentStore"
+id: pinecone-document-store
+slug: "/pinecone-document-store"
+description: "Use a Pinecone vector database with Haystack."
+---
+
+# PineconeDocumentStore
+
+Use a Pinecone vector database with Haystack.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Pinecone](/reference/integrations-pinecone)                                                      |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/pinecone |
+
+</div>
+
+[Pinecone](https://www.pinecone.io/) is a cloud-based vector database. It is fast and easy to use.
+Unlike other solutions (such as Qdrant and Weaviate), it can’t run locally on the user's machine but provides a generous free tier.
+
+### Installation
+
+You can simply install the Pinecone Haystack integration with:
+
+```shell
+pip install pinecone-haystack
+```
+
+### Initialization
+
+- To use Pinecone as a Document Store in Haystack, sign up for a free Pinecone [account](https://app.pinecone.io/) and get your API key.
+  The Pinecone API key can be explicitly provided or automatically read from the environment variable `PINECONE_API_KEY` (recommended).
+- In Haystack, each `PineconeDocumentStore` operates in a specific namespace of an index. If not provided, both index and namespace are `default`.
+  If the index already exists, the Document Store connects to it. Otherwise, it creates a new index.
+- When creating a new index, you can provide a `spec` in the form of a dictionary. This allows choosing between serverless and pod deployment options and setting additional parameters. Refer to the [Pinecone documentation](https://docs.pinecone.io/reference/api/control-plane/create_index) for more details. If not provided, a default spec with serverless deployment in the `us-east-1` region will be used (compatible with the free tier).
+- You can provide `dimension` and `metric`, but they are only taken into account if the Pinecone index does not already exist.
+
+Then, you can use the Document Store like this:
+
+```python
+from haystack import Document
+from haystack_integrations.document_stores.pinecone import PineconeDocumentStore
+
+# Make sure you have the PINECONE_API_KEY environment variable set
+document_store = PineconeDocumentStore(
+    index="default",
+    namespace="default",
+    dimension=5,
+    metric="cosine",
+    spec={"serverless": {"region": "us-east-1", "cloud": "aws"}},
+)
+
+document_store.write_documents(
+    [
+        Document(content="This is first", embedding=[0.1] * 5),
+        Document(content="This is second", embedding=[0.1, 0.2, 0.3, 0.4, 0.5]),
+    ],
+)
+print(document_store.count_documents())
+```
+
+### Supported Retrievers
+
+[`PineconeEmbeddingRetriever`](../pipeline-components/retrievers/pineconedenseretriever.mdx): Retrieves documents from the `PineconeDocumentStore` based on their dense embeddings (vectors).

docs-website/docs/document-stores/qdrant-document-store.mdx

@@ -0,0 +1,103 @@
+---
+title: "QdrantDocumentStore"
+id: qdrant-document-store
+slug: "/qdrant-document-store"
+description: "Use the Qdrant vector database with Haystack."
+---
+
+# QdrantDocumentStore
+
+Use the Qdrant vector database with Haystack.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Qdrant](/reference/integrations-qdrant)                                                        |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/qdrant |
+
+</div>
+
+Qdrant is a powerful high-performance, massive-scale vector database. The `QdrantDocumentStore` can be used with any Qdrant instance, in-memory, locally persisted, hosted, and the official Qdrant Cloud.
+
+### Installation
+
+You can simply install the Qdrant Haystack integration with:
+
+```shell
+pip install qdrant-haystack
+```
+
+### Initialization
+
+The quickest way to use `QdrantDocumentStore` is to create an in-memory instance of it:
+
+```python
+from haystack.dataclasses.document import Document
+from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
+
+document_store = QdrantDocumentStore(
+    ":memory:",
+    recreate_index=True,
+    return_embedding=True,
+    wait_result_from_api=True,
+)
+document_store.write_documents(
+    [
+        Document(content="This is first", embedding=[0.0] * 768),
+        Document(content="This is second", embedding=[0.1] * 768),
+    ],
+)
+print(document_store.count_documents())
+```
+
+:::warning[Collections Created Outside Haystack]
+
+When you create a `QdrantDocumentStore` instance, Haystack takes care of setting up the collection. In general, you cannot use a Qdrant collection created without Haystack with Haystack. If you want to migrate your existing collection, see the sample script at https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/qdrant/src/haystack_integrations/document_stores/qdrant/migrate_to_sparse.py.
+:::
+
+You can also connect directly to [Qdrant Cloud](https://cloud.qdrant.io/login). Once you have your API key and your cluster URL from the Qdrant dashboard, you can connect like this:
+
+```python
+from haystack.dataclasses.document import Document
+from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
+from haystack.utils import Secret
+
+document_store = QdrantDocumentStore(
+    url="https://XXXXXXXXX.us-east4-0.gcp.cloud.qdrant.io:6333",
+    index="your_index_name",
+    embedding_dim=5,  # based on the embedding model
+    recreate_index=True,  # enable only to recreate the index and not connect to the existing one
+    api_key=Secret.from_token("YOUR_TOKEN"),
+)
+
+document_store.write_documents(
+    [
+        Document(content="This is first", embedding=[0.0] * 5),
+        Document(content="This is second", embedding=[0.1, 0.2, 0.3, 0.4, 0.5]),
+    ],
+)
+print(document_store.count_documents())
+```
+
+:::tip[More information]
+
+You can find more ways to initialize and use QdrantDocumentStore on our [integration page](https://haystack.deepset.ai/integrations/qdrant-document-store).
+:::
+
+### Supported Retrievers
+
+- [`QdrantEmbeddingRetriever`](../pipeline-components/retrievers/qdrantembeddingretriever.mdx): Retrieves documents from the `QdrantDocumentStore` based on their dense embeddings (vectors).
+- [`QdrantSparseEmbeddingRetriever`](../pipeline-components/retrievers/qdrantsparseembeddingretriever.mdx): Retrieves documents from the `QdrantDocumentStore` based on their sparse embeddings.
+- [`QdrantHybridRetriever`](../pipeline-components/retrievers/qdranthybridretriever.mdx): Retrieves documents from the `QdrantDocumentStore` based on both dense and sparse embeddings.
+
+:::note[Sparse Embedding Support]
+
+To use Sparse Embedding support, you need to initialize the `QdrantDocumentStore` with `use_sparse_embeddings=True`, which is `False` by default.
+
+If you want to use Document Store or collection previously created with this feature disabled, you must migrate the existing data. You can do this by taking advantage of the `migrate_to_sparse_embeddings_support` utility function.
+:::
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Sparse Embedding Retrieval with Qdrant and FastEmbed](https://haystack.deepset.ai/cookbook/sparse_embedding_retrieval)

docs-website/docs/document-stores/supabasedocumentstore.mdx

@@ -0,0 +1,188 @@
+---
+title: "SupabaseDocumentStore"
+id: supabasedocumentstore
+slug: "/supabasedocumentstore"
+description: "Use Supabase as a document store in Haystack, with vector search (pgvector) or full-text search (PGroonga)."
+---
+
+# SupabaseDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Supabase](/reference/integrations-supabase)                                                       |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/supabase/ |
+
+</div>
+
+[Supabase](https://supabase.com/) is an open-source backend platform built on PostgreSQL. The Supabase integration for Haystack provides two document stores:
+
+- **`SupabasePgvectorDocumentStore`** — vector similarity search using the [pgvector](https://github.com/pgvector/pgvector) PostgreSQL extension, which comes pre-installed on Supabase.
+- **`SupabaseGroongaDocumentStore`** — multilingual full-text search using the [PGroonga](https://pgroonga.github.io/) PostgreSQL extension. No embeddings required.
+
+## Installation
+
+```shell
+pip install supabase-haystack
+```
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+## SupabasePgvectorDocumentStore
+
+`SupabasePgvectorDocumentStore` is a thin wrapper around [`PgvectorDocumentStore`](./pgvectordocumentstore.mdx) with Supabase-specific defaults:
+
+- Reads the connection string from the `SUPABASE_DB_URL` environment variable.
+- Defaults `create_extension` to `False` since pgvector is pre-installed on Supabase.
+
+### Connection
+
+Set the `SUPABASE_DB_URL` environment variable with your Supabase database connection string.
+
+:::tip[Use session mode (port 5432)]
+Supabase offers two pooler ports: transaction mode (port 6543) and session mode (port 5432). For best compatibility with pgvector operations, use session mode or a direct connection.
+:::
+
+```shell
+export SUPABASE_DB_URL="postgresql://postgres.[project-ref]:[password]@aws-0-[region].pooler.supabase.com:5432/postgres"
+```
+
+### Initialization
+
+```python
+from haystack_integrations.document_stores.supabase import SupabasePgvectorDocumentStore
+
+document_store = SupabasePgvectorDocumentStore(
+    embedding_dimension=768,
+    vector_function="cosine_similarity",
+    recreate_table=True,
+)
+```
+
+To learn more about the initialization parameters, see the [API docs](/reference/integrations-supabase#supabasepgvectordocumentstore).
+
+### Supported Retrievers
+
+- [`SupabasePgvectorEmbeddingRetriever`](/reference/integrations-supabase#supabasepgvectorembeddingretriever): Fetches documents from the store based on a query embedding.
+- [`SupabasePgvectorKeywordRetriever`](/reference/integrations-supabase#supabasepgvectorkeywordretriever): Fetches documents matching a keyword query using PostgreSQL's `ts_rank_cd` ranking.
+
+### Example: RAG pipeline
+
+```python
+from haystack import Document, Pipeline
+from haystack.document_stores.types.policy import DuplicatePolicy
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersTextEmbedder,
+    SentenceTransformersDocumentEmbedder,
+)
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.utils import Secret
+
+from haystack_integrations.document_stores.supabase import SupabasePgvectorDocumentStore
+from haystack_integrations.components.retrievers.supabase import (
+    SupabasePgvectorEmbeddingRetriever,
+)
+
+document_store = SupabasePgvectorDocumentStore(
+    embedding_dimension=768,
+    vector_function="cosine_similarity",
+    recreate_table=True,
+)
+
+# Index documents
+documents = [
+    Document(content="There are over 7,000 languages spoken around the world today."),
+    Document(
+        content="Elephants have been observed to behave in a way that indicates a high level of self-awareness.",
+    ),
+    Document(
+        content="In certain places, you can witness the phenomenon of bioluminescent waves.",
+    ),
+]
+embedder = SentenceTransformersDocumentEmbedder()
+documents_with_embeddings = embedder.run(documents)
+document_store.write_documents(
+    documents_with_embeddings["documents"],
+    policy=DuplicatePolicy.OVERWRITE,
+)
+
+# Query pipeline
+prompt_template = [
+    ChatMessage.from_system("Answer the question based on the provided context."),
+    ChatMessage.from_user(
+        "Query: {{query}}\nDocuments:\n{% for doc in documents %}{{ doc.content }}\n{% endfor %}\nAnswer:",
+    ),
+]
+
+query_pipeline = Pipeline()
+query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
+query_pipeline.add_component(
+    "retriever",
+    SupabasePgvectorEmbeddingRetriever(document_store=document_store),
+)
+query_pipeline.add_component(
+    "prompt_builder",
+    ChatPromptBuilder(
+        template=prompt_template,
+        required_variables=["query", "documents"],
+    ),
+)
+query_pipeline.add_component("generator", OpenAIChatGenerator(model="gpt-4o"))
+query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
+query_pipeline.connect("retriever.documents", "prompt_builder.documents")
+query_pipeline.connect("prompt_builder.prompt", "generator.messages")
+
+result = query_pipeline.run(
+    {
+        "text_embedder": {"text": "How many languages are there?"},
+        "prompt_builder": {"query": "How many languages are there?"},
+    },
+)
+```
+
+---
+
+## SupabaseGroongaDocumentStore
+
+`SupabaseGroongaDocumentStore` uses [PGroonga](https://pgroonga.github.io/), a PostgreSQL extension for fast, multilingual full-text search. Unlike the pgvector store, it works with plain text queries and requires no embeddings.
+
+### Prerequisites
+
+PGroonga must be enabled in your Supabase project. Run the following SQL in the Supabase SQL editor:
+
+```sql
+CREATE EXTENSION IF NOT EXISTS pgroonga;
+```
+
+You also need to create a SQL function that PGroonga uses for search. See the [integration README](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/supabase/) for the required function definition.
+
+### Initialization
+
+```python
+from haystack_integrations.document_stores.supabase import SupabaseGroongaDocumentStore
+from haystack.utils import Secret
+
+document_store = SupabaseGroongaDocumentStore(
+    supabase_url="https://<project-ref>.supabase.co",
+    supabase_key=Secret.from_env_var("SUPABASE_SERVICE_KEY"),
+    table_name="haystack_groonga_documents",
+)
+document_store.warm_up()
+```
+
+:::note
+`warm_up()` must be called before using the store. It initializes the Supabase client and creates the table and PGroonga index if they don't exist.
+:::
+
+To learn more about the initialization parameters, see the [API docs](/reference/integrations-supabase).
+
+### Supported Retrievers
+
+- [`SupabaseGroongaBM25Retriever`](/reference/integrations-supabase): Retrieves documents using PGroonga full-text search. Works without embeddings and can be combined with `SupabasePgvectorEmbeddingRetriever` for hybrid search pipelines.

docs-website/docs/document-stores/valkeydocumentstore.mdx

@@ -0,0 +1,180 @@
+---
+title: "ValkeyDocumentStore"
+id: valkeydocumentstore
+slug: "/valkeydocumentstore"
+description: "Use a Valkey database with Haystack."
+---
+
+# ValkeyDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Valkey](/reference/integrations-valkey)                                                      |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/valkey |
+
+</div>
+
+[Valkey](https://valkey.io/) is a high-performance, in-memory data structure store that you can use in Haystack pipelines with the `ValkeyDocumentStore`. Valkey operates in-memory by default for maximum performance, but can be configured with persistence options for data durability.
+
+The `ValkeyDocumentStore` connects to a Valkey server with the search module running and supports vector similarity search for RAG and other retrieval use cases. For a detailed overview of all the available methods and settings, visit the [API Reference](/reference/integrations-valkey#valkeydocumentstore).
+
+## Installation
+
+You can install the Valkey Haystack integration with:
+
+```shell
+pip install valkey-haystack
+```
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+## Initialization
+
+To use Valkey as your data storage for Haystack pipelines, you need a Valkey server with the search module running. Initialize a `ValkeyDocumentStore` like this:
+
+```python
+from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
+
+document_store = ValkeyDocumentStore(
+    nodes_list=[("localhost", 6379)],
+    index_name="my_documents",
+    embedding_dim=768,
+    distance_metric="cosine",
+)
+```
+
+### Running Valkey locally
+
+For development and testing, you can start a Valkey server with Docker:
+
+```shell
+docker run -d -p 6379:6379 valkey/valkey-bundle:latest
+```
+
+Then connect with the same initialization code above, using `nodes_list=[("localhost", 6379)]`.
+
+For more advanced configurations and clustering setups, refer to the [Valkey documentation](https://valkey.io/docs/).
+
+## Writing documents
+
+To write documents to your `ValkeyDocumentStore`, create an indexing pipeline or use the `write_documents()` method. You can use [Converters](../pipeline-components/converters.mdx), [PreProcessors](../pipeline-components/preprocessors.mdx), and other integrations to fetch and prepare data. Below is an example that indexes Markdown files into Valkey.
+
+### Indexing pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.converters import MarkdownToDocument
+from haystack.components.writers import DocumentWriter
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersDocumentEmbedder,
+)
+from haystack.components.preprocessors import DocumentSplitter
+from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
+
+document_store = ValkeyDocumentStore(
+    nodes_list=[("localhost", 6379)],
+    index_name="my_documents",
+    embedding_dim=768,
+    distance_metric="cosine",
+)
+
+indexing = Pipeline()
+indexing.add_component("converter", MarkdownToDocument())
+indexing.add_component(
+    "splitter",
+    DocumentSplitter(split_by="sentence", split_length=2),
+)
+indexing.add_component("embedder", SentenceTransformersDocumentEmbedder())
+indexing.add_component("writer", DocumentWriter(document_store))
+indexing.connect("converter", "splitter")
+indexing.connect("splitter", "embedder")
+indexing.connect("embedder", "writer")
+
+indexing.run({"converter": {"sources": ["filename.md"]}})
+```
+
+## Using Valkey in a RAG pipeline
+
+Once documents are in your `ValkeyDocumentStore`, you can use [`ValkeyEmbeddingRetriever`](../pipeline-components/retrievers/valkeyembeddingretriever.mdx) to retrieve them. The following example builds a RAG pipeline with a custom prompt:
+
+```python
+from haystack import Pipeline
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersTextEmbedder,
+)
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
+from haystack_integrations.components.retrievers.valkey import ValkeyEmbeddingRetriever
+
+document_store = ValkeyDocumentStore(
+    nodes_list=[("localhost", 6379)],
+    index_name="my_documents",
+    embedding_dim=768,
+    distance_metric="cosine",
+)
+
+prompt_template = [
+    ChatMessage.from_system(
+        "Answer the question based on the provided context. If the context does not include an answer, reply with 'I don't know'.",
+    ),
+    ChatMessage.from_user(
+        "Query: {{query}}\n"
+        "Documents:\n{% for doc in documents %}{{ doc.content }}\n{% endfor %}\n"
+        "Answer:",
+    ),
+]
+
+query_pipeline = Pipeline()
+query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
+query_pipeline.add_component(
+    "retriever",
+    ValkeyEmbeddingRetriever(document_store=document_store),
+)
+query_pipeline.add_component(
+    "prompt_builder",
+    ChatPromptBuilder(
+        template=prompt_template,
+        required_variables=["query", "documents"],
+    ),
+)
+query_pipeline.add_component(
+    "generator",
+    OpenAIChatGenerator(
+        api_key=Secret.from_token("YOUR_OPENAI_API_KEY"),
+        model="gpt-4o",
+    ),
+)
+query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
+query_pipeline.connect("retriever.documents", "prompt_builder.documents")
+query_pipeline.connect("prompt_builder.prompt", "generator.messages")
+
+query = "What is Valkey?"
+results = query_pipeline.run(
+    {
+        "text_embedder": {"text": query},
+        "prompt_builder": {"query": query},
+    },
+)
+```
+
+For more examples, see the [examples folder](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/valkey/examples) in the repository.
+
+## Performance benefits
+
+- **In-memory storage**: Fast read and write operations.
+- **High throughput**: Handles many operations per second.
+- **Low latency**: Minimal response times for document operations.
+- **Scalability**: Supports clustering for horizontal scaling.
+
+## Supported Retrievers
+
+[`ValkeyEmbeddingRetriever`](../pipeline-components/retrievers/valkeyembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query from the `ValkeyDocumentStore`.

docs-website/docs/document-stores/vespadocumentstore.mdx

@@ -0,0 +1,115 @@
+---
+title: "VespaDocumentStore"
+id: vespadocumentstore
+slug: "/vespadocumentstore"
+---
+
+# VespaDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Vespa](/reference/integrations-vespa)                                              |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/vespa |
+
+</div>
+
+[Vespa](https://vespa.ai/) is an open-source big data serving engine that supports structured, text, and vector search at scale. The `VespaDocumentStore` connects Haystack to an existing Vespa application through [pyvespa](https://vespa-engine.github.io/pyvespa/) and supports both lexical and dense vector retrieval as well as metadata filtering.
+
+Unlike most other Haystack Document Stores, the `VespaDocumentStore` does **not** create or deploy the Vespa application or schema for you. You configure Vespa with the fields and rank profiles you need, deploy it (either self-hosted or on [Vespa Cloud](https://cloud.vespa.ai/)), and then point the Document Store at the running endpoint.
+
+## Installation
+
+Install the `vespa-haystack` integration:
+
+```shell
+pip install vespa-haystack
+```
+
+To run Vespa locally, see the [Vespa quick start](https://docs.vespa.ai/en/vespa-quick-start.html). To deploy a managed Vespa application, see [Vespa Cloud](https://cloud.vespa.ai/en/getting-started).
+
+## Usage
+
+### Prerequisites: Vespa Schema
+
+Before using the `VespaDocumentStore`, you need a deployed Vespa application with a schema compatible with the fields you configure on the Document Store. By default, the integration expects:
+
+- A text field named `content` for the Document body.
+- A tensor field named `embedding` for dense vectors (when using embedding retrieval).
+- A rank profile named `bm25` for lexical retrieval (used by `VespaKeywordRetriever`).
+- A rank profile named `semantic` that ranks with `closeness(field, embedding)` (used by `VespaEmbeddingRetriever`).
+
+Field and rank profile names can be customized via the Document Store and Retriever constructors. See the [Vespa documentation](https://docs.vespa.ai/en/schemas.html) for details on writing schemas and rank profiles.
+
+### Authentication
+
+The `VespaDocumentStore` supports the authentication methods provided by `pyvespa`:
+
+- **No authentication** for local development against an unsecured Vespa endpoint.
+- **mTLS** with a data plane certificate and key (via the `cert` and `key` parameters as [Secrets](../concepts/secret-management.mdx)).
+- **Bearer token** for Vespa Cloud token endpoints (via `vespa_cloud_secret_token` or the `VESPA_CLOUD_SECRET_TOKEN` environment variable).
+
+The Vespa endpoint URL can be passed via the `url` parameter or the `VESPA_URL` environment variable:
+
+```shell
+export VESPA_URL="http://localhost"
+```
+
+For Vespa Cloud token authentication:
+
+```shell
+export VESPA_URL="https://my-app.my-tenant.aws-us-east-1c.z.vespa-app.cloud"
+export VESPA_CLOUD_SECRET_TOKEN="my-secret-token"
+```
+
+## Initialization
+
+Point the `VespaDocumentStore` at your deployed Vespa application and write Documents to it. The HTTP client is created lazily on first use:
+
+```python
+from haystack import Document
+from haystack_integrations.document_stores.vespa import VespaDocumentStore
+
+document_store = VespaDocumentStore(
+    url="http://localhost",
+    schema="doc",
+    namespace="doc",
+    content_field="content",
+    embedding_field="embedding",
+    metadata_fields=["category"],
+)
+
+document_store.write_documents(
+    [
+        Document(
+            content="Haystack integrates with Vespa for search.",
+            meta={"category": "docs"},
+        ),
+        Document(
+            content="Vespa supports lexical and vector retrieval.",
+            meta={"category": "docs"},
+        ),
+    ],
+)
+print(document_store.count_documents())
+```
+
+To learn more about the initialization parameters, see our [API docs](/reference/integrations-vespa#vespadocumentstore).
+
+To compute embeddings for your Documents, you can use a Document Embedder, such as the [`SentenceTransformersDocumentEmbedder`](../pipeline-components/embedders/sentencetransformersdocumentembedder.mdx).
+
+### Metadata Fields
+
+Vespa is strictly schema-bound: every metadata field that you want to feed or read back from Vespa must exist as a field in the deployed schema. Use the `metadata_fields` parameter to declare an allowlist of metadata keys to send to Vespa on write and to request back on read. Metadata keys that are not in this allowlist are kept on Documents in memory but are not stored in Vespa.
+
+### Metadata Filtering
+
+The `VespaDocumentStore` supports comparison operators (`==`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `not in`) and the logical operators `AND`, `OR`, and `NOT`. Filters are translated to Vespa's [YQL](https://docs.vespa.ai/en/query-language.html) where clauses whenever possible.
+
+Filters on date-typed values are evaluated client-side in Python when YQL cannot express the comparison directly. For more details on filter syntax, refer to [Metadata Filtering](../concepts/metadata-filtering.mdx).
+
+### Supported Retrievers
+
+- [`VespaEmbeddingRetriever`](../pipeline-components/retrievers/vespaembeddingretriever.mdx): A dense embedding-based Retriever that fetches Documents from Vespa using nearest-neighbor search and a configurable rank profile.
+- [`VespaKeywordRetriever`](../pipeline-components/retrievers/vespakeywordretriever.mdx): A lexical Retriever that fetches Documents from Vespa using a configurable rank profile (BM25 by default).

docs-website/docs/document-stores/weaviatedocumentstore.mdx

@@ -0,0 +1,155 @@
+---
+title: "WeaviateDocumentStore"
+id: weaviatedocumentstore
+slug: "/weaviatedocumentstore"
+---
+
+# WeaviateDocumentStore
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| API reference | [Weaviate](/reference/integrations-weaviate)                                                      |
+| GitHub link   | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weaviate |
+
+</div>
+
+Weaviate is a multi-purpose vector DB that can store both embeddings and data objects, making it a good choice for multi-modality.
+
+The `WeaviateDocumentStore` can connect to any Weaviate instance, whether it's running on Weaviate Cloud Services, Kubernetes, or a local Docker container.
+
+## Installation
+
+You can simply install the Weaviate Haystack integration with:
+
+```shell
+pip install weaviate-haystack
+```
+
+## Initialization
+
+### Weaviate Embedded
+
+To use `WeaviateDocumentStore` as a temporary instance, initialize it as ["Embedded"](https://weaviate.io/developers/weaviate/installation/embedded):
+
+```python
+from haystack_integrations.document_stores.weaviate import WeaviateDocumentStore
+from weaviate.embedded import EmbeddedOptions
+
+document_store = WeaviateDocumentStore(embedded_options=EmbeddedOptions())
+```
+
+### Docker
+
+You can use `WeaviateDocumentStore` in a local Docker container. This is what a minimal `docker-compose.yml` could look like:
+
+```yaml
+---
+version: '3.4'
+services:
+  weaviate:
+    command:
+    - --host
+    - 0.0.0.0
+    - --port
+    - '8080'
+    - --scheme
+    - http
+    image: semitechnologies/weaviate:1.30.17
+    ports:
+    - 8080:8080
+    - 50051:50051
+    volumes:
+    - weaviate_data:/var/lib/weaviate
+    restart: 'no'
+    environment:
+      QUERY_DEFAULTS_LIMIT: 25
+      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
+      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
+      DEFAULT_VECTORIZER_MODULE: 'none'
+      ENABLE_MODULES: ''
+      CLUSTER_HOSTNAME: 'node1'
+volumes:
+  weaviate_data:
+...
+```
+
+:::warning
+With this example, we explicitly enable access without authentication, so you don't need to set any username, password, or API key to connect to our local instance. That is strongly discouraged for production use. See the [authorization](#authorization) section for detailed information.
+
+:::
+
+Start your container with `docker compose up -d` and then initialize the Document Store with:
+
+```python
+from haystack_integrations.document_stores.weaviate.document_store import (
+    WeaviateDocumentStore,
+)
+from haystack import Document
+
+document_store = WeaviateDocumentStore(url="http://localhost:8080")
+document_store.write_documents(
+    [Document(content="This is first"), Document(content="This is second")],
+)
+print(document_store.count_documents())
+```
+
+### Weaviate Cloud Service
+
+To use the [Weaviate managed cloud service](https://weaviate.io/developers/wcs), first, create your Weaviate cluster.
+
+Then, initialize the `WeaviateDocumentStore` using the API Key and URL found in your [Weaviate account](https://console.weaviate.cloud/):
+
+```python
+from haystack_integrations.document_stores.weaviate import (
+    WeaviateDocumentStore,
+    AuthApiKey,
+)
+from haystack import Document
+
+import os
+
+os.environ["WEAVIATE_API_KEY"] = "YOUR-API-KEY"
+
+auth_client_secret = AuthApiKey()
+
+document_store = WeaviateDocumentStore(
+    url="YOUR-WEAVIATE-URL",
+    auth_client_secret=auth_client_secret,
+)
+```
+
+## Authorization
+
+We provide some utility classes in the `auth` package to handle authorization using different credentials. Every class stores distinct [secrets](../concepts/secret-management.mdx) and retrieves them from the environment variables when required.
+
+The default environment variables for the classes are:
+
+- **`AuthApiKey`**
+  - `WEAVIATE_API_KEY`
+- **`AuthBearerToken`**
+  - `WEAVIATE_ACCESS_TOKEN`
+  - `WEAVIATE_REFRESH_TOKEN`
+- **`AuthClientCredentials`**
+  - `WEAVIATE_CLIENT_SECRET`
+  - `WEAVIATE_SCOPE`
+- **`AuthClientPassword`**
+  - `WEAVIATE_USERNAME`
+  - `WEAVIATE_PASSWORD`
+  - `WEAVIATE_SCOPE`
+
+You can easily change environment variables if needed. In the following snippet, we instruct `AuthApiKey` to look for `MY_ENV_VAR`.
+
+```python
+from haystack_integrations.document_stores.weaviate.auth import AuthApiKey
+from haystack.utils.auth import Secret
+
+AuthApiKey(api_key=Secret.from_env_var("MY_ENV_VAR"))
+```
+
+## Supported Retrievers
+
+[`WeaviateBM25Retriever`](../pipeline-components/retrievers/weaviatebm25retriever.mdx): A keyword-based Retriever that fetches documents matching a query from the Document Store.
+
+[`WeaviateEmbeddingRetriever`](../pipeline-components/retrievers/weaviateembeddingretriever.mdx): Compares the query and document embeddings and fetches the documents most relevant to the query.

docs-website/docs/intro.mdx

@@ -0,0 +1,32 @@
+---
+title: "Introduction to Haystack"
+id: intro
+description: "Haystack is an open-source AI framework to build production-ready LLM applications such as AI Agents, powerful RAG applications and scalable multimodal search systems. Learn more about Haystack and how it works."
+---
+
+# Introduction to Haystack
+
+Haystack is an **open-source AI framework** for building production-ready **AI Agents**, **powerful RAG applications** and **scalable multimodal search systems**. Build pipelines using reusable components, each responsible for specific tasks. Customize and extend pipelines to match your requirements. Learn more about Haystack and how it works.
+
+:::tip[Welcome to Haystack]
+
+To skip the introductions and go directly to installing and creating a search app, see  [Get Started](overview/get-started.mdx).
+:::
+
+Haystack is an open-source AI orchestration framework that you can use to build powerful, production-ready applications with Large Language Models (LLMs) for various use cases. Whether you’re creating autonomous agents, multimodal apps, or scalable RAG systems, Haystack provides the tools to move from idea to production easily.
+
+Haystack is designed in a modular way, allowing you to combine the best technology from OpenAI, Google, Anthropic, and open-source projects like Hugging Face's Transformers.
+
+The core foundation of Haystack consists of components and pipelines, along with Document Stores, Agents, Tools, and many integrations. Read more about Haystack concepts in the [Haystack Concepts Overview](concepts/concepts-overview.mdx).
+
+Supported by an engaged community of developers, Haystack has grown into a comprehensive and user-friendly framework for LLM-based development.
+
+:::note[Looking to scale with confidence?]
+
+If your team needs **enterprise-grade support, best practices, and deployment guidance** to run Haystack in production, check out **Haystack Enterprise Starter**.
+
+📜 [Learn more about Haystack Enterprise Starter](https://haystack.deepset.ai/blog/announcing-haystack-enterprise)  
+🤝 [Get in touch with our team](https://www.deepset.ai/products-and-services/haystack-enterprise-starter)
+
+👉 For platform tooling to **manage data, pipelines, testing, and governance at scale**, explore the [Haystack Enterprise Platform](https://www.deepset.ai/products-and-services/haystack-enterprise-platform).
+:::

docs-website/docs/memory-stores/cogneememorystore.mdx

@@ -0,0 +1,124 @@
+---
+title: "CogneeMemoryStore"
+id: cogneememorystore
+slug: "/cogneememorystore"
+description: "A persistent memory store backed by Cognee's knowledge graph API."
+---
+
+# CogneeMemoryStore
+
+`CogneeMemoryStore` is a persistent memory store backed by Cognee's knowledge graph API. It is the shared data layer used by [`CogneeRetriever`](../pipeline-components/retrievers/cogneeretriever.mdx) and [`CogneeWriter`](../pipeline-components/writers/cogneewriter.mdx).
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Used by** | [`CogneeRetriever`](../pipeline-components/retrievers/cogneeretriever.mdx), [`CogneeWriter`](../pipeline-components/writers/cogneewriter.mdx) |
+| **Optional init variables** | `search_type`, `top_k`, `dataset_name`, `session_id`, `self_improvement`, `timeout` |
+| **API reference** | [Cognee](/reference/integrations-cognee#cogneememorystore) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/cognee |
+| **Package name** | `cognee-haystack` |
+
+</div>
+
+## Overview
+
+`CogneeMemoryStore` wraps Cognee's V2 memory API:
+
+- `add_memories` → `cognee.remember`
+- `search_memories` → `cognee.recall`
+- `improve` → `cognee.improve`
+- `delete_all_memories` → `cognee.forget`
+
+Cognee supports two memory tiers. Set `session_id` to use the **session cache** — fast writes with no LLM extraction, session-aware recall. Leave `session_id` as `None` to write to the **permanent knowledge graph**, which uses LLM extraction during ingestion and supports richer graph-completion queries.
+
+Cognee configuration (LLM provider, database, vector store) is read from environment variables. See the [Cognee documentation](https://docs.cognee.ai) for setup instructions.
+
+### Parameters
+
+- `search_type` is *optional* and defaults to `"GRAPH_COMPLETION"`. Controls which Cognee recall strategy is used. Other useful values include `"CHUNKS"` for raw retrieval and `"SUMMARIES"` for summarized graph nodes.
+- `top_k` is *optional* and defaults to `5`. Sets the default maximum number of memories returned per search.
+- `dataset_name` is *optional* and defaults to `"haystack_memory"`. Names the Cognee dataset backing this store.
+- `session_id` is *optional* and defaults to `None`. When set, reads and writes target the session-cache tier. When `None`, the permanent knowledge graph is used.
+- `self_improvement` is *optional* and defaults to `True`. When `True`, Cognee runs graph improvement inline after every write. Set to `False` when you want `improve()` to be the sole improvement trigger.
+- `timeout` is *optional* and defaults to `300`. Per-call timeout in seconds for any Cognee operation.
+
+### Installation
+
+Install the Cognee integration:
+
+```bash
+pip install cognee-haystack
+```
+
+Set your LLM API key (used by Cognee for graph extraction and queries):
+
+```bash
+export LLM_API_KEY="your-llm-api-key"
+```
+
+Optionally, set a separate embedding API key (defaults to `LLM_API_KEY` when unset):
+
+```bash
+export EMBEDDING_API_KEY="your-embedding-api-key"
+```
+
+## Usage
+
+### On its own
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.memory_stores.cognee import CogneeMemoryStore
+
+store = CogneeMemoryStore(search_type="GRAPH_COMPLETION", top_k=5)
+
+store.add_memories(
+    messages=[ChatMessage.from_user("Alice enjoys hiking and outdoor activities.")],
+    user_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+)
+
+memories = store.search_memories(
+    query="What does Alice like?",
+    user_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+)
+print([msg.text for msg in memories])
+```
+
+### Session tier vs permanent graph
+
+Use `session_id` to control which memory tier is targeted. A single store can serve both tiers — the writer's `session_id` overrides the store's `session_id` per call.
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.memory_stores.cognee import CogneeMemoryStore
+
+store = CogneeMemoryStore(dataset_name="my_agent_memory", self_improvement=False)
+
+# Write long-lived facts to the permanent graph (no session_id).
+store.add_memories(
+    messages=[ChatMessage.from_user("Alice is a senior data scientist at Acme Corp.")],
+)
+
+# Write transient session context to the session cache.
+store.add_memories(
+    messages=[ChatMessage.from_user("Alice is currently debugging a vector store issue.")],
+    session_id="alice_session_1",
+)
+
+# Promote the session cache into the permanent graph.
+store.improve(session_id="alice_session_1")
+```
+
+### Delete all memories
+
+```python
+# Delete only this store's dataset (session cache is unaffected).
+store.delete_all_memories()
+
+# To wipe everything including the session cache, call cognee directly:
+import asyncio
+import cognee
+
+asyncio.run(cognee.forget(everything=True))
+```

docs-website/docs/memory-stores/mem0memorystore.mdx

@@ -0,0 +1,103 @@
+---
+title: "Mem0MemoryStore"
+id: mem0memorystore
+slug: "/mem0memorystore"
+description: "A memory store backed by the Mem0 cloud API."
+---
+
+# Mem0MemoryStore
+
+`Mem0MemoryStore` is a memory store backed by the Mem0 cloud API. It is the shared data layer used by [`Mem0MemoryRetriever`](../pipeline-components/retrievers/mem0memoryretriever.mdx), [`Mem0MemoryWriter`](../pipeline-components/writers/mem0memorywriter.mdx), and the [Mem0 Memory Tools](../tools/ready-made-tools/mem0memorytools.mdx).
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Used by** | [`Mem0MemoryRetriever`](../pipeline-components/retrievers/mem0memoryretriever.mdx), [`Mem0MemoryWriter`](../pipeline-components/writers/mem0memorywriter.mdx), [`Mem0MemoryRetrieverTool`, `Mem0MemoryWriterTool`](../tools/ready-made-tools/mem0memorytools.mdx) |
+| **Optional init variables** | `api_key`: Defaults to `MEM0_API_KEY` environment variable |
+| **API reference** | [Mem0](/reference/integrations-mem0#mem0memorystore) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mem0 |
+| **Package name** | `mem0-haystack` |
+
+</div>
+
+## Overview
+
+`Mem0MemoryStore` wraps the Mem0 cloud API and provides two core methods:
+
+- `add_memories` — stores a list of `ChatMessage` objects as memories in Mem0.
+- `search_memories` — retrieves memories from Mem0 that are relevant to a query.
+
+Scope memories with at least one Mem0 entity ID: `user_id`, `run_id`, `agent_id`, or `app_id`. These are runtime parameters, so a single store instance can serve multiple users or sessions.
+
+The `infer` parameter on `add_memories` controls how Mem0 processes incoming messages:
+
+- `infer=True` lets Mem0 extract memories from the messages automatically. This is useful when storing a full Agent turn.
+- `infer=False` stores the supplied message text as-is. This is useful when the exact memory text has already been selected upstream.
+
+### Installation
+
+Install the Mem0 integration:
+
+```bash
+pip install mem0-haystack
+```
+
+Set your Mem0 API key:
+
+```bash
+export MEM0_API_KEY="your-mem0-api-key"
+```
+
+## Usage
+
+### On its own
+
+```python
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.memory_stores.mem0 import Mem0MemoryStore
+
+store = Mem0MemoryStore()
+
+store.add_memories(
+    messages=[ChatMessage.from_user("Alice prefers concise Python examples.")],
+    user_id="alice",
+    infer=False,
+)
+
+memories = store.search_memories(
+    query="What does Alice prefer?",
+    user_id="alice",
+    top_k=3,
+)
+print([msg.text for msg in memories])
+```
+
+### Scoping with multiple entity IDs
+
+Mem0 supports narrowing the scope of reads and writes with `user_id`, `run_id`, `agent_id`, and `app_id`. Pass any combination at call time:
+
+```python
+store.add_memories(
+    messages=[ChatMessage.from_user("Alice is working on a documentation search system.")],
+    user_id="alice",
+    run_id="docs-assistant-session-1",
+    infer=True,
+)
+
+memories = store.search_memories(
+    query="What project is Alice working on?",
+    user_id="alice",
+    run_id="docs-assistant-session-1",
+)
+print([msg.text for msg in memories])
+```
+
+### Retrieving all memories in scope
+
+Pass `query=None` to return all memories matching the provided scope without a relevance search:
+
+```python
+all_memories = store.search_memories(query=None, user_id="alice")
+print([msg.text for msg in all_memories])
+```

docs-website/docs/optimization/advanced-rag-techniques.mdx

@@ -0,0 +1,20 @@
+---
+title: "Advanced RAG Techniques"
+id: advanced-rag-techniques
+slug: "/advanced-rag-techniques"
+---
+
+# Advanced RAG Techniques
+
+This section of documentation talks about advanced RAQ techniques you can implement with Haystack.
+
+Read more about [Hypothetical Document Embeddings (HyDE)](advanced-rag-techniques/hypothetical-document-embeddings-hyde.mdx),
+
+or check out one of our cookbooks 🧑‍🍳:
+
+- [Using Hypothetical Document Embedding (HyDE) to Improve Retrieval](https://haystack.deepset.ai/cookbook/using_hyde_for_improved_retrieval)
+- [Query Decomposition and Reasoning](https://haystack.deepset.ai/cookbook/query_decomposition)
+- [Improving Retrieval by Embedding Meaningful Metadata](https://haystack.deepset.ai/cookbook/improve-retrieval-by-embedding-metadata)
+- [Query Expansion](https://haystack.deepset.ai/cookbook/query-expansion)
+- [Automated Structured Metadata Enrichment](https://haystack.deepset.ai/cookbook/metadata_enrichment)
+- [Auto-Merging and Hierarchical Document Retrieval](https://haystack.deepset.ai/cookbook/auto_merging_retriever)

docs-website/docs/optimization/advanced-rag-techniques/hypothetical-document-embeddings-hyde.mdx

@@ -0,0 +1,126 @@
+---
+title: "Hypothetical Document Embeddings (HyDE)"
+id: hypothetical-document-embeddings-hyde
+slug: "/hypothetical-document-embeddings-hyde"
+description: "Enhance the retrieval in Haystack using HyDE method by generating a mock-up hypothetical document for an initial query."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Hypothetical Document Embeddings (HyDE)
+
+Enhance the retrieval in Haystack using HyDE method by generating a mock-up hypothetical document for an initial query.
+
+## When Is It Helpful?
+
+The HyDE method is highly useful when:
+
+- The performance of the retrieval step in your pipeline is not good enough (for example, low Recall metric).
+- Your retrieval step has a query as input and returns documents from a larger document base.
+- Particularly worth a try if your data (documents or queries) come from a special domain that is very different from the typical datasets that Retrievers are trained on.
+
+## How Does It Work?
+
+Many embedding retrievers generalize poorly to new, unseen domains. This approach tries to tackle this problem. Given a query, the Hypothetical Document Embeddings (HyDE) first zero-shot prompts an instruction-following language model to generate a “fake” hypothetical document that captures relevant textual patterns from the initial query - in practice, this is done five times. Then, it encodes each hypothetical document into an embedding vector and averages them. The resulting, single embedding can be used to identify a neighbourhood in the document embedding space from which similar actual documents are retrieved based on vector similarity. As with any other retriever, these retrieved documents can then be used downstream in a pipeline (for example, in a Generator for RAG). Refer to the paper “[Precise Zero-Shot Dense Retrieval without Relevance Labels](https://aclanthology.org/2023.acl-long.99/)” for more details.
+<ClickableImage src="/img/2d00628-Untitled_2.png" alt="HyDE model architecture diagram showing how GPT generates hypothetical documents from queries in multiple languages, which are then matched with real documents via a Contriever model" size="large" />
+
+## How To Build It in Haystack?
+
+First, prepare all the components that you would need:
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+```python
+import os
+from numpy import array, mean
+from typing import List
+
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.builders import ChatPromptBuilder
+from haystack import component, Document
+from haystack.components.converters import OutputAdapter
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersDocumentEmbedder,
+)
+from haystack.dataclasses import ChatMessage
+
+# We need to ensure we have the OpenAI API key in our environment variables
+os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_KEY"
+
+# Initializing standard Haystack components
+generator = OpenAIChatGenerator(
+    model="gpt-3.5-turbo",
+    generation_kwargs={"n": 5, "temperature": 0.75, "max_tokens": 400},
+)
+prompt_builder = ChatPromptBuilder(
+    template=[
+        ChatMessage.from_user(
+            """Given a question, generate a paragraph of text that answers the question.    Question: {{question}}    Paragraph:""",
+        ),
+    ],
+    required_variables="*",
+)
+
+# The ChatGenerator returns ChatMessage replies, so we read each reply's text.
+# unsafe=True lets the adapter return actual Document objects instead of a string.
+adapter = OutputAdapter(
+    template="{{answers | build_doc}}",
+    output_type=List[Document],
+    custom_filters={"build_doc": lambda data: [Document(content=d.text) for d in data]},
+    unsafe=True,
+)
+
+embedder = SentenceTransformersDocumentEmbedder(
+    model="sentence-transformers/all-MiniLM-L6-v2",
+)
+
+
+# Adding one custom component that returns one, "average" embedding from multiple (hypothetical) document embeddings
+@component
+class HypotheticalDocumentEmbedder:
+    @component.output_types(hypothetical_embedding=List[float])
+    def run(self, documents: List[Document]):
+        stacked_embeddings = array([doc.embedding for doc in documents])
+        avg_embeddings = mean(stacked_embeddings, axis=0)
+        hyde_vector = avg_embeddings.reshape((1, len(avg_embeddings)))
+        return {"hypothetical_embedding": hyde_vector[0].tolist()}
+```
+
+Then, assemble them all into a pipeline:
+
+```python
+from haystack import Pipeline
+
+pipeline = Pipeline()
+pipeline.add_component(name="prompt_builder", instance=prompt_builder)
+pipeline.add_component(name="generator", instance=generator)
+pipeline.add_component(name="adapter", instance=adapter)
+pipeline.add_component(name="embedder", instance=embedder)
+pipeline.add_component(name="hyde", instance=HypotheticalDocumentEmbedder())
+
+pipeline.connect("prompt_builder.prompt", "generator.messages")
+pipeline.connect("generator.replies", "adapter.answers")
+pipeline.connect("adapter.output", "embedder.documents")
+pipeline.connect("embedder.documents", "hyde.documents")
+query = "What should I do if I have a fever?"
+result = pipeline.run(data={"prompt_builder": {"question": query}})
+
+# 'hypothetical_embedding': [0.0990725576877594, -0.017647066991776227, 0.05918873250484467, ...]}
+```
+
+Here's the graph of the resulting pipeline:
+<ClickableImage src="/img/74f3daa-hyde.png" alt="HyDE pipeline implementation flowchart showing prompt builder, generator, adapter, embedder, and hypothetical document embedder components" size="large"/>
+
+This pipeline example turns your query into one embedding.
+
+You can continue and feed this embedding to any [Embedding Retriever](../../pipeline-components/retrievers.mdx#dense-embedding-based-retrievers) to find similar documents in your Document Store.
+
+## Additional References
+
+📚 Article: [Optimizing Retrieval with HyDE](https://haystack.deepset.ai/blog/optimizing-retrieval-with-hyde)
+
+🧑‍🍳 Cookbook: [Using Hypothetical Document Embedding (HyDE) to Improve Retrieval](https://haystack.deepset.ai/cookbook/using_hyde_for_improved_retrieval)

docs-website/docs/optimization/evaluation.mdx

@@ -0,0 +1,64 @@
+---
+title: "Evaluation"
+id: evaluation
+slug: "/evaluation"
+description: "Learn all about pipeline or component evaluation in Haystack."
+---
+
+# Evaluation
+
+Learn all about pipeline or component evaluation in Haystack.
+
+Haystack has all the tools needed to evaluate entire pipelines or individual components like Retrievers, Readers, or Generators. This guide explains how to evaluate your pipeline in different scenarios and how to understand the metrics.
+
+Use evaluation and its results to:
+
+- Judge how well your system is performing on a given domain,
+- Compare the performance of different models,
+- Identify underperforming components in your pipeline.
+
+## Evaluation Options
+
+**Evaluating individual components or end-to-end pipelines.**
+
+Evaluating individual components can help understand performance bottlenecks and optimize one component at a time, for example, a Retriever or a prompt used with a Generator.
+
+End-to-end evaluation checks how the full pipeline is used and evaluates only the final outputs. The pipeline is approached as a black box.
+
+**Using ground-truth labels or no labels at all.**
+
+Most statistical evaluators require ground truth labels, such as the documents relevant to the query or the expected answer. In contrast, most model-based evaluators work without any labels just by following the prompt instructions. However, few-shot labels included in the prompt can improve the evaluator.
+
+**Model-based evaluation using a language model or statistical evaluation.**
+
+Model-based evaluation uses LLMs with prompt instructions or smaller fine-tuned models to score aspects of a pipeline’s outputs. Statistical evaluation requires no models and is thus a more lightweight way to score pipeline outputs. For more information, see our docs on [model-based](evaluation/model-based-evaluation.mdx) evaluation and [statistical](evaluation/statistical-evaluation.mdx) evaluation.
+
+## Evaluator Components
+
+|  |  |  |  |
+| --- | --- | --- | --- |
+| Evaluator                                                    | Evaluates Answers or Documents | Model-based or Statistical | Requires Labels |
+| [AnswerExactMatchEvaluator](../pipeline-components/evaluators/answerexactmatchevaluator.mdx) | Answers                        | Statistical                | Yes             |
+| [ContextRelevanceEvaluator](../pipeline-components/evaluators/contextrelevanceevaluator.mdx) | Documents                      | Model-based                | No              |
+| [DocumentMRREvaluator](../pipeline-components/evaluators/documentmrrevaluator.mdx)           | Documents                      | Statistical                | Yes             |
+| [DocumentMAPEvaluator](../pipeline-components/evaluators/documentmapevaluator.mdx)           | Documents                      | Statistical                | Yes             |
+| [DocumentRecallEvaluator](../pipeline-components/evaluators/documentrecallevaluator.mdx)     | Documents                      | Statistical                | Yes             |
+| [FaithfulnessEvaluator](../pipeline-components/evaluators/faithfulnessevaluator.mdx)         | Answers                        | Model-based                | No              |
+| [LLMEvaluator](../pipeline-components/evaluators/llmevaluator.mdx)                           | User-defined                   | Model-based                | No              |
+| [SASEvaluator](../pipeline-components/evaluators/sasevaluator.mdx)                           | Answers                        | Model-based                | Yes             |
+
+## Evaluator Integrations
+
+To learn more about our integration with the Ragas and DeepEval evaluation frameworks, head over to the [RagasEvaluator](../pipeline-components/evaluators/ragasevaluator.mdx) and [DeepEvalEvaluator](../pipeline-components/evaluators/deepevalevaluator.mdx) component docs.
+
+To get started using practical examples, check out our evaluation tutorial or the respective cookbooks below.
+
+## Additional References
+
+:notebook: Tutorial: [Evaluating RAG Pipelines](https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines)
+
+🧑‍🍳 Cookbooks:
+
+- [RAG Evaluation with Prometheus 2](https://haystack.deepset.ai/cookbook/prometheus2_evaluation)
+- [RAG Pipeline Evaluation Using Ragas](https://haystack.deepset.ai/cookbook/rag_eval_ragas)
+- [RAG Pipeline Evaluation Using DeepEval](https://haystack.deepset.ai/cookbook/rag_eval_deep_eval)

docs-website/docs/optimization/evaluation/model-based-evaluation.mdx

@@ -0,0 +1,127 @@
+---
+title: "Model-Based Evaluation"
+id: model-based-evaluation
+slug: "/model-based-evaluation"
+description: "Haystack supports various kinds of model-based evaluation. This page explains what model-based evaluation is and discusses the various options available with Haystack."
+---
+
+# Model-Based Evaluation
+
+Haystack supports various kinds of model-based evaluation. This page explains what model-based evaluation is and discusses the various options available with Haystack.
+
+## What is Model-Based Evaluation
+
+Model-based evaluation in Haystack uses a language model to check the results of a Pipeline. This method is easy to use because it usually doesn't need labels for the outputs. It's often used with Retrieval-Augmented Generative (RAG) Pipelines, but can work with any Pipeline.
+
+Currently, Haystack supports the end-to-end, model-based evaluation of a complete RAG Pipeline.
+
+### Using LLMs for Evaluation
+
+A common strategy for model-based evaluation involves using a Language Model (LLM), such as OpenAI's GPT models, as the evaluator model, often referred to as the _golden_ model. The most frequently used golden model is GPT-4. We utilize this model to evaluate a RAG Pipeline by providing it with the Pipeline's results and sometimes additional information, along with a prompt that outlines the evaluation criteria.
+
+This method of using an LLM as the evaluator is very flexible as it exposes a number of metrics to you. Each of these metrics is ultimately a well-crafted prompt describing to the LLM how to evaluate and score results. Common metrics are faithfulness, context relevance, and so on.
+
+### Using Local LLMs
+
+To use the model-based Evaluators with a local model, you need to pass the `api_base_url` and `model` in the `api_params` parameter when initializing the Evaluator.
+
+The following example shows how this would work with an Ollama model.
+
+First, make sure that Ollama is running locally:
+
+```curl
+curl http://localhost:11434/api/generate -d '{
+  "model": "llama3",
+  "prompt":"Why is the sky blue?"
+}'
+```
+
+Then, your pipeline would look like this:
+
+```python
+from haystack.components.evaluators import FaithfulnessEvaluator
+from haystack.utils import Secret
+
+questions = ["Who created the Python language?"]
+contexts = [
+    [
+        (
+            "Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming "
+            "language. Its design philosophy emphasizes code readability, and its language constructs aim to help "
+            "programmers write clear, logical code for both small and large-scale software projects."
+        ),
+    ],
+]
+predicted_answers = [
+    "Python is a high-level general-purpose programming language that was created by George Lucas.",
+]
+local_endpoint = "http://localhost:11434/v1"
+
+evaluator = FaithfulnessEvaluator(
+    api_key=Secret.from_token("just-a-placeholder"),
+    api_params={"api_base_url": local_endpoint, "model": "llama3"},
+)
+
+result = evaluator.run(
+    questions=questions,
+    contexts=contexts,
+    predicted_answers=predicted_answers,
+)
+```
+
+### Using Small Cross-Encoder Models for Evaluation
+
+Alongside LLMs for evaluation, we can also use small cross-encoder models. These models can calculate, for example, semantic answer similarity. In contrast to metrics based on LLMs, the metrics based on smaller models don’t require an API key of a model provider.
+
+This method of using small cross-encoder models as evaluators is faster and cheaper to run but is less flexible in terms of what aspect you can evaluate. You can only evaluate what the small model was trained to evaluate.
+
+## Model-Based Evaluation Pipelines in Haystack
+
+There are two ways of performing model-based evaluation in Haystack, both of which leverage [Pipelines](../../concepts/pipelines.mdx) and [Evaluator](../../pipeline-components/evaluators.mdx) components.
+
+- You can create and run an evaluation Pipeline independently. This means you’ll have to provide the required inputs to the evaluation Pipeline manually. We recommend this way because the separation of your RAG Pipeline and your evaluation Pipeline allows you to store the results of your RAG Pipeline and try out different evaluation metrics afterward without needing to re-run your RAG Pipeline every time.
+- As another option, you can add an evaluator component to the end of a RAG Pipeline. This means you run both a RAG Pipeline and evaluation on top of it in a single `pipeline.run()`  call.
+
+### Model-based Evaluation of Retrieved Documents
+
+#### [ContextRelevanceEvaluator](../../pipeline-components/evaluators/contextrelevanceevaluator.mdx)
+
+Context relevance refers to how relevant the retrieved documents are to the query. An LLM is used to judge that aspect. It first extracts statements from the documents and then checks how many of them are relevant for answering the query.
+
+### Model-based Evaluation of Generated or Extracted Answers
+
+#### [FaithfulnessEvaluator](../../pipeline-components/evaluators/faithfulnessevaluator.mdx)
+
+Faithfulness, also called groundedness, evaluates to what extent a generated answer is based on retrieved documents. An LLM is used to extract statements from the answer and check the faithfulness for each separately. If the answer is not based on the documents, the answer, or at least parts of it, is called a hallucination.
+
+#### [SASEvaluator](../../pipeline-components/evaluators/sasevaluator.mdx) (Semantic Answer Similarity)
+
+Semantic answer similarity uses a transformer-based, cross-encoder architecture to evaluate the semantic similarity of two answers rather than their lexical overlap. While F1 and EM would both score _one hundred percent_ as sharing zero similarity with _100 %_, SAS is trained to assign a high score to such cases. SAS is particularly useful for seeking out cases where F1 doesn't give a good indication of the validity of a predicted answer. You can read more about SAS in [Semantic Answer Similarity for Evaluating Question-Answering Models paper](https://arxiv.org/abs/2108.06130).
+
+### Evaluation Framework Integrations
+
+Currently, Haystack has integrations with [DeepEval](https://docs.confident-ai.com/docs/metrics-introduction) and [Ragas](https://docs.ragas.io/en/stable/index.html). There is an Evaluator component available for each of these frameworks:
+
+- [RagasEvaluator](../../pipeline-components/evaluators/ragasevaluator.mdx)
+- [DeepEvalEvaluator](../../pipeline-components/evaluators/deepevalevaluator.mdx)
+
+|  |  |  |
+| --- | --- | --- |
+| Feature/Integration | RagasEvaluator | DeepEvalEvaluator |
+| Evaluator Models | All GPT models from OpenAI  <br />Google VertexAI Models  <br />Azure OpenAI Models  <br />Amazon Bedrock Models | All GPT models from OpenAI |
+| Supported metrics | ANSWER_CORRECTNESS, FAITHFULNESS, ANSWER_SIMILARITY, CONTEXT_PRECISION, CONTEXT_UTILIZATION,CONTEXT_RECALL, ASPECT_CRITIQUE, CONTEXT_RELEVANCY, ANSWER_RELEVANCY | ANSWER_RELEVANCY, FAITHFULNESS, CONTEXTUAL_PRECISION, CONTEXTUAL_RECALL, CONTEXTUAL_RELEVANCE |
+| Customizable prompt for response evaluation | ✅, with ASPECT_CRITIQUE metric | ❌ |
+| Explanations of scores | ❌ | ✅ |
+| Monitoring dashboard | ❌ | ❌ |
+
+:::info[Framework Documentation]
+
+You can find more information about the metrics in the documentation of the respective evaluation frameworks:
+
+- Ragas metrics: https://docs.ragas.io/en/latest/concepts/metrics/index.html
+- DeepEval metrics: https://docs.confident-ai.com/docs/metrics-introduction
+:::
+
+## Additional References
+
+:notebook: Tutorial: [Evaluating RAG Pipelines](https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines)

docs-website/docs/optimization/evaluation/statistical-evaluation.mdx

@@ -0,0 +1,49 @@
+---
+title: "Statistical Evaluation"
+id: statistical-evaluation
+slug: "/statistical-evaluation"
+description: "Haystack supports various statistical evaluation metrics. This page explains what statistical evaluation is and discusses the various options available within Haystack."
+---
+
+# Statistical Evaluation
+
+Haystack supports various statistical evaluation metrics. This page explains what statistical evaluation is and discusses the various options available within Haystack.
+
+## Introduction
+
+Statistical evaluation in Haystack compares ground truth labels with pipeline predictions, typically using metrics such as precision or recall. It's often used to evaluate the Retriever component within Retrieval-Augmented Generative (RAG) pipelines, but this methodology can be adapted for any pipeline if ground truth labels of relevant documents are available.
+
+When evaluating answers, such as those predicted by an extractive question answering pipeline, the ground truth labels of expected answers are compared to the pipeline's predictions.
+
+For assessing answers generated by LLMs with one of Haystack’s Generator components, we recommend model-based evaluation instead. It can incorporate measures of semantic similarity or coherence and is better suited to evaluate predictions that might differ in wording from the ground truth labels.
+
+## Statistical Evaluation Pipelines in Haystack
+
+There are two ways of performing model-based evaluation in Haystack, both of which leverage [pipelines](../../concepts/pipelines.mdx) and [Evaluator](../../pipeline-components/evaluators.mdx) components:
+
+- You can create and run an evaluation pipeline independently. This means you’ll have to provide the required inputs to the evaluation pipeline manually. We recommend this way because the separation of your RAG pipeline and your evaluation pipeline allows you to store the results of your RAG pipeline and try out different evaluation metrics afterward without needing to re-run your pipeline every time.
+- As another option, you can add an Evaluator to the end of a RAG pipeline. This means you run both a RAG pipeline and evaluation on top of it in a single `pipeline.run()`  call.
+
+## Statistical Evaluation of Retrieved Documents
+
+### [DocumentRecallEvaluator](../../pipeline-components/evaluators/documentrecallevaluator.mdx)
+
+Recall measures how often the correct document was among the retrieved documents over a set of queries. For a single query, the output is binary: either the correct document is contained in the selection, or it is not. Over the entire dataset, the recall score amounts to a number between zero (no query retrieved the right document) and one (all queries retrieved the right documents).
+
+In some scenarios, there can be multiple correct documents for one query. The metric `recall_single_hit` considers whether at least one of the correct documents is retrieved, whereas `recall_multi_hit` takes into account how many of the multiple correct documents for one query are retrieved.
+
+Note that recall is affected by the number of documents that the Retriever returns. If the Retriever returns few documents, it means that it is difficult to retrieve the correct documents. Make sure to set the Retriever's `top_k` to an appropriate value in the pipeline that you're evaluating.
+
+### [DocumentMRREvaluator](../../pipeline-components/evaluators/documentmrrevaluator.mdx) (Mean Reciprocal Rank)
+
+In contrast to the recall metric, mean reciprocal rank takes the position of the top correctly retrieved document (the “rank”) into account. It does this to account for the fact that a query elicits multiple responses of varying relevance. Like recall, MRR can be a value between zero (no matches) and one (the system retrieved a correct document for all queries as the top result). For more details, check out [Mean Reciprocal Rank wiki page](https://en.wikipedia.org/wiki/Mean_reciprocal_rank).
+
+### [DocumentMAPEvaluator](../../pipeline-components/evaluators/documentmapevaluator.mdx) (Mean Average Precision)
+
+Mean average precision is similar to mean reciprocal rank but takes into account the position of every correctly retrieved document. Like MRR, mAP can be a value between zero (no matches) and one (the system retrieved correct documents for all top results). mAP is particularly useful in cases where there is more than one correct answer to be retrieved. For more details, check out [Mean Average Precision wiki page](https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Mean_average_precision).
+
+## Statistical Evaluation of Extracted or Generated Answers
+
+### [AnswerExactMatchEvaluator](../../pipeline-components/evaluators/answerexactmatchevaluator.mdx)
+
+Exact match measures the proportion of cases where the predicted Answer is identical to the correct Answer. For example, for the annotated question-answer pair “What is Haystack?" + "A question answering library in Python”, even a predicted answer like “A Python question answering library” would yield a zero score because it does not match the expected answer 100%.

docs-website/docs/overview/breaking-change-policy.mdx

@@ -0,0 +1,88 @@
+---
+title: "Breaking Change Policy"
+id: breaking-change-policy
+slug: "/breaking-change-policy"
+description: "This document outlines the breaking change policy for Haystack, including the definition of breaking changes, versioning conventions, and the deprecation process for existing features."
+---
+
+# Breaking Change Policy
+
+This document outlines the breaking change policy for Haystack, including the definition of breaking changes, versioning conventions, and the deprecation process for existing features.
+
+Haystack is under active development, which means that functionalities are being added, deprecated, or removed rather frequently. This policy aims to minimize the impact of these changes on current users and deployments. It provides a clear schedule and outlines the necessary steps before upgrading to a new Haystack version.
+
+## Breaking Change Definition
+
+A breaking change occurs when: 
+
+- A Component is removed, renamed, or the Python import path is changed.
+- A parameter is renamed, removed, or changed from optional to mandatory.
+- A new mandatory parameter is added.
+
+Existing deployments might break, and the change is deemed a _breaking change_. The decision to declare a change as breaking has nothing to do with its potential impact: while the change might only impact a tiny subset of applications using a specific Haystack feature, it would still be treated as a breaking change.
+
+The following cases are **not** considered a breaking change:
+
+- A new functionality is added (for example, a new Component).
+- A component, class, or utility function gets a new optional parameter.
+- An existing parameter gets changed from mandatory to optional.
+
+Existing deployments are not impacted, and the change is deemed non-breaking. Release notes will mention the change and possibly provide an upgrade path, but upgrading Haystack won’t break existing applications.
+
+## Versioning
+
+Haystack releases are labeled with a series of three numbers separated by dots, for example, `2.0.1`. Each number has a specific meaning: 
+
+- `2` is the Major version
+- `0` is the Minor version
+- `1` is the Patch version
+
+:::info
+Albeit similar, Haystack DOES NOT follow the principles of [Semantic Versioning](https://semver.org). Read on to see the differences.
+:::
+
+Given a Haystack release with a version number of type `MAJOR.MINOR.PATCH`, you should expect:
+
+1. **For Major version change:** fundamental, incompatible API changes. In this case, you would most likely need a migration process before being able to update Haystack. Major releases happen no more than once a year, changes are extensively documented, and a migration path is provided.
+2. **For Minor version change:** addition or removal of functionalities that might not be backward compatible. Most of the time, you will be able to upgrade your Haystack installation seamlessly, but always refer to the [release notes](https://github.com/deepset-ai/haystack/releases) for guidance. Deprecated components are the most common breaking change shipped in a Minor version release.
+3. **For Patch version change:** bugfixes. You can safely upgrade Haystack to the new version without concerns that your program will break.
+
+## Deprecation of Existing Features
+
+Haystack strives for robustness. To achieve this, we clean up our code by removing old features that are no longer used. This helps us maintain the codebase, improve security, and make it easier to keep everything running smoothly. Before we remove a feature, component, class, or utility function, we go through a process called deprecation.
+
+A Major or Minor (but not Patch) version may deprecate certain features from previous releases, and this is what you should expect:
+
+- If a feature is deprecated in Haystack version `X.Y`, it will continue to work but the Python code will raise warnings detailing the steps to take in order to upgrade.
+- Features deprecated in Haystack version `X.Y` will be removed in Haystack `X.Y+1`, giving affected users a timeframe of roughly a month to prepare the upgrade.
+
+### Example
+
+To clarify the process, here’s an example:
+
+At some point, we decide to remove a `FooComponent` and declare it deprecated in Haystack version `2.99.0`. This is what will happen:
+
+1. `FooComponent` keeps working as usual In Haystack `2.99.0`, but using the component raises a `FutureWarning` message in the code.
+2. In Haystack version `2.100.0`, we remove the `FooComponent` from the codebase. Trying to use it produces an error.
+
+## Discontinuing an Integration
+
+When existing features are changed or removed, integrations go through the same deprecation process as detailed on this page for Haystack. It’s important to note that integrations are independent and distributed with their own packages. In certain cases, a special form of deprecation may occur where the integration is discontinued and subsequently removed from the Core Integrations repository.
+
+To give our community the opportunity to take over the integration and keep it maintained before being discontinued Core Integrations gradually go through different states, as detailed below:
+
+- **Staged**
+  - The source code of the integration is moved from `main` to a special `staging` branch of the Core Integrations repository.
+  - The documentation pages are removed from the Haystack documentation website.
+  - The main README of the Core Integrations repository shows a disclaimer explaining how the integration can be adopted from the community.
+  - The integration tile is removed (it can be re-added later by the maintainer who adopted the integration).
+  - The integration package on PyPI remains available.
+  - A grace period of 3 months starts.
+- **Adopted**
+  - An organization or an individual from the community accepts to take over the ownership of the Staged integration.
+  - The adopter creates their own repository, and the source code of the discontinued integration is removed from the `staging` branch.
+  - Ownership of the PyPI package is transferred to the new maintainer.
+  - The adopter will create a new integration tile in [haystack-integrations](https://github.com/deepset-ai/haystack-integrations).
+- **Discontinued**
+  - If the grace period expires and nobody adopts the Staged Integration, its source code is removed from the `staging` branch.
+  - The PyPI package of the integration won’t be removed but won’t be further updated.

docs-website/docs/overview/docs-mcp-server.mdx

@@ -0,0 +1,88 @@
+---
+title: "Using Haystack Docs in Your Coding Agent"
+id: docs-mcp-server
+slug: "/docs-mcp-server"
+description: "Connect your coding agent to the Haystack documentation through the public MCP server. Includes setup instructions for Claude Code, Cursor, and GitHub Copilot."
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Using Haystack Docs in Your Coding Agent
+
+Haystack publishes a public [Model Context Protocol (MCP)](https://modelcontextprotocol.io) server that lets coding agents search the official Haystack documentation. Pointing your agent at it means it answers questions from up-to-date docs instead of relying on training data, which can lag behind the framework.
+
+The server exposes a single tool, `search_haystack_docs`, that returns relevant documentation sections with source URLs. No API key or sign-up is needed.
+
+## Server URL
+
+```
+https://docs.haystack.deepset.ai/api/mcp
+```
+
+The server speaks **HTTP** transport. Most agents auto-detect this. If yours asks you to choose, pick `http`.
+
+## Setup
+
+<Tabs>
+<TabItem value="claude-code" label="Claude Code" default>
+
+Add the server with the `claude mcp` CLI:
+
+```shell
+claude mcp add --transport http haystack-docs https://docs.haystack.deepset.ai/api/mcp
+```
+
+Restart your Claude Code session. Verify it's connected by running `/mcp` — you should see `haystack-docs` listed with the `search_haystack_docs` tool.
+
+For more options (project vs. user scope, SSE transport, headers), see the [Claude Code MCP docs](https://code.claude.com/docs/en/mcp).
+
+</TabItem>
+<TabItem value="cursor" label="Cursor">
+
+Open Cursor settings → **Tools & MCPs** → **Add new MCP server**, or edit `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (per-project) directly:
+
+```json
+{
+  "mcpServers": {
+    "haystack-docs": {
+      "url": "https://docs.haystack.deepset.ai/api/mcp"
+    }
+  }
+}
+```
+
+Save the file and reload Cursor. The tool appears in the **Available Tools** list inside the chat panel.
+
+See the [Cursor MCP docs](https://cursor.com/docs/context/mcp) for the full configuration reference.
+
+</TabItem>
+<TabItem value="copilot" label="GitHub Copilot">
+
+In VS Code, create or edit `.vscode/mcp.json` in your workspace:
+
+```json
+{
+  "servers": {
+    "haystack-docs": {
+      "type": "http",
+      "url": "https://docs.haystack.deepset.ai/api/mcp"
+    }
+  }
+}
+```
+
+Open the Copilot Chat panel, switch to **Agent** mode, then click the tools icon and enable `haystack-docs`. You can also register the server globally from the command palette via **MCP: Add Server**.
+
+See [Add and manage MCP servers in VS Code](https://code.visualstudio.com/docs/copilot/chat/mcp-servers) for the full configuration reference.
+
+</TabItem>
+</Tabs>
+
+## Verifying it works
+
+Ask your agent a question that requires current Haystack knowledge, for example:
+
+> What are the required methods on a Haystack custom component?
+
+The agent should call `search_haystack_docs` and cite source URLs under `docs.haystack.deepset.ai` in its answer. If it answers without calling the tool, prompt it explicitly: *"Use the haystack-docs MCP server to answer."*

docs-website/docs/overview/faq.mdx

@@ -0,0 +1,44 @@
+---
+title: "FAQ"
+id: faq
+slug: "/faq"
+description: "Here are the answers to the questions people frequently ask about Haystack."
+---
+
+# FAQ
+
+Here are the answers to the questions people frequently ask about Haystack.
+
+### How can I make sure that my GPU is being engaged when I use Haystack?
+
+You will want to ensure that a CUDA enabled GPU is being engaged when Haystack is running (you can check by running `nvidia-smi -l` on your command line). Components which can be sped up by GPU have a `device` argument in their constructor. For more details, check the [Device Management](../concepts/device-management.mdx) page.
+
+### Are you tracking my Haystack usage?
+
+We only collect _anonymous_ usage statistics of Haystack pipeline components. Read more about telemetry in Haystack or how you can opt out on the [Telemetry](telemetry.mdx) page.
+
+### How can I ask my questions around Haystack?
+
+For general questions, we recommend joining the [Haystack Discord ](https://discord.com/invite/xYvH6drSmA)or using [GitHub discussions](https://github.com/deepset-ai/haystack/discussions), where the community and maintainers can help. You can also explore [tutorials](https://haystack.deepset.ai/tutorials/40_building_chat_application_with_function_calling) and [examples](https://haystack.deepset.ai/cookbook/tools_support) on website to find more info.
+
+### How can I get expert support for Haystack?
+
+If you’re a team running Haystack in production or want to move faster and scale with confidence, we recommend [Haystack Enterprise Starter](https://haystack.deepset.ai/blog/announcing-haystack-enterprise). It gives you direct access to the Haystack team, proven best practices, and hands-on support to help you go from prototype to production smoothly.
+
+👉 [Get in touch with our team to explore Haystack Enterprise Starter](https://www.deepset.ai/products-and-services/haystack-enterprise)
+
+### Where can I find documentation for older Haystack versions?
+
+The website only hosts documentation for the 5 most recent Haystack versions.
+
+For older versions (up to 2.18), you can access the documentation on GitHub: https://github.com/deepset-ai/haystack/tree/main/docs-website/versioned_docs.
+
+### Where can I find tutorials and documentation for Haystack 1.x?
+
+You can access old tutorials in the [GitHub history](https://github.com/deepset-ai/haystack-tutorials/tree/5917718cbfbb61410aab4121ee6fe754040a5dc7) and download the Haystack 1.x documentation as a [ZIP file](https://core-engineering.s3.eu-central-1.amazonaws.com/public/docs/haystack-v1-docs.zip).
+
+The ZIP file contains documentation for all minor releases from version 1.0 to 1.26.
+
+To download documentation for a specific release, replace the version number in the following URL: `https://core-engineering.s3.eu-central-1.amazonaws.com/public/docs/v1.26.zip`.
+
+Learn how to migrate to Haystack version 2.x with our [migration guide](migration.mdx).

docs-website/docs/overview/get-started.mdx

@@ -0,0 +1,581 @@
+---
+title: "Get Started"
+id: get-started
+slug: "/get-started"
+description: "Learn how to quickly get up and running with Haystack. Build your first RAG pipeline and tool-calling Agent with step-by-step examples for multiple LLM providers."
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Get Started
+
+Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing Haystack, building your first RAG pipeline, and creating a tool-calling Agent.
+
+## Build your first RAG application
+
+Let's build your first Retrieval Augmented Generation (RAG) pipeline and see how Haystack answers questions.
+
+First, install the minimal form of Haystack:
+
+```shell
+pip install haystack-ai
+```
+
+In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). Choose your preferred LLM provider from the tabs below. For easier use, you can also set the API key as an environment variable.
+
+<Tabs>
+<TabItem value="openai" label="OpenAI" default>
+
+[OpenAIChatGenerator](../pipeline-components/generators/openaichatgenerator.mdx) is included in the `haystack-ai` package.
+
+```python
+from haystack import Pipeline, Document
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.builders import ChatPromptBuilder
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+
+prompt_template = [
+    ChatMessage.from_system(
+        """
+        Given these documents, answer the question.
+        Documents:
+        {% for doc in documents %}
+            {{ doc.content }}
+        {% endfor %}
+        """,
+    ),
+    ChatMessage.from_user("{{question}}"),
+]
+
+retriever = InMemoryBM25Retriever(document_store=document_store)
+prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
+llm = OpenAIChatGenerator(
+    api_key=Secret.from_env_var("OPENAI_API_KEY"),
+    model="gpt-4o-mini",
+)
+
+rag_pipeline = Pipeline()
+rag_pipeline.add_component("retriever", retriever)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+
+question = "Who lives in Paris?"
+results = rag_pipeline.run(
+    {
+        "retriever": {"query": question},
+        "prompt_builder": {"question": question},
+    },
+)
+
+print(results["llm"]["replies"])
+```
+
+</TabItem>
+<TabItem value="huggingface" label="Hugging Face">
+
+[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `huggingface-api-haystack` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API.
+
+The examples on this page use the Hugging Face API components and the SerperDev web search component, which have moved to the `huggingface-api-haystack` and `serperdev-haystack` packages. Install them to run the examples:
+
+```shell
+pip install huggingface-api-haystack serperdev-haystack
+```
+
+```python
+from haystack import Pipeline, Document
+from haystack_integrations.components.generators.huggingface_api import (
+    HuggingFaceAPIChatGenerator,
+)
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.builders import ChatPromptBuilder
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+
+prompt_template = [
+    ChatMessage.from_system(
+        """
+        Given these documents, answer the question.
+        Documents:
+        {% for doc in documents %}
+            {{ doc.content }}
+        {% endfor %}
+        """,
+    ),
+    ChatMessage.from_user("{{question}}"),
+]
+
+retriever = InMemoryBM25Retriever(document_store=document_store)
+prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
+llm = HuggingFaceAPIChatGenerator(
+    api_type="serverless_inference_api",
+    api_params={"model": "Qwen/Qwen2.5-72B-Instruct"},
+    token=Secret.from_env_var("HF_API_TOKEN"),
+)
+
+rag_pipeline = Pipeline()
+rag_pipeline.add_component("retriever", retriever)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+
+question = "Who lives in Paris?"
+results = rag_pipeline.run(
+    {
+        "retriever": {"query": question},
+        "prompt_builder": {"question": question},
+    },
+)
+
+print(results["llm"]["replies"])
+```
+
+</TabItem>
+<TabItem value="anthropic" label="Anthropic">
+
+Install the [Anthropic integration](https://haystack.deepset.ai/integrations/anthropic):
+
+```bash
+pip install anthropic-haystack
+```
+
+See the [AnthropicChatGenerator](../pipeline-components/generators/anthropicchatgenerator.mdx) docs for more details.
+
+```python
+from haystack import Pipeline, Document
+from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.builders import ChatPromptBuilder
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+
+prompt_template = [
+    ChatMessage.from_system(
+        """
+        Given these documents, answer the question.
+        Documents:
+        {% for doc in documents %}
+            {{ doc.content }}
+        {% endfor %}
+        """,
+    ),
+    ChatMessage.from_user("{{question}}"),
+]
+
+retriever = InMemoryBM25Retriever(document_store=document_store)
+prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
+llm = AnthropicChatGenerator(
+    api_key=Secret.from_env_var("ANTHROPIC_API_KEY"),
+    model="claude-sonnet-4-5-20250929",
+)
+
+rag_pipeline = Pipeline()
+rag_pipeline.add_component("retriever", retriever)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+
+question = "Who lives in Paris?"
+results = rag_pipeline.run(
+    {
+        "retriever": {"query": question},
+        "prompt_builder": {"question": question},
+    },
+)
+
+print(results["llm"]["replies"])
+```
+
+</TabItem>
+<TabItem value="amazon-bedrock" label="Amazon Bedrock">
+
+Install the [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock):
+
+```bash
+pip install amazon-bedrock-haystack
+```
+
+See the [AmazonBedrockChatGenerator](../pipeline-components/generators/amazonbedrockchatgenerator.mdx) docs for more details.
+
+```python
+import os
+from haystack import Pipeline, Document
+from haystack_integrations.components.generators.amazon_bedrock import (
+    AmazonBedrockChatGenerator,
+)
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID"
+os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY"
+os.environ["AWS_DEFAULT_REGION"] = "YOUR_AWS_REGION"
+
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+
+prompt_template = [
+    ChatMessage.from_system(
+        """
+        Given these documents, answer the question.
+        Documents:
+        {% for doc in documents %}
+            {{ doc.content }}
+        {% endfor %}
+        """,
+    ),
+    ChatMessage.from_user("{{question}}"),
+]
+
+retriever = InMemoryBM25Retriever(document_store=document_store)
+prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
+llm = AmazonBedrockChatGenerator(model="anthropic.claude-3-5-sonnet-20240620-v1:0")
+
+rag_pipeline = Pipeline()
+rag_pipeline.add_component("retriever", retriever)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+
+question = "Who lives in Paris?"
+results = rag_pipeline.run(
+    {
+        "retriever": {"query": question},
+        "prompt_builder": {"question": question},
+    },
+)
+
+print(results["llm"]["replies"])
+```
+
+</TabItem>
+<TabItem value="google-gemini" label="Google Gemini">
+
+Install the [Google Gen AI integration](https://haystack.deepset.ai/integrations/google-genai):
+
+```bash
+pip install google-genai-haystack
+```
+
+See the [GoogleGenAIChatGenerator](../pipeline-components/generators/googlegenaichatgenerator.mdx) docs for more details.
+
+```python
+from haystack import Pipeline, Document
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.builders import ChatPromptBuilder
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+
+prompt_template = [
+    ChatMessage.from_system(
+        """
+        Given these documents, answer the question.
+        Documents:
+        {% for doc in documents %}
+            {{ doc.content }}
+        {% endfor %}
+        """,
+    ),
+    ChatMessage.from_user("{{question}}"),
+]
+
+retriever = InMemoryBM25Retriever(document_store=document_store)
+prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
+llm = GoogleGenAIChatGenerator(
+    api_key=Secret.from_env_var("GOOGLE_API_KEY"),
+    model="gemini-2.5-flash",
+)
+
+rag_pipeline = Pipeline()
+rag_pipeline.add_component("retriever", retriever)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+
+question = "Who lives in Paris?"
+results = rag_pipeline.run(
+    {
+        "retriever": {"query": question},
+        "prompt_builder": {"question": question},
+    },
+)
+
+print(results["llm"]["replies"])
+```
+
+</TabItem>
+<TabItem value="more-providers" label="More Providers">
+
+<div style={{backgroundColor: 'var(--ifm-color-emphasis-100)', padding: '1.5rem', borderRadius: '8px'}}>
+
+Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options.
+
+Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx).
+
+You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page.
+
+</div>
+
+</TabItem>
+</Tabs>
+
+### Next Steps
+
+Ready to dive deeper? Check out the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial for a step-by-step guide on building a complete RAG pipeline with your own data.
+
+## Build your first Agent
+
+Agents are AI systems that can use tools to gather information, perform actions, and interact with external systems. Let's build an agent that can search the web to answer questions.
+
+This example requires a [SerperDev API key](https://serper.dev/) for web search. Set it as the `SERPERDEV_API_KEY` environment variable.
+
+<Tabs>
+<TabItem value="openai" label="OpenAI" default>
+
+[OpenAIChatGenerator](../pipeline-components/generators/openaichatgenerator.mdx) is included in the `haystack-ai` package.
+
+```python
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.tools import ComponentTool
+from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
+from haystack.utils import Secret
+
+search_tool = ComponentTool(component=SerperDevWebSearch())
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(
+        api_key=Secret.from_env_var("OPENAI_API_KEY"),
+        model="gpt-4o-mini",
+    ),
+    tools=[search_tool],
+    system_prompt="You are a helpful assistant that can search the web for information.",
+)
+
+result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")])
+
+print(result["last_message"].text)
+```
+
+</TabItem>
+<TabItem value="huggingface" label="Hugging Face">
+
+[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `huggingface-api-haystack` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API.
+
+```python
+from haystack.components.agents import Agent
+from haystack_integrations.components.generators.huggingface_api import (
+    HuggingFaceAPIChatGenerator,
+)
+from haystack.dataclasses import ChatMessage
+from haystack.tools import ComponentTool
+from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
+from haystack.utils import Secret
+
+search_tool = ComponentTool(component=SerperDevWebSearch())
+
+agent = Agent(
+    chat_generator=HuggingFaceAPIChatGenerator(
+        api_type="serverless_inference_api",
+        api_params={"model": "Qwen/Qwen2.5-72B-Instruct"},
+        token=Secret.from_env_var("HF_API_TOKEN"),
+    ),
+    tools=[search_tool],
+    system_prompt="You are a helpful assistant that can search the web for information.",
+)
+
+result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")])
+
+print(result["last_message"].text)
+```
+
+</TabItem>
+<TabItem value="anthropic" label="Anthropic">
+
+Install the [Anthropic integration](https://haystack.deepset.ai/integrations/anthropic):
+
+```bash
+pip install anthropic-haystack
+```
+
+See the [AnthropicChatGenerator](../pipeline-components/generators/anthropicchatgenerator.mdx) docs for more details.
+
+```python
+from haystack.components.agents import Agent
+from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.tools import ComponentTool
+from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
+from haystack.utils import Secret
+
+search_tool = ComponentTool(component=SerperDevWebSearch())
+
+agent = Agent(
+    chat_generator=AnthropicChatGenerator(
+        api_key=Secret.from_env_var("ANTHROPIC_API_KEY"),
+        model="claude-sonnet-4-5-20250929",
+    ),
+    tools=[search_tool],
+    system_prompt="You are a helpful assistant that can search the web for information.",
+)
+
+result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")])
+
+print(result["last_message"].text)
+```
+
+</TabItem>
+<TabItem value="amazon-bedrock" label="Amazon Bedrock">
+
+Install the [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock):
+
+```bash
+pip install amazon-bedrock-haystack
+```
+
+See the [AmazonBedrockChatGenerator](../pipeline-components/generators/amazonbedrockchatgenerator.mdx) docs for more details.
+
+```python
+import os
+from haystack.components.agents import Agent
+from haystack_integrations.components.generators.amazon_bedrock import (
+    AmazonBedrockChatGenerator,
+)
+from haystack.dataclasses import ChatMessage
+from haystack.tools import ComponentTool
+from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
+
+os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID"
+os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY"
+os.environ["AWS_DEFAULT_REGION"] = "YOUR_AWS_REGION"
+
+search_tool = ComponentTool(component=SerperDevWebSearch())
+
+agent = Agent(
+    chat_generator=AmazonBedrockChatGenerator(
+        model="anthropic.claude-3-5-sonnet-20240620-v1:0",
+    ),
+    tools=[search_tool],
+    system_prompt="You are a helpful assistant that can search the web for information.",
+)
+
+result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")])
+
+print(result["last_message"].text)
+```
+
+</TabItem>
+<TabItem value="google-gemini" label="Google Gemini">
+
+Install the [Google Gen AI integration](https://haystack.deepset.ai/integrations/google-genai):
+
+```bash
+pip install google-genai-haystack
+```
+
+See the [GoogleGenAIChatGenerator](../pipeline-components/generators/googlegenaichatgenerator.mdx) docs for more details.
+
+```python
+from haystack.components.agents import Agent
+from haystack_integrations.components.generators.google_genai import (
+    GoogleGenAIChatGenerator,
+)
+from haystack.dataclasses import ChatMessage
+from haystack.tools import ComponentTool
+from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
+from haystack.utils import Secret
+
+search_tool = ComponentTool(component=SerperDevWebSearch())
+
+agent = Agent(
+    chat_generator=GoogleGenAIChatGenerator(
+        api_key=Secret.from_env_var("GOOGLE_API_KEY"),
+        model="gemini-2.5-flash",
+    ),
+    tools=[search_tool],
+    system_prompt="You are a helpful assistant that can search the web for information.",
+)
+
+result = agent.run(messages=[ChatMessage.from_user("What is Haystack AI?")])
+
+print(result["last_message"].text)
+```
+
+</TabItem>
+<TabItem value="more-providers" label="More Providers">
+
+<div style={{backgroundColor: 'var(--ifm-color-emphasis-100)', padding: '1.5rem', borderRadius: '8px'}}>
+
+Haystack supports many more model providers including **Cohere**, **Mistral**, **NVIDIA**, **Ollama**, and others—both cloud-hosted and local options.
+
+Browse the full list of supported models and chat generators in the [Generators documentation](../pipeline-components/generators.mdx).
+
+You can also explore all available integrations on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page.
+
+</div>
+
+</TabItem>
+</Tabs>
+
+### Next Steps
+
+For a hands-on guide on creating a tool-calling agent that can use both components and pipelines as tools, check out the [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent) tutorial.

docs-website/docs/overview/installation.mdx

@@ -0,0 +1,51 @@
+---
+title: "Installation"
+id: installation
+slug: "/installation"
+description: "See how to quickly install Haystack with pip, uv, or conda."
+---
+
+# Installation
+
+See how to quickly install Haystack with pip, uv, or conda.
+
+## Package Installation
+
+Use [pip](https://github.com/pypa/pip) to install the [Haystack PyPI package](https://pypi.org/project/haystack-ai/):
+
+```shell
+pip install haystack-ai
+```
+
+Alternatively, you can use [uv](https://docs.astral.sh/uv/) to install Haystack:
+
+```shell
+uv pip install haystack-ai
+```
+
+Or add it as a dependency to your project:
+
+```shell
+uv add haystack-ai
+```
+
+You can also use [conda](https://docs.conda.io/projects/conda/en/stable/) to install the [Haystack conda package](https://anaconda.org/conda-forge/haystack-ai):
+
+```shell
+conda install conda-forge::haystack-ai
+```
+
+### Optional Dependencies
+
+Some components in Haystack rely on additional optional dependencies.
+To keep the installation lightweight, these are not included by default – only the essentials are installed.
+If you use a feature that requires an optional dependency that hasn't been installed, Haystack will raise an error that instructs you to install missing dependencies, for example:
+
+```shell
+ImportError: "Haystack failed to import the optional dependency 'pypdf'. Run 'pip install pypdf'.
+```
+
+## Contributing to Haystack
+
+If you would like to contribute to the Haystack, check our [Contributor Guidelines](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md) first
+and follow the instructions [here](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md#setting-up-your-development-environment) to set up your development environment.

docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx

@@ -0,0 +1,669 @@
+---
+title: "Migrating from LangGraph/LangChain to Haystack"
+id: migrating-from-langgraphlangchain-to-haystack
+slug: "/migrating-from-langgraphlangchain-to-haystack"
+description: "Whether you're planning to migrate to Haystack or just comparing LangChain/LangGraph and Haystack to choose the proper framework for your AI application, this guide will help you map common patterns between frameworks."
+---
+
+import CodeBlock from '@theme/CodeBlock';
+
+# Migrating from LangGraph/LangChain to Haystack
+
+Whether you're planning to migrate to Haystack or just comparing **LangChain/LangGraph** and **Haystack** to choose the proper framework for your AI application, this guide will help you map common patterns between frameworks.
+
+In this guide, you'll learn how to translate core LangGraph concepts, like nodes, edges, and state, into Haystack components, pipelines, and agents. The goal is to preserve your existing logic while leveraging Haystack's flexible, modular ecosystem.
+
+It's most accurate to think of Haystack as covering both **LangChain** and **LangGraph** territory: Haystack provides the building blocks for everything from simple sequential flows to fully agentic workflows with custom logic.
+
+## Why you might explore or migrate to Haystack
+
+You might consider Haystack if you want to build your AI applications on a **stable, actively maintained foundation** with an intuitive developer experience.
+
+* **Unified orchestration framework.** Haystack supports both deterministic pipelines and adaptive agentic flows, letting you combine them with the right level of autonomy in a single system.
+* **High-quality codebase and design.** Haystack is engineered for clarity and reliability with well-tested components, predictable APIs, and a modular architecture that simply works.
+* **Ease of customization.** Extend core components, add your own logic, or integrate custom tools with minimal friction.
+* **Reduced cognitive overhead.** Haystack extends familiar ideas rather than introducing new abstractions, helping you stay focused on applying concepts, not learning them.
+* **Comprehensive documentation and learning resources.** Every concept, from components and pipelines to agents and tools, is supported by detailed and well-maintained docs, tutorials, and educational content.
+* **Frequent release cycles.** New features, improvements, and bug fixes are shipped regularly, ensuring that the framework evolves quickly while maintaining backward compatibility.
+* **Scalable from prototype to production.** Start small and expand easily. The same code you use for a proof of concept can power enterprise-grade deployments through the whole Haystack ecosystem.
+
+## Concept mapping: LangGraph/LangChain → Haystack
+
+Here's a table of key concepts and their approximate equivalents between the two frameworks. Use this when auditing your LangGraph/Langchain architecture and planning the migration.
+
+| LangGraph/LangChain concept | Haystack equivalent | Notes |
+| --- | --- | --- |
+| Node | Component | A unit of logic in both frameworks. In Haystack, a [Component](../concepts/components.mdx) can run standalone, in a pipeline, or as a tool with agent. You can [create custom components](../concepts/components/custom-components.mdx) or use built-in ones like Generators and Retrievers. |
+| Edge / routing logic | Connection / Branching / Looping | [Pipelines](../concepts/pipelines.mdx) connect component inputs and outputs with type-checked links. They support branching, routing, and loops for flexible flow control. |
+| Graph / Workflow (nodes + edges) | Pipeline or Agent | LangGraph explicitly defines graphs; Haystack achieves similar orchestration through pipelines or [Agents](../concepts/agents.mdx) when adaptive logic is needed. |
+| Subgraphs | SuperComponent | A [SuperComponent](../concepts/components/supercomponents.mdx) wraps a full pipeline and exposes it as a single reusable component |
+| Models / LLMs | ChatGenerator Components | Haystack's [ChatGenerators](../pipeline-components/generators.mdx) unify access to open and proprietary models, with support for streaming, structured outputs, and multimodal data. |
+| Agent Creation (`create_agent`, multi-agent from LangChain) | Agent Component | Haystack provides a simple, pipeline-based [Agent](../concepts/agents.mdx) abstraction that handles reasoning, tool use, and multi-step execution. |
+| Tool (Langchain) | [Tool](../tools/tool.mdx) / [PipelineTool](../tools/pipelinetool.mdx) / [ComponentTool](../tools/componenttool.mdx) / [MCPTool](../tools/mcptool.mdx) | Haystack exposes Python functions, pipelines, components,  external APIs and MCP servers as agent tools. |
+| Multi-Agent Collaboration (LangChain) | Multi-Agent System | Using [`ComponentTool`](../tools/componenttool.mdx), agents can use other agents as tools, enabling [multi-agent architectures](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system) within one framework. |
+| Model Context Protocol `load_mcp_tools` `MultiServerMCPClient` | Model Context Protocol - `MCPTool`, `MCPToolset`, `StdioServerInfo`, `StreamableHttpServerInfo` | Haystack provides [various MCP primitives](https://haystack.deepset.ai/integrations/mcp) for connecting multiple MCP servers and organizing MCP toolsets. |
+| Memory (State, short-term, long-term) | Memory (Agent State, short-term, long-term) | Agent [State](../pipeline-components/agents-1/state.mdx) provides a structured way to share data between tools and store intermediate results during agent execution. For short-term memory, Haystack offers a [ChatMessage Store](/reference/experimental-chatmessage-store-api) to persist chat history. More memory options are coming soon. |
+| Time travel (Checkpoints) | Breakpoints (Breakpoint, PipelineSnapshot) | [Breakpoints](../concepts/pipelines/pipeline-breakpoints.mdx) let you pause, inspect, modify, and resume a pipeline for debugging or iterative development. |
+| Human-in-the-Loop (Interrupts / Commands) | Human-in-the-loop (`ConfirmationHook` with confirmation strategies) | Haystack applies [confirmation strategies](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent) through a `ConfirmationHook` registered under the Agent's `before_tool` [hook point](../pipeline-components/agents-1/hooks.mdx) to pause or block the execution to gather user feedback |
+
+## Ecosystem and Tooling Mapping: LangChain → Haystack
+
+At deepset, we're building the tools to make LLMs truly usable in production, open source and beyond.
+
+* [Haystack, AI Orchestration Framework](https://github.com/deepset-ai/haystack) → Open Source AI framework for building production-ready, AI-powered agents and applications, on your own or with community support.
+* [Haystack Enterprise Starter](https://www.deepset.ai/products-and-services/haystack-enterprise) → Private and secure engineering support, advanced pipeline templates, deployment guides, and early access features for teams needing more support and guidance.
+* [Haystack Enterprise Platform](https://www.deepset.ai/products-and-services/deepset-ai-platform) → An enterprise-ready platform for teams running Gen AI apps in production, with security, governance, and scalability built in with [a free version](https://www.deepset.ai/deepset-studio).
+
+Here's the product equivalent of two ecosystems:
+
+| **LangChain Ecosystem** | **Haystack Ecosystem** | **Notes** |
+| --- | --- | --- |
+| **LangChain, LangGraph, Deep Agents** | **Haystack** | **Core AI orchestration framework for components, pipelines, and agents**. Supports deterministic workflows and agentic execution with explicit, modular building blocks. |
+| **LangSmith (Observability)** | **Haystack Enterprise Platform** | **Integrated tooling for building, debugging and iterating.** Assemble agents and pipelines visually with the **Builder**, which includes component validation, testing and debugging. The **Prompt Explorer** is used to iterate and evaluate models and prompts. Built-in chat interfaces to enable fast SME and stakeholder feedback. Collaborative building environment for engineers and business. |
+| **LangSmith (Deployment)** | **Hayhooks** **Haystack Enterprise Starter** (deployment guides + advanced best practice templates) **Haystack Enterprise Platform** (1-click deployment, on-prem/VPC options) | Multiple deployment paths: lightweight API exposure via [Hayhooks](https://github.com/deepset-ai/hayhooks), structured enterprise deployment patterns through Haystack Enterprise Starter, and full managed or self-hosted deployment through the Haystack Enterprise Platform. |
+
+## Code Comparison
+
+### Agentic Flows with Haystack vs LangGraph
+
+Here's an example **graph-based agent** with access to a list of tools, comparing the LangGraph and Haystack APIs.
+
+**Step 1: Define tools**
+
+Both frameworks use a `@tool` decorator to expose Python functions as tools the LLM can call. The function signature and docstring define the tool's interface, which the LLM uses to understand when and how to invoke each tool.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`# pip install haystack-ai anthropic-haystack
+
+from haystack.tools import tool
+
+# Define tools
+@tool
+def multiply(a: int, b: int) -> int:
+    """Multiply \`a\` and \`b\`.
+
+    Args:
+        a: First int
+        b: Second int
+    """
+    return a * b
+
+@tool
+def add(a: int, b: int) -> int:
+    """Adds \`a\` and \`b\`.
+
+    Args:
+        a: First int
+        b: Second int
+    """
+    return a + b
+
+@tool
+def divide(a: int, b: int) -> float:
+    """Divide \`a\` and \`b\`.
+
+    Args:
+        a: First int
+        b: Second int
+    """
+    return a / b`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`# pip install langchain-anthropic langgraph langchain
+
+from langchain.tools import tool
+
+# Define tools
+@tool
+def multiply(a: int, b: int) -> int:
+    """Multiply \`a\` and \`b\`.
+
+    Args:
+        a: First int
+        b: Second int
+    """
+    return a * b
+
+@tool
+def add(a: int, b: int) -> int:
+    """Adds \`a\` and \`b\`.
+
+    Args:
+        a: First int
+        b: Second int
+    """
+    return a + b
+
+@tool
+def divide(a: int, b: int) -> float:
+    """Divide \`a\` and \`b\`.
+
+    Args:
+        a: First int
+        b: Second int
+    """
+    return a / b`}</CodeBlock>
+  </div>
+</div>
+
+**Step 2: Initialize the LLM with tools**
+
+Both frameworks connect tools to the LLM, but with different APIs. In Haystack, tools are passed directly to the `ChatGenerator` component during initialization. In LangGraph, you first initialize the model, then bind tools using `.bind_tools()` to create a tool-enabled LLM instance.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+
+# Augment the LLM with tools
+tools = [add, multiply, divide]
+model = AnthropicChatGenerator(
+    model="claude-sonnet-4-5-20250929",
+    generation_kwargs={"temperature": 0},
+    tools=tools,
+)`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`from langchain.chat_models import init_chat_model
+
+# Augment the LLM with tools
+model = init_chat_model(
+    "claude-sonnet-4-5-20250929",
+    temperature=0,
+)
+tools = [add, multiply, divide]
+tools_by_name = {tool.name: tool for tool in tools}
+llm_with_tools = model.bind_tools(tools)`}</CodeBlock>
+  </div>
+</div>
+
+**Step 3: Set up message handling and LLM invocation**
+
+This is where the frameworks diverge more significantly. In Haystack you'll use a custom component (`MessageCollector`) to accumulate conversation history across the agentic loop. LangGraph instead defines a node function (`llm_call`) that operates on `MessagesState` - a built-in state container that automatically manages message history.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`from typing import Any, Dict, List
+
+from haystack import component
+from haystack.core.component.types import Variadic
+from haystack.dataclasses import ChatMessage
+
+# Components
+
+# Custom component to temporarily store the messages
+@component()
+class MessageCollector:
+    def __init__(self):
+        self._messages = []
+
+    @component.output_types(messages=List[ChatMessage])
+    def run(self, messages: Variadic[List[ChatMessage]]) -> Dict[str, Any]:
+        self._messages.extend([msg for inner in messages for msg in inner])
+        return {"messages": self._messages}
+
+    def clear(self):
+        self._messages = []
+
+message_collector = MessageCollector()`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`from langgraph.graph import MessagesState
+from langchain.messages import SystemMessage, ToolMessage
+from typing import Literal
+
+# Nodes
+def llm_call(state: MessagesState):
+    # LLM decides whether to call a tool or not
+
+    return {
+        "messages": [
+            llm_with_tools.invoke(
+                [
+                    SystemMessage(
+                        content="You are a helpful assistant tasked with performing arithmetic on a set of inputs."
+                    )
+                ]
+                + state["messages"]
+            )
+        ]
+    }`}</CodeBlock>
+  </div>
+</div>
+
+**Step 4: Execute tool calls**
+
+When the LLM decides to use a tool, it must be invoked and its result returned. Haystack provides a built-in `ToolInvoker` component that handles this automatically. LangGraph requires you to define a custom node function that iterates over tool calls, invokes each tool, and wraps the results in `ToolMessage` objects.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`from haystack.components.tools import ToolInvoker
+
+# Tool invoker component to execute a tool call
+tool_invoker = ToolInvoker(tools=tools)`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`def tool_node(state: dict):
+    # Performs the tool call
+    result = []
+    for tool_call in state["messages"][-1].tool_calls:
+        tool = tools_by_name[tool_call["name"]]
+        observation = tool.invoke(tool_call["args"])
+        result.append(ToolMessage(content=observation, tool_call_id=tool_call["id"]))
+    return {"messages": result}`}</CodeBlock>
+  </div>
+</div>
+
+**Step 5: Implement conditional routing logic**
+
+After the LLM responds, we need to decide whether to continue the loop (if tools were called) or finish (if the LLM provided a final answer). Haystack uses a `ConditionalRouter` component with declarative route conditions written in Jinja2 templates. LangGraph uses a conditional edge function (`should_continue`) that inspects the state and returns the next node or `END`.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`from haystack.components.routers import ConditionalRouter
+
+# ConditionalRouter component to route to the tool invoker or end user based upon whether the LLM made a tool call
+routes = [
+    {
+        "condition": "{{replies[0].tool_calls | length > 0}}",
+        "output": "{{replies}}",
+        "output_name": "there_are_tool_calls",
+        "output_type": List[ChatMessage],
+    },
+    {
+        "condition": "{{replies[0].tool_calls | length == 0}}",
+        "output": "{{replies}}",
+        "output_name": "final_replies",
+        "output_type": List[ChatMessage],
+    },
+]
+router = ConditionalRouter(routes, unsafe=True)`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`from langgraph.graph import END
+
+# Conditional edge function to route to the tool node or end based upon whether the LLM made a tool call
+def should_continue(state: MessagesState) -> Literal["tool_node", END]:
+    # Decide if we should continue the loop or stop based upon whether the LLM made a tool call
+
+    messages = state["messages"]
+    last_message = messages[-1]
+
+    # If the LLM makes a tool call, then perform an action
+    if last_message.tool_calls:
+        return "tool_node"
+
+    # Otherwise, we stop (reply to the user)
+    return END`}</CodeBlock>
+  </div>
+</div>
+
+**Step 6: Assemble the workflow**
+
+This is where you wire together all the components or nodes. Haystack uses a `Pipeline` where you explicitly add components and connect their inputs and outputs, creating a directed graph with loops. LangGraph uses a `StateGraph` where you add nodes and edges, then compile the graph into an executable agent. Both approaches achieve the same agentic loop, but with different levels of explicitness.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`from haystack import Pipeline
+
+# Build pipeline
+agent_pipe = Pipeline()
+
+# Add components
+agent_pipe.add_component("message_collector", message_collector)
+agent_pipe.add_component("llm", model)
+agent_pipe.add_component("router", router)
+agent_pipe.add_component("tool_invoker", tool_invoker)
+
+# Add connections
+agent_pipe.connect("message_collector", "llm.messages")
+agent_pipe.connect("llm.replies", "router")
+agent_pipe.connect("router.there_are_tool_calls", "tool_invoker") # If there are tool calls, send them to the ToolInvoker
+agent_pipe.connect("router.there_are_tool_calls", "message_collector")
+agent_pipe.connect("tool_invoker.tool_messages", "message_collector")`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`from langgraph.graph import StateGraph, START
+
+# Build workflow
+agent_builder = StateGraph(MessagesState)
+
+# Add nodes
+agent_builder.add_node("llm_call", llm_call)
+agent_builder.add_node("tool_node", tool_node)
+
+# Add edges to connect nodes
+agent_builder.add_edge(START, "llm_call")
+agent_builder.add_conditional_edges(
+    "llm_call",
+    should_continue,
+    ["tool_node", END]
+)
+agent_builder.add_edge("tool_node", "llm_call")
+
+# Compile the agent
+agent = agent_builder.compile()`}</CodeBlock>
+  </div>
+</div>
+
+**Step 7: Run the agent**
+
+Finally, we execute the agent with a user message. Haystack calls `.run()` on the pipeline with initial messages, while LangGraph calls `.invoke()` on the compiled agent. Both return the conversation history.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`# Run the pipeline
+messages = [
+    ChatMessage.from_system(text="You are a helpful assistant tasked with performing arithmetic on a set of inputs."),
+    ChatMessage.from_user(text="Add 3 and 4.")
+]
+result = agent_pipe.run({"messages": messages})
+result`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`from langchain.messages import HumanMessage
+
+# Invoke
+messages = [
+    HumanMessage(content="Add 3 and 4.")
+]
+messages = agent.invoke({"messages": messages})
+for m in messages["messages"]:
+    m.pretty_print()`}</CodeBlock>
+  </div>
+</div>
+
+### Creating Agents
+
+The [Agentic Flows](#agentic-flows-with-haystack-vs-langgraph) walkthrough above showed how to assemble an agent loop manually from pipeline primitives. Haystack also provides a high-level `Agent` class that wraps the full loop  -  LLM calls, tool invocation, and iteration  -  into a single component. LangGraph offers an equivalent shortcut through `create_react_agent` in `langgraph.prebuilt`. Both produce a ReAct-style agent that handles tool calling and multi-step reasoning automatically.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`# pip install haystack-ai anthropic-haystack
+
+from haystack.components.agents import Agent
+from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.tools import tool
+
+@tool
+def multiply(a: int, b: int) -> int:
+    """Multiply \`a\` and \`b\`."""
+    return a * b
+
+@tool
+def add(a: int, b: int) -> int:
+    """Add \`a\` and \`b\`."""
+    return a + b
+
+# Create an agent  -  the agentic loop is handled automatically
+agent = Agent(
+    chat_generator=AnthropicChatGenerator(
+        model="claude-sonnet-4-5-20250929",
+        generation_kwargs={"temperature": 0},
+    ),
+    tools=[multiply, add],
+    system_prompt="You are a helpful assistant that performs arithmetic.",
+)
+
+result = agent.run(messages=[
+    ChatMessage.from_user("What is 3 multiplied by 7, then add 5?")
+])
+print(result["messages"][-1].text) # or print(result["last_message"].text)`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`# pip install langchain-anthropic langgraph
+
+from langchain_anthropic import ChatAnthropic
+from langchain_core.tools import tool
+from langchain.agents import create_agent
+from langchain_core.messages import HumanMessage, SystemMessage
+
+@tool
+def multiply(a: int, b: int) -> int:
+    """Multiply \`a\` and \`b\`."""
+    return a * b
+
+@tool
+def add(a: int, b: int) -> int:
+    """Add \`a\` and \`b\`."""
+    return a + b
+
+# Create an agent - the agentic loop is handled automatically
+model = ChatAnthropic(
+    model="claude-sonnet-4-5-20250929",
+    temperature=0,
+)
+agent = create_agent(
+    model,
+    tools=[multiply, add],
+    system_prompt=SystemMessage(
+        content="You are a helpful assistant that performs arithmetic."
+    ),
+)
+
+result = agent.invoke({
+    "messages": [HumanMessage(content="What is 3 multiplied by 7, then add 5?")]
+})
+print(result["messages"][-1].content)`}</CodeBlock>
+  </div>
+</div>
+
+### Connecting to Document Stores
+
+Document stores are the foundation of retrieval-augmented generation (RAG). In Haystack, document stores integrate natively with pipeline components like Retrievers and Prompt Builders via explicit typed connections. LangChain centers retrieval around its vector store abstraction composed using LCEL (LangChain Expression Language).
+
+Both frameworks offer in-memory stores for prototyping and a wide range of production backends (Elasticsearch, Qdrant, Weaviate, Pinecone, and more) via integrations.
+
+**Step 1: Create a document store and add documents**
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`# pip install haystack-ai sentence-transformers-haystack
+
+from haystack import Document
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersDocumentEmbedder
+
+# Embed and write documents to the document store
+document_store = InMemoryDocumentStore()
+
+doc_embedder = SentenceTransformersDocumentEmbedder(
+    model="sentence-transformers/all-MiniLM-L6-v2"
+)
+
+docs = [
+    Document(content="Paris is the capital of France."),
+    Document(content="Berlin is the capital of Germany."),
+    Document(content="Tokyo is the capital of Japan."),
+]
+docs_with_embeddings = doc_embedder.run(docs)["documents"]
+document_store.write_documents(docs_with_embeddings)`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangChain">{`# pip install langchain-community langchain-huggingface sentence-transformers
+
+from langchain_huggingface import HuggingFaceEmbeddings
+from langchain_community.vectorstores import InMemoryVectorStore
+from langchain_core.documents import Document
+
+# Embed and add documents to the vector store
+embeddings = HuggingFaceEmbeddings(
+    model_name="sentence-transformers/all-MiniLM-L6-v2"
+)
+vectorstore = InMemoryVectorStore(embedding=embeddings)
+vectorstore.add_documents([
+    Document(page_content="Paris is the capital of France."),
+    Document(page_content="Berlin is the capital of Germany."),
+    Document(page_content="Tokyo is the capital of Japan."),
+])`}</CodeBlock>
+  </div>
+</div>
+
+**Step 2: Build a RAG pipeline**
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`from haystack import Pipeline
+from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder
+from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+
+# ChatPromptBuilder expects a List[ChatMessage] as template
+template = [ChatMessage.from_user("""
+Given the following documents, answer the question.
+{% for doc in documents %}{{ doc.content }}{% endfor %}
+Question: {{ question }}
+""")]
+
+rag_pipeline = Pipeline()
+rag_pipeline.add_component(
+    "text_embedder",
+    SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
+)
+rag_pipeline.add_component(
+    "retriever", InMemoryEmbeddingRetriever(document_store=document_store)
+)
+rag_pipeline.add_component(
+    "prompt_builder", ChatPromptBuilder(template=template)
+)
+rag_pipeline.add_component(
+    "llm", AnthropicChatGenerator(model="claude-sonnet-4-5-20250929")
+)
+
+rag_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
+rag_pipeline.connect("retriever.documents", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
+
+result = rag_pipeline.run({
+    "text_embedder": {"text": "What is the capital of France?"},
+    "prompt_builder": {"question": "What is the capital of France?"},
+})
+print(result["llm"]["replies"][0].text)`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangChain">{`from langchain_anthropic import ChatAnthropic
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.output_parsers import StrOutputParser
+from langchain_core.runnables import RunnablePassthrough
+
+def format_docs(docs):
+    return "\\n".join(doc.page_content for doc in docs)
+
+retriever = vectorstore.as_retriever()
+model = ChatAnthropic(model="claude-sonnet-4-5-20250929")
+
+template = """
+Given the following documents, answer the question.
+{context}
+Question: {question}
+"""
+prompt = ChatPromptTemplate.from_template(template)
+
+rag_chain = (
+    {"context": retriever | format_docs, "question": RunnablePassthrough()}
+    | prompt
+    | model
+    | StrOutputParser()
+)
+
+result = rag_chain.invoke("What is the capital of France?")
+print(result)`}</CodeBlock>
+  </div>
+</div>
+
+### Using MCP Tools
+
+Both frameworks support the [Model Context Protocol (MCP)](https://modelcontextprotocol.io), letting agents connect to external tools and services exposed by MCP servers. Haystack provides [`MCPTool`](https://docs.haystack.deepset.ai/docs/mcptool) and [`MCPToolset`](https://docs.haystack.deepset.ai/docs/mcptoolset) through the `mcp-haystack` integration package, which plug directly into the `Agent` component. LangChain's MCP support relies on the separate `langchain-mcp-adapters` package and requires an async workflow throughout.
+
+<div className="code-comparison">
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="Haystack">{`# pip install haystack-ai mcp-haystack anthropic-haystack
+
+from haystack_integrations.tools.mcp import MCPToolset, StdioServerInfo
+from haystack.components.agents import Agent
+from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
+from haystack.dataclasses import ChatMessage
+
+# Connect to an MCP server - tools are auto-discovered
+toolset = MCPToolset(
+    server_info=StdioServerInfo(
+        command="uvx",
+        args=["mcp-server-fetch"],
+    )
+)
+
+agent = Agent(
+    chat_generator=AnthropicChatGenerator(model="claude-sonnet-4-5-20250929"),
+    tools=toolset,
+    system_prompt="You are a helpful assistant that can fetch web content.",
+)
+
+result = agent.run(messages=[
+    ChatMessage.from_user("Fetch the content from https://haystack.deepset.ai")
+])
+print(result["messages"][-1].text) # or print(result["last_message"].text)`}</CodeBlock>
+  </div>
+  <div className="code-comparison__column">
+    <CodeBlock language="python" title="LangGraph + LangChain">{`# pip install langchain-mcp-adapters langgraph langchain-anthropic
+
+import asyncio
+from langchain_mcp_adapters.client import MultiServerMCPClient
+from langchain.agents import create_agent
+from langchain_anthropic import ChatAnthropic
+from langchain_core.messages import HumanMessage, SystemMessage
+
+model = ChatAnthropic(model="claude-sonnet-4-5-20250929")
+
+async def run():
+    client = MultiServerMCPClient(
+        {
+            "fetch": {
+                "command": "uvx",
+                "args": ["mcp-server-fetch"],
+                "transport": "stdio",
+            }
+        }
+    )
+    tools = await client.get_tools()
+    agent = create_agent(
+        model,
+        tools,
+        system_prompt=SystemMessage(
+            content="You are a helpful assistant that can fetch web content."
+        ),
+    )
+    result = await agent.ainvoke(
+        {
+            "messages": [
+                HumanMessage(content="Fetch the content from https://haystack.deepset.ai")
+            ]
+        }
+    )
+    print(result["messages"][-1].content)
+
+
+asyncio.run(run())`}</CodeBlock>
+  </div>
+</div>
+
+## Hear from Haystack Users
+
+See how teams across industries use Haystack to power their production AI systems, from RAG applications to agentic workflows.
+
+> "_Haystack allows its users a production ready, easy to use framework that covers just about all of your needs, and allows you to write integrations easily for those it doesn't._"
+> **- Josh Longenecker, GenAI Specialist at AWS**
+>
+> _"Haystack's design philosophy significantly accelerates development and improves the robustness of AI applications, especially when heading towards production. The emphasis on explicit, modular components truly pays off in the long run."_
+> **- Rima Hajou, Data & AI Technical Lead at Accenture**
+
+### Featured Stories
+
+* [TELUS Agriculture & Consumer Goods Built an Agentic Chatbot with Haystack to Transform Trade Promotions Workflows](https://haystack.deepset.ai/blog/telus-user-story)
+* [Lufthansa Industry Solutions Uses Haystack to Power Enterprise RAG](https://haystack.deepset.ai/blog/lufthansa-user-story)
+
+## Start Building with Haystack
+
+**👉 Thinking about migrating or evaluating Haystack?** Jump right in with the [Haystack Get Started guide](https://haystack.deepset.ai/overview/quick-start) or [contact our team](https://www.deepset.ai/products-and-services/haystack-enterprise-starter), we'd love to support you.

docs-website/docs/overview/migration.mdx

@@ -0,0 +1,508 @@
+---
+title: "Migration Guide"
+id: migration
+slug: "/migration"
+description: "Learn how to make the move to Haystack 2.x from Haystack 1.x."
+---
+
+# Migration Guide
+
+Learn how to make the move to Haystack 2.x from Haystack 1.x.
+
+This guide is designed for those with previous experience with Haystack and who are interested in understanding the differences between Haystack 1.x and Haystack 2.x. If you're new to Haystack, skip this page and proceed directly to Haystack 2.x [documentation](get-started.mdx).
+
+## Major Changes
+
+Haystack 2.x represents a significant overhaul of Haystack 1.x, and it's important to note that certain key concepts outlined in this section don't have a direct correlation between the two versions.
+
+### Package Name
+
+Haystack 1.x was distributed with a package called `farm-haystack`. To migrate your application, you must uninstall `farm-haystack` and install the new `haystack-ai` package for Haystack 2.x.
+
+:::warning
+Two versions of the project cannot coexist in the same Python environment.
+
+One of the options is to remove both packages if they are installed in the same environment, followed by installing only one of them:
+
+```bash
+pip uninstall -y farm-haystack haystack-ai
+pip install haystack-ai
+```
+:::
+
+### Nodes
+
+While Haystack 2.x continues to rely on the `Pipeline` abstraction, the elements linked in a pipeline graph are now referred to as just _components_, replacing the terms _nodes_ and _pipeline components_ used in the previous versions. The [_Migrating Components_](#migrating-components) paragraph below outlines which component in Haystack 2.x can be used as a replacement for a specific 1.x node.
+
+### Pipelines
+
+Pipelines continue to serve as the fundamental structure of all Haystack applications. While the concept of `Pipeline` abstraction remains consistent, Haystack 2.x introduces significant enhancements that address various limitations of its predecessor. For instance, the pipelines now support loops. Pipelines also offer greater flexibility in their input, which is no longer restricted to queries. The pipeline now allows to route the output of a component to multiple recipients. This increases flexibility, however, comes with notable differences in the pipeline definition process in Haystack 2.x compared to the previous version.
+
+In Haystack 1.x, a pipeline was built by adding one node after the other. In the resulting pipeline graph, edges are automatically added to connect those nodes in the order they were added.
+
+Building a pipeline in Haystack 2.x is a two-step process:
+
+1. Initially, components are added to the pipeline without any specific order by calling the `add_component` method.
+2. Subsequently, the components must be explicitly connected by calling the `connect` method to define the final graph.
+
+To migrate an existing pipeline, the first step is to go through the nodes and identify their counterparts in Haystack 2.x (see the following section,  [_Migrating Components_](#migrating-components), for guidance). If all the nodes can be replaced by corresponding components, they have to be added to the pipeline with `add_component` and explicitly connected with the appropriate calls to `connect`. Here is an example:
+
+**Haystack 1.x**
+
+```python
+pipeline = Pipeline()
+
+node_1 = SomeNode()
+node_2 = AnotherNode()
+
+pipeline.add_node(node_1, name="Node_1", inputs=["Query"])
+pipeline.add_node(node_2, name="Node_2", inputs=["Node_1"])
+```
+
+**Haystack 2.x**
+
+```python
+pipeline = Pipeline()
+
+component_1 = SomeComponent()
+component_2 = AnotherComponent()
+
+pipeline.add_component("Comp_1", component_1)
+pipeline.add_component("Comp_2", component_2)
+
+pipeline.connect("Comp_1", "Comp_2")
+```
+
+In case a specific replacement component is not available for one of your nodes, migrating the pipeline might still be possible by:
+
+- Either [creating a custom component](../concepts/components/custom-components.mdx), or
+- Changing the pipeline logic, as the last resort.
+
+:::info
+Check out the [Pipelines](../concepts/pipelines.mdx) section of our 2.x documentation to understand how new pipelines work more granularly.
+:::
+
+### Document Stores
+
+The fundamental concept of Document Stores as gateways to access text and metadata stored in a database didn’t change in Haystack 2.x, but there are significant differences against Haystack 1.x.
+
+In Haystack 1.x, Document Stores were a special type of node that you can use in two ways:
+
+- As the last node in an indexing pipeline (such as a pipeline whose ultimate goal is storing data in a database).
+- As a normal Python instance passed to a Retriever node.
+
+In Haystack 2.x, the Document Store is not a component, so to migrate the two use cases above to version 2.x, you can respectively:
+
+- Replace the Document Store at the end of the pipeline with a [`DocumentWriter`](../pipeline-components/writers/documentwriter.mdx)  component.
+- Identify the right Retriever component and create it passing the Document Store instance, same as it is in Haystack 1.x.
+
+### Retrievers
+
+Haystack 1.x provided a set of nodes that filter relevant documents from different data sources according to a given query. Each of those nodes implements a certain retrieval algorithm and supports one or more types of Document Stores. For example, the `BM25Retriever` node in Haystack 1.x can work seamlessly with OpenSearch and Elasticsearch but not with Qdrant; the `EmbeddingRetriever`, on the contrary, can work with all the three databases.
+
+In Haystack 2.x, the concept is flipped, and each Document Store provides one or more retriever components, depending on which retrieval methods the underlying vector database supports. For example, the `OpenSearchDocumentStore` comes with [two Retriever components](../document-stores/opensearch-document-store.mdx#supported-retrievers), one relying on BM25, and the other on vector similarity.
+
+To migrate a 1.x retrieval pipeline to 2.x, the first step is to identify the Document Store being used and replace the Retriever node with the corresponding Retriever component from Haystack 2.x with the Document Store of choice. For example, a `BM25Retriever` node using Elasticsearch in a Haystack 1.x pipeline should be replaced with the [`ElasticsearchBM25Retriever`](../pipeline-components/retrievers/elasticsearchbm25retriever.mdx)  component.
+
+### PromptNode
+
+The `PromptNode`  in Haystack 1.x represented the gateway to any Large Language Model (LLM) inference provider, whether it is locally available or remote. Based on the name of the model, Haystack infers the right provider to call and forward the query.
+
+In Haystack 2.x, the task of using LLMs is assigned to [Generators](../pipeline-components/generators.mdx). These are a set of components that are highly specialized and tailored for each inference provider.
+
+The first step when migrating a pipeline with a `PromptNode` is to identify the model provider used and to replace the node with two components:
+
+- A Generator component for the model provider of choice,
+- A `PromptBuilder` or `ChatPromptBuilder` component to build the prompt to be used.
+
+The [_Migration examples_](#migration-examples) section below shows how to port a `PromptNode` using OpenAI with a prompt template to a corresponding Haystack 2.x pipeline using the `OpenAIGenerator` in conjunction with a `PromptBuilder` component.
+
+### Agents
+
+The agentic approach facilitates the answering of questions that are significantly more complex than those typically addressed by extractive or generative question answering techniques.
+
+Haystack 1.x provided Agents, enabling the use of LLMs in a loop.
+
+Currently in Haystack 2.x, you can build Agents using three main elements in a pipeline: Chat Generators, ToolInvoker component, and Tools. A standalone Agent abstraction in Haystack 2.x is in an experimental phase.
+
+:::note[Agents Documentation Page]
+
+Take a look at our 2.x [Agents](../concepts/agents.mdx) documentation page for more information and detailed examples.
+:::
+
+### REST API
+
+Haystack 1.x enabled the deployment of pipelines through a RESTful API over HTTP. This feature is facilitated by a separate application named `rest_api` which is exclusively accessible in the form of a [source code on GitHub](https://github.com/deepset-ai/haystack/tree/v1.x/rest_api).
+
+Haystack 2.x takes the same RESTful approach, but in this case, the application to be used to deploy pipelines is called [Hayhooks](../development/hayhooks.mdx) and can be installed with `pip install hayhooks`.
+
+At the moment, porting an existing Haystack 1.x deployment using the `rest_api` project to Hayhooks would require a complete rewrite of the application.
+
+## Dependencies
+
+In order to minimize runtime errors, Haystack 1.x was distributed in a package that’s quite large, as it tries to set up the Python environment with as many dependencies as possible.
+
+In contrast, Haystack 2.x strives for a more streamlined approach, offering a minimal set of dependencies right out of the box. It features a system that issues a warning when an additional dependency is required, thereby providing the user with the necessary instructions.
+
+To make sure all the dependencies are satisfied when migrating a Haystack 1.x application to version 2.x, a good strategy is to run end-to-end tests and cover all the execution paths to ensure all the required dependencies are available in the target Python environment.
+
+## Migrating Components
+
+This table outlines which component (or a group of components) can be used to replace a certain node when porting a Haystack 1.x pipeline to the latest 2.x version. It’s important to note that when a Haystack 2.x replacement is not available, this doesn’t necessarily mean we are planning this feature.
+
+If you need help migrating a 1.x node without a 2.x counterpart, open an [issue](https://github.com/deepset-ai/haystack/issues) in Haystack GitHub repository.
+
+### Data Handling
+
+| Haystack 1.x               | Description                                                                                                                                                                             | Haystack 2.x                                                                         |
+| --- | --- | --- |
+| Crawler                    | Scrapes text from websites. **Example usage:** To run searches on your website content.                                                                                                 | Not Available                                                                        |
+| DocumentClassifier         | Classifies documents by attaching metadata to them. **Example usage:** Labeling documents by their characteristic (for example, sentiment).                                             | [TransformersZeroShotDocumentClassifier](../pipeline-components/classifiers/transformerszeroshotdocumentclassifier.mdx) |
+| DocumentLanguageClassifier | Detects the language of the documents you pass to it and adds it to the document metadata.                                                                                              | [DocumentLanguageClassifier](../pipeline-components/classifiers/documentlanguageclassifier.mdx)                       |
+| EntityExtractor            | Extracts predefined entities out of a piece of text. **Example usage:** Named entity extraction (NER).                                                                                  | [NamedEntityExtractor](../pipeline-components/extractors/transformersnamedentityextractor.mdx)                                   |
+| FileClassifier             | Distinguishes between text, PDF, Markdown, Docx, and HTML files. **Example usage:** Routing files to appropriate converters (for example, it routes PDF files to `PDFToTextConverter`). | [FileTypeRouter](../pipeline-components/routers/filetyperouter.mdx)                                               |
+| FileConverter              | Cleans and splits documents in different formats. **Example usage:** In indexing pipelines, extracting text from a file and casting it into the Document class format.                  | [Converters](../pipeline-components/converters.mdx)                                                       |
+| PreProcessor               | Cleans and splits documents. **Example usage:** Normalizing white spaces, getting rid of headers and footers, splitting documents into smaller ones.                                    | [PreProcessors](../pipeline-components/preprocessors.mdx)                                                 |
+
+### Semantic Search
+
+| Haystack 1.x      | Description                                                                                                                                                                                                                 | Haystack 2.x                                                                            |
+| --- | --- | --- |
+| Ranker            | Orders documents based on how relevant they are to the query. **Example usage:** In a query pipeline, after a keyword-based Retriever to rank the documents it returns.                                                     | [Rankers](../pipeline-components/rankers.mdx)                                                                |
+| Reader            | Finds an answer by selecting a text span in documents. **Example usage:** In a query pipeline when you want to know the location of the answer.                                                                             | [TransformersExtractiveReader](../pipeline-components/readers/transformersextractivereader.mdx)                                              |
+| Retriever         | Fetches relevant documents from the Document Store. **Example usage:** Coupling Retriever with a Reader in a query pipeline to speed up the search (the Reader only goes through the documents it gets from the Retriever). | [Retrievers](../pipeline-components/retrievers.mdx)                                                          |
+| QuestionGenerator | When given a document, it generates questions this document can answer. **Example usage:** Auto-suggested questions in your search app.                                                                                     | Prompt [Builders](../pipeline-components/builders.mdx) with dedicated prompt, [Generators](../pipeline-components/generators.mdx) |
+
+### Prompts and LLMs
+
+| Haystack 1.x | Description                                                                                                                                                                                                                   | Haystack 2.x                                                     |
+| --- | --- | --- |
+| PromptNode   | Uses large language models to perform various NLP tasks in a pipeline or on its own. **Example usage:** It's a very versatile component that can perform tasks like summarization, question answering, translation, and more. | Prompt [Builders](../pipeline-components/builders.mdx),[Generators](../pipeline-components/generators.mdx) |
+
+### Routing
+
+| Haystack 1.x | Description | Haystack 2.x |
+| --- | --- | --- |
+| QueryClassifier | Categorizes queries. **Example usage:** Distinguishing between keyword queries and natural language questions and routing them to the Retrievers that can handle them best. | [TransformersZeroShotTextRouter](../pipeline-components/routers/transformerszeroshottextrouter.mdx)  <br />[TransformersTextRouter](../pipeline-components/routers/transformerstextrouter.mdx) |
+| RouteDocuments | Routes documents to different branches of your pipeline based on their content type or metadata field. **Example usage:** Routing table data to `TableReader` and text data to `TransfomersReader` for better handling. | [Routers](../pipeline-components/routers.mdx) |
+
+### Utility Components
+
+| Haystack 1.x            | Description                                                                                                                                                                                                                                                                                                                                            | Haystack 2.x                                                                            |
+| --- | --- | --- |
+| DocumentMerger          | Concatenates multiple documents into a single one. **Example usage: **Merge the documents to summarize in a summarization pipeline.                                                                                                                                                                                                                    | Prompt [Builders](../pipeline-components/builders.mdx)                                                       |
+| Docs2Answers            | Converts Documents into Answers. **Example usage:** When using REST API for document retrieval. REST API expects Answer as output, you can use `Doc2Answer` as the last node to convert the retrieved documents to answers.                                                                                                                            | [AnswerBuilder](../pipeline-components/builders/answerbuilder.mdx)                                                    |
+| JoinAnswers             | Takes answers returned by multiple components and joins them in a single list of answers. **Example usage:** For running queries on different document types (for example, tables and text), where the documents are routed to different readers, and each reader returns a separate list of answers.                                                  | [AnswerJoiner](../pipeline-components/joiners/answerjoiner.mdx)                                                        |
+| JoinDocuments           | Takes documents returned by different components and joins them to form one list of documents. **Example usage:** In document retrieval pipelines, where there are different types of documents, each routed to a different Retriever. Each Retriever returns a separate list of documents, and you can join them into one list using `JoinDocuments`. | [DocumentJoiner](../pipeline-components/joiners/documentjoiner.mdx)                                                  |
+| Shaper                  | Currently functions mostly as `PromptNode` helper making sure the `PromptNode` input or output is correct. **Example usage:** In a question answering pipeline using `PromptNode`, where the `PromptTemplate` expects questions as input, while Haystack pipelines use query. You can use Shaper to rename queries to questions.                       | Prompt [Builders](../pipeline-components/builders.mdx)                                                       |
+| Summarizer              | Creates an overview of a document. **Example usage:** To get a glimpse of the documents the Retriever is returning.                                                                                                                                                                                                                                    | Prompt [Builders](../pipeline-components/builders.mdx) with dedicated prompt, [Generators](../pipeline-components/generators.mdx) |
+| TransformersImageToText | Generates captions for images. **Example usage:** Automatically generate captions for a list of images that you can later use in your knowledge base.                                                                                                                                                                                                  | [VertexAIImageQA](../pipeline-components/generators/vertexaiimageqa.mdx)                                                  |
+| Translator              | Translates text from one language into another. **Example usage:** Running searches on documents in other languages.                                                                                                                                                                                                                                   | Prompt [Builders](../pipeline-components/builders.mdx) with dedicated prompt, [Generators](../pipeline-components/generators.mdx) |
+
+### Extras
+
+| Haystack 1.x     | Description                                                                                                                                                                      | Haystack 2.x                                                                   |
+| --- | --- | --- |
+| AnswerToSpeech   | Converts text answers into speech answers. **Example usage:** Improving accessibility of your search system by providing a way to have the answer and its context read out loud. | [ElevenLabs](https://haystack.deepset.ai/integrations/elevenlabs) Integration  |
+| DocumentToSpeech | Converts text documents to speech documents. **Example usage:** Improving accessibility of a document retrieval pipeline by providing the option to read documents out loud.     | [ElevenLabs](https://haystack.deepset.ai/integrations/elevenlabs)  Integration |
+
+## Migration examples
+
+:::info
+This section might grow as we assist users with their use cases.
+:::
+
+### Indexing Pipeline
+
+<details>
+
+<summary>Haystack 1.x</summary>
+
+```python
+from haystack.document_stores import InMemoryDocumentStore
+from haystack.nodes.file_classifier import FileTypeClassifier
+from haystack.nodes.file_converter import TextConverter
+from haystack.nodes.preprocessor import PreProcessor
+from haystack.pipelines import Pipeline
+
+# Initialize a DocumentStore
+document_store = InMemoryDocumentStore()
+
+# Indexing Pipeline
+indexing_pipeline = Pipeline()
+
+# Makes sure the file is a TXT file (FileTypeClassifier node)
+classifier = FileTypeClassifier()
+indexing_pipeline.add_node(classifier, name="Classifier", inputs=["File"])
+
+# Converts a file into text and performs basic cleaning (TextConverter node)
+text_converter = TextConverter(remove_numeric_tables=True)
+indexing_pipeline.add_node(
+    text_converter,
+    name="Text_converter",
+    inputs=["Classifier.output_1"],
+)
+
+# Pre-processes the text by performing splits and adding metadata to the text (Preprocessor node)
+preprocessor = PreProcessor(
+    clean_whitespace=True,
+    clean_empty_lines=True,
+    split_length=100,
+    split_overlap=50,
+    split_respect_sentence_boundary=True,
+)
+indexing_pipeline.add_node(preprocessor, name="Preprocessor", inputs=["Text_converter"])
+
+# - Writes the resulting documents into the document store
+indexing_pipeline.add_node(
+    document_store,
+    name="Document_Store",
+    inputs=["Preprocessor"],
+)
+
+# Then we run it with the documents and their metadata as input
+result = indexing_pipeline.run(file_paths=file_paths, meta=files_metadata)
+```
+
+</details>
+
+<details>
+
+<summary>Haystack 2.x</summary>
+
+```python
+from haystack import Pipeline
+from haystack.components.routers import FileTypeRouter
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.converters import TextFileToDocument
+from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
+from haystack.components.writers import DocumentWriter
+
+# Initialize a DocumentStore
+document_store = InMemoryDocumentStore()
+
+# Indexing Pipeline
+indexing_pipeline = Pipeline()
+
+# Makes sure the file is a TXT file (FileTypeRouter component)
+classifier = FileTypeRouter(mime_types=["text/plain"])
+indexing_pipeline.add_component("file_type_router", classifier)
+
+# Converts a file into a Document (TextFileToDocument component)
+text_converter = TextFileToDocument()
+indexing_pipeline.add_component("text_converter", text_converter)
+
+# Performs basic cleaning (DocumentCleaner component)
+cleaner = DocumentCleaner(
+    remove_empty_lines=True,
+    remove_extra_whitespaces=True,
+)
+indexing_pipeline.add_component("cleaner", cleaner)
+
+# Pre-processes the text by performing splits and adding metadata to the text (DocumentSplitter component)
+preprocessor = DocumentSplitter(split_by="passage", split_length=100, split_overlap=50)
+indexing_pipeline.add_component("preprocessor", preprocessor)
+
+# - Writes the resulting documents into the document store
+indexing_pipeline.add_component("writer", DocumentWriter(document_store))
+
+# Connect all the components
+indexing_pipeline.connect("file_type_router.text/plain", "text_converter")
+indexing_pipeline.connect("text_converter", "cleaner")
+indexing_pipeline.connect("cleaner", "preprocessor")
+indexing_pipeline.connect("preprocessor", "writer")
+
+# Then we run it with the documents and their metadata as input
+result = indexing_pipeline.run({"file_type_router": {"sources": file_paths}})
+```
+
+</details>
+
+### Query Pipeline
+
+<details>
+
+<summary>Haystack 1.x</summary>
+
+```python
+from haystack.document_stores import InMemoryDocumentStore
+from haystack.pipelines import ExtractiveQAPipeline
+from haystack import Document
+from haystack.nodes import BM25Retriever
+from haystack.nodes import FARMReader
+
+document_store = InMemoryDocumentStore(use_bm25=True)
+document_store.write_documents(
+    [
+        Document(content="Paris is the capital of France."),
+        Document(content="Berlin is the capital of Germany."),
+        Document(content="Rome is the capital of Italy."),
+        Document(content="Madrid is the capital of Spain."),
+    ],
+)
+
+retriever = BM25Retriever(document_store=document_store)
+reader = FARMReader(model_name_or_path="deepset/roberta-base-squad2")
+extractive_qa_pipeline = ExtractiveQAPipeline(reader, retriever)
+
+query = "What is the capital of France?"
+result = extractive_qa_pipeline.run(
+    query=query,
+    params={"Retriever": {"top_k": 10}, "Reader": {"top_k": 5}},
+)
+```
+
+</details>
+
+<details>
+
+<summary>Haystack 2.x</summary>
+
+```python
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack import Document, Pipeline
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.readers import ExtractiveReader
+
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="Paris is the capital of France."),
+        Document(content="Berlin is the capital of Germany."),
+        Document(content="Rome is the capital of Italy."),
+        Document(content="Madrid is the capital of Spain."),
+    ],
+)
+
+retriever = InMemoryBM25Retriever(document_store)
+reader = ExtractiveReader(model="deepset/roberta-base-squad2")
+extractive_qa_pipeline = Pipeline()
+extractive_qa_pipeline.add_component("retriever", retriever)
+extractive_qa_pipeline.add_component("reader", reader)
+extractive_qa_pipeline.connect("retriever", "reader")
+
+query = "What is the capital of France?"
+result = extractive_qa_pipeline.run(
+    data={
+        "retriever": {"query": query, "top_k": 3},
+        "reader": {"query": query, "top_k": 2},
+    },
+)
+```
+
+</details>
+
+### RAG Pipeline
+
+<details>
+
+<summary>Haystack 1.x</summary>
+
+```python
+from datasets import load_dataset
+
+from haystack.pipelines import Pipeline
+from haystack.document_stores import InMemoryDocumentStore
+from haystack.nodes import EmbeddingRetriever, PromptNode, PromptTemplate, AnswerParser
+
+document_store = InMemoryDocumentStore(embedding_dim=384)
+dataset = load_dataset("bilgeyucel/seven-wonders", split="train")
+document_store.write_documents(dataset)
+retriever = EmbeddingRetriever(
+    embedding_model="sentence-transformers/all-MiniLM-L6-v2",
+    document_store=document_store,
+    top_k=2,
+)
+document_store.update_embeddings(retriever)
+
+rag_prompt = PromptTemplate(
+    prompt="""Synthesize a comprehensive answer from the following text for the given question.
+                             Provide a clear and concise response that summarizes the key points and information presented in the text.
+                             Your answer should be in your own words and be no longer than 50 words.
+                             \n\n Related text: {join(documents)} \n\n Question: {query} \n\n Answer:""",
+    output_parser=AnswerParser(),
+)
+
+prompt_node = PromptNode(
+    model_name_or_path="gpt-3.5-turbo",
+    api_key=OPENAI_API_KEY,
+    default_prompt_template=rag_prompt,
+)
+
+pipe = Pipeline()
+pipe.add_node(component=retriever, name="retriever", inputs=["Query"])
+pipe.add_node(component=prompt_node, name="prompt_node", inputs=["retriever"])
+
+output = pipe.run(query="What does Rhodes Statue look like?")
+```
+
+</details>
+
+<details>
+
+<summary>Haystack 2.x</summary>
+
+```python
+from datasets import load_dataset
+
+from haystack import Document, Pipeline
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.builders import PromptBuilder
+from haystack.components.generators import OpenAIGenerator
+from haystack.components.embedders import SentenceTransformersDocumentEmbedder
+from haystack.components.embedders import SentenceTransformersTextEmbedder
+from haystack.components.retrievers import InMemoryEmbeddingRetriever
+
+document_store = InMemoryDocumentStore()
+dataset = load_dataset("bilgeyucel/seven-wonders", split="train")
+embedder = SentenceTransformersDocumentEmbedder(
+    "sentence-transformers/all-MiniLM-L6-v2",
+)
+embedder.warm_up()
+output = embedder.run([Document(**ds) for ds in dataset])
+document_store.write_documents(output.get("documents"))
+
+template = """
+Given the following information, answer the question.
+
+Context:
+{% for document in documents %}
+    {{ document.content }}
+{% endfor %}
+
+Question: {{question}}
+Answer:
+"""
+prompt_builder = PromptBuilder(template=template)
+
+retriever = InMemoryEmbeddingRetriever(document_store=document_store, top_k=2)
+generator = OpenAIGenerator(model="gpt-3.5-turbo")
+query_embedder = SentenceTransformersTextEmbedder(
+    model="sentence-transformers/all-MiniLM-L6-v2",
+)
+
+basic_rag_pipeline = Pipeline()
+basic_rag_pipeline.add_component("text_embedder", query_embedder)
+basic_rag_pipeline.add_component("retriever", retriever)
+basic_rag_pipeline.add_component("prompt_builder", prompt_builder)
+basic_rag_pipeline.add_component("llm", generator)
+
+basic_rag_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
+basic_rag_pipeline.connect("retriever", "prompt_builder.documents")
+basic_rag_pipeline.connect("prompt_builder", "llm")
+
+query = "What does Rhodes Statue look like?"
+output = basic_rag_pipeline.run(
+    {"text_embedder": {"text": query}, "prompt_builder": {"question": query}},
+)
+```
+
+</details>
+
+## Documentation and Tutorials for Haystack 1.x
+
+You can access old tutorials in the [GitHub history](https://github.com/deepset-ai/haystack-tutorials/tree/5917718cbfbb61410aab4121ee6fe754040a5dc7) and download the Haystack 1.x documentation as a [ZIP file](https://core-engineering.s3.eu-central-1.amazonaws.com/public/docs/haystack-v1-docs.zip).
+
+The ZIP file contains documentation for all minor releases from version 1.0 to 1.26.
+
+To download documentation for a specific release, replace the version number in the following URL: `https://core-engineering.s3.eu-central-1.amazonaws.com/public/docs/v1.26.zip`.

docs-website/docs/overview/platform-components.mdx

@@ -0,0 +1,514 @@
+---
+title: "Haystack Enterprise Components"
+id: platform-components
+slug: "/platform-components"
+description: "A complete list of Haystack components available on the Haystack Enterprise Platform, grouped by integration partner."
+---
+
+# Haystack Enterprise Components
+
+The Haystack Enterprise Platform currently supports **224 components** and **55 integrations**. The following table lists them grouped by integration partner.
+
+## Core Components
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [Agent](https://docs.haystack.deepset.ai/docs/agent) | Component | ✅ Available |
+| [AnswerBuilder](https://docs.haystack.deepset.ai/docs/answerbuilder) | Builder | ✅ Available |
+| [AnswerJoiner](https://docs.haystack.deepset.ai/docs/answerjoiner) | Joiner | ✅ Available |
+| [AutoMergingRetriever](https://docs.haystack.deepset.ai/docs/automergingretriever) | Retriever | ✅ Available |
+| [AzureOCRDocumentConverter](https://docs.haystack.deepset.ai/docs/azureocrdocumentconverter) | Converter | ✅ Available |
+| [AzureOpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/azureopenaichatgenerator) | Generator | ✅ Available |
+| [AzureOpenAIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/azureopenaidocumentembedder) | Embedder | ✅ Available |
+| [AzureOpenAIResponsesChatGenerator](https://docs.haystack.deepset.ai/docs/azureopenairesponseschatgenerator) | Generator | ✅ Available |
+| [AzureOpenAITextEmbedder](https://docs.haystack.deepset.ai/docs/azureopenaitextembedder) | Embedder | ✅ Available |
+| [BranchJoiner](https://docs.haystack.deepset.ai/docs/branchjoiner) | Joiner | ✅ Available |
+| [CacheChecker](https://docs.haystack.deepset.ai/docs/cachechecker) | Component | ✅ Available |
+| [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder) | Builder | ✅ Available |
+| [ConditionalRouter](https://docs.haystack.deepset.ai/docs/conditionalrouter) | Router | ✅ Available |
+| [CSVDocumentCleaner](https://docs.haystack.deepset.ai/docs/csvdocumentcleaner) | Preprocessor | ✅ Available |
+| [CSVDocumentSplitter](https://docs.haystack.deepset.ai/docs/csvdocumentsplitter) | Preprocessor | ✅ Available |
+| [CSVToDocument](https://docs.haystack.deepset.ai/docs/csvtodocument) | Converter | ✅ Available |
+| [DocumentCleaner](https://docs.haystack.deepset.ai/docs/documentcleaner) | Preprocessor | ✅ Available |
+| [DocumentJoiner](https://docs.haystack.deepset.ai/docs/documentjoiner) | Joiner | ✅ Available |
+| [DocumentLanguageClassifier](https://docs.haystack.deepset.ai/docs/documentlanguageclassifier) | Classifier | ✅ Available |
+| [DocumentLengthRouter](https://docs.haystack.deepset.ai/docs/documentlengthrouter) | Router | ✅ Available |
+| [DocumentPreprocessor](https://docs.haystack.deepset.ai/docs/documentpreprocessor) | Preprocessor | ✅ Available |
+| [DocumentSplitter](https://docs.haystack.deepset.ai/docs/documentsplitter) | Preprocessor | ✅ Available |
+| [DocumentToImageContent](https://docs.haystack.deepset.ai/docs/documenttoimagecontent) | Converter | ✅ Available |
+| [DocumentTypeRouter](https://docs.haystack.deepset.ai/docs/documenttyperouter) | Router | ✅ Available |
+| [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter) | Writer | ✅ Available |
+| [DOCXToDocument](https://docs.haystack.deepset.ai/docs/docxtodocument) | Converter | ✅ Available |
+| [EmbeddingBasedDocumentSplitter](https://docs.haystack.deepset.ai/docs/embeddingbaseddocumentsplitter) | Preprocessor | ✅ Available |
+| [FallbackChatGenerator](https://docs.haystack.deepset.ai/docs/fallbackchatgenerator) | Generator | ✅ Available |
+| [FileToFileContent](https://docs.haystack.deepset.ai/docs/filetofilecontent) | Converter | ✅ Available |
+| [FileTypeRouter](https://docs.haystack.deepset.ai/docs/filetyperouter) | Router | ✅ Available |
+| [FilterRetriever](https://docs.haystack.deepset.ai/docs/filterretriever) | Retriever | ✅ Available |
+| [HierarchicalDocumentSplitter](https://docs.haystack.deepset.ai/docs/hierarchicaldocumentsplitter) | Preprocessor | ✅ Available |
+| [HTMLToDocument](https://docs.haystack.deepset.ai/docs/htmltodocument) | Converter | ✅ Available |
+| [HuggingFaceAPIChatGenerator](https://docs.haystack.deepset.ai/docs/huggingfaceapichatgenerator) | Generator | ✅ Available |
+| [HuggingFaceAPIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/huggingfaceapidocumentembedder) | Embedder | ✅ Available |
+| [HuggingFaceAPITextEmbedder](https://docs.haystack.deepset.ai/docs/huggingfaceapitextembedder) | Embedder | ✅ Available |
+| [HuggingFaceLocalChatGenerator](https://docs.haystack.deepset.ai/docs/huggingfacelocalchatgenerator) | Generator | ✅ Available |
+| [HuggingFaceTEIRanker](https://docs.haystack.deepset.ai/docs/huggingfaceteiranker) | Ranker | ✅ Available |
+| [ImageFileToDocument](https://docs.haystack.deepset.ai/docs/imagefiletodocument) | Converter | ✅ Available |
+| [ImageFileToImageContent](https://docs.haystack.deepset.ai/docs/imagefiletoimagecontent) | Converter | ✅ Available |
+| [JSONConverter](https://docs.haystack.deepset.ai/docs/jsonconverter) | Converter | ✅ Available |
+| [JsonSchemaValidator](https://docs.haystack.deepset.ai/docs/jsonschemavalidator) | Validator | ✅ Available |
+| [LinkContentFetcher](https://docs.haystack.deepset.ai/docs/linkcontentfetcher) | Fetcher | ✅ Available |
+| [ListJoiner](https://docs.haystack.deepset.ai/docs/listjoiner) | Joiner | ✅ Available |
+| [LLMDocumentContentExtractor](https://docs.haystack.deepset.ai/docs/llmdocumentcontentextractor) | Extractor | ✅ Available |
+| [LLMMessagesRouter](https://docs.haystack.deepset.ai/docs/llmmessagesrouter) | Router | ✅ Available |
+| [LLMMetadataExtractor](https://docs.haystack.deepset.ai/docs/llmmetadataextractor) | Extractor | ✅ Available |
+| [LLMRanker](https://docs.haystack.deepset.ai/docs/llmranker) | Ranker | ✅ Available |
+| [LostInTheMiddleRanker](https://docs.haystack.deepset.ai/docs/lostinthemiddleranker) | Ranker | ✅ Available |
+| [MarkdownHeaderSplitter](https://docs.haystack.deepset.ai/docs/markdownheadersplitter) | Preprocessor | ✅ Available |
+| [MarkdownToDocument](https://docs.haystack.deepset.ai/docs/markdowntodocument) | Converter | ✅ Available |
+| [MetadataRouter](https://docs.haystack.deepset.ai/docs/metadatarouter) | Router | ✅ Available |
+| [MetaFieldGroupingRanker](https://docs.haystack.deepset.ai/docs/metafieldgroupingranker) | Ranker | ✅ Available |
+| [MetaFieldRanker](https://docs.haystack.deepset.ai/docs/metafieldranker) | Ranker | ✅ Available |
+| [MSGToDocument](https://docs.haystack.deepset.ai/docs/msgtodocument) | Converter | ✅ Available |
+| [MultiFileConverter](https://docs.haystack.deepset.ai/docs/multifileconverter) | Converter | ✅ Available |
+| [MultiQueryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/multiqueryembeddingretriever) | Retriever | ✅ Available |
+| [MultiQueryTextRetriever](https://docs.haystack.deepset.ai/docs/multiquerytextretriever) | Retriever | ✅ Available |
+| [MultiRetriever](https://docs.haystack.deepset.ai/docs/multiretriever) | Retriever | ✅ Available |
+| [NamedEntityExtractor](https://docs.haystack.deepset.ai/docs/namedentityextractor) | Extractor | ✅ Available |
+| [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator) | Generator | ✅ Available |
+| [OpenAIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/openaidocumentembedder) | Embedder | ✅ Available |
+| [OpenAIResponsesChatGenerator](https://docs.haystack.deepset.ai/docs/openairesponseschatgenerator) | Generator | ✅ Available |
+| [OpenAITextEmbedder](https://docs.haystack.deepset.ai/docs/openaitextembedder) | Embedder | ✅ Available |
+| [OpenAPIConnector](https://docs.haystack.deepset.ai/docs/openapiconnector) | Connector | ✅ Available |
+| [OpenAPIServiceConnector](https://docs.haystack.deepset.ai/docs/openapiserviceconnector) | Connector | ✅ Available |
+| [OpenAPIServiceToFunctions](https://docs.haystack.deepset.ai/docs/openapiservicetofunctions) | Converter | ✅ Available |
+| [OutputAdapter](https://docs.haystack.deepset.ai/docs/outputadapter) | Converter | ✅ Available |
+| [PDFMinerToDocument](https://docs.haystack.deepset.ai/docs/pdfminertodocument) | Converter | ✅ Available |
+| [PDFToImageContent](https://docs.haystack.deepset.ai/docs/pdftoimagecontent) | Converter | ✅ Available |
+| [PPTXToDocument](https://docs.haystack.deepset.ai/docs/pptxtodocument) | Converter | ✅ Available |
+| [PromptBuilder](https://docs.haystack.deepset.ai/docs/promptbuilder) | Builder | ✅ Available |
+| [PyPDFToDocument](https://docs.haystack.deepset.ai/docs/pypdftodocument) | Converter | ✅ Available |
+| [PythonCodeSplitter](https://docs.haystack.deepset.ai/docs/pythoncodesplitter) | Preprocessor | ✅ Available |
+| [QueryExpander](https://docs.haystack.deepset.ai/docs/queryexpander) | Component | ✅ Available |
+| [RecursiveDocumentSplitter](https://docs.haystack.deepset.ai/docs/recursivesplitter) | Preprocessor | ✅ Available |
+| [RegexTextExtractor](https://docs.haystack.deepset.ai/docs/regextextextractor) | Extractor | ✅ Available |
+| [RemoteWhisperTranscriber](https://docs.haystack.deepset.ai/docs/remotewhispertranscriber) | Audio | ✅ Available |
+| [SearchApiWebSearch](https://docs.haystack.deepset.ai/docs/searchapiwebsearch) | Component | ✅ Available |
+| [SentenceTransformersDiversityRanker](https://docs.haystack.deepset.ai/docs/sentencetransformersdiversityranker) | Ranker | ✅ Available |
+| [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder) | Embedder | ✅ Available |
+| [SentenceTransformersDocumentImageEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentimageembedder) | Embedder | ✅ Available |
+| [SentenceTransformersSimilarityRanker](https://docs.haystack.deepset.ai/docs/sentencetransformerssimilarityranker) | Ranker | ✅ Available |
+| [SentenceTransformersSparseDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerssparsedocumentembedder) | Embedder | ✅ Available |
+| [SentenceTransformersSparseTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerssparsetextembedder) | Embedder | ✅ Available |
+| [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder) | Embedder | ✅ Available |
+| [SentenceWindowRetriever](https://docs.haystack.deepset.ai/docs/sentencewindowretriever) | Retriever | ✅ Available |
+| [SerperDevWebSearch](https://docs.haystack.deepset.ai/docs/serperdevwebsearch) | Component | ✅ Available |
+| [StringJoiner](https://docs.haystack.deepset.ai/docs/stringjoiner) | Joiner | ✅ Available |
+| [TextCleaner](https://docs.haystack.deepset.ai/docs/textcleaner) | Preprocessor | ✅ Available |
+| [TextEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/textembeddingretriever) | Retriever | ✅ Available |
+| [TextFileToDocument](https://docs.haystack.deepset.ai/docs/textfiletodocument) | Converter | ✅ Available |
+| [TextLanguageRouter](https://docs.haystack.deepset.ai/docs/textlanguagerouter) | Router | ✅ Available |
+| [TikaDocumentConverter](https://docs.haystack.deepset.ai/docs/tikadocumentconverter) | Converter | ✅ Available |
+| [ToolInvoker](https://docs.haystack.deepset.ai/docs/toolinvoker) | Component | ✅ Available |
+| [TopPSampler](https://docs.haystack.deepset.ai/docs/toppsampler) | Sampler | ✅ Available |
+| [TransformersExtractiveReader](https://docs.haystack.deepset.ai/docs/transformersextractivereader) | Reader | ✅ Available |
+| [TransformersSimilarityRanker](https://docs.haystack.deepset.ai/docs/transformerssimilarityranker) | Ranker | ✅ Available |
+| [TransformersTextRouter](https://docs.haystack.deepset.ai/docs/transformerstextrouter) | Router | ✅ Available |
+| [TransformersZeroShotDocumentClassifier](https://docs.haystack.deepset.ai/docs/transformerszeroshotdocumentclassifier) | Classifier | ✅ Available |
+| [TransformersZeroShotTextRouter](https://docs.haystack.deepset.ai/docs/transformerszeroshottextrouter) | Router | ✅ Available |
+| [XLSXToDocument](https://docs.haystack.deepset.ai/docs/xlsxtodocument) | Converter | ✅ Available |
+
+## AIML API
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [AIMLAPIChatGenerator](https://docs.haystack.deepset.ai/docs/aimllapichatgenerator) | Generator | ✅ Available |
+
+## AlloyDB
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [AlloyDBEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/alloydbembeddingretriever) | Retriever | ✅ Available |
+| [AlloyDBKeywordRetriever](https://docs.haystack.deepset.ai/docs/alloydbkeywordretriever) | Retriever | ✅ Available |
+
+## Amazon Bedrock
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [AmazonBedrockChatGenerator](https://docs.haystack.deepset.ai/docs/amazonbedrockchatgenerator) | Generator | ✅ Available |
+| [AmazonBedrockDocumentEmbedder](https://docs.haystack.deepset.ai/docs/amazonbedrockdocumentembedder) | Embedder | ✅ Available |
+| [AmazonBedrockDocumentImageEmbedder](https://docs.haystack.deepset.ai/docs/amazonbedrockdocumentimageembedder) | Embedder | ✅ Available |
+| [AmazonBedrockRanker](https://docs.haystack.deepset.ai/docs/amazonbedrockranker) | Ranker | ✅ Available |
+| [AmazonBedrockTextEmbedder](https://docs.haystack.deepset.ai/docs/amazonbedrocktextembedder) | Embedder | ✅ Available |
+
+## Amazon S3
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [S3Downloader](https://docs.haystack.deepset.ai/docs/s3downloader) | Component | ✅ Available |
+
+## Amazon Textract
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [AmazonTextractConverter](https://docs.haystack.deepset.ai/docs/amazontextractconverter) | Converter | ✅ Available |
+
+## Anthropic
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [AnthropicChatGenerator](https://docs.haystack.deepset.ai/docs/anthropicchatgenerator) | Generator | ✅ Available |
+| [AnthropicFoundryChatGenerator](https://docs.haystack.deepset.ai/docs/anthropicfoundrychatgenerator) | Generator | ✅ Available |
+| [AnthropicVertexChatGenerator](https://docs.haystack.deepset.ai/docs/anthropicvertexchatgenerator) | Generator | ✅ Available |
+
+## ArcadeDB
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [ArcadeDBEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/arcadedbembeddingretriever) | Retriever | ✅ Available |
+
+## Astra
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [AstraEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/astraretriever) | Retriever | ✅ Available |
+
+## Azure AI Search
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [AzureAISearchBM25Retriever](https://docs.haystack.deepset.ai/docs/azureaisearchbm25retriever) | Retriever | ✅ Available |
+| [AzureAISearchEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/azureaisearchembeddingretriever) | Retriever | ✅ Available |
+| [AzureAISearchHybridRetriever](https://docs.haystack.deepset.ai/docs/azureaisearchhybridretriever) | Retriever | ✅ Available |
+
+## Azure Document Intelligence
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [AzureDocumentIntelligenceConverter](https://docs.haystack.deepset.ai/docs/azuredocumentintelligenceconverter) | Converter | ✅ Available |
+
+## Brave
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [BraveWebSearch](https://docs.haystack.deepset.ai/docs/bravewebsearch) | Component | ✅ Available |
+
+## Chroma
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [ChromaEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/chromaembeddingretriever) | Retriever | ✅ Available |
+| [ChromaQueryTextRetriever](https://docs.haystack.deepset.ai/docs/chromaqueryretriever) | Retriever | ✅ Available |
+
+## Cohere
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [CohereChatGenerator](https://docs.haystack.deepset.ai/docs/coherechatgenerator) | Generator | ✅ Available |
+| [CohereDocumentEmbedder](https://docs.haystack.deepset.ai/docs/coheredocumentembedder) | Embedder | ✅ Available |
+| [CohereRanker](https://docs.haystack.deepset.ai/docs/cohereranker) | Ranker | ✅ Available |
+| [CohereTextEmbedder](https://docs.haystack.deepset.ai/docs/coheretextembedder) | Embedder | ✅ Available |
+
+## Docling
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [DoclingConverter](https://docs.haystack.deepset.ai/docs/doclingconverter) | Converter | ✅ Available |
+| [DoclingServeConverter](https://docs.haystack.deepset.ai/docs/doclingserveconverter) | Converter | ✅ Available |
+
+## Elasticsearch
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [ElasticsearchBM25Retriever](https://docs.haystack.deepset.ai/docs/elasticsearchbm25retriever) | Retriever | ✅ Available |
+| [ElasticsearchEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/elasticsearchembeddingretriever) | Retriever | ✅ Available |
+| [ElasticsearchHybridRetriever](https://docs.haystack.deepset.ai/docs/elasticsearchhybridretriever) | Retriever | ✅ Available |
+| [ElasticsearchSQLRetriever](https://docs.haystack.deepset.ai/docs/elasticsearchsqlretriever) | Retriever | ✅ Available |
+
+## FalkorDB
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [FalkorDBCypherRetriever](https://docs.haystack.deepset.ai/docs/falkordbcypherretriever) | Retriever | ✅ Available |
+| [FalkorDBEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/falkordbembeddingretriever) | Retriever | ✅ Available |
+
+## FastEmbed
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [FastembedDocumentEmbedder](https://docs.haystack.deepset.ai/docs/fastembeddocumentembedder) | Embedder | ✅ Available |
+| [FastembedRanker](https://docs.haystack.deepset.ai/docs/fastembedranker) | Ranker | ✅ Available |
+| [FastembedSparseDocumentEmbedder](https://docs.haystack.deepset.ai/docs/fastembedsparsedocumentembedder) | Embedder | ✅ Available |
+| [FastembedSparseTextEmbedder](https://docs.haystack.deepset.ai/docs/fastembedsparsetextembedder) | Embedder | ✅ Available |
+| [FastembedTextEmbedder](https://docs.haystack.deepset.ai/docs/fastembedtextembedder) | Embedder | ✅ Available |
+
+## Firecrawl
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [FirecrawlCrawler](https://docs.haystack.deepset.ai/docs/firecrawlcrawler) | Fetcher | ✅ Available |
+| [FirecrawlWebSearch](https://docs.haystack.deepset.ai/docs/firecrawlwebsearch) | Component | ✅ Available |
+
+## GitHub
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [GitHubFileEditor](https://docs.haystack.deepset.ai/docs/githubfileeditor) | Connector | ✅ Available |
+| [GitHubIssueCommenter](https://docs.haystack.deepset.ai/docs/githubissuecommenter) | Connector | ✅ Available |
+| [GitHubIssueViewer](https://docs.haystack.deepset.ai/docs/githubissueviewer) | Connector | ✅ Available |
+| [GitHubPRCreator](https://docs.haystack.deepset.ai/docs/githubprcreator) | Connector | ✅ Available |
+| [GitHubRepoForker](https://docs.haystack.deepset.ai/docs/githubrepoforker) | Connector | ✅ Available |
+| [GitHubRepoViewer](https://docs.haystack.deepset.ai/docs/githubrepoviewer) | Connector | ✅ Available |
+
+## Google Generative AI
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [GoogleGenAIChatGenerator](https://docs.haystack.deepset.ai/docs/googlegenaichatgenerator) | Generator | ✅ Available |
+| [GoogleGenAIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/googlegenaidocumentembedder) | Embedder | ✅ Available |
+| [GoogleGenAITextEmbedder](https://docs.haystack.deepset.ai/docs/googlegenaitextembedder) | Embedder | ✅ Available |
+
+## Hugging Face API
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [HuggingFaceAPIChatGenerator](https://docs.haystack.deepset.ai/docs/huggingfaceapichatgenerator) | Generator | ✅ Available |
+| [HuggingFaceAPIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/huggingfaceapidocumentembedder) | Embedder | ✅ Available |
+| [HuggingFaceAPITextEmbedder](https://docs.haystack.deepset.ai/docs/huggingfaceapitextembedder) | Embedder | ✅ Available |
+| [HuggingFaceTEIRanker](https://docs.haystack.deepset.ai/docs/huggingfaceteiranker) | Ranker | ✅ Available |
+
+## Jina AI
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [JinaDocumentEmbedder](https://docs.haystack.deepset.ai/docs/jinadocumentembedder) | Embedder | ✅ Available |
+| [JinaRanker](https://docs.haystack.deepset.ai/docs/jinaranker) | Ranker | ✅ Available |
+| [JinaReaderConnector](https://docs.haystack.deepset.ai/docs/jinareaderconnector) | Connector | ✅ Available |
+| [JinaTextEmbedder](https://docs.haystack.deepset.ai/docs/jinatextembedder) | Embedder | ✅ Available |
+
+## Langfuse
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [LangfuseConnector](https://docs.haystack.deepset.ai/docs/langfuseconnector) | Connector | ✅ Available |
+
+## Lara
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [LaraDocumentTranslator](https://docs.haystack.deepset.ai/docs/laradocumenttranslator) | Component | ✅ Available |
+
+## LiteLLM
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [LiteLLMChatGenerator](https://docs.haystack.deepset.ai/docs/litellmchatgenerator) | Generator | ✅ Available |
+
+## Llama Stack
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [LlamaStackChatGenerator](https://docs.haystack.deepset.ai/docs/llamastackchatgenerator) | Generator | ✅ Available |
+
+## Llama.cpp
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [LlamaCppChatGenerator](https://docs.haystack.deepset.ai/docs/llamacppchatgenerator) | Generator | ✅ Available |
+
+## MarkItDown
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [MarkItDownConverter](https://docs.haystack.deepset.ai/docs/markitdownconverter) | Converter | ✅ Available |
+
+## Mem0
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [Mem0MemoryRetriever](https://docs.haystack.deepset.ai/docs/mem0memoryretriever) | Retriever | ✅ Available |
+| [Mem0MemoryWriter](https://docs.haystack.deepset.ai/docs/mem0memorywriter) | Writer | ✅ Available |
+
+## Meta Llama
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [MetaLlamaChatGenerator](https://docs.haystack.deepset.ai/docs/metallamachatgenerator) | Generator | ✅ Available |
+
+## Mistral
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [MistralChatGenerator](https://docs.haystack.deepset.ai/docs/mistralchatgenerator) | Generator | ✅ Available |
+| [MistralDocumentEmbedder](https://docs.haystack.deepset.ai/docs/mistraldocumentembedder) | Embedder | ✅ Available |
+| [MistralOCRDocumentConverter](https://docs.haystack.deepset.ai/docs/mistralocrdocumentconverter) | Converter | ✅ Available |
+| [MistralTextEmbedder](https://docs.haystack.deepset.ai/docs/mistraltextembedder) | Embedder | ✅ Available |
+
+## MongoDB Atlas
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [MongoDBAtlasEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/mongodbatlasembeddingretriever) | Retriever | ✅ Available |
+| [MongoDBAtlasFullTextRetriever](https://docs.haystack.deepset.ai/docs/mongodbatlasfulltextretriever) | Retriever | ✅ Available |
+
+## NVIDIA
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [NvidiaChatGenerator](https://docs.haystack.deepset.ai/docs/nvidiachatgenerator) | Generator | ✅ Available |
+| [NvidiaDocumentEmbedder](https://docs.haystack.deepset.ai/docs/nvidiadocumentembedder) | Embedder | ✅ Available |
+| [NvidiaRanker](https://docs.haystack.deepset.ai/docs/nvidiaranker) | Ranker | ✅ Available |
+| [NvidiaTextEmbedder](https://docs.haystack.deepset.ai/docs/nvidiatextembedder) | Embedder | ✅ Available |
+
+## Ollama
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [OllamaChatGenerator](https://docs.haystack.deepset.ai/docs/ollamachatgenerator) | Generator | ✅ Available |
+| [OllamaDocumentEmbedder](https://docs.haystack.deepset.ai/docs/ollamadocumentembedder) | Embedder | ✅ Available |
+| [OllamaTextEmbedder](https://docs.haystack.deepset.ai/docs/ollamatextembedder) | Embedder | ✅ Available |
+
+## OpenRouter
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [OpenRouterChatGenerator](https://docs.haystack.deepset.ai/docs/openrouterchatgenerator) | Generator | ✅ Available |
+
+## OpenSearch
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [OpenSearchBM25Retriever](https://docs.haystack.deepset.ai/docs/opensearchbm25retriever) | Retriever | ✅ Available |
+| [OpenSearchEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/opensearchembeddingretriever) | Retriever | ✅ Available |
+| [OpenSearchHybridRetriever](https://docs.haystack.deepset.ai/docs/opensearchhybridretriever) | Retriever | ✅ Available |
+| [OpenSearchMetadataRetriever](https://docs.haystack.deepset.ai/docs/opensearchmetadataretriever) | Retriever | ✅ Available |
+| [OpenSearchSQLRetriever](https://docs.haystack.deepset.ai/docs/opensearchsqlretriever) | Retriever | ✅ Available |
+
+## Oracle
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [OracleEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/oracleembeddingretriever) | Retriever | ✅ Available |
+| [OracleKeywordRetriever](https://docs.haystack.deepset.ai/docs/oraclekeywordretriever) | Retriever | ✅ Available |
+
+## Perplexity
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [PerplexityChatGenerator](https://docs.haystack.deepset.ai/docs/perplexitychatgenerator) | Generator | ✅ Available |
+| [PerplexityDocumentEmbedder](https://docs.haystack.deepset.ai/docs/perplexitydocumentembedder) | Embedder | ✅ Available |
+| [PerplexityTextEmbedder](https://docs.haystack.deepset.ai/docs/perplexitytextembedder) | Embedder | ✅ Available |
+| [PerplexityWebSearch](https://docs.haystack.deepset.ai/docs/perplexitywebsearch) | Component | ✅ Available |
+
+## pgvector
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [PgvectorEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/pgvectorembeddingretriever) | Retriever | ✅ Available |
+| [PgvectorKeywordRetriever](https://docs.haystack.deepset.ai/docs/pgvectorkeywordretriever) | Retriever | ✅ Available |
+
+## Pinecone
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [PineconeEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/pineconedenseretriever) | Retriever | ✅ Available |
+
+## Presidio
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [PresidioDocumentCleaner](https://docs.haystack.deepset.ai/docs/presidiodocumentcleaner) | Preprocessor | ✅ Available |
+| [PresidioEntityExtractor](https://docs.haystack.deepset.ai/docs/presidioentityextractor) | Extractor | ✅ Available |
+| [PresidioTextCleaner](https://docs.haystack.deepset.ai/docs/presidiotextcleaner) | Preprocessor | ✅ Available |
+
+## Pyversity
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [PyversityRanker](https://docs.haystack.deepset.ai/docs/pyversityranker) | Ranker | ✅ Available |
+
+## Qdrant
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [QdrantEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/qdrantembeddingretriever) | Retriever | ✅ Available |
+| [QdrantHybridRetriever](https://docs.haystack.deepset.ai/docs/qdranthybridretriever) | Retriever | ✅ Available |
+| [QdrantSparseEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/qdrantsparseembeddingretriever) | Retriever | ✅ Available |
+
+## Snowflake
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [SnowflakeTableRetriever](https://docs.haystack.deepset.ai/docs/snowflaketableretriever) | Retriever | ✅ Available |
+
+## SQLAlchemy
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| SQLAlchemyTableRetriever | Retriever | ✅ Available |
+
+## STACKIT
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [STACKITChatGenerator](https://docs.haystack.deepset.ai/docs/stackitchatgenerator) | Generator | ✅ Available |
+| [STACKITDocumentEmbedder](https://docs.haystack.deepset.ai/docs/stackitdocumentembedder) | Embedder | ✅ Available |
+| [STACKITTextEmbedder](https://docs.haystack.deepset.ai/docs/stackittextembedder) | Embedder | ✅ Available |
+
+## Supabase
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [SupabasePgvectorEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/supabasepgvectorembeddingretriever) | Retriever | ✅ Available |
+| [SupabasePgvectorKeywordRetriever](https://docs.haystack.deepset.ai/docs/supabasepgvectorkeywordretriever) | Retriever | ✅ Available |
+
+## Tavily
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [TavilyWebSearch](https://docs.haystack.deepset.ai/docs/tavilywebsearch) | Component | ✅ Available |
+
+## TogetherAI
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [TogetherAIChatGenerator](https://docs.haystack.deepset.ai/docs/togetheraichatgenerator) | Generator | ✅ Available |
+
+## Unstructured
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [UnstructuredFileConverter](https://docs.haystack.deepset.ai/docs/unstructuredfileconverter) | Converter | ✅ Available |
+
+## Valkey
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [ValkeyEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/valkeyembeddingretriever) | Retriever | ✅ Available |
+
+## Vespa
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [VespaEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/vespaembeddingretriever) | Retriever | ✅ Available |
+| [VespaKeywordRetriever](https://docs.haystack.deepset.ai/docs/vespakeywordretriever) | Retriever | ✅ Available |
+
+## vLLM
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [VLLMChatGenerator](https://docs.haystack.deepset.ai/docs/vllmchatgenerator) | Generator | ✅ Available |
+| [VLLMDocumentEmbedder](https://docs.haystack.deepset.ai/docs/vllmdocumentembedder) | Embedder | ✅ Available |
+| [VLLMRanker](https://docs.haystack.deepset.ai/docs/vllmranker) | Ranker | ✅ Available |
+| [VLLMTextEmbedder](https://docs.haystack.deepset.ai/docs/vllmtextembedder) | Embedder | ✅ Available |
+
+## Weaviate
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [WeaviateBM25Retriever](https://docs.haystack.deepset.ai/docs/weaviatebm25retriever) | Retriever | ✅ Available |
+| [WeaviateEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/weaviateembeddingretriever) | Retriever | ✅ Available |
+
+## Weights & Biases (Weave)
+
+| Component | Type | Haystack Enterprise Platform |
+|-----------|------|------------------------------|
+| [WeaveConnector](https://docs.haystack.deepset.ai/docs/weaveconnector) | Connector | ✅ Available |

docs-website/docs/overview/telemetry.mdx

@@ -0,0 +1,76 @@
+---
+title: "Telemetry"
+id: telemetry
+slug: "/telemetry"
+description: "Haystack relies on anonymous usage statistics to continuously improve. That's why some basic information, like the type of Document Store used, is shared automatically."
+---
+
+# Telemetry
+
+Haystack relies on anonymous usage statistics to continuously improve. That's why some basic information, like the type of Document Store used, is shared automatically.
+
+## What Information Is Shared?
+
+Telemetry in Haystack comprises anonymous usage statistics of base components, such as `DocumentStore`, `Retriever`, `Reader`, or any other pipeline component. We receive an event every time these components are initialized. This way, we know which components are most relevant to our community. For the same reason, an event is also sent when one of the tutorials is executed.
+
+Each event contains an anonymous, randomly generated user ID (`uuid`)  and a collection of properties about your execution environment. They **never** contain properties that can be used to identify you, such as:
+
+- IP addresses
+- Hostnames
+- File paths
+- Queries
+- Document contents
+
+By taking the above steps, we ensure that only anonymized data is transmitted to our telemetry server.
+
+Here is an exemplary event that is sent when tutorial 1 is executed by running `Tutorial1_Basic_QA_Pipeline.py`:
+
+```json
+{
+    "event": "tutorial 1 executed",
+    "distinct_id": "9baab867-3bc8-438c-9974-a192c9d53cd1",
+    "properties": {
+        "os_family": "Darwin",
+        "os_machine": "arm64",
+        "os_version": "21.3.0",
+        "haystack_version": "1.0.0",
+        "python_version": "3.9.6",
+        "torch_version": "1.9.0",
+        "transformers_version": "4.13.0",
+        "execution_env": "script",
+        "n_gpu": 0,
+    },
+}
+```
+
+Our telemetry code can be directly inspected on [GitHub](https://github.com/deepset-ai/haystack/blob/5d66d040cc303ab49225587cd61290f1987a5d1f/haystack/telemetry/_telemetry.py).
+
+## How Does Telemetry Help?
+
+Thanks to telemetry, we can understand the needs of the community: _"What pipeline nodes are most popular?", "Should we focus on supporting one specific Document Store?", "How many people use Haystack on Windows?"_ are some of the questions telemetry helps us answer. Metadata about the operating system and installed dependencies allows us to quickly identify and address issues caused by specific setups.
+
+In short, by sharing this information, you enable us to continuously improve Haystack for everyone.
+
+## How Can I Opt Out?
+
+You can disable telemetry with one of the following methods:
+
+### Through an Environment Variable
+
+You can disable telemetry by setting the environment variable `HAYSTACK_TELEMETRY_ENABLED` to `"False"` .
+
+### Using a Bash Shell
+
+If you are using a bash shell, add the following line to the file `~/.bashrc` to disable telemetry: `export HAYSTACK_TELEMETRY_ENABLED=False`.
+
+### Using zsh
+
+If you are using zsh as your shell, for example, on macOS, add  the following line to the file `~/.zshrc`: `export HAYSTACK_TELEMETRY_ENABLED=False`.
+
+### On Windows
+
+To disable telemetry on Windows, set a user-level environment variable by running this command in the standard command prompt: `setx HAYSTACK_TELEMETRY_ENABLED "False"`.
+
+Alternatively, run the following command in Windows PowerShell: `[Environment]::SetEnvironmentVariable("HAYSTACK_TELEMETRY_ENABLED","False","User")`.
+
+You might need to restart the operating system for the command to take effect.

docs-website/docs/pipeline-components/agents-1/agent.mdx

@@ -0,0 +1,492 @@
+---
+title: "Agent"
+id: agent
+slug: "/agent"
+description: "The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`."
+---
+
+# Agent
+
+The `Agent` component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively. It can execute external tools, manage state across multiple LLM calls, and stop execution based on configurable `exit_conditions`.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or user input                     |
+| **Mandatory init variables**           | `chat_generator`: An instance of a Chat Generator that supports tools                  |
+| **Mandatory run variables**            | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)s                              |
+| **Output variables**                   | `messages`: Chat history with tool and model responses                                 |
+| **API reference**                      | [Agents](/reference/agents-api)                                                               |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack/blob/main/haystack/components/agents/agent.py |
+| **Package name** | `haystack-ai` |
+
+</div>
+
+## Overview
+
+The `Agent` component is a loop-based system that uses a chat-based large language model (LLM) and external tools to solve complex user queries.
+It works iteratively—calling tools, updating state, and generating prompts—until one of the configurable `exit_conditions` is met.
+
+It can:
+
+- Dynamically select tools based on user input,
+- Maintain and validate runtime state using a schema,
+- Stream token-level outputs from the LLM.
+
+The `Agent` returns a dictionary containing:
+
+- `messages`: the full conversation history,
+- `last_message`: the final `ChatMessage` from the agent,
+- `step_count`: the number of steps the agent ran,
+- `token_usage`: aggregated token usage summed across every LLM call in the run,
+- `tool_call_counts`: how many times each tool was invoked, keyed by tool name,
+- Additional dynamic keys based on `state_schema`.
+
+### Run Metadata
+
+The `step_count`, `token_usage`, and `tool_call_counts` outputs are populated automatically during a run. They are added to the agent's `state_schema` behind the scenes, so tools registered with `inputs_from_state` can read them mid-run. They are outputs only — they cannot be passed as inputs to `run()` or `run_async()`, and using them as keys in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for details.
+
+```python
+response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")])
+
+print(response["step_count"])  # 2
+print(
+    response["token_usage"],
+)  # {"prompt_tokens": 512, "completion_tokens": 86, ...}
+print(response["tool_call_counts"])  # {"calculator": 1}
+```
+
+## Parameters
+
+`chat_generator` is the only mandatory parameter — an instance of a Chat Generator that supports tools. All other parameters are optional.
+
+- `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx). Tool names must be unique; duplicate names are detected at the start of each agent step, before the chat generator is called.
+- `system_prompt`: A plain string or Jinja2 template used as the system message for every run. If the template contains Jinja2 variables, those variables become additional inputs to `run()`.
+- `user_prompt`: A Jinja2 template appended to the user-provided messages on each run. Template variables become additional inputs to `run()`. Use `required_variables` to enforce which variables must be provided.
+- `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`. Exit conditions are evaluated at runtime rather than validated at initialization, so a condition can name a tool that is only loaded later — for example, a tool passed at runtime via `run(tools=...)` or one discovered by a [`SearchableToolset`](../../tools/searchabletoolset.mdx).
+- `state_schema`: Defines the agent's runtime state — a dict mapping key names to type configs (e.g. `{“docs”: {“type”: list[Document]}}`). Tools can read from and write to state keys via `inputs_from_state` and `outputs_to_state`. See [State](./state.mdx) for full details.
+- `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output.
+- `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`.
+- `raise_on_tool_invocation_failure`: If `True`, raises an exception when a tool call fails. If `False` (default), the error is passed back to the LLM as a message so it can recover.
+- `hooks`: A dict mapping a hook point (`"before_llm"`, `"before_tool"`, `"after_tool"`, `"on_exit"`) to a list of hooks the agent runs at that point. Hooks receive the live `State` and influence the run by mutating it — for example, to build run-time context or require human confirmation of tool calls. See [Hooks](./hooks.mdx) and [Human in the Loop](./human-in-the-loop.mdx).
+- `tool_concurrency_limit`: Maximum number of tool calls to execute at the same time. Defaults to `4`; set to `1` to disable parallel tool execution.
+- `tool_streaming_callback_passthrough`: If `True`, passes the streaming callback to tools that accept it.
+
+### Runtime overrides
+
+`run()` also accepts parameters that override the init-time configuration for a single call:
+
+- `tools`: Pass a list of `Tool`/`Toolset` objects, or a list of tool name strings to select a subset of the agent's configured tools for this run.
+- `generation_kwargs`: Additional keyword arguments forwarded to the LLM, overriding any set at init time (e.g. `{“temperature”: 0.2}`).
+- `hook_context`: A dict of request-scoped resources made available to [hooks](./hooks.mdx) via `state.data["hook_context"]` — for example, a user ID or a WebSocket connection.
+
+:::info
+For the full parameter reference, see the [Agents API Documentation](/reference/agents-api).
+:::
+
+## Usage
+
+### On its own
+
+```python
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.tools import tool
+from haystack.components.agents import Agent
+from typing import Annotated
+
+
+@tool(outputs_to_state={"calc_result": {"source": "result"}})
+def calculator(
+    expression: Annotated[str, "Math expression to evaluate, e.g. '7 * (4 + 2)'"],
+) -> dict:
+    """Evaluate basic math expressions."""
+    try:
+        result = eval(expression, {"__builtins__": {}})
+        return {"result": result}
+    except Exception as e:
+        return {"error": str(e)}
+
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[calculator],
+    system_prompt="You are a helpful assistant. Always use the calculator tool to evaluate math expressions.",
+    state_schema={"calc_result": {"type": int}},
+)
+
+response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")])
+
+print(response["last_message"].text)
+print("Calc Result:", response.get("calc_result"))
+```
+
+### In a pipeline
+
+The example pipeline below creates a database assistant using `OpenAIChatGenerator`, `LinkContentFetcher`, and custom database tool.
+It reads the given URL and processes the page content, then builds a prompt for the AI.
+The assistant uses this information to write people's names and titles from the given page to the database.
+
+```python
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
+from haystack.components.converters.html import HTMLToDocument
+from haystack.components.fetchers.link_content import LinkContentFetcher
+from haystack import Document, Pipeline
+from haystack.dataclasses import ChatMessage
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.tools import tool
+from typing import Annotated, Optional
+
+document_store = InMemoryDocumentStore()  # create a document store or an SQL database
+
+
+@tool
+def add_database_tool(
+    name: Annotated[str, "First name of the person"],
+    surname: Annotated[str, "Last name of the person"],
+    job_title: Annotated[Optional[str], "Job title or role of the person"] = None,
+    other: Annotated[Optional[str], "Any other relevant information"] = None,
+) -> str:
+    """Add a person to the database with information about them."""
+    document_store.write_documents(
+        [
+            Document(
+                content=name + " " + surname + " " + (job_title or ""),
+                meta={"other": other},
+            ),
+        ],
+    )
+    # Returning a confirmation lets the agent know the tool call succeeded
+    return f"Successfully added {name} {surname} to the database."
+
+
+database_assistant = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[add_database_tool],
+    system_prompt="""
+    You are a database assistant.
+    Your task is to extract the names of people mentioned in the given context and add them to a knowledge base,
+    along with additional relevant information about them that can be extracted from the context.
+    Do not use your own knowledge, stay grounded to the given context.
+    Do not ask the user for confirmation.
+    Instead, automatically update the knowledge base and return a brief summary of the people added,
+    including the information stored for each.
+    """,
+)
+
+extraction_agent = Pipeline()
+extraction_agent.add_component("fetcher", LinkContentFetcher())
+extraction_agent.add_component("converter", HTMLToDocument())
+extraction_agent.add_component(
+    "builder",
+    ChatPromptBuilder(
+        template=[
+            ChatMessage.from_user("""
+    {% for doc in docs %}
+    {{ doc.content|default|truncate(25000) }}
+    {% endfor %}
+    """),
+        ],
+        required_variables=["docs"],
+    ),
+)
+
+extraction_agent.add_component("database_agent", database_assistant)
+extraction_agent.connect("fetcher.streams", "converter.sources")
+extraction_agent.connect("converter.documents", "builder.docs")
+extraction_agent.connect("builder", "database_agent")
+
+agent_output = extraction_agent.run(
+    {
+        "fetcher": {
+            "urls": ["https://github.com/deepset-ai/haystack/releases/tag/v2.27.0"],
+        },
+    },
+)
+
+print(agent_output["database_agent"]["last_message"].text)
+
+# Inspect what was written to the document store
+written_docs = document_store.filter_documents()
+print(f"\n{len(written_docs)} people added to the database:")
+for doc in written_docs:
+    print(f"  - {doc.content}")
+```
+
+### In YAML
+The example pipeline below fetches a webpage, converts its HTML to text, and builds a chat prompt combining the page content with a user query.
+The `Agent` then answers the question based on the provided content and can use its web search tool to find additional information if needed.
+
+<details>
+<summary>View YAML</summary>
+
+```yaml
+components:
+  agent:
+    init_parameters:
+      chat_generator:
+        init_parameters:
+          api_base_url: null
+          api_key:
+            env_vars:
+            - OPENAI_API_KEY
+            strict: true
+            type: env_var
+          generation_kwargs: {}
+          http_client_kwargs: null
+          max_retries: null
+          model: gpt-5.4-nano
+          organization: null
+          streaming_callback: null
+          timeout: null
+          tools: null
+          tools_strict: false
+        type: haystack.components.generators.chat.openai.OpenAIChatGenerator
+      exit_conditions:
+      - text
+      hooks: null
+      max_agent_steps: 5
+      raise_on_tool_invocation_failure: false
+      required_variables: null
+      state_schema: {}
+      streaming_callback: null
+      system_prompt: You are a helpful assistant. Use the web search tool to find
+        information when needed.
+      tool_concurrency_limit: 4
+      tool_streaming_callback_passthrough: false
+      tools:
+      - data:
+          component:
+            init_parameters:
+              allowed_domains: null
+              api_key:
+                env_vars:
+                - SERPERDEV_API_KEY
+                strict: true
+                type: env_var
+              exclude_subdomains: false
+              search_params: {}
+              top_k: 3
+            type: haystack_integrations.components.websearch.serperdev.websearch.SerperDevWebSearch
+          description: Search the web for current information on any topic
+          inputs_from_state: null
+          name: web_search
+          outputs_to_state: null
+          outputs_to_string: null
+          parameters: null
+        type: haystack.tools.component_tool.ComponentTool
+      user_prompt: null
+    type: haystack.components.agents.agent.Agent
+  converter:
+    init_parameters:
+      extraction_kwargs: {}
+      store_full_path: false
+    type: haystack.components.converters.html.HTMLToDocument
+  fetcher:
+    init_parameters:
+      client_kwargs:
+        follow_redirects: true
+        timeout: 3
+      http2: false
+      raise_on_failure: true
+      request_headers: {}
+      retry_attempts: 2
+      timeout: 3
+      user_agents:
+      - haystack/LinkContentFetcher/2.27.0rc0
+    type: haystack.components.fetchers.link_content.LinkContentFetcher
+  prompt_builder:
+    init_parameters:
+      required_variables:
+      - docs
+      - query
+      template:
+      - content:
+        - text: 'Based on the following content:
+
+            {% for doc in docs %}
+
+            {{ doc.content }}
+
+            {% endfor %}
+
+            Answer this question: {{ query }}'
+        meta: {}
+        name: null
+        role: user
+      variables: null
+    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
+connection_type_validation: true
+connections:
+- receiver: converter.sources
+  sender: fetcher.streams
+- receiver: prompt_builder.docs
+  sender: converter.documents
+- receiver: agent.messages
+  sender: prompt_builder.prompt
+max_runs_per_component: 100
+metadata: {}
+```
+
+</details>
+
+## Streaming
+
+You can stream output as it's generated. Pass a callback to `streaming_callback`.
+Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).
+
+```python
+from haystack.components.generators.utils import print_streaming_chunk
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[...],
+    system_prompt="...",
+    streaming_callback=print_streaming_chunk,
+)
+```
+
+See our [Streaming Support](../generators/guides-to-generators/choosing-the-right-generator.mdx#streaming-support) docs to learn more how `StreamingChunk` works and how to write a custom callback.
+
+Give preference to `print_streaming_chunk` by default.
+Write a custom callback only if you need a specific transport (for example, SSE/WebSocket) or custom UI formatting.
+
+## Multimodal Inputs
+
+Agents support multimodal inputs when paired with a vision-capable model such as `gpt-5` (OpenAI) or `gemini-2.5-flash` (Google).
+Pass images alongside text by including `ImageContent` objects in the `content_parts` of a `ChatMessage`:
+
+```python
+from haystack.dataclasses import ChatMessage, ImageContent
+
+image = ImageContent.from_url("https://example.com/chart.png")
+result = agent.run(
+    messages=[
+        ChatMessage.from_user(content_parts=["What does this chart show?", image]),
+    ],
+)
+```
+
+Tools can also return `ImageContent` directly, letting the agent fetch and reason about images dynamically during its loop.
+Two things are required: set `outputs_to_string={"raw_result": True}` so the `ToolInvoker` skips string conversion, and return a `list[ImageContent]` (the tool result type is `str | Sequence[TextContent | ImageContent]`).
+
+The standard Chat Completions API doesn't support images in tool results — use `OpenAIResponsesChatGenerator` (OpenAI's Responses API) instead:
+
+```python
+from typing import Annotated
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIResponsesChatGenerator
+from haystack.dataclasses import ChatMessage, ImageContent
+from haystack.tools import tool
+
+
+@tool(outputs_to_string={"raw_result": True})
+def fetch_image(
+    url: Annotated[str, "URL of the image to fetch and analyze"],
+) -> list[ImageContent]:
+    """Fetch an image from a URL so the agent can analyze its contents."""
+    return [ImageContent.from_url(url)]
+
+
+agent = Agent(
+    chat_generator=OpenAIResponsesChatGenerator(model="gpt-5"),
+    tools=[fetch_image],
+    system_prompt="You are a helpful assistant that can fetch and analyze images from URLs.",
+)
+
+result = agent.run(
+    messages=[
+        ChatMessage.from_user(
+            "Fetch the image at https://picsum.photos/seed/haystack/640/480 and describe what you see.",
+        ),
+    ],
+)
+print(result["last_message"].text)
+```
+
+`ImageContent` can be created from a URL, a local file path, or a PDF page using the `PDFToImageContent` converter.
+
+### In a pipeline
+
+When an `Agent` sits inside a pipeline, use `ChatPromptBuilder` with its string template format and the `| templatize_part` filter to pass images as structured content parts:
+
+```python
+from haystack import Pipeline
+from haystack.components.agents import Agent
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ImageContent
+
+template = """
+{% message role="user" %}
+{{ question }}
+{{ image | templatize_part }}
+{% endmessage %}
+"""
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5"),
+    system_prompt="You are a helpful assistant that can analyze images.",
+)
+prompt_builder = ChatPromptBuilder(
+    template=template,
+    required_variables=["question", "image"],
+)
+
+pipeline = Pipeline()
+pipeline.add_component("prompt_builder", prompt_builder)
+pipeline.add_component("agent", agent)
+pipeline.connect("prompt_builder.prompt", "agent.messages")
+
+# Download or provide your own chart image as "chart.png"
+image = ImageContent.from_file_path("chart.png")
+result = pipeline.run(
+    {
+        "prompt_builder": {"question": "What does this chart show?", "image": image},
+    },
+)
+print(result["agent"]["last_message"].text)
+```
+
+:::tip
+See these cookbooks for complete multimodal agent examples:
+- [Multimodal Agents](https://haystack.deepset.ai/cookbook/multimodal_intro#multimodal-agent) — image inputs and tool use with agents
+- [Gemma Chat RAG](https://haystack.deepset.ai/cookbook/gemma_chat_rag) — vision model in a RAG pipeline
+:::
+
+## Multi-Agent Systems
+
+You can wrap an `Agent` as a tool to build multi-agent systems where specialist agents handle focused subtasks and a coordinator agent plans and delegates.
+
+See [Multi-Agent Systems](../../concepts/agents/multi-agent-systems.mdx) for a full guide, including the recommended `@tool` decorator approach for full interface control and `ComponentTool` for declarative configuration.
+
+## MCP Integration
+
+Agents work with MCP in two directions:
+
+- **Consuming MCP tools**: Pass `MCPTool` or `MCPToolset` instances in the `tools` list to call tools on any MCP-compatible server (filesystem, browser, databases, and more). See [MCPTool](../../tools/mcptool.mdx) and [MCPToolset](../../tools/mcptoolset.mdx).
+- **Exposing as an MCP server**: Use [Hayhooks](../../development/hayhooks.mdx) to deploy your agent and expose it as an MCP server, making it callable from any MCP-compatible client such as Claude Desktop or Cursor.
+
+## Additional References
+
+📖 Related docs:
+
+- [State](./state.mdx) — managing shared data between tools
+- [Hooks](./hooks.mdx) — running custom logic at defined points of the run loop
+- [Human in the Loop](./human-in-the-loop.mdx) — intercepting tool calls for human review
+- [Tool Result Offloading](./tool-result-offloading.mdx) — keeping large tool results out of the context window
+
+📚 Tutorials:
+
+- [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)
+- [Creating a Multi-Agent System](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system)
+- [Human-in-the-Loop with Haystack Agents](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent/)
+
+🧑‍🍳 Cookbook:
+
+- [Build a GitHub Issue Resolver Agent](https://haystack.deepset.ai/cookbook/github_issue_resolver_agent)
+- [Multimodal Agents](https://haystack.deepset.ai/cookbook/multimodal_intro#multimodal-agent)
+- [Gemma Chat RAG](https://haystack.deepset.ai/cookbook/gemma_chat_rag)

docs-website/docs/pipeline-components/agents-1/hooks.mdx

@@ -0,0 +1,202 @@
+---
+title: "Hooks"
+id: hooks
+slug: "/hooks"
+description: "Hooks let you run custom logic at defined points of an Agent's run loop — before each LLM call, before and after tool execution, and on exit."
+---
+
+# Hooks
+
+Hooks let you run custom logic at defined points of an [`Agent`](./agent.mdx)'s run loop — before each LLM call, before and after tool execution, and on exit.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Configured on**       | The [`Agent`](./agent.mdx) component via the `hooks` parameter |
+| **Key classes**         | `hook` (decorator), `FunctionHook`, `Hook` (protocol) |
+| **Import path**         | `haystack.hooks` |
+| **API reference**       | [Hooks](/reference/hooks-api) |
+| **GitHub link**         | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/ |
+| **Package name** | `haystack-ai` |
+
+</div>
+
+## Overview
+
+Pass `hooks` to the `Agent` as a dictionary mapping a *hook point* to a list of hooks the Agent runs at that point. Each hook receives the live [`State`](./state.mdx) and influences the run by mutating it in place. Hooks for a hook point run in list order, and the same hook can be registered under multiple hook points.
+
+This enables patterns such as building run-time system context, retrieving memories before the first LLM call, auditing or intercepting tool calls, and requiring a condition to hold before the Agent is allowed to finish.
+
+### Hook points
+
+- `before_llm`: Runs before each chat-generator call.
+- `before_tool`: Runs after the model requests tool calls, before any tools run. After these hooks run, the Agent re-reads the current last message from `state.data["messages"]`. If that message contains tool calls, those calls are executed. If it does not, no tools run for that step, no tool-based exit condition is triggered, and the Agent loops back to the next LLM call unless `max_agent_steps` has been reached.
+- `after_tool`: Runs after tools execute, once their result messages are in `state.data["messages"]`, before the exit-condition check and the next LLM call. Use it to rewrite the freshly produced tool-result messages — for example, to offload, redact, truncate, or summarize results. It does not run on the plain-text exit step. It does still run when a `before_tool` hook removed the pending tool calls: no tools executed on that step, so don't assume the last message is a fresh tool result.
+- `on_exit`: Runs when the Agent is about to stop on an exit condition. An `on_exit` hook can keep the Agent running by setting the `continue_run` control flag (`state.set("continue_run", True)`), usually alongside a message telling the model what to do next. `on_exit` hooks run when the Agent stops on an exit condition, but not when it stops because `max_agent_steps` is reached.
+
+Registering a hook under an unknown hook point raises a `ValueError` at construction. A hook class can declare an `allowed_hook_points` attribute listing the hook points it supports; the Agent validates it and fails fast if the hook is registered somewhere it doesn't belong.
+
+### State keys for hooks
+
+The Agent manages a few state keys that hooks interact with. Like the run-metadata keys (`step_count`, `token_usage`, `tool_call_counts`), they are reserved — using any of them in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for the full list:
+
+- `continue_run`: Set by an `on_exit` hook to keep the Agent running.
+- `tools`: The tools available in the current step, for hooks to inspect.
+- `hook_context`: Request-scoped resources passed to `Agent.run(hook_context={...})` / `run_async(hook_context={...})`. Hooks read it with `state.data["hook_context"]` or `state.data.get("hook_context")` — use it for per-request resources such as a user ID, a WebSocket, or a database client. Avoid the plain `state.get("hook_context")` here: `State.get` returns a deep copy of the value, which often fails for the kinds of resources stored in this dict (such as a WebSocket or a database client).
+
+Hooks can also read the automatically tracked run metadata: `step_count`, `token_usage`, and `tool_call_counts`.
+
+## Creating hooks
+
+### With the `@hook` decorator
+
+The `@hook` decorator wraps a function taking a single `State` argument into a hook. A regular function becomes the hook's sync path, a coroutine function its async path. To give a single hook both paths, construct a `FunctionHook` directly with both `function` and `async_function`.
+
+The example below registers a hook at each of `before_llm`, `before_tool`, and `on_exit` to show what hooks can do:
+
+```python
+from datetime import datetime, timezone
+from typing import Annotated
+
+from haystack.components.agents import Agent
+from haystack.components.agents.state import State, replace_values
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.hooks import hook
+from haystack.tools import tool
+
+
+@tool
+def search(query: Annotated[str, "The search query"]) -> str:
+    """Search the web."""
+    # Placeholder: would call a real search API
+    return "Fusion startups reported net-energy-gain milestones this year."
+
+
+@hook
+def build_context(state: State) -> None:
+    # before_llm: build run-time system context once, before the first model call.
+    if state.get("step_count") == 0:
+        now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
+        system = ChatMessage.from_system(
+            f"You are a research assistant. The current time is {now}.",
+        )
+        state.set(
+            "messages",
+            [system, *state.data["messages"]],
+            handler_override=replace_values,
+        )
+
+
+@hook
+def audit_tool_calls(state: State) -> None:
+    # before_tool: see which tools the model is about to run.
+    pending = state.data["messages"][-1].tool_calls
+    print(f"about to run: {[tc.tool_name for tc in pending]}")
+
+
+@hook
+def require_search(state: State) -> None:
+    # on_exit: keep going until the agent has actually searched.
+    if state.get("tool_call_counts", {}).get("search", 0) == 0:
+        state.set("messages", [ChatMessage.from_system("Search before answering.")])
+        state.set("continue_run", True)
+
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[search],
+    hooks={
+        "before_llm": [build_context],
+        "before_tool": [audit_tool_calls],
+        "on_exit": [require_search],
+    },
+)
+
+result = agent.run(
+    messages=[
+        ChatMessage.from_user("What are the latest developments in fusion energy?"),
+    ],
+)
+print(result["last_message"].text)
+```
+
+### Class-based hooks
+
+A hook is any object with a `run(state)` method; it may additionally define `run_async(state)` for true async behavior. Class-based hooks may also implement the optional lifecycle methods `warm_up` / `warm_up_async` and `close` / `close_async`. The Agent calls them from its own `warm_up` / `close`, so a hook can defer opening clients or reading credentials until warm-up and release them on close.
+
+When a class-based hook should be serializable (so an Agent using it can be serialized), implement `to_dict` / `from_dict`: store serializable constructor arguments on the hook and rebuild runtime clients from those values.
+
+The example below is an `on_exit` hook that grades the Agent's answer with its own LLM and asks the Agent to improve a weak answer before finishing:
+
+```python
+from typing import Any
+
+from haystack.components.agents import Agent
+from haystack.components.agents.state import State
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.core.serialization import default_from_dict, default_to_dict
+from haystack.dataclasses import ChatMessage
+
+
+class GradeFinalAnswer:
+    """Grade the Agent's answer with an LLM and ask it to improve a weak answer before finishing."""
+
+    def __init__(self, model: str = "gpt-5.4-nano"):
+        self.model = model
+        self._judge = OpenAIChatGenerator(model=self.model)
+
+    def warm_up(self) -> None:
+        # Warm up the judge's own client during the Agent's warm-up.
+        self._judge.warm_up()
+
+    def close(self) -> None:
+        # Release the judge's client during the Agent's close.
+        self._judge.close()
+
+    def run(self, state: State) -> None:
+        answer = state.data["messages"][-1].text or ""
+        verdict = (
+            self._judge.run(
+                messages=[
+                    ChatMessage.from_user(
+                        f"Reply with only PASS or FAIL. Is this answer complete?\n\n{answer}",
+                    ),
+                ],
+            )["replies"][0].text
+            or ""
+        )
+        if "FAIL" in verdict.upper():
+            state.set(
+                "messages",
+                [
+                    ChatMessage.from_user(
+                        "Your answer was incomplete. Please improve it.",
+                    ),
+                ],
+            )
+            state.set("continue_run", True)
+
+    def to_dict(self) -> dict[str, Any]:
+        return default_to_dict(self, model=self.model)
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any]) -> "GradeFinalAnswer":
+        return default_from_dict(cls, data)
+
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    hooks={"on_exit": [GradeFinalAnswer()]},
+)
+result = agent.run(messages=[ChatMessage.from_user("Explain how vaccines work.")])
+print(result["last_message"].text)
+```
+
+## Ready-made hooks
+
+Haystack ships two ready-made hooks:
+
+- `ConfirmationHook`: A `before_tool` hook that applies Human-in-the-Loop confirmation strategies to pending tool calls — a human can confirm, modify, or reject the tool calls the model requested before they run. See [Human in the Loop](./human-in-the-loop.mdx).
+- `ToolResultOffloadHook`: An `after_tool` hook that offloads tool results to a `ToolResultStore` (such as `FileSystemToolResultStore`) and replaces them in the conversation with a compact pointer, so the next LLM call sees a reference instead of the full result. Per-tool policies (`AlwaysOffload`, `NeverOffload`, `OffloadOverChars`) control which results are offloaded. See [Tool Result Offloading](./tool-result-offloading.mdx).

docs-website/docs/pipeline-components/agents-1/human-in-the-loop.mdx

@@ -0,0 +1,321 @@
+---
+title: "Human in the Loop"
+id: human-in-the-loop
+slug: "/human-in-the-loop"
+description: "Human-in-the-loop allows you to intercept agent tool calls before execution, letting a human confirm, reject, or modify the tool parameters."
+---
+
+# Human in the Loop
+
+Human-in-the-loop (HITL) lets you intercept an agent's tool calls before they are executed.
+A human can **confirm**, **reject**, or **modify** the parameters of each tool call in real time.
+This is useful for high-stakes operations - such as sending emails, modifying databases, or making API calls - where you want a human to review the action first.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Configured on**       | The [`Agent`](./agent.mdx) component, as a `ConfirmationHook` registered under the `before_tool` [hook point](./hooks.mdx) |
+| **Key classes**         | `ConfirmationHook`, `BlockingConfirmationStrategy`, `AlwaysAskPolicy`, `AskOncePolicy`, `NeverAskPolicy`, `RichConsoleUI`, `SimpleConsoleUI` |
+| **Import path**         | `haystack.human_in_the_loop`                                                                     |
+| **GitHub link**         | https://github.com/deepset-ai/haystack/blob/main/haystack/human_in_the_loop/                    |
+| **Package name** | `haystack-ai` |
+
+</div>
+
+## Overview
+
+HITL is one application of the Agent's general [hooks](./hooks.mdx) mechanism: a `ConfirmationHook` registered under the `before_tool` hook point intercepts the tool calls the model requested before they run, and confirms, modifies, or rejects them by rewriting the conversation in the Agent's `State`.
+
+The HITL system is composed of these layers:
+
+- **`ConfirmationHook`** - the `before_tool` hook that applies your confirmation strategies to pending tool calls. Its `confirmation_strategies` mapping accepts a single tool name, a tuple of tool names, or the wildcard `"*"` that applies to any tool without a more specific entry.
+- **Strategy** - decides what to do when a tool is about to be called. The built-in `BlockingConfirmationStrategy` pauses execution and asks a human.
+- **Policy** - decides *when* to ask. Built-in policies: `AlwaysAskPolicy`, `NeverAskPolicy`, `AskOncePolicy`.
+- **UI** - the interface used to ask the human. Built-in UIs: `RichConsoleUI` (requires `rich`) and `SimpleConsoleUI` (stdlib only).
+
+When the agent is about to invoke a tool, the strategy checks the policy.
+If the policy says to ask, the UI prompts the human with the tool name, description, and parameters. The human can:
+
+- **Confirm** (`y`) - execute as-is
+- **Reject** (`n`) - skip execution and feed rejection feedback back to the LLM
+- **Modify** (`m`) - edit the parameters before execution
+
+The agent then continues with the human's decision.
+
+:::info
+Strategies see only the arguments the model produced for a tool call. Values injected from [`State`](./state.mdx) via a tool's `inputs_from_state` mapping are not included in what is presented for confirmation — that injection happens at tool execution time.
+:::
+
+## Usage
+
+### Basic setup
+
+```python
+from typing import Annotated
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.human_in_the_loop import (
+    AlwaysAskPolicy,
+    BlockingConfirmationStrategy,
+    ConfirmationHook,
+    SimpleConsoleUI,
+)
+from haystack.tools import tool
+
+
+@tool
+def send_email(
+    to: Annotated[str, "The recipient email address"],
+    subject: Annotated[str, "The email subject line"],
+    body: Annotated[str, "The email body"],
+) -> str:
+    """Send an email to a recipient."""
+    return f"Email sent to {to}."
+
+
+strategy = BlockingConfirmationStrategy(
+    confirmation_policy=AlwaysAskPolicy(),
+    confirmation_ui=SimpleConsoleUI(),
+)
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"),
+    tools=[send_email],
+    hooks={
+        "before_tool": [
+            ConfirmationHook(confirmation_strategies={"send_email": strategy}),
+        ],
+    },
+)
+
+result = agent.run(
+    messages=[ChatMessage.from_user("Send a welcome email to alice@example.com")],
+)
+```
+
+When the agent calls `send_email`, the terminal will pause and show:
+
+```
+--- Tool Execution Request ---
+Tool: send_email
+Description: Send an email to a recipient.
+Arguments:
+  to: alice@example.com
+  subject: Welcome!
+  body: Hi Alice, welcome aboard!
+------------------------------
+Confirm execution? (y=confirm / n=reject / m=modify):
+```
+
+### Using RichConsoleUI
+
+`RichConsoleUI` provides a styled terminal prompt using the [`rich`](https://github.com/Textualize/rich) library:
+
+```shell
+pip install rich
+```
+
+```python
+from haystack.human_in_the_loop import RichConsoleUI
+
+strategy = BlockingConfirmationStrategy(
+    confirmation_policy=AlwaysAskPolicy(),
+    confirmation_ui=RichConsoleUI(),
+)
+```
+
+### Applying strategies to multiple tools
+
+You can configure different strategies per tool, share one strategy across a group of tools using a tuple key, or set a default for all tools with the wildcard `"*"` (applied to any tool without a more specific entry):
+
+```python
+@tool
+def delete_record(record_id: Annotated[str, "The ID of the record to delete"]) -> str:
+    """Delete a record from the database."""
+    return f"Record {record_id} deleted."
+
+
+@tool
+def update_record(
+    record_id: Annotated[str, "The ID of the record to update"],
+    data: Annotated[str, "The new data as a JSON string"],
+) -> str:
+    """Update a record in the database."""
+    return f"Record {record_id} updated."
+
+
+@tool
+def search(query: Annotated[str, "The search query"]) -> str:
+    """Search the knowledge base."""
+    return f"Results for: {query}"
+
+
+ask_strategy = BlockingConfirmationStrategy(
+    confirmation_policy=AlwaysAskPolicy(),
+    confirmation_ui=SimpleConsoleUI(),
+)
+
+confirmation_hook = ConfirmationHook(
+    confirmation_strategies={
+        # Share one strategy across multiple sensitive tools using a tuple key
+        ("send_email", "delete_record", "update_record"): ask_strategy,
+        # search has no strategy - always executes without asking
+    },
+)
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"),
+    tools=[send_email, delete_record, update_record, search],
+    hooks={"before_tool": [confirmation_hook]},
+)
+```
+
+### Customizing feedback messages
+
+When a tool call is rejected or modified, `BlockingConfirmationStrategy` sends a message back to the LLM explaining what happened. Three optional template parameters control these messages — each has a sensible default, so you only need to set them if you want different wording:
+
+- `reject_template`: Sent to the LLM when the user rejects a tool call. Must include a `{tool_name}` placeholder. Default: `"Tool execution for '{tool_name}' was rejected by the user."`
+- `modify_template`: Sent when the user modifies the parameters. Must include `{tool_name}` and `{final_tool_params}` placeholders. Default: `"The parameters for tool '{tool_name}' were updated by the user to:\n{final_tool_params}"`
+- `user_feedback_template`: Appends the user's optional free-text feedback to either message. Must include a `{feedback}` placeholder. Default: `"With user feedback: {feedback}"`
+
+```python
+strategy = BlockingConfirmationStrategy(
+    confirmation_policy=AlwaysAskPolicy(),
+    confirmation_ui=SimpleConsoleUI(),
+    reject_template="Skipping '{tool_name}' — rejected by operator.",
+    modify_template="Updated parameters for '{tool_name}': {final_tool_params}",
+    user_feedback_template="Reason: {feedback}",
+)
+```
+
+## Policies
+
+Policies control *when* the human is asked.
+
+| Policy | Behavior |
+| --- | --- |
+| `AlwaysAskPolicy` | Ask every time the tool is called |
+| `NeverAskPolicy` | Never ask - always proceed (useful for toggling HITL off without removing the strategy) |
+| `AskOncePolicy` | Ask once per unique `(tool_name, parameters)` combination. Remembers confirmed calls and skips asking on repeats. |
+
+### Custom policy
+
+You can implement your own policy by subclassing `ConfirmationPolicy` from `haystack.human_in_the_loop.types`:
+
+```python
+from haystack.human_in_the_loop.types import ConfirmationPolicy, ConfirmationUIResult
+from typing import Any
+
+
+class AskForSensitiveParamsPolicy(ConfirmationPolicy):
+    """Only ask when the 'to' parameter looks like an external email domain."""
+
+    def should_ask(
+        self,
+        tool_name: str,
+        tool_description: str,
+        tool_params: dict[str, Any],
+    ) -> bool:
+        to = tool_params.get("to", "")
+        return not to.endswith("@mycompany.com")
+```
+
+For stateful policies, also implement `update_after_confirmation`.
+It is called after the user responds and receives the full `ConfirmationUIResult`, letting you update internal state based on the outcome.
+The following policy asks once per tool name and skips re-asking for any tool the user has already confirmed:
+
+```python
+from haystack.human_in_the_loop.types import ConfirmationPolicy
+from haystack.human_in_the_loop import ConfirmationUIResult
+from typing import Any
+
+
+class AskOncePerToolPolicy(ConfirmationPolicy):
+    """Ask once per tool name, regardless of parameters. Skip on repeat confirmed calls."""
+
+    def __init__(self) -> None:
+        self._confirmed_tools: set[str] = set()
+
+    def should_ask(
+        self,
+        tool_name: str,
+        tool_description: str,
+        tool_params: dict[str, Any],
+    ) -> bool:
+        return tool_name not in self._confirmed_tools
+
+    def update_after_confirmation(
+        self,
+        tool_name: str,
+        tool_description: str,
+        tool_params: dict[str, Any],
+        confirmation_result: ConfirmationUIResult,
+    ) -> None:
+        if confirmation_result.action == "confirm":
+            self._confirmed_tools.add(tool_name)
+```
+
+## Dataclasses
+
+### `ConfirmationUIResult`
+
+Returned by the UI after the human responds.
+
+| Field | Type | Description |
+| --- | --- | --- |
+| `action` | `str` | `"confirm"`, `"reject"`, or `"modify"` |
+| `feedback` | `str \| None` | Optional free-text feedback from the human |
+| `new_tool_params` | `dict \| None` | Replacement parameters when action is `"modify"` |
+
+### `ToolExecutionDecision`
+
+Returned by the strategy to the agent.
+
+| Field | Type | Description |
+| --- | --- | --- |
+| `tool_name` | `str` | Name of the tool |
+| `execute` | `bool` | Whether to execute the tool |
+| `tool_call_id` | `str \| None` | ID of the tool call |
+| `feedback` | `str \| None` | Feedback message passed back to the LLM on rejection or modification |
+| `final_tool_params` | `dict \| None` | Final parameters to use for execution |
+
+## Example: HITL with Hayhooks and Open WebUI
+
+The [hitl-hayhooks-redis-openwebui](https://github.com/deepset-ai/hitl-hayhooks-redis-openwebui) repository shows a full production-style HITL setup using a Haystack Agent served via [Hayhooks](https://github.com/deepset-ai/hayhooks) with approval dialogs rendered in [Open WebUI](https://github.com/open-webui/open-webui).
+
+The key pattern it demonstrates is a custom `RedisConfirmationStrategy` that receives per-request resources - a Redis client and an async event queue - at runtime. Pass such resources via the generic `hook_context` run argument (`agent.run(messages=[...], hook_context={"redis": client})`). `ConfirmationHook` reads this dict from state with `state.data["hook_context"]` (not `state.get`, which returns a deep copy that fails for live resources like clients and queues - see [Hooks](./hooks.mdx)) and passes it to each strategy's `run()` as the `confirmation_strategy_context` keyword argument, which is how a custom strategy receives the Redis client and event queue:
+
+- When a tool call is about to execute, the strategy emits a `tool_call_start` SSE event and blocks on `Redis BLPOP` waiting for an approval decision.
+- The Open WebUI Pipe function receives the SSE event, shows the user a confirmation dialog, then writes `approved` or `rejected` to Redis via `LPUSH`.
+- Once Redis unblocks, the strategy returns a `ToolExecutionDecision` and the agent continues.
+
+This is a good reference if you need non-blocking HITL in a web or server environment where `SimpleConsoleUI` and `RichConsoleUI` are not suitable.
+
+## Custom UI
+
+Implement `ConfirmationUI` from `haystack.human_in_the_loop.types` to build your own interface - for example, a web-based approval queue:
+
+```python
+from haystack.human_in_the_loop.types import ConfirmationUI
+from haystack.human_in_the_loop import ConfirmationUIResult
+from typing import Any
+
+
+class WebhookApprovalUI(ConfirmationUI):
+    """Sends a webhook and waits for an async approval response."""
+
+    def get_user_confirmation(
+        self,
+        tool_name: str,
+        tool_description: str,
+        tool_params: dict[str, Any],
+    ) -> ConfirmationUIResult:
+        # Send approval request to your system and wait for response
+        response = send_approval_request_and_wait(tool_name, tool_params)
+        return ConfirmationUIResult(
+            action=response["action"],
+            feedback=response.get("feedback"),
+        )
+```

docs-website/docs/pipeline-components/agents-1/state.mdx

@@ -0,0 +1,424 @@
+---
+title: "State"
+id: state
+slug: "/state"
+description: "`State` is a container for storing shared information during Agent and Tool execution. It provides a structured way to share data between tools, accumulate results across multiple tool calls, and surface them alongside the agent's final answer."
+---
+
+# State
+
+`State` is a container for storing shared information during Agent and Tool execution.
+It provides a structured way to share data between tools, accumulate results across multiple tool calls, and surface them alongside the agent's final answer.
+
+## Overview
+
+When building agents that use multiple tools, you often need tools to share information or accumulate results across iterations.
+State provides centralized storage that all tools can read from and write to.
+For example, a search tool called multiple times can append its results to a shared `documents` list, which is then returned alongside the agent's final answer for source inspection.
+
+State uses a schema-based approach where you define:
+
+- What data can be stored,
+- The type of each piece of data,
+- How values are merged when updated.
+
+The Agent creates and manages the `State` object internally. You shouldn't need to instantiate it directly.
+You interact with it through tool definitions (`inputs_from_state`, `outputs_to_state`, or a `state: State` parameter) and read results from the agent's output dict.
+
+### Supported Types
+
+State supports standard Python types:
+
+- Basic types: `str`, `int`, `float`, `bool`, `dict`
+- List types: `list`, `list[str]`, `list[int]`, `list[Document]`
+- Union types: `str | int`, `str | None`
+- Custom classes and data classes.
+
+### Automatic Message Handling
+
+State automatically includes a `messages` field that stores the full conversation history during execution.
+You don't need to define this in your schema.
+It uses `list[ChatMessage]` type with the `merge_lists` handler, so new messages are appended on each iteration.
+
+### State API
+
+| Method | Description |
+| --- | --- |
+| `state.get(key, default=None)` | Read a value; returns `default` if the key doesn't exist |
+| `state.set(key, value)` | Write a value, merged using the schema's handler |
+| `state.has(key)` | Returns `True` if the key exists in state |
+| `state.data` | Returns a snapshot of all current state as a `dict` |
+
+## Schema Definition
+
+The schema defines what data can be stored and how values are updated. Each schema entry consists of:
+
+- `type` (required): The Python type for this field (for example, `str`, `int`, `list`)
+- `handler` (optional): A callable that determines how new values are merged when `set()` is called
+
+```python
+{
+    "parameter_name": {
+        "type": SomeType,  # Required: expected Python type
+        "handler": some_func,  # Optional: merge function
+    },
+}
+```
+
+If you don't specify a handler, State automatically assigns a default based on the type.
+
+:::info Reserved keys
+The `Agent` manages some state keys itself and rejects them in a user-provided `state_schema` with a `ValueError`:
+
+- The run-metadata keys `step_count`, `token_usage`, and `tool_call_counts`, which the Agent populates automatically during a run: tools can read them mid-run via `inputs_from_state`, and they are returned in the result dictionary.
+- The hook-facing keys `continue_run` (set by an `on_exit` hook to keep the Agent running), `tools` (the tools available in the current step, for hooks to inspect), and `hook_context` (the request-scoped resources passed to `Agent.run(hook_context={...})`).
+
+If one of your state keys clashes, rename it (for example, `my_token_usage`).
+:::
+
+### Default Handlers
+
+State provides two built-in merge behaviors (importable from `haystack.components.agents.state`):
+
+- **`merge_lists`**: Appends to the existing list (default for list types)
+- **`replace_values`**: Overwrites the existing value (default for non-list types)
+
+```python
+from haystack.components.agents import State
+
+schema = {
+    "documents": {"type": list},  # uses merge_lists by default
+    "user_name": {"type": str},  # uses replace_values by default
+}
+
+state = State(schema=schema)
+
+state.set("documents", [1, 2])
+state.set("documents", [3, 4])
+print(state.get("documents"))  # [1, 2, 3, 4]
+
+state.set("user_name", "Alice")
+state.set("user_name", "Bob")
+print(state.get("user_name"))  # "Bob"
+```
+
+### Custom Handlers
+
+Custom handlers are useful when the default `merge_lists` or `replace_values` behaviors don't fit your needs.
+A handler takes the current state value and the new value and returns the merged result.
+
+The example below uses a deduplication handler, useful when multiple tool calls might return overlapping results and you want to avoid accumulating duplicates in state:
+
+```python
+def deduplicate(current_value: list | None, new_value: list) -> list:
+    """Append new items, skipping any already in the list."""
+    existing = set(current_value or [])
+    return (current_value or []) + [item for item in new_value if item not in existing]
+
+
+schema = {"doc_ids": {"type": list, "handler": deduplicate}}
+
+state = State(schema=schema)
+state.set("doc_ids", ["doc-1", "doc-2"])
+state.set("doc_ids", ["doc-2", "doc-3"])
+print(state.get("doc_ids"))  # ["doc-1", "doc-2", "doc-3"]
+```
+
+You can also override the handler for a single `set()` call:
+
+```python
+from haystack.components.agents import State
+
+
+def concatenate_strings(current: str | None, new: str) -> str:
+    return f"{current}-{new}" if current else new
+
+
+state = State(schema={"user_name": {"type": str}})
+state.set("user_name", "Alice")
+state.set("user_name", "Bob", handler_override=concatenate_strings)
+print(state.get("user_name"))  # "Alice-Bob"
+```
+
+## Using State
+
+Define a `state_schema` when creating the Agent.
+State keys declared in `state_schema` are exposed as output keys on the agent's result dict alongside `messages` and `last_message`.
+
+Tools interact with State through three mechanisms:
+
+- **`outputs_to_state`**: Write tool results to state keys after the tool runs.
+- **`inputs_from_state`**: Inject state values into tool parameters before the tool runs.
+- **Direct `State` injection**: Add a `state: State` parameter to your tool function's signature. The Agent detects the `State` annotation and injects the live `State` object automatically, so you can read or write any key defined in the schema. The `State` object is never exposed to the LLM's parameter schema.
+
+### Reading from State: `inputs_from_state`
+
+`inputs_from_state` maps state keys to function parameter names using the format `{"state_key": "param_name"}`.
+The value is injected from state before the tool runs, so the LLM never needs to provide it.
+
+Parameters mapped via `inputs_from_state` are automatically excluded from the LLM's parameter schema.
+The model never sees or provides them:
+
+```python
+from typing import Annotated
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.dataclasses import ChatMessage
+from haystack.tools import tool
+
+
+@tool(inputs_from_state={"user_name": "user_context"})
+def search_documents(
+    query: Annotated[str, "The search query"],
+    user_context: str,  # injected from state; excluded from LLM schema
+) -> dict:
+    """Search documents using query and user context."""
+    return {"results": [f"Found results for '{query}' (user: {user_context})"]}
+
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[search_documents],
+    system_prompt="Use the search_documents tool to find information.",
+    streaming_callback=print_streaming_chunk,
+    state_schema={"user_name": {"type": str}},
+)
+
+result = agent.run(
+    messages=[ChatMessage.from_user("Search for Python tutorials")],
+    user_name="Alice",  # state key "user_name" is pre-populated by passing user_name= to agent.run()
+)
+
+print(result["last_message"].text)
+```
+
+### Writing to State: `outputs_to_state`
+
+The `outputs_to_state` parameter maps tool output keys to state keys. Each entry supports two optional fields:
+
+```python
+{
+    "state_key": {
+        "source": "tool_output_key",  # which key to read from the tool's return dict; omit to store the entire dict
+        "handler": some_func,  # override the schema's merge handler for this mapping only
+    },
+}
+```
+
+```python
+from typing import Annotated
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.dataclasses import ChatMessage
+from haystack.tools import tool
+
+
+@tool(
+    outputs_to_state={
+        "documents": {"source": "documents"},
+        "result_count": {"source": "count"},
+        "last_query": {"source": "query"},
+    },
+)
+def retrieve_documents(
+    query: Annotated[str, "The search query"],
+) -> dict:
+    """Retrieve relevant documents."""
+    return {
+        "documents": [
+            {"title": "Doc 1", "content": "Content about Python"},
+            {"title": "Doc 2", "content": "More about Python"},
+        ],
+        "count": 2,
+        "query": query,
+    }
+
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[retrieve_documents],
+    system_prompt="Use the retrieve_documents tool to find information.",
+    streaming_callback=print_streaming_chunk,
+    state_schema={
+        "documents": {"type": list},
+        "result_count": {"type": int},
+        "last_query": {"type": str},
+    },
+)
+
+result = agent.run(messages=[ChatMessage.from_user("Find information about Python")])
+
+print(f"Documents: {result['documents']}")
+print(f"Result count: {result['result_count']}")
+print(f"Last query: {result['last_query']}")
+```
+
+If you omit `source`, the entire tool result dict is stored under the state key:
+
+```python
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.dataclasses import ChatMessage
+from haystack.tools import tool
+
+
+@tool(outputs_to_state={"user_info": {}})
+def get_user_info() -> dict:
+    """Get user information."""
+    return {"name": "Alice", "email": "alice@example.com", "role": "admin"}
+
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[get_user_info],
+    system_prompt="Use the get_user_info tool to look up user details.",
+    streaming_callback=print_streaming_chunk,
+    state_schema={"user_info": {"type": dict}},
+)
+
+result = agent.run(messages=[ChatMessage.from_user("What are the user's details?")])
+
+print(result["last_message"].text)
+print(f"User info: {result['user_info']}")
+```
+
+### Combining Inputs and Outputs
+
+Tools can both read from and write to State, enabling tool chaining across iterations.
+This example builds on `retrieve_documents` from the previous section:
+
+```python
+@tool(
+    inputs_from_state={"documents": "documents"},
+    outputs_to_state={
+        "final_docs": {"source": "processed_docs"},
+        "final_count": {"source": "processed_count"},
+    },
+)
+def process_documents(
+    max_results: Annotated[int, "Maximum number of documents to return"],
+    documents: list = None,  # injected from state; LLM does not provide this
+) -> dict:
+    """Process retrieved documents and return a filtered subset."""
+    processed = (documents or [])[:max_results]
+    return {"processed_docs": processed, "processed_count": len(processed)}
+
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[retrieve_documents, process_documents],  # chained through state
+    system_prompt="Use the available tools to retrieve and process documents.",
+    streaming_callback=print_streaming_chunk,
+    state_schema={
+        "documents": {"type": list},
+        "result_count": {"type": int},
+        "last_query": {"type": str},
+        "final_docs": {"type": list},
+        "final_count": {"type": int},
+    },
+)
+
+result = agent.run(
+    messages=[ChatMessage.from_user("Find and process 3 documents about Python")],
+)
+print(f"Processed {result['final_count']} documents")
+```
+
+### Injecting State Directly into Tools
+
+As an alternative to `inputs_from_state` and `outputs_to_state`, a tool can declare a `state: State` parameter to receive the live `State` object at invocation time.
+This lets the tool read from and write to any number of state keys without declaring mappings upfront.
+
+The ToolInvoker detects the `State` annotation and injects the object automatically.
+It is excluded from the LLM-facing schema. The model is never asked to supply it.
+Both `State` and `State | None` annotations are supported.
+
+For function-based tools, add the `state` parameter and use the `@tool` decorator:
+
+```python
+from typing import Annotated
+from haystack.components.agents import Agent, State
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.dataclasses import ChatMessage, Document
+from haystack.tools import tool
+
+
+@tool
+def retrieve_and_store(
+    query: Annotated[str, "The search query"],
+    state: State,
+) -> str:
+    """Retrieve documents and store them directly in state."""
+    documents = [Document(content=f"Result for '{query}'")]
+    state.set("documents", documents)
+    user_name = state.get("user_name", "unknown")
+    return f"Retrieved {len(documents)} document(s) for {user_name}"
+
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[retrieve_and_store],
+    system_prompt="Use the retrieve_and_store tool to find documents.",
+    streaming_callback=print_streaming_chunk,
+    state_schema={"documents": {"type": list[Document]}, "user_name": {"type": str}},
+)
+
+result = agent.run(
+    messages=[ChatMessage.from_user("Find documents about Python")],
+    user_name="Alice",
+)
+
+print(result["last_message"].text)
+print(result["documents"])
+```
+
+For component-based tools, declare a `State` input socket on the `run` method and wrap it with `ComponentTool`:
+
+```python
+from haystack import component
+from haystack.components.agents import Agent, State
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.generators.utils import print_streaming_chunk
+from haystack.dataclasses import ChatMessage, Document
+from haystack.tools import ComponentTool
+
+
+@component
+class DocumentRetriever:
+    """Retrieve documents and store them in state."""
+
+    @component.output_types(reply=str)
+    def run(self, query: str, state: State) -> dict[str, str]:
+        """
+        Retrieve documents based on query and store them in state.
+
+        :param query: The search query
+        """
+        documents = [Document(content=f"Result for '{query}'")]
+        state.set("documents", documents)
+        return {"reply": f"Retrieved {len(documents)} document(s)"}
+
+
+retriever_tool = ComponentTool(
+    component=DocumentRetriever(),
+    name="retrieve",
+    description="Retrieve documents based on a search query",
+)
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[retriever_tool],
+    system_prompt="Use the retrieve tool to find documents.",
+    streaming_callback=print_streaming_chunk,
+    state_schema={"documents": {"type": list[Document]}},
+)
+
+result = agent.run(messages=[ChatMessage.from_user("Find documents about Python")])
+
+print(result["last_message"].text)
+print(result["documents"])
+```

docs-website/docs/pipeline-components/agents-1/tool-result-offloading.mdx

@@ -0,0 +1,224 @@
+---
+title: "Tool Result Offloading"
+id: tool-result-offloading
+slug: "/tool-result-offloading"
+description: "Tool result offloading writes large tool results to a store and replaces them in the conversation with a compact pointer, keeping the Agent's context window small."
+---
+
+# Tool Result Offloading
+
+Tool result offloading writes selected tool results to a store and replaces them in the conversation with a compact pointer — a reference plus a short preview — so the next LLM call sees a reference instead of the full result.
+This keeps the context window small when tools return large outputs (web pages, file contents, query results), and it is a step towards letting an Agent operate on offloaded results with follow-up tools, such as a file-reading tool that opens the referenced files.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Configured on**       | The [`Agent`](./agent.mdx) component, as a `ToolResultOffloadHook` registered under the `after_tool` [hook point](./hooks.mdx) |
+| **Key classes**         | `ToolResultOffloadHook`, `FileSystemToolResultStore`, `AlwaysOffload`, `NeverOffload`, `OffloadOverChars` |
+| **Import path**         | `haystack.hooks.tool_result_offloading` |
+| **API reference**       | [Hooks](/reference/hooks-api) |
+| **GitHub link**         | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/tool_result_offloading/ |
+| **Package name** | `haystack-ai` |
+
+</div>
+
+## Overview
+
+Tool result offloading is one application of the Agent's general [hooks](./hooks.mdx) mechanism: a `ToolResultOffloadHook` registered under the `after_tool` hook point runs after each step's tools execute and rewrites the freshly produced tool-result messages in the Agent's [`State`](./state.mdx). It only considers the current step's results; earlier conversation history is left untouched.
+
+The system is composed of these layers:
+
+- **`ToolResultOffloadHook`** - the `after_tool` hook that applies your offload strategies to fresh tool results. Its `offload_strategies` mapping accepts a single tool name, a tuple of tool names, or the wildcard `"*"` that applies to any tool without a more specific entry.
+- **Policy** - decides *whether* a given result is offloaded. Built-in policies: `AlwaysOffload`, `NeverOffload`, `OffloadOverChars`.
+- **Store** - decides *where* the full result lives. The built-in `FileSystemToolResultStore` writes results to the local file system.
+
+When a result is offloaded, the hook writes the full text to the store and rebuilds the message with a one-line pointer in its place:
+
+```
+Tool result offloaded to '/abs/path/tool_results/2_search_call-123.txt' (18234 characters). Preview: Fusion startups reported...
+```
+
+The pointer carries the store reference, the original length, and a preview of the first `preview_chars` characters (200 by default, configurable on the hook), so the model knows roughly what was offloaded and where to find it.
+
+## Usage
+
+### Basic setup
+
+The example below offloads any tool result longer than 4,000 characters to files under a local `tool_results` directory:
+
+```python
+from typing import Annotated
+
+from haystack.components.agents import Agent
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack.hooks.tool_result_offloading import (
+    FileSystemToolResultStore,
+    OffloadOverChars,
+    ToolResultOffloadHook,
+)
+from haystack.tools import tool
+
+
+@tool
+def search(query: Annotated[str, "The search query"]) -> str:
+    """Search the web and return the (potentially large) results."""
+    # Placeholder: would call a real search API
+    return f"... large result for {query} ..."
+
+
+offload_hook = ToolResultOffloadHook(
+    store=FileSystemToolResultStore(root="tool_results"),
+    offload_strategies={"*": OffloadOverChars(4000)},
+)
+
+agent = Agent(
+    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
+    tools=[search],
+    hooks={"after_tool": [offload_hook]},
+)
+
+result = agent.run(messages=[ChatMessage.from_user("Summarize today's tech news")])
+```
+
+### Configuring what gets offloaded per tool
+
+Each key in `offload_strategies` may be a single tool name, a tuple of tool names sharing one policy, or the wildcard `"*"`. More specific keys win over `"*"`, and a tool with no matching key (and no `"*"`) is never offloaded:
+
+```python
+from haystack.hooks.tool_result_offloading import (
+    AlwaysOffload,
+    FileSystemToolResultStore,
+    NeverOffload,
+    OffloadOverChars,
+    ToolResultOffloadHook,
+)
+
+offload_hook = ToolResultOffloadHook(
+    store=FileSystemToolResultStore(root="tool_results"),
+    offload_strategies={
+        "web_search": AlwaysOffload(),  # force offload
+        "get_time": NeverOffload(),  # opt out of the wildcard default
+        ("read_file", "list_dir"): OffloadOverChars(4000),  # tuple key: shared policy
+        "*": OffloadOverChars(8000),  # default for any unlisted tool
+    },
+)
+```
+
+### What is offloaded
+
+The hook only offloads **successful, text** tool results:
+
+- Error results — including rejections produced by a `before_tool` [Human-in-the-Loop](./human-in-the-loop.mdx) hook — are always left in context, so the model sees what went wrong.
+- Non-text results (image or file content) are left in context; supporting only text is a deliberate choice for now. A warning is logged when a non-text result has a matching offload policy.
+- Each result is offloaded at most once, even though the hook runs on every tool step. This also means two offload hooks registered under `after_tool` won't offload each other's pointers.
+
+## Policies
+
+Policies control *whether* a result is offloaded.
+
+| Policy | Behavior |
+| --- | --- |
+| `AlwaysOffload` | Offload every result of the tool it is assigned to |
+| `NeverOffload` | Never offload - keep the full result in context (useful to opt a tool out of a wildcard default) |
+| `OffloadOverChars(threshold)` | Offload only when the result is longer than `threshold` characters |
+
+### Custom policy
+
+Subclass the `OffloadPolicy` protocol from `haystack.hooks.tool_result_offloading` for custom conditions. A policy needs a `should_offload` method, which receives the tool name, the result text, and the Agent's live [`State`](./state.mdx), so it can also decide based on run context:
+
+```python
+from haystack.components.agents.state import State
+from haystack.hooks.tool_result_offloading import OffloadPolicy
+
+
+class OffloadLateSteps(OffloadPolicy):
+    """Offload results only once the run is several steps deep and context pressure builds up."""
+
+    def should_offload(self, tool_name: str, result: str, state: State) -> bool:
+        return state.data.get("step_count", 0) >= 3 and len(result) > 1000
+```
+
+The protocol provides default `to_dict` / `from_dict` implementations, so a policy like this one, whose constructor takes no arguments, is serializable as-is. A policy with constructor arguments should implement both methods itself, following `OffloadOverChars` as an example.
+
+## Stores
+
+### `FileSystemToolResultStore`
+
+`FileSystemToolResultStore(root=...)` writes each offloaded result to a file under its root directory and returns the absolute file path as the reference. The directory is created on first write. Store keys are derived from the step count, tool name, and tool call ID (for example `2_search_call-123.txt`), so results from different tools and steps do not collide. A key that would resolve outside the root directory is rejected.
+
+### Custom store
+
+Subclass the `ToolResultStore` protocol to target other backends, such as object storage or an isolated sandbox file system. A store needs two methods: `write(key=..., content=...)` persists the content and returns an opaque reference string, and `read(reference)` resolves that reference back to the content:
+
+```python
+from haystack.hooks.tool_result_offloading import ToolResultStore
+
+
+class InMemoryToolResultStore(ToolResultStore):
+    """Keep offloaded results in a dict - useful for tests."""
+
+    def __init__(self) -> None:
+        self._data: dict[str, str] = {}
+
+    def write(self, *, key: str, content: str) -> str:
+        self._data[key] = content
+        return key
+
+    def read(self, reference: str) -> str:
+        return self._data[reference]
+```
+
+Like `OffloadPolicy`, the protocol provides default `to_dict` / `from_dict` implementations covering stores whose constructor takes no arguments; implement both methods for stores with constructor arguments.
+
+### Per-run stores via `hook_context`
+
+The constructor `store` is shared by every run - fine for single-user or local use. In a multi-user server, give each run its own isolated store (for example, a per-session directory) by passing it in the Agent's generic `hook_context` run argument under the key `RESULT_STORE_CONTEXT_KEY`. It overrides the constructor store for that run:
+
+```python
+from haystack.hooks.tool_result_offloading import (
+    RESULT_STORE_CONTEXT_KEY,
+    FileSystemToolResultStore,
+)
+
+per_request_store = FileSystemToolResultStore(root=f"tool_results/{session_id}")
+result = agent.run(
+    messages=[ChatMessage.from_user("...")],
+    hook_context={RESULT_STORE_CONTEXT_KEY: per_request_store},
+)
+```
+
+Isolating the store per run keeps concurrent users from colliding on store keys or reading each other's offloaded results — especially important when a file-reading tool is scoped to the store. The hook itself keeps no mutable state, so a single instance is safe to share across concurrent runs.
+
+## Letting the Agent read offloaded results back
+
+The pointer left in the conversation tells the model where the full result lives, but the model can only act on it if the Agent has a tool that can read from the store. With `FileSystemToolResultStore`, that can be a simple file-reading tool:
+
+```python
+from typing import Annotated
+
+from haystack.tools import tool
+
+
+@tool
+def read_offloaded_result(
+    path: Annotated[str, "Absolute path of an offloaded tool result"],
+) -> str:
+    """Read back the full content of an offloaded tool result."""
+    return FileSystemToolResultStore(root="tool_results").read(path)
+```
+
+With this tool available, the Agent can work with a compact conversation and selectively re-read only the offloaded results it actually needs — instead of carrying every full result in context on every LLM call.
+
+## Serialization
+
+`ToolResultOffloadHook` implements `to_dict` / `from_dict`, so an Agent using it can be serialized as long as the configured store and policies are serializable too. The built-in store and policies all are; for custom ones, see the notes in [Policies](#custom-policy) and [Stores](#custom-store) above.
+
+## Additional References
+
+📖 Related docs:
+
+- [Hooks](./hooks.mdx) — the general mechanism behind this feature, including the `after_tool` hook point
+- [Human in the Loop](./human-in-the-loop.mdx) — another ready-made hook, intercepting tool calls for human review
+- [State](./state.mdx) — the live run state hooks and policies receive

docs-website/docs/pipeline-components/audio.mdx

@@ -0,0 +1,16 @@
+---
+title: "Audio"
+id: audio
+slug: "/audio"
+description: "Use these components to work with audio in Haystack by transcribing files or converting text to audio."
+---
+
+# Audio
+
+Use these components to work with audio in Haystack by transcribing files or converting text to audio.
+
+| Name                                                       | Description                                                                                   |
+| --- | --- |
+| [FunASRTranscriber](audio/funasrtranscriber.mdx)               | Transcribe audio files using FunASR — a local, open-source speech recognition toolkit supporting 50+ languages. |
+| [LocalWhisperTranscriber](audio/localwhispertranscriber.mdx)   | Transcribe audio files using OpenAI's Whisper model using your local installation of Whisper. |
+| [RemoteWhisperTranscriber](audio/remotewhispertranscriber.mdx) | Transcribe audio files using OpenAI's Whisper model.                                          |

docs-website/docs/pipeline-components/audio/external-integrations-audio.mdx

@@ -0,0 +1,15 @@
+---
+title: "External Integrations"
+id: external-integrations-audio
+slug: "/external-integrations-audio"
+description: "External integrations that enable working with audio in Haystack by transcribing files or converting text to audio."
+---
+
+# External Integrations
+
+External integrations that enable working with audio in Haystack by transcribing files or converting text to audio.
+
+| Name | Description |
+| --- | --- |
+| [AssemblyAI](https://haystack.deepset.ai/integrations/assemblyai) | Perform speech recognition, speaker diarization and summarization. |
+| [Elevenlabs](https://haystack.deepset.ai/integrations/elevenlabs) | Convert text to speech using ElevenLabs’ API.                      |

docs-website/docs/pipeline-components/audio/funasrtranscriber.mdx

@@ -0,0 +1,66 @@
+---
+title: "FunASRTranscriber"
+id: funasrtranscriber
+slug: "/funasrtranscriber"
+description: "Transcribe audio files to Documents using FunASR — a local, open-source speech recognition toolkit supporting 50+ languages."
+---
+
+# FunASRTranscriber
+
+Transcribe audio files to Haystack Documents using FunASR — a local, open-source speech recognition toolkit supporting 50+ languages.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | As the first component in an indexing pipeline |
+| **Mandatory run variables** | `sources`: A list of audio file paths (`str` or `Path`) or `ByteStream` objects |
+| **Output variables** | `documents`: A list of Haystack Documents, one per source, with transcript text in `content` |
+| **API reference** | [FunASR integration](/reference/integrations-funasr) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/funasr/src/haystack_integrations/components/audio/funasr/transcriber.py |
+
+</div>
+
+## Overview
+
+`FunASRTranscriber` uses [FunASR](https://github.com/modelscope/FunASR), an open-source speech recognition toolkit from Alibaba DAMO Academy, to transcribe audio files into Haystack `Document` objects. It runs entirely locally — no API key required.
+
+The default model is `iic/SenseVoiceSmall`, a multilingual model supporting 50+ languages that is 5–10x faster than Whisper. Models are downloaded from ModelScope on first use and cached in `~/.cache/modelscope`.
+
+The component accepts audio file paths (`str` or `Path`) as well as `ByteStream` objects. The model is loaded into memory automatically the first time the component runs.
+
+## Usage
+
+### On its own
+
+```python
+from haystack_integrations.components.audio.funasr import FunASRTranscriber
+
+transcriber = FunASRTranscriber()
+
+result = transcriber.run(sources=["speech.wav"])
+print(result["documents"][0].content)
+```
+
+### In a pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.fetchers import LinkContentFetcher
+from haystack_integrations.components.audio.funasr import FunASRTranscriber
+
+pipe = Pipeline()
+pipe.add_component("fetcher", LinkContentFetcher())
+pipe.add_component("transcriber", FunASRTranscriber())
+
+pipe.connect("fetcher", "transcriber")
+
+result = pipe.run(
+    data={
+        "fetcher": {
+            "urls": ["https://example.com/interview.wav"],
+        },
+    },
+)
+print(result["transcriber"]["documents"][0].content)
+```

docs-website/docs/pipeline-components/audio/localwhispertranscriber.mdx

@@ -0,0 +1,90 @@
+---
+title: "LocalWhisperTranscriber"
+id: localwhispertranscriber
+slug: "/localwhispertranscriber"
+description: "Use `LocalWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model using your local installation of Whisper."
+---
+
+# LocalWhisperTranscriber
+
+Use `LocalWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model using your local installation of Whisper.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | As the first component in an indexing pipeline                                                |
+| **Mandatory run variables**            | `sources`: A list of paths or binary streams that you want to transcribe                      |
+| **Output variables**                   | `documents`: A list of documents                                                              |
+| **API reference**                      | [Whisper](/reference/integrations-whisper)                                                                        |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/whisper |
+| **Package name** | `whisper-haystack` |
+
+</div>
+
+## Overview
+
+The component also needs to know which Whisper model to work with. Specify this in the `model` parameter when initializing the component. All transcription is completed on the executing machine, and the audio is never sent to a third-party provider.
+
+See other optional parameters you can specify in our [API documentation](/reference/integrations-whisper).
+
+See the [Whisper API documentation](https://platform.openai.com/docs/guides/speech-to-text) and the official Whisper [GitHub repo](https://github.com/openai/whisper) for the supported audio formats and languages.
+
+The `LocalWhisperTranscriber` is part of the `whisper-haystack` integration package. To work with it, install the package along with [Whisper](https://github.com/openai/whisper) (which also pulls in torch) using the following commands:
+
+```python
+pip install whisper-haystack
+pip install -U openai-whisper
+```
+
+## Usage
+
+### On its own
+
+Here’s an example of how to use `LocalWhisperTranscriber` on its own:
+
+```python
+import requests
+from haystack_integrations.components.audio.whisper import LocalWhisperTranscriber
+
+response = requests.get(
+    "https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
+)
+with open("kennedy_speech.mp3", "wb") as file:
+    file.write(response.content)
+
+transcriber = LocalWhisperTranscriber(model="tiny")
+
+transcription = transcriber.run(sources=["./kennedy_speech.mp3"])
+print(transcription["documents"][0].content)
+```
+
+### In a pipeline
+
+The pipeline below fetches an audio file from a specified URL and transcribes it. It first retrieves the audio file using `LinkContentFetcher`, then transcribes the audio into text with `LocalWhisperTranscriber`, and finally outputs the transcription text.
+
+```python
+from haystack_integrations.components.audio.whisper import LocalWhisperTranscriber
+from haystack.components.fetchers import LinkContentFetcher
+from haystack import Pipeline
+
+pipe = Pipeline()
+pipe.add_component("fetcher", LinkContentFetcher())
+pipe.add_component("transcriber", LocalWhisperTranscriber(model="tiny"))
+
+pipe.connect("fetcher", "transcriber")
+result = pipe.run(
+    data={
+        "fetcher": {
+            "urls": [
+                "https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
+            ],
+        },
+    },
+)
+print(result["transcriber"]["documents"][0].content)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)

docs-website/docs/pipeline-components/audio/remotewhispertranscriber.mdx

@@ -0,0 +1,104 @@
+---
+title: "RemoteWhisperTranscriber"
+id: remotewhispertranscriber
+slug: "/remotewhispertranscriber"
+description: "Use `RemoteWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model."
+---
+
+# RemoteWhisperTranscriber
+
+Use `RemoteWhisperTranscriber` to transcribe audio files using OpenAI's Whisper model.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | As the first component in an indexing pipeline                                                 |
+| **Mandatory init variables**           | `api_key`: An OpenAI API key. Can be set with an environment variable `OPENAI_API_KEY`.        |
+| **Mandatory run variables**            | `sources`: A list of paths or binary streams that you want to transcribe                       |
+| **Output variables**                   | `documents`: A list of documents                                                               |
+| **API reference**                      | [Whisper](/reference/integrations-whisper)                                                                         |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/whisper |
+| **Package name** | `whisper-haystack` |
+
+</div>
+
+## Overview
+
+The `RemoteWhisperTranscriber` is part of the `whisper-haystack` integration package. Install it with:
+
+```python
+pip install whisper-haystack
+```
+
+`RemoteWhisperTranscriber` works with OpenAI-compatible clients and isn't limited to just OpenAI as a provider. For example, [Groq](https://console.groq.com/docs/speech-text) offers a drop-in replacement that can be used as well. You can set the API key in one of two ways:
+
+1. Through the `api_key` initialization parameter, where the key is resolved using [Secret API](../../concepts/secret-management.mdx).
+2. By setting it in the `OPENAI_API_KEY` environment variable, which the system will use to access the key.
+
+```python
+from haystack_integrations.components.audio.whisper import RemoteWhisperTranscriber
+
+transcriber = RemoteWhisperTranscriber()
+```
+
+Additionally, the component requires the following parameters to work:
+
+- `model` specifies the Whisper model.
+- `api_base_url` specifies the OpenAI base URL and defaults to `"https://api.openai.com/v1"`. If you are using Whisper provider other than OpenAI set this parameter according to provider's documentation.
+
+See other optional parameters in our [API documentation](/reference/integrations-whisper).
+
+See the [Whisper API documentation](https://platform.openai.com/docs/guides/speech-to-text) and the official Whisper [GitHub repo](https://github.com/openai/whisper) for the supported audio formats and languages.
+
+## Usage
+
+### On its own
+
+Here’s an example of how to use `RemoteWhisperTranscriber` to transcribe a local file:
+
+```python
+import requests
+from haystack_integrations.components.audio.whisper import RemoteWhisperTranscriber
+
+response = requests.get(
+    "https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
+)
+with open("kennedy_speech.mp3", "wb") as file:
+    file.write(response.content)
+
+transcriber = RemoteWhisperTranscriber()
+transcription = transcriber.run(sources=["./kennedy_speech.mp3"])
+
+print(transcription["documents"][0].content)
+```
+
+### In a pipeline
+
+The pipeline below fetches an audio file from a specified URL and transcribes it. It first retrieves the audio file using `LinkContentFetcher`, then transcribes the audio into text with `RemoteWhisperTranscriber`, and finally outputs the transcription text.
+
+```python
+from haystack_integrations.components.audio.whisper import RemoteWhisperTranscriber
+from haystack.components.fetchers import LinkContentFetcher
+from haystack import Pipeline
+
+pipe = Pipeline()
+pipe.add_component("fetcher", LinkContentFetcher())
+pipe.add_component("transcriber", RemoteWhisperTranscriber())
+
+pipe.connect("fetcher", "transcriber")
+result = pipe.run(
+    data={
+        "fetcher": {
+            "urls": [
+                "https://ia903102.us.archive.org/19/items/100-Best--Speeches/EK_19690725_64kb.mp3",
+            ],
+        },
+    },
+)
+print(result["transcriber"]["documents"][0].content)
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Multilingual RAG from a podcast with Whisper, Qdrant and Mistral](https://haystack.deepset.ai/cookbook/multilingual_rag_podcast)

docs-website/docs/pipeline-components/builders.mdx

@@ -0,0 +1,13 @@
+---
+title: "Builders"
+id: builders
+slug: "/builders"
+---
+
+# Builders
+
+| Component                                                  | Description                                                                          |
+| --- | --- |
+| [AnswerBuilder](builders/answerbuilder.mdx)                       | Creates `GeneratedAnswer` objects from the query and the answer.                     |
+| [PromptBuilder](builders/promptbuilder.mdx)                       | Renders prompt templates with given parameters.                                      |
+| [ChatPromptBuilder](builders/chatpromptbuilder.mdx)               | PromptBuilder for chat messages.                                                     |

docs-website/docs/pipeline-components/builders/answerbuilder.mdx

@@ -0,0 +1,114 @@
+---
+title: "AnswerBuilder"
+id: answerbuilder
+slug: "/answerbuilder"
+description: "Use this component in pipelines that contain a Generator to parse its replies."
+---
+
+# AnswerBuilder
+
+Use this component in pipelines that contain a Generator to parse its replies.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | Use in pipelines (such as a RAG pipeline) after a [Generator](../generators.mdx)  component to create [`GeneratedAnswer`](../../concepts/data-classes.mdx#generatedanswer)   objects from its replies. |
+| **Mandatory run variables** | `query`: A query string  <br /> <br />`replies`: A list of strings, or a list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)   objects that are replies from a Generator |
+| **Output variables** | `answers`:  A list of `GeneratedAnswer` objects |
+| **API reference** | [Builders](/reference/builders-api) |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/answer_builder.py |
+| **Package name** | `haystack-ai` |
+
+</div>
+
+## Overview
+
+`AnswerBuilder` takes a query and the replies a Generator returns as input and parses them into `GeneratedAnswer` objects. Optionally, it also takes documents and metadata from the Generator as inputs to enrich the `GeneratedAnswer` objects.
+
+The `AnswerBuilder` works with both Chat and non-Chat Generators.
+
+The optional `pattern` parameter defines how to extract answer texts from replies. It needs to be a regular expression with a maximum of one capture group. If a capture group is present, the text matched by the capture group is used as the answer. If no capture group is present, the whole match is used as the answer. If no `pattern` is set, the whole reply is used as the answer text.
+
+The optional `reference_pattern` parameter can be set to a regular expression that parses referenced documents from the replies so that only those referenced documents are listed in the `GeneratedAnswer` objects. Haystack assumes that documents are referenced by their index in the list of input documents and that indices start at 1. For example, if you set the `reference_pattern` to _`\\[(\\d+)\\]`,_ it finds “1” in a string "This is an answer[1]". If `reference_pattern` is not set, all input documents are listed in the `GeneratedAnswer` objects.
+
+## Usage
+
+### On its own
+
+Below is an example where we’re using the `AnswerBuilder` to parse a string that could be the reply received from a Generator using a custom regular expression. Any text other than the answer will not be included in the `GeneratedAnswer` object constructed by the builder.
+
+```python
+from haystack.components.builders import AnswerBuilder
+
+builder = AnswerBuilder(pattern="Answer: (.*)")
+builder.run(
+    query="What's the answer?",
+    replies=["This is an argument. Answer: This is the answer."],
+)
+```
+
+### In a pipeline
+
+Below is an example of a RAG pipeline where we use an `AnswerBuilder` to create `GeneratedAnswer` objects from the replies returned by a Generator. In addition to the text of the reply, these objects also hold the query, the referenced docs, and metadata returned by the Generator.
+
+```python
+from haystack import Pipeline
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
+from haystack.components.builders.answer_builder import AnswerBuilder
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+from haystack.dataclasses import Document
+
+prompt_template = [
+    ChatMessage.from_system("You are a helpful assistant."),
+    ChatMessage.from_user(
+        "Given these documents, answer the question.\nDocuments:\n"
+        "{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
+        "Question: {{query}}\nAnswer:",
+    ),
+]
+
+docs = [
+    Document(content="The capital of France is Paris"),
+    Document(content="The capital of England is London"),
+]
+document_store = InMemoryDocumentStore()
+document_store.write_documents(docs)
+
+p = Pipeline()
+p.add_component(
+    instance=InMemoryBM25Retriever(document_store=document_store),
+    name="retriever",
+)
+p.add_component(
+    instance=ChatPromptBuilder(
+        template=prompt_template,
+        required_variables={"query", "documents"},
+    ),
+    name="prompt_builder",
+)
+p.add_component(
+    instance=OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
+    name="llm",
+)
+p.add_component(instance=AnswerBuilder(), name="answer_builder")
+p.connect("retriever", "prompt_builder.documents")
+p.connect("prompt_builder", "llm.messages")
+p.connect("llm.replies", "answer_builder.replies")
+p.connect("retriever", "answer_builder.documents")
+
+query = "What is the capital of France?"
+result = p.run(
+    {
+        "retriever": {"query": query},
+        "prompt_builder": {"query": query},
+        "answer_builder": {"query": query},
+    },
+)
+
+print(result)
+```

docs-website/docs/pipeline-components/builders/chatpromptbuilder.mdx

@@ -0,0 +1,477 @@
+---
+title: "ChatPromptBuilder"
+id: chatpromptbuilder
+slug: "/chatpromptbuilder"
+description: "This component constructs prompts dynamically by processing chat messages."
+---
+
+# ChatPromptBuilder
+
+This component constructs prompts dynamically by processing chat messages.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | Before a [Generator](../generators.mdx)                                                                                                         |
+| **Mandatory init variables**           | `template`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects or a special string template. Needs to be provided either during init or run. |
+| **Mandatory run variables**            | `**kwargs`: Any strings that should be used to render the prompt template. See [Variables](#variables) section for more details.             |
+| **Output variables**                   | `prompt`: A dynamically constructed prompt                                                                                                     |
+| **API reference**                      | [Builders](/reference/builders-api)                                                                                                                   |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/chat_prompt_builder.py                                         |
+| **Package name** | `haystack-ai` |
+
+</div>
+
+## Overview
+
+The `ChatPromptBuilder` component creates prompts using static or dynamic templates written in [Jinja2](https://palletsprojects.com/p/jinja/) syntax, by processing a list of chat messages or a special string template. The templates contain placeholders like `{{ variable }}` that are filled with values provided during runtime. You can use it for static prompts set at initialization or change the templates and variables dynamically while running.
+
+To use it, start by providing a list of `ChatMessage` objects or a special string as the template.
+
+[`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) is a data class that includes message content, a role (who generated the message, such as `user`, `assistant`, `system`, `tool`), and optional metadata.
+
+The builder looks for placeholders in the template and identifies the required variables. You can also list these variables manually. During runtime, the `run` method takes the template and the variables, fills in the placeholders, and returns the completed prompt. If required variables are missing. If the template is invalid, the builder raises an error.
+
+For example, you can create a simple translation prompt:
+
+```python
+template = [ChatMessage.from_user("Translate to {{ target_language }}: {{ text }}")]
+builder = ChatPromptBuilder(template=template)
+result = builder.run(target_language="French", text="Hello, how are you?")
+```
+
+Or you can also replace the template at runtime with a new one:
+
+```python
+new_template = [
+    ChatMessage.from_user("Summarize in {{ target_language }}: {{ content }}"),
+]
+result = builder.run(
+    template=new_template,
+    target_language="English",
+    content="A detailed paragraph.",
+)
+```
+
+### Variables
+
+The template variables found in the init template are used as input types for the component. If there are no `required_variables` set, all variables are considered optional by default. In this case, any missing variables are replaced with empty strings, which can lead to unintended behavior, especially in complex pipelines.
+
+Use `required_variables` and `variables` to specify the input types and required variables:
+
+- `required_variables`
+  - Defines which template variables must be provided when the component runs.
+  - If any required variable is missing, the component raises an error and halts execution.
+  - You can:
+    - Pass a list of required variable names (such as `["name"]`), or
+    - Use `"*"` to mark all variables in the template as required.
+
+- `variables`
+  - Lists all variables that can appear in the template, whether required or optional.
+  - Optional variables that aren't provided are replaced with an empty string in the rendered prompt.
+  - This allows partial prompts to be constructed without errors, unless a variable is marked as required.
+
+In the example below, only _name_ is required to run the component, while _topic_ is only an optional variable:
+
+```python
+template = [
+    ChatMessage.from_user("Hello, {{ name }}. How can I assist you with {{ topic }}?"),
+]
+
+builder = ChatPromptBuilder(
+    template=template,
+    required_variables=["name"],
+    variables=["name", "topic"],
+)
+
+result = builder.run(name="Alice")
+# Output: "Hello, Alice. How can I assist you with ?"
+```
+
+The component only waits for the required inputs before running.
+
+### Roles
+
+A [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) represents a single message in the conversation and can have one of three class methods that build the chat messages: `from_user`, `from_system`, or `from_assistant`. `from_user` messages are inputs provided by the user, such as a query or request. `from_system` messages provide context or instructions to guide the LLM’s behavior, such as setting a tone or purpose for the conversation. `from_assistant` defines the expected or actual response from the LLM.
+
+Here’s how the roles work together in a `ChatPromptBuilder`:
+
+```python
+system_message = ChatMessage.from_system(
+    "You are an assistant helping tourists in {{ language }}.",
+)
+
+user_message = ChatMessage.from_user("What are the best places to visit in {{ city }}?")
+
+assistant_message = ChatMessage.from_assistant(
+    "The best places to visit in {{ city }} include the Eiffel Tower, Louvre Museum, and Montmartre.",
+)
+```
+
+### String Templates
+
+Instead of a list of `ChatMessage` objects, you can also express the template as a special string.
+
+This template format allows you to define `ChatMessage` sequences using Jinja2 syntax. Each `{% message %}` block defines a single message with a specific role, and you can insert dynamic content using `{{ variables }}`.
+
+Compared to using a list of `ChatMessage`s, this format is more flexible and allows including structured parts like images in the templatized `ChatMessage`; to better understand this use case, check out the [multimodal example](#multimodal) in the Usage section below.
+
+#### The `insert` Tag
+
+String templates also support an `{% insert %}` tag. It is a placeholder that evaluates an expression to a `ChatMessage` or a list of `ChatMessage` objects and expands it into the prompt, so messages provided at runtime can be interleaved with literal `{% message %}` blocks. For example, you can wrap the runtime messages with a system message above and a templated user message below, then pass the messages (and any template variables) at run time:
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+template = """
+{% message role="system" %}You are a helpful assistant.{% endmessage %}
+{% insert messages %}
+{% message role="user" %}{{ query }}{% endmessage %}
+"""
+
+builder = ChatPromptBuilder(template=template)
+result = builder.run(
+    messages=[ChatMessage.from_user("Hi"), ChatMessage.from_assistant("Hello!")],
+    query="What's the weather?",
+)
+# result["prompt"] -> [system, user "Hi", assistant "Hello!", user "What's the weather?"]
+```
+
+All content types (tool calls, tool call results, images, reasoning, `name`, and `meta`) round trip without loss. A missing or empty value expands to nothing.
+
+The expression can be a plain variable (`{% insert messages %}`), a slice or index (`{% insert messages[-1:] %}`, `{% insert messages[-1] %}`), or a combination of variables (`{% insert previous + current %}`). Multiple `{% insert %}` tags can be used in a single template, so the runtime messages can be split, reordered, or repeated across different positions.
+
+### Jinja2 Time Extension
+
+`PromptBuilder` supports the Jinja2 TimeExtension, which allows you to work with datetime formats.
+
+The Time Extension provides two main features:
+
+1. A `now` tag that gives you access to the current time,
+2. Date/time formatting capabilities through Python's datetime module.
+
+To use the Jinja2 TimeExtension, you need to install a dependency with:
+
+```shell
+pip install arrow>=1.3.0
+```
+
+#### The `now` Tag
+
+The `now` tag creates a datetime object representing the current time, which you can then store in a variable:
+
+```jinja2
+{% now 'utc' as current_time %}
+The current UTC time is: {{ current_time }}
+```
+
+You can specify different timezones:
+
+```jinja2
+{% now 'America/New_York' as ny_time %}
+The time in New York is: {{ ny_time }}
+```
+
+If you don't specify a timezone, your system's local timezone will be used:
+
+```jinja2
+{% now as local_time %}
+Local time: {{ local_time }}
+```
+
+#### Date Formatting
+
+You can format the datetime objects using Python's `strftime` syntax:
+
+```jinja2
+{% now as current_time %}
+Formatted date: {{ current_time.strftime('%Y-%m-%d %H:%M:%S') }}
+```
+
+The common format codes are:
+
+- `%Y`: 4-digit year (for example, 2025)
+- `%m`: Month as a zero-padded number (01-12)
+- `%d`: Day as a zero-padded number (01-31)
+- `%H`: Hour (24-hour clock) as a zero-padded number (00-23)
+- `%M`: Minute as a zero-padded number (00-59)
+- `%S`: Second as a zero-padded number (00-59)
+
+#### Example
+
+```python
+from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+template = [
+    ChatMessage.from_user("Current date is: {% now 'UTC' %}"),
+    ChatMessage.from_assistant("Thank you for providing the date"),
+    ChatMessage.from_user("Yesterday was: {% now 'UTC' - 'days=1' %}"),
+]
+builder = ChatPromptBuilder(template=template)
+
+result = builder.run()["prompt"]
+```
+
+## Usage
+
+### On its own
+
+#### With static template
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+template = [
+    ChatMessage.from_user(
+        "Translate to {{ target_language }}. Context: {{ snippet }}; Translation:",
+    ),
+]
+builder = ChatPromptBuilder(template=template)
+builder.run(target_language="spanish", snippet="I can't speak spanish.")
+```
+
+#### With special string template
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+template = """
+{% message role="user" %}
+Hello, my name is {{name}}!
+{% endmessage %}
+"""
+
+builder = ChatPromptBuilder(template=template)
+result = builder.run(name="John")
+
+assert result["prompt"] == [ChatMessage.from_user("Hello, my name is John!")]
+```
+
+#### Specifying name and meta in a ChatMessage
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+template = """
+{% message role="user" name="John" meta={"key": "value"} %}
+Hello from {{country}}!
+{% endmessage %}
+"""
+
+builder = ChatPromptBuilder(template=template)
+result = builder.run(country="Italy")
+assert result["prompt"] == [
+    ChatMessage.from_user("Hello from Italy!", name="John", meta={"key": "value"}),
+]
+```
+
+#### Multiple ChatMessages with different roles
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+template = """
+{% message role="system" %}
+You are a {{adjective}} assistant.
+{% endmessage %}
+
+{% message role="user" %}
+Hello, my name is {{name}}!
+{% endmessage %}
+
+{% message role="assistant" %}
+Hello, {{name}}! How can I help you today?
+{% endmessage %}
+"""
+
+builder = ChatPromptBuilder(template=template)
+result = builder.run(name="John", adjective="helpful")
+assert result["prompt"] == [
+    ChatMessage.from_system("You are a helpful assistant."),
+    ChatMessage.from_user("Hello, my name is John!"),
+    ChatMessage.from_assistant("Hello, John! How can I help you today?"),
+]
+```
+
+#### Overriding static template at runtime
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+template = [
+    ChatMessage.from_user(
+        "Translate to {{ target_language }}. Context: {{ snippet }}; Translation:",
+    ),
+]
+builder = ChatPromptBuilder(template=template)
+builder.run(target_language="spanish", snippet="I can't speak spanish.")
+
+summary_template = [
+    ChatMessage.from_user(
+        "Translate to {{ target_language }} and summarize. Context: {{ snippet }}; Summary:",
+    ),
+]
+builder.run(
+    target_language="spanish",
+    snippet="I can't speak spanish.",
+    template=summary_template,
+)
+```
+
+#### Multimodal
+
+The `| templatize_part` filter in the example below tells the template engine to insert structured (non-text) objects, such as images, into the message content. These are treated differently from plain text and are rendered as special content parts in the final `ChatMessage`.
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage, ImageContent
+
+template = """
+{% message role="user" meta={"key": "value"}%}
+Hello! I am {{user_name}}. What's the difference between the following images?
+{% for image in images %}
+{{ image | templatize_part }}
+{% endfor %}
+{% endmessage %}
+"""
+builder = ChatPromptBuilder(template=template)
+images = [
+    ImageContent.from_file_path("apple.jpg"),
+    ImageContent.from_file_path("kiwi.jpg"),
+]
+result = builder.run(user_name="John", images=images)
+
+assert result["prompt"] == [
+    ChatMessage.from_user(
+        content_parts=[
+            "Hello! I am John. What's the difference between the following images?",
+            *images,
+        ],
+        meta={"key": "value"},
+    ),
+]
+```
+
+### In a pipeline
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+from haystack.utils import Secret
+
+# no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+llm = OpenAIChatGenerator()
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("llm", llm)
+pipe.connect("prompt_builder.prompt", "llm.messages")
+
+location = "Berlin"
+language = "English"
+system_message = ChatMessage.from_system(
+    "You are an assistant giving information to tourists in {{language}}",
+)
+messages = [system_message, ChatMessage.from_user("Tell me about {{location}}")]
+
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location, "language": language},
+            "template": messages,
+        },
+    },
+)
+print(res)
+```
+
+Then, you could ask about the weather forecast for the said location. The `ChatPromptBuilder` fills in the template with the new `day_count` variable and passes it to an LLM once again:
+
+```python
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.dataclasses import ChatMessage
+from haystack import Pipeline
+from haystack.utils import Secret
+
+# no parameter init, we don't use any runtime template variables
+prompt_builder = ChatPromptBuilder()
+llm = OpenAIChatGenerator()
+
+pipe = Pipeline()
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("llm", llm)
+pipe.connect("prompt_builder.prompt", "llm.messages")
+
+location = "Berlin"
+
+messages = [
+    system_message,
+    ChatMessage.from_user(
+        "What's the weather forecast for {{location}} in the next {{day_count}} days?",
+    ),
+]
+res = pipe.run(
+    data={
+        "prompt_builder": {
+            "template_variables": {"location": location, "day_count": "5"},
+            "template": messages,
+        },
+    },
+)
+
+print(res)
+```
+
+### In YAML
+
+This is the YAML representation of the pipeline shown above. It dynamically constructs a prompt and generates an answer using a chat model.
+
+```yaml
+components:
+  llm:
+    init_parameters:
+      api_base_url: null
+      api_key:
+        env_vars:
+        - OPENAI_API_KEY
+        strict: true
+        type: env_var
+      generation_kwargs: {}
+      http_client_kwargs: null
+      max_retries: null
+      model: gpt-4o-mini
+      organization: null
+      streaming_callback: null
+      timeout: null
+      tools: null
+      tools_strict: false
+    type: haystack.components.generators.chat.openai.OpenAIChatGenerator
+  prompt_builder:
+    init_parameters:
+      required_variables: null
+      template: null
+      variables: null
+    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
+connection_type_validation: true
+connections:
+- receiver: llm.messages
+  sender: prompt_builder.prompt
+max_runs_per_component: 100
+metadata: {}
+```
+
+## Additional References
+
+🧑‍🍳 Cookbook: [Advanced Prompt Customization for Anthropic](https://haystack.deepset.ai/cookbook/prompt_customization_for_anthropic)

docs-website/docs/pipeline-components/builders/promptbuilder.mdx

@@ -0,0 +1,306 @@
+---
+title: "PromptBuilder"
+id: promptbuilder
+slug: "/promptbuilder"
+description: "Use this component in pipelines before a Generator to render a prompt template and fill in variable values."
+---
+
+# PromptBuilder
+
+Use this component in pipelines before a Generator to render a prompt template and fill in variable values.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | In a querying pipeline, before a [Generator](../generators.mdx)                                                                      |
+| **Mandatory init variables**           | `template`: A prompt template string that uses Jinja2 syntax                                                                        |
+| **Mandatory run variables**            | `**kwargs`: Any strings that should be used to render the prompt template. See [Variables](#variables)  section for more details. |
+| **Output variables**                   | `prompt`: A string that represents the rendered prompt template                                                                     |
+| **API reference**                      | [Builders](/reference/builders-api)                                                                                                        |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack/blob/main/haystack/components/builders/prompt_builder.py                                   |
+| **Package name** | `haystack-ai` |
+
+</div>
+
+## Overview
+
+`PromptBuilder` is initialized with a prompt template and renders it by filling in parameters passed through keyword arguments, `kwargs`. With `kwargs`, you can pass a variable number of keyword arguments so that any variable used in the prompt template can be specified with the desired value. Values for all variables appearing in the prompt template need to be provided through the `kwargs`.
+
+The template that is provided to the `PromptBuilder` during initialization needs to conform to the [Jinja2](https://palletsprojects.com/p/jinja/) template language.
+
+### Variables
+
+The template variables found in the init template are used as input types for the component. If there are no `required_variables` set, all variables are considered optional by default. In this case, any missing variables are replaced with empty strings, which can lead to unintended behavior, especially in complex pipelines.
+
+Use `required_variables` and `variables` to specify the input types and required variables:
+
+- `required_variables`
+  - Defines which template variables must be provided when the component runs.
+  - If any required variable is missing, the component raises an error and halts execution.
+  - You can:
+    - Pass a list of required variable names (such as `["query"]`), or
+    - Use `"*"` to mark all variables in the template as required.
+
+- `variables`
+  - Lists all variables that can appear in the template, whether required or optional.
+  - Optional variables that aren't provided are replaced with an empty string in the rendered prompt.
+  - This allows partial prompts to be constructed without errors, unless a variable is marked as required.
+
+```python
+from haystack.components.builders import PromptBuilder
+
+# All variables optional (default to empty string)
+builder = PromptBuilder(
+    template="Hello {{name}}! {{greeting}}",
+    required_variables=[],  # or omit this parameter entirely
+)
+
+# Some variables required
+builder = PromptBuilder(
+    template="Hello {{name}}! {{greeting}}",
+    required_variables=["name"],  # 'greeting' remains optional
+)
+```
+
+The component only waits for the required inputs before running.
+
+### Jinja2 Time Extension
+
+`PromptBuilder` supports the Jinja2 TimeExtension, which allows you to work with datetime formats.
+
+The Time Extension provides two main features:
+
+1. A `now` tag that gives you access to the current time,
+2. Date/time formatting capabilities through Python's datetime module.
+
+To use the Jinja2 TimeExtension, you need to install a dependency with:
+
+```shell
+pip install arrow>=1.3.0
+```
+
+#### The `now` Tag
+
+The `now` tag creates a datetime object representing the current time, which you can then store in a variable:
+
+```jinja2
+{% now 'utc' as current_time %}
+The current UTC time is: {{ current_time }}
+```
+
+You can specify different timezones:
+
+```jinja2
+{% now 'America/New_York' as ny_time %}
+The time in New York is: {{ ny_time }}
+```
+
+If you don't specify a timezone, your system's local timezone will be used:
+
+```jinja2
+{% now as local_time %}
+Local time: {{ local_time }}
+```
+
+#### Date Formatting
+
+You can format the datetime objects using Python's `strftime` syntax:
+
+```jinja2
+{% now as current_time %}
+Formatted date: {{ current_time.strftime('%Y-%m-%d %H:%M:%S') }}
+```
+
+The common format codes are:
+
+- `%Y`: 4-digit year (for example, 2025)
+- `%m`: Month as a zero-padded number (01-12)
+- `%d`: Day as a zero-padded number (01-31)
+- `%H`: Hour (24-hour clock) as a zero-padded number (00-23)
+- `%M`: Minute as a zero-padded number (00-59)
+- `%S`: Second as a zero-padded number (00-59)
+
+#### Example
+
+```python
+from haystack.components.builders import PromptBuilder
+
+# Define template using Jinja-style formatting
+template = """
+Current date is: {% now 'UTC' %}
+Thank you for providing the date
+Yesterday was: {% now 'UTC' - 'days=1' %}
+"""
+
+builder = PromptBuilder(template=template)
+
+result = builder.run()["prompt"]
+```
+
+## Usage
+
+### On its own
+
+Below is an example of using the `PromptBuilder` to render a prompt template and fill it with `target_language` and `snippet`. The PromptBuilder returns a prompt with the string `Translate the following context to spanish. Context: I can't speak spanish.; Translation:`.
+
+```python
+from haystack.components.builders import PromptBuilder
+
+template = "Translate the following context to {{ target_language }}. Context: {{ snippet }}; Translation:"
+builder = PromptBuilder(template=template)
+builder.run(target_language="spanish", snippet="I can't speak spanish.")
+```
+
+### In a pipeline
+
+Below is an example of a RAG pipeline where we use a `PromptBuilder` to render a custom prompt template and fill it with the contents of retrieved documents and a query. The rendered prompt is then sent to a Generator.
+
+```python
+from haystack import Pipeline, Document
+from haystack.utils import Secret
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.builders.prompt_builder import PromptBuilder
+
+# in a real world use case documents could come from a retriever, web, or any other source
+documents = [
+    Document(content="Joe lives in Berlin"),
+    Document(content="Joe is a software engineer"),
+]
+prompt_template = """
+    Given these documents, answer the question.\nDocuments:
+    {% for doc in documents %}
+        {{ doc.content }}
+    {% endfor %}
+
+    \nQuestion: {{query}}
+    \nAnswer:
+    """
+p = Pipeline()
+p.add_component(instance=PromptBuilder(template=prompt_template), name="prompt_builder")
+p.add_component(
+    instance=OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
+    name="llm",
+)
+p.connect("prompt_builder", "llm")
+
+question = "Where does Joe live?"
+result = p.run({"prompt_builder": {"documents": documents, "query": question}})
+print(result)
+```
+
+#### Changing the template at runtime (Prompt Engineering)
+
+`PromptBuilder` allows you to switch the prompt template of an existing pipeline. The example below builds on top of the existing pipeline in the previous section. We are invoking the existing pipeline with a new prompt template:
+
+```python
+documents = [
+    Document(content="Joe lives in Berlin", meta={"name": "doc1"}),
+    Document(content="Joe is a software engineer", meta={"name": "doc1"}),
+]
+new_template = """
+    You are a helpful assistant.
+    Given these documents, answer the question.
+    Documents:
+    {% for doc in documents %}
+        Document {{ loop.index }}:
+        Document name: {{ doc.meta['name'] }}
+        {{ doc.content }}
+    {% endfor %}
+
+    Question: {{ query }}
+    Answer:
+    """
+p.run(
+    {
+        "prompt_builder": {
+            "documents": documents,
+            "query": question,
+            "template": new_template,
+        },
+    },
+)
+```
+
+If you want to use different variables during prompt engineering than in the default template, you can do so by setting `PromptBuilder`'s variables init parameter accordingly.
+
+#### Overwriting variables at runtime
+
+In case you want to overwrite the values of variables, you can use `template_variables` during runtime, as shown below:
+
+```python
+language_template = """
+    You are a helpful assistant.
+    Given these documents, answer the question.
+    Documents:
+    {% for doc in documents %}
+        Document {{ loop.index }}:
+        Document name: {{ doc.meta['name'] }}
+        {{ doc.content }}
+    {% endfor %}
+
+    Question: {{ query }}
+    Please provide your answer in {{ answer_language | default('English') }}
+    Answer:
+    """
+p.run(
+    {
+        "prompt_builder": {
+            "documents": documents,
+            "query": question,
+            "template": language_template,
+            "template_variables": {"answer_language": "German"},
+        },
+    },
+)
+```
+
+Note that `language_template` introduces `answer_language` variable which is not bound to any pipeline variable. If not set otherwise, it would use its default value, "English". In this example, we overwrite its value to "German".
+The `template_variables` allows you to overwrite pipeline variables (such as documents) as well.
+
+### In YAML
+
+This is the YAML representation of the RAG pipeline shown above. It renders a custom prompt template by filling it with the contents of retrieved documents and a query, then sends the rendered prompt to a generator.
+
+```yaml
+components:
+  llm:
+    init_parameters:
+      api_base_url: null
+      api_key:
+        env_vars:
+        - OPENAI_API_KEY
+        strict: true
+        type: env_var
+      generation_kwargs: {}
+      http_client_kwargs: null
+      max_retries: null
+      model: gpt-5-mini
+      organization: null
+      streaming_callback: null
+      timeout: null
+      tools: null
+      tools_strict: false
+    type: haystack.components.generators.chat.openai.OpenAIChatGenerator
+  prompt_builder:
+    init_parameters:
+      required_variables: '*'
+      template: "\n    Given these documents, answer the question.\nDocuments:\n \
+        \   {% for doc in documents %}\n        {{ doc.content }}\n    {% endfor %}\n\
+        \n    \nQuestion: {{query}}\n    \nAnswer:\n    "
+      variables: null
+    type: haystack.components.builders.prompt_builder.PromptBuilder
+connection_type_validation: true
+connections:
+- receiver: llm.messages
+  sender: prompt_builder.prompt
+max_runs_per_component: 100
+metadata: {}
+```
+
+## Additional References
+
+🧑‍🍳 Cookbooks:
+
+- [Advanced Prompt Customization for Anthropic](https://haystack.deepset.ai/cookbook/prompt_customization_for_anthropic)
+- [Prompt Optimization with DSPy](https://haystack.deepset.ai/cookbook/prompt_optimization_with_dspy)

docs-website/docs/pipeline-components/caching/cachechecker.mdx

@@ -0,0 +1,106 @@
+---
+title: "CacheChecker"
+id: cachechecker
+slug: "/cachechecker"
+description: "This component checks for the presence of documents in a Document Store based on a specified cache field."
+---
+
+# CacheChecker
+
+This component checks for the presence of documents in a Document Store based on a specified cache field.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | Flexible |
+| **Mandatory init variables** | `document_store`: A Document Store instance  <br /> <br />`cache_field`: Name of the document's metadata field |
+| **Mandatory run variables** | `items`: A list of values associated with the `cache_field` in documents |
+| **Output variables** | `hits`: A list of documents that were found with the specified value in cache  <br /> <br />`misses`: A list of values that could not be found |
+| **API reference** | [Caching](/reference/caching-api) |
+| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/caching/cache_checker.py |
+| **Package name** | `haystack-ai` |
+
+</div>
+
+## Overview
+
+`CacheChecker` checks if a Document Store contains any document with a value in the `cache_field` that matches any of the values provided in the `items` input variable. It returns a dictionary with two keys: `"hits"` and `"misses"`. The values are lists of documents that were found in the cache and items that were not, respectively.
+
+## Usage
+
+### On its own
+
+```python
+from haystack.components.caching import CacheChecker
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+
+my_doc_store = InMemoryDocumentStore()
+
+# For URL-based caching
+cache_checker = CacheChecker(document_store=my_doc_store, cache_field="url")
+cache_check_results = cache_checker.run(
+    items=[
+        "https://example.com/resource",
+        "https://another_example.com/other_resources",
+    ],
+)
+print(
+    cache_check_results["hits"],
+)  # List of Documents that were found in the cache: all of these have 'url': <one of the above> in the metadata
+print(
+    cache_check_results["misses"],
+)  # URLs that were not found in the cache, like ["https://example.com/resource"]
+
+# For caching based on a custom identifier
+cache_checker = CacheChecker(document_store=my_doc_store, cache_field="metadata_field")
+cache_check_results = cache_checker.run(items=["12345", "ABCDE"])
+print(
+    cache_check_results["hits"],
+)  # Documents that were found in the cache: all of these have 'metadata_field': <one of the above> in the metadata
+print(
+    cache_check_results["misses"],
+)  # Values that were not found in the cache, like: ["ABCDE"]
+```
+
+### In a pipeline
+
+```python
+from haystack import Pipeline
+from haystack.components.converters import TextFileToDocument
+from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
+from haystack.components.writers import DocumentWriter
+from haystack.components.caching import CacheChecker
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+
+pipeline = Pipeline()
+document_store = InMemoryDocumentStore()
+pipeline.add_component(
+    instance=CacheChecker(document_store, cache_field="meta.file_path"),
+    name="cache_checker",
+)
+pipeline.add_component(instance=TextFileToDocument(), name="text_file_converter")
+pipeline.add_component(instance=DocumentCleaner(), name="cleaner")
+pipeline.add_component(
+    instance=DocumentSplitter(split_by="sentence", split_length=250, split_overlap=30),
+    name="splitter",
+)
+pipeline.add_component(
+    instance=DocumentWriter(document_store=document_store),
+    name="writer",
+)
+pipeline.connect("cache_checker.misses", "text_file_converter.sources")
+pipeline.connect("text_file_converter.documents", "cleaner.documents")
+pipeline.connect("cleaner.documents", "splitter.documents")
+pipeline.connect("splitter.documents", "writer.documents")
+
+pipeline.draw("pipeline.png")
+
+# Take the current directory as input and run the pipeline
+result = pipeline.run({"cache_checker": {"items": ["code_of_conduct_1.txt"]}})
+print(result)
+
+# The second execution skips the files that were already processed
+result = pipeline.run({"cache_checker": {"items": ["code_of_conduct_1.txt"]}})
+print(result)
+```

docs-website/docs/pipeline-components/classifiers.mdx

@@ -0,0 +1,15 @@
+---
+title: "Classifiers"
+id: classifiers
+slug: "/classifiers"
+description: "Use Classifiers to classify your documents by specific traits and update the metadata."
+---
+
+# Classifiers
+
+Use Classifiers to classify your documents by specific traits and update the metadata.
+
+| Classifier                                                                           | Description                                           |
+| --- | --- |
+| [DocumentLanguageClassifier](classifiers/documentlanguageclassifier.mdx)                       | Classify documents by language.                       |
+| [TransformersZeroShotDocumentClassifier](classifiers/transformerszeroshotdocumentclassifier.mdx) | Classify the documents based on the provided labels. |

docs-website/docs/pipeline-components/classifiers/documentlanguageclassifier.mdx

@@ -0,0 +1,127 @@
+---
+title: "DocumentLanguageClassifier"
+id: documentlanguageclassifier
+slug: "/documentlanguageclassifier"
+description: "Use this component to classify documents by language and add language information to metadata."
+---
+
+# DocumentLanguageClassifier
+
+Use this component to classify documents by language and add language information to metadata.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | Before [`MetadataRouter`](../routers/metadatarouter.mdx)                                                                    |
+| **Mandatory run variables**            | `documents`:  A list of documents                                                                                  |
+| **Output variables**                   | `documents`:  A list of documents                                                                                  |
+| **API reference**                      | [Langdetect](/reference/integrations-langdetect)                                                                                 |
+| **GitHub link**                        | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langdetect |
+| **Package name** | `langdetect-haystack` |
+
+</div>
+
+## Overview
+
+`DocumentLanguageClassifier` classifies the language of documents and adds the detected language to their metadata. If a document's text does not match any of the languages specified at initialization, it is classified as "unmatched". By default, the classifier classifies for English (”en”) documents, with the rest being classified as “unmatched”.
+
+The set of supported languages can be specified in the init method with the `languages` variable, using ISO codes.
+
+To route your documents to various branches of the pipeline based on the language, use `MetadataRouter` component right after `DocumentLanguageClassifier`.
+
+For classifying and then routing plain text using the same logic, use the `TextLanguageRouter` component instead.
+
+## Usage
+
+Install the `langdetect-haystack` package to use the `DocumentLanguageClassifier` component:
+
+```shell
+pip install langdetect-haystack
+```
+
+### On its own
+
+Below, we are using the `DocumentLanguageClassifier` to classify English and German documents:
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+```python
+from haystack_integrations.components.classifiers.langdetect import (
+    DocumentLanguageClassifier,
+)
+from haystack import Document
+
+documents = [
+    Document(content="Mein Name ist Jean und ich wohne in Paris."),
+    Document(content="Mein Name ist Mark und ich wohne in Berlin."),
+    Document(content="Mein Name ist Giorgio und ich wohne in Rome."),
+    Document(content="My name is Pierre and I live in Paris"),
+    Document(content="My name is Paul and I live in Berlin."),
+    Document(content="My name is Alessia and I live in Rome."),
+]
+
+document_classifier = DocumentLanguageClassifier(languages=["en", "de"])
+document_classifier.run(documents=documents)
+```
+
+### In a pipeline
+
+Below, we are using the `DocumentLanguageClassifier` in an indexing pipeline that indexes English and German documents into two difference indexes in an `InMemoryDocumentStore`, using embedding models for each language.
+
+```python
+from haystack import Pipeline
+from haystack import Document
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack_integrations.components.classifiers.langdetect import (
+    DocumentLanguageClassifier,
+)
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersDocumentEmbedder,
+)
+from haystack.components.writers import DocumentWriter
+from haystack.components.routers import MetadataRouter
+
+document_store_en = InMemoryDocumentStore()
+document_store_de = InMemoryDocumentStore()
+
+document_classifier = DocumentLanguageClassifier(languages=["en", "de"])
+metadata_router = MetadataRouter(
+    rules={"en": {"language": {"$eq": "en"}}, "de": {"language": {"$eq": "de"}}},
+)
+english_embedder = SentenceTransformersDocumentEmbedder()
+german_embedder = SentenceTransformersDocumentEmbedder(
+    model="PM-AI/bi-encoder_msmarco_bert-base_german",
+)
+en_writer = DocumentWriter(document_store=document_store_en)
+de_writer = DocumentWriter(document_store=document_store_de)
+
+indexing_pipeline = Pipeline()
+indexing_pipeline.add_component(document_classifier, name="document_classifier")
+indexing_pipeline.add_component(metadata_router, name="metadata_router")
+indexing_pipeline.add_component(english_embedder, name="english_embedder")
+indexing_pipeline.add_component(german_embedder, name="german_embedder")
+indexing_pipeline.add_component(en_writer, name="en_writer")
+indexing_pipeline.add_component(de_writer, name="de_writer")
+
+indexing_pipeline.connect("document_classifier.documents", "metadata_router.documents")
+indexing_pipeline.connect("metadata_router.en", "english_embedder.documents")
+indexing_pipeline.connect("metadata_router.de", "german_embedder.documents")
+indexing_pipeline.connect("english_embedder", "en_writer")
+indexing_pipeline.connect("german_embedder", "de_writer")
+
+indexing_pipeline.run(
+    {
+        "document_classifier": {
+            "documents": [
+                Document(content="This is an English sentence."),
+                Document(content="Dies ist ein deutscher Satz."),
+            ],
+        },
+    },
+)
+```

docs-website/docs/pipeline-components/classifiers/transformerszeroshotdocumentclassifier.mdx

@@ -0,0 +1,115 @@
+---
+title: "TransformersZeroShotDocumentClassifier"
+id: transformerszeroshotdocumentclassifier
+slug: "/transformerszeroshotdocumentclassifier"
+description: "Classifies the documents based on the provided labels and adds them to their metadata."
+---
+
+# TransformersZeroShotDocumentClassifier
+
+Classifies the documents based on the provided labels and adds them to their metadata.
+
+<div className="key-value-table">
+
+|  |  |
+| --- | --- |
+| **Most common position in a pipeline** | Before a [MetadataRouter](../routers/metadatarouter.mdx) |
+| **Mandatory init variables** | `model`: The name or path of a Hugging Face model for zero shot document classification  <br /> <br />`labels`: The set of possible class labels to classify each document into, for example, [`positive`, `negative`]. The labels depend on the selected model. |
+| **Mandatory run variables** | `documents`: A list of documents to classify |
+| **Output variables** | `documents`: A list of processed documents with an added `classification` metadata field |
+| **API reference** | [Transformers](/reference/integrations-transformers) |
+| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/transformers |
+| **Package name** | `transformers-haystack` |
+
+</div>
+
+## Overview
+
+The `TransformersZeroShotDocumentClassifier` component performs zero-shot classification of documents based on the labels that you set and adds the predicted label to their metadata.
+
+The component uses a Hugging Face pipeline for zero-shot classification.
+To initialize the component, provide the model and the set of labels to be used for categorization.
+You can additionally configure the component to allow multiple labels to be true with the `multi_label` boolean set to True.
+
+Classification is run on the document's content field by default. If you want it to run on another field, set the`classification_field` to one of the document's metadata fields.
+
+The classification results are stored in the `classification` dictionary within each document's metadata. If `multi_label` is set to `True`, you will find the scores for each label under the `details` key within the `classification` dictionary.
+
+Available models for the task of zero-shot-classification are:
+    - `valhalla/distilbart-mnli-12-3`
+    - `cross-encoder/nli-distilroberta-base`
+    - `cross-encoder/nli-deberta-v3-xsmall`
+
+## Usage
+
+Install the `transformers-haystack` package to use the `TransformersZeroShotDocumentClassifier`:
+
+```shell
+pip install transformers-haystack
+```
+
+### On its own
+
+```python
+from haystack import Document
+from haystack_integrations.components.classifiers.transformers import (
+    TransformersZeroShotDocumentClassifier,
+)
+
+documents = [
+    Document(id="0", content="Cats don't get teeth cavities."),
+    Document(id="1", content="Cucumbers can be grown in water."),
+]
+
+document_classifier = TransformersZeroShotDocumentClassifier(
+    model="cross-encoder/nli-deberta-v3-xsmall",
+    labels=["animals", "food"],
+)
+
+document_classifier.run(documents=documents)
+```
+
+### In a pipeline
+
+The following is a pipeline that classifies documents based on predefined classification labels
+retrieved from a search pipeline:
+
+```python
+from haystack import Document
+from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.core.pipeline import Pipeline
+from haystack_integrations.components.classifiers.transformers import (
+    TransformersZeroShotDocumentClassifier,
+)
+
+documents = [
+    Document(id="0", content="Today was a nice day!"),
+    Document(id="1", content="Yesterday was a bad day!"),
+]
+
+document_store = InMemoryDocumentStore()
+retriever = InMemoryBM25Retriever(document_store=document_store)
+document_classifier = TransformersZeroShotDocumentClassifier(
+    model="cross-encoder/nli-deberta-v3-xsmall",
+    labels=["positive", "negative"],
+)
+
+document_store.write_documents(documents)
+
+pipeline = Pipeline()
+pipeline.add_component(name="retriever", instance=retriever)
+pipeline.add_component(name="document_classifier", instance=document_classifier)
+pipeline.connect("retriever", "document_classifier")
+
+queries = ["How was your day today?", "How was your day yesterday?"]
+expected_predictions = ["positive", "negative"]
+
+for idx, query in enumerate(queries):
+    result = pipeline.run({"retriever": {"query": query, "top_k": 1}})
+    classified_docs = result["document_classifier"]["documents"]
+    assert classified_docs[0].id == str(idx)
+    assert (
+        classified_docs[0].meta["classification"]["label"] == expected_predictions[idx]
+    )
+```