chore: import upstream snapshot with attribution

docs-website/docs/concepts/pipelines/creating-pipelines.mdx

@@ -0,0 +1,258 @@
+---
+title: "Creating Pipelines"
+id: creating-pipelines
+slug: "/creating-pipelines"
+description: "Learn the general principles of creating a pipeline."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Creating Pipelines
+
+Learn the general principles of creating a pipeline.
+
+You can use these instructions to create both indexing and query pipelines.
+
+This task uses an example of a semantic document search pipeline.
+
+## Prerequisites
+
+For each component you want to use in your pipeline, you must know the names of its input and output. You can check them on the documentation page for a specific component or in the component's `run()` method. For more information, see [Components: Input and Output](../components.mdx#input-and-output).
+
+## Steps to Create a Pipeline
+
+### 1\. Import dependencies
+
+Import all the dependencies, like pipeline, documents, Document Store, and all the components you want to use in your pipeline.
+For example, to create a semantic document search pipelines, you need the `Document` object, the pipeline, the Document Store, Embedders, and a Retriever:
+
+The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:
+
+```shell
+pip install sentence-transformers-haystack
+```
+
+```python
+from haystack import Document, Pipeline
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack_integrations.components.embedders.sentence_transformers import (
+    SentenceTransformersTextEmbedder,
+)
+from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
+```
+
+### 2\. Initialize components
+
+Initialize the components, passing any parameters you want to configure:
+
+```python
+document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")
+text_embedder = SentenceTransformersTextEmbedder()
+retriever = InMemoryEmbeddingRetriever(document_store=document_store)
+```
+
+### 3\. Create the pipeline
+
+```python
+query_pipeline = Pipeline()
+```
+
+### 4\. Add components
+
+Add components to the pipeline one by one. The order in which you do this doesn't matter:
+
+```python
+query_pipeline.add_component("component_name", component_type)
+
+# Here is an example of how you'd add the components initialized in step 2 above:
+query_pipeline.add_component("text_embedder", text_embedder)
+query_pipeline.add_component("retriever", retriever)
+
+# You could also add components without initializing them before:
+query_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
+query_pipeline.add_component(
+    "retriever",
+    InMemoryEmbeddingRetriever(document_store=document_store),
+)
+```
+
+### 5\. Connect components
+
+Connect the components by indicating which output of a component should be connected to the input of the next component. If a component has only one input or output and the connection is obvious, you can just pass the component name without specifying the input or output.
+
+To understand what inputs are expected to run your pipeline, use an `.inputs()` pipeline function. See a detailed examples in the [Pipeline Inputs](#pipeline-inputs) section below.
+
+Here's a more visual explanation within the code:
+
+```python
+# This is the syntax to connect components. Here you're connecting output1 of component1 to input1 of component2:
+pipeline.connect("component1.output1", "component2.input1")
+
+# If both components have only one output and input, you can just pass their names:
+pipeline.connect("component1", "component2")
+
+# If one of the components has only one output but the other has multiple inputs,
+# you can pass just the name of the component with a single output, but for the component with
+# multiple inputs, you must specify which input you want to connect
+
+# Here, component1 has only one output, but component2 has multiple inputs:
+pipeline.connect("component1", "component2.input1")
+
+# And here's how it should look like for the semantic document search pipeline we're using as an example:
+pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
+# Because the InMemoryEmbeddingRetriever only has one input, this is also correct:
+pipeline.connect("text_embedder.embedding", "retriever")
+```
+
+You need to link all the components together, connecting them gradually in pairs. Here's an explicit example for the pipeline we're assembling:
+
+```python
+# Imagine this pipeline has four components: text_embedder, retriever, prompt_builder and llm.
+# Here's how you would connect them into a pipeline:
+
+query_pipeline.connect("text_embedder.embedding", "retriever")
+query_pipeline.connect("retriever", "prompt_builder.documents")
+query_pipeline.connect("prompt_builder", "llm")
+```
+
+### 6\. Run the pipeline
+
+Wait for the pipeline to validate the components and connections. If everything is OK, you can now run the pipeline. `Pipeline.run()` can be called in two ways, either passing a dictionary of the component names and their inputs, or by directly passing just the inputs. When passed directly, the pipeline resolves inputs to the correct components.
+
+```python
+# Here's one way of calling the run() method
+results = pipeline.run({"component1": {"input1_value": value1, "input2_value": value2}})
+
+# The inputs can also be passed directly without specifying component names
+results = pipeline.run({"input1_value": value1, "input2_value": value2})
+
+# This is how you'd run the semantic document search pipeline we're using as an example:
+query = "Here comes the query text"
+results = query_pipeline.run({"text_embedder": {"text": query}})
+```
+
+## Pipeline Inputs
+
+If you need to understand what component inputs are expected to run your pipeline, Haystack features a useful pipeline function `.inputs()` that lists all the required inputs for the components.
+
+This is how it works:
+
+```python
+# A short pipeline example that converts webpages into documents
+from haystack import Pipeline
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.fetchers import LinkContentFetcher
+from haystack.components.converters import HTMLToDocument
+from haystack.components.writers import DocumentWriter
+
+document_store = InMemoryDocumentStore()
+fetcher = LinkContentFetcher()
+converter = HTMLToDocument()
+writer = DocumentWriter(document_store=document_store)
+
+pipeline = Pipeline()
+pipeline.add_component(instance=fetcher, name="fetcher")
+pipeline.add_component(instance=converter, name="converter")
+pipeline.add_component(instance=writer, name="writer")
+
+pipeline.connect("fetcher.streams", "converter.sources")
+pipeline.connect("converter.documents", "writer.documents")
+
+# Requesting a list of required inputs
+pipeline.inputs()
+
+# {'fetcher': {'urls': {'type': typing.List[str], 'is_mandatory': True}},
+# 'converter': {'meta': {'type': typing.Union[typing.Dict[str, typing.Any], typing.List[typing.Dict[str, typing.Any]], NoneType],
+# 'is_mandatory': False,
+# 'default_value': None},
+# 'extraction_kwargs': {'type': typing.Optional[typing.Dict[str, typing.Any]],
+# 'is_mandatory': False,
+# 'default_value': None}},
+# 'writer': {'policy': {'type': typing.Optional[haystack.document_stores.types.policy.DuplicatePolicy],
+# 'is_mandatory': False,
+# 'default_value': None}}}
+```
+
+From the above response, you can see that the `urls` input is mandatory for `LinkContentFetcher`. This is how you would then run this pipeline:
+
+```python
+pipeline.run(
+    data={"fetcher": {"urls": ["https://docs.haystack.deepset.ai/docs/pipelines"]}},
+)
+```
+
+## Example
+
+The following example walks you through creating a RAG pipeline.
+
+```python
+# import necessary dependencies
+from haystack import Pipeline, Document
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.components.builders import ChatPromptBuilder
+from haystack.utils import Secret
+from haystack.dataclasses import ChatMessage
+
+# create a document store and write documents to it
+document_store = InMemoryDocumentStore()
+document_store.write_documents(
+    [
+        Document(content="My name is Jean and I live in Paris."),
+        Document(content="My name is Mark and I live in Berlin."),
+        Document(content="My name is Giorgio and I live in Rome."),
+    ],
+)
+
+# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer.
+prompt_template = [
+    ChatMessage.from_system(
+        """
+        Given these documents, answer the question.
+        Documents:
+        {% for doc in documents %}
+            {{ doc.content }}
+        {% endfor %}
+        Question:
+        """,
+    ),
+    ChatMessage.from_user("{{question}}"),
+    ChatMessage.from_system("Answer:"),
+]
+
+# create the components adding the necessary parameters
+retriever = InMemoryBM25Retriever(document_store=document_store)
+prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
+llm = OpenAIChatGenerator(
+    api_key=Secret.from_env_var("OPENAI_API_KEY"),
+    model="gpt-4o-mini",
+)
+
+# Create the pipeline and add the components to it. The order doesn't matter.
+# At this stage, the Pipeline validates the components without running them yet.
+rag_pipeline = Pipeline()
+rag_pipeline.add_component("retriever", retriever)
+rag_pipeline.add_component("prompt_builder", prompt_builder)
+rag_pipeline.add_component("llm", llm)
+
+# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name").
+rag_pipeline.connect("retriever", "prompt_builder.documents")
+rag_pipeline.connect("prompt_builder", "llm")
+
+# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components.
+question = "Who lives in paris?"
+results = rag_pipeline.run(
+    {
+        "retriever": {"query": question},
+        "prompt_builder": {"question": question},
+    },
+)
+
+print(results["llm"]["replies"])
+```
+
+Here's what a [visualized Mermaid graph](visualizing-pipelines.mdx) of this pipeline would look like:
+
+<br />
+<ClickableImage src="/img/vizualised-rag-pipeline.png" alt="RAG pipeline diagram with three connected components: InMemoryBM25Retriever receives a query string and outputs documents, ChatPromptBuilder combines the documents with a question input to create prompt messages, and OpenAIChatGenerator processes the messages to produce replies. Each component box displays its class name and optional input parameters." size="large" />

docs-website/docs/concepts/pipelines/debugging-pipelines.mdx

@@ -0,0 +1,125 @@
+---
+title: "Debugging Pipelines"
+id: debugging-pipelines
+slug: "/debugging-pipelines"
+description: "Learn how to debug and troubleshoot your Haystack pipelines."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Debugging Pipelines
+
+Learn how to debug and troubleshoot your Haystack pipelines.
+
+There are several options available to you to debug your pipelines:
+
+- [Inspect your components' outputs](#inspecting-component-outputs)
+- [Adjust logging](#logging)
+- [Set up tracing](#tracing)
+- [Try one of the monitoring tool integrations](#monitoring-tools)
+
+## Inspecting Component Outputs
+
+To view outputs from specific pipeline components, add the `include_outputs_from` parameter when executing your pipeline. Place it after the input dictionary and set it to the name of the component whose output you want included in the result.
+
+For example, here’s how you can print the output of `PromptBuilder` in this pipeline:
+
+```python
+from haystack import Pipeline, Document
+from haystack.utils import Secret
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
+from haystack.dataclasses import ChatMessage
+
+# Documents
+documents = [
+    Document(content="Joe lives in Berlin"),
+    Document(content="Joe is a software engineer"),
+]
+
+# Define prompt template
+prompt_template = [
+    ChatMessage.from_system("You are a helpful assistant."),
+    ChatMessage.from_user(
+        "Given these documents, answer the question.\nDocuments:\n"
+        "{% for doc in documents %}{{ doc.content }}{% endfor %}\n"
+        "Question: {{query}}\nAnswer:",
+    ),
+]
+
+# Define pipeline
+p = Pipeline()
+p.add_component(
+    instance=ChatPromptBuilder(
+        template=prompt_template,
+        required_variables={"query", "documents"},
+    ),
+    name="prompt_builder",
+)
+p.add_component(
+    instance=OpenAIChatGenerator(api_key=Secret.from_env_var("OPENAI_API_KEY")),
+    name="llm",
+)
+p.connect("prompt_builder", "llm.messages")
+
+# Define question
+question = "Where does Joe live?"
+
+# Execute pipeline
+result = p.run(
+    {"prompt_builder": {"documents": documents, "query": question}},
+    include_outputs_from="prompt_builder",
+)
+
+# Print result
+print(result)
+```
+
+## Logging
+
+Adjust the logging format according to your debugging needs. See our [Logging](../../development/logging.mdx) documentation for details.
+
+## Real-Time Pipeline Logging
+
+Use Haystack's [`LoggingTracer`](https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py) logs to inspect the data that's flowing through your pipeline in real-time.
+
+This feature is particularly helpful during experimentation and prototyping, as you don’t need to set up any tracing backend beforehand.
+
+Here’s how you can enable this tracer. In this example, we are adding color tags (this is optional) to highlight the components' names and inputs:
+
+```python
+import logging
+from haystack import tracing
+from haystack.tracing.logging_tracer import LoggingTracer
+
+logging.basicConfig(
+    format="%(levelname)s - %(name)s -  %(message)s",
+    level=logging.WARNING,
+)
+logging.getLogger("haystack").setLevel(logging.DEBUG)
+
+tracing.tracer.is_content_tracing_enabled = (
+    True  # to enable tracing/logging content (inputs/outputs)
+)
+tracing.enable_tracing(
+    LoggingTracer(
+        tags_color_strings={
+            "haystack.component.input": "\x1b[1;31m",
+            "haystack.component.name": "\x1b[1;34m",
+        },
+    ),
+)
+```
+
+Here’s what the resulting log would look like when a pipeline is run:
+<ClickableImage src="/img/55c3d5c84282d726c95fb3350ec36be49a354edca8a6164f5dffdab7121cec58-image_2.png" alt="Console output showing Haystack pipeline execution with DEBUG level tracing logs including component names, types, and input/output specifications" />
+
+## Tracing
+
+To get a bigger picture of the pipeline’s performance, try tracing it with [Langfuse](../../development/tracing/langfuse.mdx).
+
+Our [Tracing](../../development/tracing.mdx) page has more about other tracing solutions for Haystack.
+
+## Monitoring Tools
+
+Take a look at available tracing and monitoring [integrations](https://haystack.deepset.ai/integrations?type=Monitoring+Tool&version=2.0) for Haystack pipelines, such as Arize AI or Arize Phoenix.

docs-website/docs/concepts/pipelines/pipeline-breakpoints.mdx

@@ -0,0 +1,160 @@
+---
+title: "Pipeline Breakpoints"
+id: pipeline-breakpoints
+slug: "/pipeline-breakpoints"
+description: "Learn how to pause and resume Haystack pipeline execution using breakpoints to debug, inspect, and continue workflows from saved snapshots."
+---
+
+# Pipeline Breakpoints
+
+Learn how to pause and resume Haystack pipeline execution using breakpoints to debug, inspect, and continue workflows from saved snapshots.
+
+## Introduction
+
+Haystack pipelines support breakpoints for debugging complex execution flows. A `Breakpoint` allows you to pause the execution at specific components, inspect the pipeline state, and resume execution from saved snapshots. This feature works for any regular component as well as an `Agent` component.
+
+You can set a `Breakpoint` on any component in a pipeline with a specific visit count. When triggered, the system stops the execution of the `Pipeline` and captures a snapshot of the current pipeline state. The state can be saved to a JSON file when snapshot file saving is enabled, see [Snapshot file saving](#snapshot-file-saving) below. You can inspect and modify the snapshot and use it to resume execution from the exact point where it stopped.
+
+## Setting a `Breakpoint` on a Regular Component
+
+Create a `Breakpoint` by specifying the component name and the visit count at which to trigger it. This is useful for pipelines with loops. The default `visit_count` value is 0.
+
+```python
+from haystack.dataclasses.breakpoints import Breakpoint
+from haystack.core.errors import BreakpointException
+
+# Create a breakpoint that triggers on the first visit to the "llm" component
+break_point = Breakpoint(
+    component_name="llm",
+    visit_count=0,  # 0 = first visit, 1 = second visit, etc.
+    snapshot_file_path="/path/to/snapshots",  # Optional: save snapshot to file
+)
+
+# Run pipeline with breakpoint
+try:
+    result = pipeline.run(data=input_data, break_point=break_point)
+except BreakpointException as e:
+    print(f"Breakpoint triggered at component: {e.component}")
+    print(f"Component inputs: {e.inputs}")
+    print(f"Pipeline results so far: {e.results}")
+```
+
+A `BreakpointException` is raised containing the component inputs and the outputs of the pipeline up until the moment where the execution was interrupted, such as just before the execution of component associated with the breakpoint – the `llm` in the example above.
+
+If a `snapshot_file_path` is specified in the `Breakpoint` and snapshot file saving is enabled, the system saves a JSON snapshot with the same information as in the `BreakpointException`. Snapshot file saving to disk is disabled by default; see [Snapshot file saving](#snapshot-file-saving) below.
+
+To access the pipeline state during the breakpoint we can both catch the exception raised by the breakpoint as well as specify where the JSON file should be saved, note that file saving is enabled must be enabled.
+
+## Using a custom snapshot callback
+
+You can pass a `snapshot_callback` to `Pipeline.run()` to handle snapshots yourself instead of saving to a file. When a breakpoint is triggered or a snapshot is created on error, the callback is invoked with the `PipelineSnapshot` object. This is useful for saving snapshots to a database, sending them to a remote service, or custom logging.
+
+```python
+from haystack.core.errors import BreakpointException
+from haystack.dataclasses.breakpoints import Breakpoint, PipelineSnapshot
+
+
+def my_snapshot_callback(snapshot: PipelineSnapshot) -> None:
+    # Custom handling: e.g. save to DB, send to API, or log
+    print(f"Snapshot at component: {snapshot.break_point}")
+
+
+break_point = Breakpoint(component_name="llm", visit_count=0)
+try:
+    result = pipeline.run(
+        data=input_data,
+        break_point=break_point,
+        snapshot_callback=my_snapshot_callback,
+    )
+except BreakpointException as e:
+    print(f"Breakpoint triggered: {e.component}")
+```
+
+When `snapshot_callback` is provided, file-saving is skipped and the callback is responsible for handling the snapshot.
+
+## Snapshot file saving
+
+Snapshot file saving to disk is **disabled by default**. To save snapshots as JSON files when a breakpoint is triggered or on pipeline failure, set the environment variable `HAYSTACK_PIPELINE_SNAPSHOT_SAVE_ENABLED` to `"true"` or `"1"` (case-insensitive). When enabled, snapshots are written to the path given by `snapshot_file_path` on the breakpoint, or to the default directory in [Error Recovery with Snapshots](#error-recovery-with-snapshots) when a run fails.
+
+Custom `snapshot_callback` functions are always invoked when provided, regardless of this setting.
+
+```python
+import os
+
+# Enable saving snapshot files to disk
+os.environ["HAYSTACK_PIPELINE_SNAPSHOT_SAVE_ENABLED"] = "true"
+
+break_point = Breakpoint(
+    component_name="llm",
+    visit_count=0,
+    snapshot_file_path="/path/to/snapshots",
+)
+# When the breakpoint triggers, a JSON file will be written to /path/to/snapshots/
+```
+
+## Resuming a Pipeline Execution from a Breakpoint
+
+To resume the execution of a pipeline from the breakpoint, pass the path to the generated JSON file at the run time of the pipeline, using the `pipeline_snapshot`.
+
+Use the `load_pipeline_snapshot()` to first load the JSON and then pass it to the pipeline.
+
+```python
+from haystack.core.pipeline.breakpoint import load_pipeline_snapshot
+
+# Load the snapshot
+snapshot = load_pipeline_snapshot("llm_2025_05_03_11_23_23.json")
+
+# Resume execution from the snapshot
+result = pipeline.run(data={}, pipeline_snapshot=snapshot)
+print(result["llm"]["replies"])
+```
+
+## Error Recovery with Snapshots
+
+Pipelines automatically create a snapshot of the last valid state if a run fails. The snapshot contains inputs, visit counts, and intermediate outputs up to the failure. You can inspect it, fix the issue, and resume execution from that checkpoint instead of restarting the whole run.
+
+### Access the Snapshot on Failure
+
+Wrap `pipeline.run()` in a `try`/`except` block and retrieve the snapshot from the raised `PipelineRuntimeError`:
+
+```python
+from haystack.core.errors import PipelineRuntimeError
+
+try:
+    pipeline.run(data=input_data)
+except PipelineRuntimeError as e:
+    snapshot = e.pipeline_snapshot
+    if snapshot is not None:
+        intermediate_outputs = snapshot.pipeline_state.pipeline_outputs
+        # Inspect intermediate_outputs to diagnose the failure
+```
+
+When snapshot file saving is enabled (see [Snapshot file saving](#snapshot-file-saving)), Haystack also saves the same snapshot as a JSON file on disk.
+The directory is chosen automatically in this order:
+
+- `~/.haystack/pipeline_snapshot`
+- `/tmp/haystack/pipeline_snapshot`
+- `./.haystack/pipeline_snapshot`
+
+Filenames will have the following pattern: `{component_name}_{visit_nr}_{YYYY_MM_DD_HH_MM_SS}.json`.
+
+### Resume from a Snapshot
+
+You can resume directly from the in-memory snapshot or load it from disk.
+
+Resume from memory:
+
+```python
+result = pipeline.run(data={}, pipeline_snapshot=snapshot)
+```
+
+Resume from disk:
+
+```python
+from haystack.core.pipeline.breakpoint import load_pipeline_snapshot
+
+snapshot = load_pipeline_snapshot(
+    "/path/to/.haystack/pipeline_snapshot/reader_0_2025_09_20_12_33_10.json",
+)
+result = pipeline.run(data={}, pipeline_snapshot=snapshot)
+```

docs-website/docs/concepts/pipelines/pipeline-loops.mdx

@@ -0,0 +1,253 @@
+---
+title: "Pipeline Loops"
+id: pipeline-loops
+slug: "/pipeline-loops"
+description: "Understand how loops work in Haystack pipelines, how they terminate, and how to use them safely for feedback and self-correction."
+---
+
+# Pipeline Loops
+
+Learn how loops work in Haystack pipelines, how they terminate, and how to use them for feedback and self-correction.
+
+Haystack pipelines support **loops**: cycles in the component graph where the output of a later component is fed back into an earlier one.
+This enables feedback flows such as self-correction, validation, or iterative refinement, as well as more advanced [agentic behavior](../pipelines.mdx#agentic-pipelines).
+
+At runtime, the pipeline re-runs a component whenever all of its required inputs are ready again.
+You control when loops stop either by designing your graph and routing logic carefully or by using built-in [safety limits](#loop-termination-and-safety-limits).
+
+## Multiple Runs of the Same Component
+
+If a component participates in a loop, it can be run multiple times within a single `Pipeline.run()` call.
+The pipeline keeps an internal visit counter for each component:
+
+- Each time the component runs, its visit count increases by 1.
+- You can use this visit count in debugging tools like [breakpoints](./pipeline-breakpoints.mdx) to inspect specific iterations of a loop.
+
+In the final pipeline result:
+
+- For each component that ran, the pipeline returns **only the last-produced output**.
+- To capture outputs from intermediate components (for example, a validator or a router) in the final result dictionary, use the `include_outputs_from` argument of `Pipeline.run()`.
+
+## Loop Termination and Safety Limits
+
+Loops must eventually stop so that a pipeline run can complete.
+There are two main ways a loop ends:
+
+1. **Natural completion**: No more components are runnable  
+   The pipeline finishes when the work queue is empty and no component can run again (for example, the router stops feeding inputs back into the loop).
+
+2. **Reaching the maximum run count**  
+   Every pipeline has a per-component run limit, controlled by the `max_runs_per_component` parameter of the `Pipeline` constructor, which is `100` by default. If any component exceeds this limit, Haystack raises a `PipelineMaxComponentRuns` error.
+
+   You can set this limit to a lower value:
+
+   ```python
+   from haystack import Pipeline
+
+   pipe = Pipeline(max_runs_per_component=5)
+   ```
+
+   The limit is checked before each execution, so a component with a limit of 3 will complete 3 runs successfully before the error is raised on the 4th attempt.
+
+   This safeguard is especially important when experimenting with new loops or complex routing logic.
+   If your loop condition is wrong or never satisfied, the error prevents the pipeline from running indefinitely.
+
+## Example: Feedback Loop for Self-Correction
+
+The following example shows a simple feedback loop where:
+
+- A `ChatPromptBuilder` creates a prompt that includes previous incorrect replies.
+- An `OpenAIChatGenerator` produces an answer.
+- A `ConditionalRouter` checks if the answer is correct:
+  - If correct, it sends the answer to `final_answer` and the loop ends.
+  - If incorrect, it sends the answer back to the `ChatPromptBuilder`, which triggers another iteration.
+
+```python
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+from haystack.components.routers import ConditionalRouter
+from haystack.dataclasses import ChatMessage
+
+template = [
+    ChatMessage.from_system(
+        "Answer the following question concisely with just the answer, no punctuation.",
+    ),
+    ChatMessage.from_user(
+        "{% if previous_replies %}"
+        "Previously you replied incorrectly: {{ previous_replies[0].text }}\n"
+        "{% endif %}"
+        "Question: {{ query }}",
+    ),
+]
+
+prompt_builder = ChatPromptBuilder(template=template, required_variables=["query"])
+generator = OpenAIChatGenerator()
+
+router = ConditionalRouter(
+    routes=[
+        {
+            # End the loop when the answer is correct
+            "condition": "{{ 'Rome' in replies[0].text }}",
+            "output": "{{ replies }}",
+            "output_name": "final_answer",
+            "output_type": list[ChatMessage],
+        },
+        {
+            # Loop back when the answer is incorrect
+            "condition": "{{ 'Rome' not in replies[0].text }}",
+            "output": "{{ replies }}",
+            "output_name": "previous_replies",
+            "output_type": list[ChatMessage],
+        },
+    ],
+    unsafe=True,  # Required to handle ChatMessage objects
+)
+
+pipe = Pipeline(max_runs_per_component=3)
+
+pipe.add_component("prompt_builder", prompt_builder)
+pipe.add_component("generator", generator)
+pipe.add_component("router", router)
+
+pipe.connect("prompt_builder.prompt", "generator.messages")
+pipe.connect("generator.replies", "router.replies")
+pipe.connect("router.previous_replies", "prompt_builder.previous_replies")
+
+result = pipe.run(
+    {
+        "prompt_builder": {
+            "query": "What is the capital of Italy? If the statement 'Previously you replied incorrectly:' is missing "
+            "above then answer with Milan.",
+        },
+    },
+    include_outputs_from={"router", "prompt_builder"},
+)
+
+print(result["prompt_builder"]["prompt"][1].text)  # Shows the last prompt used
+print(result["router"]["final_answer"][0].text)  # Rome
+```
+
+### What Happens During This Loop
+
+1. **First iteration**
+   - `prompt_builder` runs with `query="What is the capital of Italy?"` and no previous replies.
+   - `generator` returns a `ChatMessage` with the LLM's answer.
+   - The router evaluates its conditions and checks if `"Rome"` is in the reply.
+   - If the answer is incorrect, `previous_replies` is fed back into `prompt_builder.previous_replies`.
+
+2. **Subsequent iterations** (if needed)
+   - `prompt_builder` runs again, now including the previous incorrect reply in the user message.
+   - `generator` produces a new answer with the additional context.
+   - The router checks again whether the answer contains `"Rome"`.
+
+3. **Termination**
+   - When the router routes to `final_answer`, no more inputs are fed back into the loop.
+   - The queue empties and the pipeline run finishes successfully.
+
+Because we used `max_runs_per_component=3`, any unexpected behavior that causes the loop to continue would raise a `PipelineMaxComponentRuns` error instead of looping forever.
+
+## Components for Building Loops
+
+Two components are particularly useful for building loops:
+
+- **[`ConditionalRouter`](../../pipeline-components/routers/conditionalrouter.mdx)**: Routes data to different outputs based on conditions. Use it to decide whether to exit the loop or continue iterating. The example above uses this pattern.
+
+- **[`BranchJoiner`](../../pipeline-components/joiners/branchjoiner.mdx)**: Merges inputs from multiple sources into a single output. Use it when a component inside the loop needs to receive both the initial input (on the first iteration) and looped-back values (on subsequent iterations). For example, you might use `BranchJoiner` to feed both user input and validation errors into the same Generator. See the [BranchJoiner documentation](../../pipeline-components/joiners/branchjoiner.mdx#enabling-loops) for a complete loop example.
+
+## Greedy vs. Lazy Variadic Sockets in Loops
+
+Some components support variadic inputs that can receive multiple values on a single socket.
+In loops, variadic behavior controls how inputs are consumed across iterations.
+
+- **Greedy variadic sockets**  
+  Consume exactly one value at a time and remove it after the component runs.
+  This includes user-provided inputs, which prevents them from retriggering the component indefinitely.
+  Most variadic sockets are greedy by default.
+
+- **Lazy variadic sockets**  
+  Accumulate all values received from predecessors across iterations.
+  Useful when you need to collect multiple partial results over time (for example, gathering outputs from several loop iterations before proceeding).
+
+For most loop scenarios it's sufficient to just connect components as usual and use `max_runs_per_component` to protect against mistakes.
+
+## Troubleshooting Loops
+
+If your pipeline seems stuck or runs longer than expected, here are common causes and how to debug them.
+
+### Common Causes of Infinite Loops
+
+1. **Condition never satisfied**: Your exit condition (for example, `"Rome" in reply`) might never be true due to LLM behavior or data issues. Always set a reasonable `max_runs_per_component` as a safety net.
+
+2. **Relying on optional outputs**: When a component has multiple output sockets but only returns some of them, the unreturned outputs don't trigger their downstream connections. This can cause confusion in loops.
+
+   For example, this pattern can be problematic:
+
+   ```python
+   @component
+   class Validator:
+       @component.output_types(valid=str, invalid=Optional[str])
+       def run(self, text: str):
+           if is_valid(text):
+               return {"valid": text}  # "invalid" is never returned
+           else:
+               return {"invalid": text}
+   ```
+
+   If you connect `invalid` back to an upstream component for retry, but also have other connections that keep the loop alive, you might get unexpected behavior.
+
+   Instead, use a `ConditionalRouter` with explicit, mutually exclusive conditions:
+
+   ```python
+   router = ConditionalRouter(
+       routes=[
+           {"condition": "{{ is_valid }}", "output": "{{ text }}", "output_name": "valid", ...},
+           {"condition": "{{ not is_valid }}", "output": "{{ text }}", "output_name": "invalid", ...},
+       ]
+   )
+   ```
+
+3. **User inputs retriggering the loop**: If a user-provided input is connected to a socket inside the loop, it might cause the loop to restart unexpectedly.
+
+   ```python
+   # Problematic: user input goes directly to a component inside the loop
+   result = pipe.run({
+       "generator": {"prompt": query},  # This input persists and may retrigger the loop
+   })
+
+   # Better: use an entry-point component outside the loop
+   result = pipe.run({
+       "prompt_builder": {"query": query},  # Entry point feeds into the loop once
+   })
+   ```
+
+   See [Greedy vs. Lazy Variadic Sockets](#greedy-vs-lazy-variadic-sockets-in-loops) for details on how inputs are consumed.
+
+4. **Multiple paths feeding the same component**: If a component inside the loop receives inputs from multiple sources, it runs whenever *any* path provides input.
+
+   ```python
+   # Component receives from two sources – runs when either provides input
+   pipe.connect("source_a.output", "processor.input")
+   pipe.connect("source_b.output", "processor.input")  # Variadic input
+   ```
+
+   Ensure you understand when each path produces output, or use `BranchJoiner` to explicitly control the merge point.
+
+### Debugging Tips
+
+1. **Start with a low limit**: When developing loops, set `max_runs_per_component=3` or similar. This helps you catch issues early with a clear error instead of waiting for a timeout.
+
+2. **Use `include_outputs_from`**: Add intermediate components (like your router) to see what's happening at each step:
+   ```python
+   result = pipe.run(data, include_outputs_from={"router", "validator"})
+   ```
+
+3. **Enable tracing**: Use tracing to see every component execution, including inputs and outputs. This makes it easy to follow each iteration of the loop. For quick debugging, use `LoggingTracer` ([setup instructions](./debugging-pipelines.mdx#real-time-pipeline-logging)). For deeper analysis, integrate with tools like Langfuse or other [tracing backends](../../development/tracing.mdx).
+
+4. **Visualize the pipeline**: Use `pipe.draw()` or `pipe.show()` to see the graph structure and verify your connections are correct. See the [Pipeline Visualization](./visualizing-pipelines.mdx) documentation for details.
+
+5. **Use breakpoints**: Set a `Breakpoint` on a specific component and visit count to inspect the state at that iteration. See [Pipeline Breakpoints](./pipeline-breakpoints.mdx) for details.
+
+6. **Check for blocked pipelines**: If you see a `PipelineComponentsBlockedError`, it means no components can run. This typically indicates a missing connection or a circular dependency. Check that all required inputs are provided.
+
+By combining careful graph design, per-component run limits, and these debugging tools, you can build robust feedback loops in your Haystack pipelines.

docs-website/docs/concepts/pipelines/serialization.mdx

@@ -0,0 +1,276 @@
+---
+title: "Serializing Pipelines"
+id: serialization
+slug: "/serialization"
+description: "Save your pipelines into a custom format and explore the serialization options."
+---
+
+# Serializing Pipelines
+
+Save your pipelines into a custom format and explore the serialization options.
+
+Serialization means converting a pipeline to a format that you can save on your disk and load later.
+
+Haystack supports YAML format for pipeline serialization.
+
+## Converting a Pipeline to YAML
+
+Use the `dumps()` method to convert a Pipeline object to YAML:
+
+```python
+from haystack import Pipeline
+
+pipe = Pipeline()
+print(pipe.dumps())
+
+# Prints:
+#
+# components: {}
+# connections: []
+# max_runs_per_component: 100
+# metadata: {}
+```
+
+You can also use `dump()` method to save the YAML representation of a pipeline in a file:
+
+```python
+with open("/content/test.yml", "w") as file:
+    pipe.dump(file)
+```
+
+## Converting a Pipeline Back to Python
+
+You can convert a YAML pipeline back into Python. Use the `loads()` method to convert a string representation of a pipeline (`str`, `bytes` or `bytearray`)  or the `load()` method to convert a pipeline represented in a file-like object into a corresponding Python object.
+
+Both loading methods support callbacks that let you modify components during the deserialization process. Deserialization is gated by a trusted-module allowlist, so pipelines referencing classes outside of it fail to load until you extend the allowlist — see [Deserialization Security](#deserialization-security) below.
+
+Here is an example script:
+
+```python
+from haystack import Pipeline
+from haystack.core.serialization import DeserializationCallbacks
+from typing import Type, Dict, Any
+
+# This is the YAML you want to convert to Python:
+pipeline_yaml = """
+components:
+  cleaner:
+    init_parameters:
+      remove_empty_lines: true
+      remove_extra_whitespaces: true
+      remove_regex: null
+      remove_repeated_substrings: false
+      remove_substrings: null
+    type: haystack.components.preprocessors.document_cleaner.DocumentCleaner
+  converter:
+    init_parameters:
+      encoding: utf-8
+    type: haystack.components.converters.txt.TextFileToDocument
+connections:
+- receiver: cleaner.documents
+  sender: converter.documents
+max_runs_per_component: 100
+metadata: {}
+"""
+
+
+def component_pre_init_callback(
+    component_name: str,
+    component_cls: Type,
+    init_params: Dict[str, Any],
+):
+    # This function gets called every time a component is deserialized.
+    if component_name == "cleaner":
+        assert "DocumentCleaner" in component_cls.__name__
+        # Modify the init parameters. The modified parameters are passed to
+        # the init method of the component during deserialization.
+        init_params["remove_empty_lines"] = False
+        print("Modified 'remove_empty_lines' to False in 'cleaner' component")
+    else:
+        print(f"Not modifying component {component_name} of class {component_cls}")
+
+
+pipe = Pipeline.loads(
+    pipeline_yaml,
+    callbacks=DeserializationCallbacks(component_pre_init_callback),
+)
+```
+
+## Deserialization Security
+
+Loading a pipeline instantiates the classes referenced in the serialized data. To prevent a crafted YAML file from importing and instantiating arbitrary classes, `Pipeline.load`, `Pipeline.loads`, and `Pipeline.from_dict` refuse to import classes from modules outside a trusted-module allowlist and raise a `DeserializationError` instead.
+
+By default, the allowlist contains `haystack`, `haystack_integrations`, `haystack_experimental`, `builtins`, `typing`, and `collections`. Dangerous builtins such as `eval`, `exec`, `compile`, `__import__`, `open`, and `getattr` are blocked even though `builtins` is allowlisted.
+
+### Allowing Custom Modules
+
+Pipelines that reference custom components or callables in other packages fail to load until you add the modules to the allowlist. You can extend it in three ways:
+
+```python
+from haystack import Pipeline
+
+# 1. Per call: pass additional module patterns for this deserialization only
+pipe = Pipeline.load(open("pipeline.yaml"), allowed_modules=["mypkg.*"])
+
+# 2. Process-wide: extend the allowlist programmatically
+from haystack.core.serialization import allow_deserialization_module
+
+allow_deserialization_module("mypkg")
+```
+
+```shell
+# 3. Environment variable with comma-separated patterns, read on every deserialization call
+export HAYSTACK_DESERIALIZATION_ALLOWLIST="mypkg.*,otherpkg.*"
+```
+
+Patterns are matched as prefixes by default (`"mypkg"` matches `mypkg` and any of its submodules), or as `fnmatch` globs if they contain `*`, `?`, or `[` somewhere other than a trailing `.*`. A trailing `.*` is treated as a prefix match, so `"mypkg"` and `"mypkg.*"` behave identically.
+
+If the source of the serialized data is fully trusted, you can bypass the allowlist entirely with `unsafe=True`:
+
+```python
+pipe = Pipeline.load(open("pipeline.yaml"), unsafe=True)
+```
+
+Only use `unsafe=True` when you fully trust where the serialized pipeline comes from — it also lifts the block on dangerous builtins.
+
+### Nested Init Parameter Validation
+
+As an additional safeguard, deserialization validates the keys of `init_parameters` against the class's `__init__` signature before recursing into any nested `{"type": "...", "init_parameters": {...}}` dictionary. A nested dictionary whose key is not an accepted parameter name is rejected with a `DeserializationError` *before* the nested type is imported, which blocks attempts to smuggle untrusted classes into unused parameter slots. Classes whose constructor takes `**kwargs` are exempt, since their accepted parameter set cannot be statically determined.
+
+This validation may surface pre-existing bugs in YAML files — for example typos, leftovers from renamed or removed parameters, or stale snapshots from older Haystack versions. The fix is to update the YAML so each nested-component key matches a real `__init__` parameter of the parent class.
+
+## Default Serialization Behavior
+
+The serialization system uses `default_to_dict` and `default_from_dict` to handle many object types automatically. You typically do **not** need to implement custom `to_dict`/`from_dict` for:
+
+- **Secrets**: serialized and deserialized automatically so that sensitive values aren't stored in plain text.
+- **ComponentDevice**: device configuration is detected and restored automatically.
+- **Objects with their own `to_dict`/`from_dict`**: any init parameter whose type defines `to_dict()` is serialized by calling it; any dict in `init_parameters` with a `type` key pointing to a class with `from_dict()` is deserialized automatically.
+
+To serialize or deserialize a single component, you can use `component_to_dict` and `component_from_dict` from `haystack.core.serialization`. They use the default behavior above as a fallback when the component doesn't define custom `to_dict`/`from_dict`:
+
+```python
+from haystack import component
+from haystack.core.serialization import component_from_dict, component_to_dict
+
+
+@component
+class Greeter:
+    def __init__(self, message: str = "Hello"):
+        self.message = message
+
+    @component.output_types(greeting=str)
+    def run(self, name: str):
+        return {"greeting": f"{self.message}, {name}!"}
+
+
+# Serialize a component instance to a dictionary
+greeter = Greeter(message="Hi")
+data = component_to_dict(greeter, "my_greeter")
+
+# Deserialize back to a component instance
+restored = component_from_dict(Greeter, data, "my_greeter")
+assert restored.message == greeter.message
+```
+
+:::caution[Init parameters must be stored as instance attributes]
+
+Default serialization only works when there is a **1:1 mapping** between init parameter names and instance attributes. For every argument in `__init__`, the component must assign it to an attribute with the same name. For example, if you have `def __init__(self, prompt: str)`, you must have `self.prompt = prompt` in the class. Otherwise the serialization logic can't find the value to serialize and raises an error or uses the default value if the parameter has one.
+:::
+
+## Performing Custom Serialization
+
+Pipelines and components in Haystack can serialize simple components, including custom ones, out of the box. Code like this just works:
+
+```python
+from haystack import component
+
+
+@component
+class RepeatWordComponent:
+    def __init__(self, times: int):
+        self.times = times
+
+    @component.output_types(result=str)
+    def run(self, word: str):
+        return word * self.times
+```
+
+On the other hand, this code doesn't work if the final format is JSON, as the `set` type is not JSON-serializable:
+
+```python
+from haystack import component
+
+
+@component
+class SetIntersector:
+    def __init__(self, intersect_with: set):
+        self.intersect_with = intersect_with
+
+    @component.output_types(result=set)
+    def run(self, data: set):
+        return data.intersection(self.intersect_with)
+```
+
+In such cases, you can provide your own implementation  `from_dict` and `to_dict` to components:
+
+```python
+from haystack import component, default_from_dict, default_to_dict
+
+
+class SetIntersector:
+    def __init__(self, intersect_with: set):
+        self.intersect_with = intersect_with
+
+    @component.output_types(result=set)
+    def run(self, data: set):
+        return data.intersect(self.intersect_with)
+
+    def to_dict(self):
+        return default_to_dict(self, intersect_with=list(self.intersect_with))
+
+    @classmethod
+    def from_dict(cls, data):
+        # convert the set into a list for the dict representation,
+        # so it can be converted to JSON
+        data["intersect_with"] = set(data["intersect_with"])
+        return default_from_dict(cls, data)
+```
+
+## Saving a Pipeline to a Custom Format
+
+Once a pipeline is available in its dictionary format, the last step of serialization is to convert that dictionary into a format you can store or send over the wire. Haystack supports YAML out of the box, but if you need a different format, you can write a custom Marshaller.
+
+A `Marshaller` is a Python class responsible for converting text to a dictionary and a dictionary to text according to a certain format. Marshallers must respect the `Marshaller` [protocol](https://github.com/deepset-ai/haystack/blob/main/haystack/marshal/protocol.py), providing the methods `marshal` and `unmarshal`.
+
+This is the code for a custom TOML marshaller that relies on the `rtoml` library:
+
+```python
+# This code requires a `pip install rtoml`
+from typing import Dict, Any, Union
+import rtoml
+
+
+class TomlMarshaller:
+    def marshal(self, dict_: Dict[str, Any]) -> str:
+        return rtoml.dumps(dict_)
+
+    def unmarshal(self, data_: Union[str, bytes]) -> Dict[str, Any]:
+        return dict(rtoml.loads(data_))
+```
+
+You can then pass a Marshaller instance to the methods `dump`, `dumps`, `load`, and `loads`:
+
+```python
+from haystack import Pipeline
+from my_custom_marshallers import TomlMarshaller
+
+pipe = Pipeline()
+pipe.dumps(TomlMarshaller())
+# prints:
+# 'max_runs_per_component = 100\nconnections = []\n\n[metadata]\n\n[components]\n'
+```
+
+## Additional References
+
+:notebook: Tutorial: [Serializing LLM Pipelines](https://haystack.deepset.ai/tutorials/29_serializing_pipelines)

docs-website/docs/concepts/pipelines/smart-pipeline-connections.mdx

@@ -0,0 +1,148 @@
+---
+title: "Smart Pipeline Connections"
+id: smart-pipeline-connections
+slug: "/smart-pipeline-connections"
+description: "Learn how Haystack pipelines simplify connections through implicit joining and flexible type adaptation, reducing the need for glue components."
+---
+
+# Smart Pipeline Connections
+
+Haystack pipelines support smarter connection semantics that reduce boilerplate and make pipeline definitions easier to read and maintain.
+These features focus on simplifying how components are connected, without changing component behavior.
+
+Smart connections help eliminate common glue components such as `Joiners` and `OutputAdapters` in many pipelines.
+
+## Implicit List Joining
+
+Pipelines natively support connecting multiple component outputs directly to a single component input, without requiring an explicit `Joiner` component.
+
+This works when:
+
+* The target input is typed as `list`, `list | None`, or a union of list types (e.g. `list[int] | list[str]`).
+* All connected outputs are compatible list types.
+
+When multiple outputs are connected to the same input, the pipeline implicitly concatenates the lists from the outputs into a single list for the input.
+
+### Example
+
+Multiple converters can write directly into a single `DocumentWriter` without using a `DocumentJoiner`:
+
+<details>
+
+<summary>Expand to see the pipeline graph</summary>
+<ClickableImage src="/img/pipeline-illustration-auto-joiner.png" alt="Pipeline architecture diagram showing a DocumentWriter receiving inputs from multiple converters without a Joiner" size="large" />
+
+</details>
+
+```python
+from haystack import Pipeline
+from haystack.components.converters import HTMLToDocument, TextFileToDocument
+from haystack.components.routers import FileTypeRouter
+from haystack.components.writers import DocumentWriter
+from haystack.dataclasses import ByteStream
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+
+sources = [
+    ByteStream.from_string(text="Text file content", mime_type="text/plain"),
+    ByteStream.from_string(
+        text="<html><body>Some content</body></html>",
+        mime_type="text/html",
+    ),
+]
+
+doc_store = InMemoryDocumentStore()
+
+pipe = Pipeline()
+pipe.add_component("router", FileTypeRouter(mime_types=["text/plain", "text/html"]))
+pipe.add_component("txt_converter", TextFileToDocument())
+pipe.add_component("html_converter", HTMLToDocument())
+pipe.add_component("writer", DocumentWriter(doc_store))
+pipe.connect("router.text/plain", "txt_converter.sources")
+pipe.connect("router.text/html", "html_converter.sources")
+pipe.connect("txt_converter.documents", "writer.documents")
+pipe.connect("html_converter.documents", "writer.documents")
+
+result = pipe.run({"router": {"sources": sources}})
+```
+
+This pattern is especially useful when routing files, documents, or results across multiple parallel branches.
+
+## Flexible Type Connections
+
+To further streamline pipeline definitions, Haystack pipelines support limited implicit type adaptation at connection time.
+This makes pipeline connections more flexible and reduces the need for `OutputAdapter` components.
+
+**Supported adaptations**
+
+| Source Type              | Target Type          | Behavior                                                      |
+|--------------------------|--------------------|---------------------------------------------------------------|
+| `str`                    | `ChatMessage`       | Wrapped into a `ChatMessage` with user role.                 |
+| `ChatMessage`            | `str`               | Extracts `ChatMessage.text`; raises `PipelineRuntimeError` if `None`. |
+| `T`                      | `list[T]`           | Wraps the item into a single-element list.                   |
+| `list[str] or list[ChatMessage]`| `str` or `ChatMessage` | Extracts the first item; raises `PipelineRuntimeError` if the list is empty. |
+
+
+All adaptations are checked at connection time to ensure type safety, but applied at runtime during pipeline execution.
+
+When multiple connections are possible, strict type matching is prioritized over implicit conversion.
+This preserves backward compatibility with earlier versions of Haystack, where flexible type connections were not supported.
+
+### Example
+
+Pipeline connecting the Chat Generator `messages` output (`list[ChatMessage]`) to the retriever `query` input (`str`)
+without using an `OutputAdapter`:
+
+```python
+from haystack.document_stores.in_memory import InMemoryDocumentStore
+from haystack.dataclasses import Document
+from haystack.components.retrievers import InMemoryBM25Retriever
+from haystack import Pipeline
+from haystack.components.builders import ChatPromptBuilder
+from haystack.components.generators.chat import OpenAIChatGenerator
+
+document_store = InMemoryDocumentStore()
+
+documents = [
+    Document(content="Bob lives in Paris."),
+    Document(content="Alice lives in London."),
+    Document(content="Ivy lives in Melbourne."),
+    Document(content="Kate lives in Brisbane."),
+    Document(content="Liam lives in Adelaide."),
+]
+
+document_store.write_documents(documents)
+
+template = """{% message role="user" %}
+Rewrite the following query to be used for keyword search.
+{{ query }}
+{% endmessage %}
+"""
+
+p = Pipeline()
+p.add_component("prompt_builder", ChatPromptBuilder(template=template))
+p.add_component("llm", OpenAIChatGenerator(model="gpt-4.1-mini"))
+p.add_component(
+    "retriever",
+    InMemoryBM25Retriever(document_store=document_store, top_k=3),
+)
+
+p.connect("prompt_builder", "llm")
+# implicitly converts list[ChatMessage] -> str
+p.connect("llm", "retriever")
+
+query = """Someday I'd love to visit Brisbane, but for now I just want
+to know the names of the people who live there."""
+
+result = p.run(data={"prompt_builder": {"query": query}})
+```
+
+## When You Still Need `Joiners` or `OutputAdapters`
+
+Explicit `Joiners` or `OutputAdapters` are still useful when you need:
+
+- Custom aggregation logic beyond simple list concatenation
+- Type conversions not covered by implicit adaptation
+- Explicit control over formatting or ordering
+
+Smart connections reduce the need for glue components, but they do not remove them entirely.
+When in doubt, explicit components provide clarity and more control.

docs-website/docs/concepts/pipelines/visualizing-pipelines.mdx

@@ -0,0 +1,88 @@
+---
+title: "Visualizing Haystack Pipelines"
+id: visualizing-pipelines
+slug: "/visualizing-pipelines"
+description: "You can visualize your Haystack AI pipelines as graphs to better understand how the components are connected."
+---
+
+import ClickableImage from "@site/src/components/ClickableImage";
+
+# Visualizing Haystack Pipelines
+
+You can visualize your pipelines as graphs to better understand how the components are connected.
+
+Haystack pipelines have  `draw()` and `show()` methods that enable you to visualize the pipeline as a graph using Mermaid graphs.
+
+:::note[Data Privacy Notice]
+
+Exercise caution with sensitive data when using pipeline visualization.
+
+This feature is based on Mermaid graphs web service that doesn't have clear terms of data retention or privacy policy.
+:::
+
+## Prerequisites
+
+To use Mermaid graphs, you must have an internet connection to reach the Mermaid graph renderer at https://mermaid.ink.
+
+## Displaying a Graph
+
+Use the pipeline's `show()` method to display the diagram in Jupyter notebooks.
+
+```python
+my_pipeline.show()
+```
+
+## Saving a Graph
+
+Use the pipeline's `draw()` method passing the path where you want to save the diagram and the diagram format. Possible formats are:  `mermaid-text` and `mermaid-image` (default).
+
+```python
+my_pipeline.draw(path=local_path)
+```
+
+## Visualizing SuperComponents
+
+To show the internal structure of [SuperComponents](../components/supercomponents.mdx) in your digram instead of a black box component, set the `super_component_expansion` parameter to `True`:
+
+```python
+my_pipeline.show(super_component_expansion=True)
+
+# or
+
+my_pipeline.draw(path=local_path, super_component_expansion=True)
+```
+
+## Visualizing Locally
+
+If you don't have an internet connection or don't want to send your pipeline data to the remote https://mermaid.ink, you can install a local mermaid.ink server and use it to render your pipeline.
+
+Let's run a local mermaid.ink server using their official Docker images from https://github.com/jihchi/mermaid.ink/pkgs/container/mermaid.ink.
+
+In this case, let's install one for a system running a MacOS M3 chip and expose it on port 3000:
+
+```dockerfile
+docker run --platform linux/amd64 --publish 3000:3000 --cap-add=SYS_ADMIN ghcr.io/jihchi/mermaid.ink
+```
+
+Check that the local mermaid.ink server is running by going to http://localhost:3000/.
+
+You should see a local server running, and now you can simply render the image using your local mermaid.ink server by specifying the URL when calling the`show()`or `draw()`  method:
+
+```python
+my_pipeline.show(server_url="http://localhost:3000")
+# or
+my_pipeline.draw("my_pipeline.png", server_url="http://localhost:3000")
+```
+
+## Example
+
+This is an example of what a pipeline graph may look like:
+<ClickableImage src="/img/46a8989-Untitled.png" alt="RAG pipeline flowchart showing the data flow from query through retriever, prompt builder, language model, and answer builder components" size="large" />
+
+<br />
+
+## Importing a Pipeline to Haystack Enterprise Platform
+
+You can import your Haystack pipeline into Haystack Enterprise Platform and continue visually building your pipeline
+
+To do that, follow the steps described in Haystack Enterprise Platform [documentation](https://docs.cloud.deepset.ai/docs/import-a-pipeline#import-your-pipeline).