chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,82 @@
|
||||
---
|
||||
title: "Custom Tracer"
|
||||
id: custom-tracer
|
||||
slug: "/tracing-custom-tracer"
|
||||
description: "Learn how to connect Haystack to a custom tracing backend by implementing the Tracer interface."
|
||||
---
|
||||
|
||||
# Custom Tracer
|
||||
|
||||
Learn how to connect Haystack to a custom tracing backend by implementing the `Tracer` interface.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Base classes** | `Tracer` and `Span` |
|
||||
| **How to enable** | Implement the `Tracer` interface, then `tracing.enable_tracing(your_tracer)` |
|
||||
| **Content tracing** | Optional. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
|
||||
| **Package** | Built into Haystack |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/tracer.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
If your tracing backend isn't supported out of the box, you can connect it to Haystack by implementing the `Tracer` interface. This gives you full control over how spans are created and how tags are recorded.
|
||||
|
||||
## Usage
|
||||
|
||||
1. Implement the `Tracer` interface. The following code snippet provides an example using the OpenTelemetry package:
|
||||
|
||||
```python
|
||||
import contextlib
|
||||
from typing import Optional, Dict, Any, Iterator
|
||||
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.trace import NonRecordingSpan
|
||||
|
||||
from haystack.tracing import Tracer, Span
|
||||
from haystack.tracing import utils as tracing_utils
|
||||
import opentelemetry.trace
|
||||
|
||||
class OpenTelemetrySpan(Span):
|
||||
def __init__(self, span: opentelemetry.trace.Span) -> None:
|
||||
self._span = span
|
||||
|
||||
def set_tag(self, key: str, value: Any) -> None:
|
||||
# Tracing backends usually don't support any tag value
|
||||
# `coerce_tag_value` forces the value to either be a Python
|
||||
# primitive (int, float, boolean, str) or tries to dump it as string.
|
||||
coerced_value = tracing_utils.coerce_tag_value(value)
|
||||
self._span.set_attribute(key, coerced_value)
|
||||
|
||||
class OpenTelemetryTracer(Tracer):
|
||||
def __init__(self, tracer: opentelemetry.trace.Tracer) -> None:
|
||||
self._tracer = tracer
|
||||
|
||||
@contextlib.contextmanager
|
||||
def trace(self, operation_name: str, tags: Optional[Dict[str, Any]] = None) -> Iterator[Span]:
|
||||
with self._tracer.start_as_current_span(operation_name) as span:
|
||||
span = OpenTelemetrySpan(span)
|
||||
if tags:
|
||||
span.set_tags(tags)
|
||||
|
||||
yield span
|
||||
|
||||
def current_span(self) -> Optional[Span]:
|
||||
current_span = trace.get_current_span()
|
||||
if isinstance(current_span, NonRecordingSpan):
|
||||
return None
|
||||
|
||||
return OpenTelemetrySpan(current_span)
|
||||
```
|
||||
|
||||
2. Tell Haystack to use your custom tracer:
|
||||
|
||||
```python
|
||||
from haystack import tracing
|
||||
|
||||
haystack_tracer = OpenTelemetryTracer(tracer)
|
||||
tracing.enable_tracing(haystack_tracer)
|
||||
```
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
title: "Datadog"
|
||||
id: datadog
|
||||
slug: "/tracing-datadog"
|
||||
description: "Learn how to trace your Haystack pipelines with Datadog."
|
||||
---
|
||||
|
||||
# Datadog
|
||||
|
||||
Learn how to trace your Haystack pipelines with Datadog.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Tracer class** | `DatadogTracer` |
|
||||
| **How to enable** | Enable the tracer with `tracing.enable_tracing(DatadogTracer(ddtrace.tracer))`, or add the `DatadogConnector` component to your pipeline |
|
||||
| **Content tracing** | Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
|
||||
| **Package** | `datadog-haystack` |
|
||||
| **API reference** | [datadog](/reference/integrations-datadog) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/datadog |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
Trace your Haystack pipelines with [Datadog](https://www.datadoghq.com/) through [Datadog's tracing library `ddtrace`](https://ddtrace.readthedocs.io/en/stable/). Haystack captures detailed information about pipeline runs, like API calls, context data, and prompts, so you can see the complete trace of your pipeline execution in Datadog.
|
||||
|
||||
## Installation
|
||||
|
||||
Install the `datadog-haystack` package:
|
||||
|
||||
```shell
|
||||
pip install datadog-haystack
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. A way to receive traces, such as a running [Datadog Agent](https://docs.datadoghq.com/agent/). `ddtrace` sends traces to the Datadog Agent at `localhost:8126` by default.
|
||||
2. Configure `ddtrace` through the standard mechanisms, for example the `DD_SERVICE`, `DD_ENV`, and `DD_VERSION` environment variables, or by running your application with the `ddtrace-run` command. See the [ddtrace documentation](https://ddtrace.readthedocs.io/en/stable/) for more details.
|
||||
|
||||
## Usage
|
||||
|
||||
Enable the `DatadogTracer` directly to trace any Haystack pipeline, without adding a component to it. Make sure to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable before importing any Haystack components.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||||
|
||||
import ddtrace
|
||||
|
||||
from haystack import Pipeline, tracing
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
from haystack_integrations.tracing.datadog import DatadogTracer
|
||||
|
||||
# Enable the Datadog tracer
|
||||
tracing.enable_tracing(DatadogTracer(ddtrace.tracer))
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("prompt_builder", ChatPromptBuilder())
|
||||
pipe.add_component("llm", OpenAIChatGenerator())
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
messages = [
|
||||
ChatMessage.from_system(
|
||||
"Always respond in German even if some input data is in other languages.",
|
||||
),
|
||||
ChatMessage.from_user("Tell me about {{location}}"),
|
||||
]
|
||||
|
||||
response = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"location": "Berlin"},
|
||||
"template": messages,
|
||||
},
|
||||
},
|
||||
)
|
||||
print(response["llm"]["replies"][0])
|
||||
```
|
||||
|
||||
Each pipeline run produces a trace that includes the entire execution context, including prompts, completions, and metadata. You can then view the traces in your Datadog dashboard.
|
||||
|
||||
## Alternative: the DatadogConnector component
|
||||
|
||||
If you prefer to manage tracing as part of your pipeline definition (for example, so it serializes to YAML), you can add the `DatadogConnector` component instead. It enables the same Datadog tracing as soon as it is initialized.
|
||||
|
||||
:::info
|
||||
See the [`DatadogConnector` documentation page](../../pipeline-components/connectors/datadogconnector.mdx) for full usage examples, or check out the [integration page](https://haystack.deepset.ai/integrations/datadog).
|
||||
:::
|
||||
@@ -0,0 +1,110 @@
|
||||
---
|
||||
title: "Langfuse"
|
||||
id: langfuse
|
||||
slug: "/tracing-langfuse"
|
||||
description: "Learn how to trace your Haystack pipelines with Langfuse."
|
||||
---
|
||||
|
||||
import ClickableImage from "@site/src/components/ClickableImage";
|
||||
|
||||
# Langfuse
|
||||
|
||||
Learn how to trace your Haystack pipelines with Langfuse.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Tracer class** | `LangfuseTracer` |
|
||||
| **How to enable** | Enable the tracer with `tracing.enable_tracing(LangfuseTracer(langfuse))`, or add the `LangfuseConnector` component to your pipeline |
|
||||
| **Content tracing** | Required. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` |
|
||||
| **Package** | `langfuse-haystack` |
|
||||
| **API reference** | [langfuse](/reference/integrations-langfuse) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langfuse |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
Trace your Haystack pipelines with the [Langfuse](https://langfuse.com/) UI. Langfuse captures detailed information about pipeline runs, like API calls, context data, prompts, and more. Use it to monitor model performance such as token usage and cost, find areas for improvement, and create datasets from your pipeline executions.
|
||||
|
||||
## Installation
|
||||
|
||||
Install the `langfuse-haystack` package:
|
||||
|
||||
```shell
|
||||
pip install langfuse-haystack
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. An active Langfuse [account](https://cloud.langfuse.com/).
|
||||
2. Set the `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` environment variables with your Langfuse secret and public keys, found in your account profile.
|
||||
3. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` to enable tracing.
|
||||
|
||||
:::info[Usage Notice]
|
||||
To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import. An even better practice is to set these environment variables in your shell before running the script.
|
||||
:::
|
||||
|
||||
## Usage
|
||||
|
||||
Enable the `LangfuseTracer` directly to trace any Haystack pipeline, without adding a component to it.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com"
|
||||
os.environ["LANGFUSE_SECRET_KEY"] = "<your-secret-key>"
|
||||
os.environ["LANGFUSE_PUBLIC_KEY"] = "<your-public-key>"
|
||||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||||
|
||||
from langfuse import Langfuse
|
||||
|
||||
from haystack import Pipeline, tracing
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
from haystack_integrations.tracing.langfuse import LangfuseTracer
|
||||
|
||||
# Enable the Langfuse tracer. The client reads your keys from the environment.
|
||||
langfuse = Langfuse()
|
||||
langfuse_tracer = LangfuseTracer(langfuse, name="Chat example")
|
||||
tracing.enable_tracing(langfuse_tracer)
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("prompt_builder", ChatPromptBuilder())
|
||||
pipe.add_component("llm", OpenAIChatGenerator())
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
messages = [
|
||||
ChatMessage.from_system(
|
||||
"Always respond in German even if some input data is in other languages.",
|
||||
),
|
||||
ChatMessage.from_user("Tell me about {{location}}"),
|
||||
]
|
||||
|
||||
response = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"location": "Berlin"},
|
||||
"template": messages,
|
||||
},
|
||||
},
|
||||
)
|
||||
print(response["llm"]["replies"][0])
|
||||
|
||||
# Flush any pending spans before the program exits
|
||||
langfuse_tracer.flush()
|
||||
```
|
||||
|
||||
Each pipeline run produces one trace that includes the entire execution context, including prompts, completions, and metadata. You can then view the trace in the Langfuse UI.
|
||||
<ClickableImage src="/img/11cec4f-langfuse-generation-span.png" alt="Langfuse trace detail view showing generation span with input prompt, output, metadata, latency, and cost information for a language model call" />
|
||||
|
||||
## Alternative: the LangfuseConnector component
|
||||
|
||||
If you prefer to manage tracing as part of your pipeline definition, you can add the `LangfuseConnector` component instead. It enables the same Langfuse tracing, exposes the `trace_url` as an output, and supports a custom `SpanHandler` for advanced span processing.
|
||||
|
||||
:::info
|
||||
See the [`LangfuseConnector` documentation page](../../pipeline-components/connectors/langfuseconnector.mdx) for full usage examples and advanced span customization, or read the [blog post](https://haystack.deepset.ai/blog/langfuse-integration) for a complete walkthrough.
|
||||
:::
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
title: "LoggingTracer"
|
||||
id: logging-tracer
|
||||
slug: "/tracing-logging-tracer"
|
||||
description: "Learn how to inspect the data flowing through your Haystack pipelines in real time with the LoggingTracer."
|
||||
---
|
||||
|
||||
import ClickableImage from "@site/src/components/ClickableImage";
|
||||
|
||||
# LoggingTracer
|
||||
|
||||
Learn how to inspect the data flowing through your Haystack pipelines in real time with the `LoggingTracer`.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Tracer class** | `LoggingTracer` |
|
||||
| **How to enable** | `tracing.enable_tracing(LoggingTracer(...))` |
|
||||
| **Content tracing** | Required to log inputs and outputs. Set `tracing.tracer.is_content_tracing_enabled = True` |
|
||||
| **Package** | Built into Haystack |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
Use Haystack's [`LoggingTracer`](https://github.com/deepset-ai/haystack/blob/main/haystack/tracing/logging_tracer.py) logs to inspect the data that's flowing through your pipeline in real time.
|
||||
|
||||
This feature is particularly helpful during experimentation and prototyping, as you don’t need to set up any tracing backend beforehand.
|
||||
|
||||
## Usage
|
||||
|
||||
Here’s how you can enable this tracer. In this example, we are adding color tags (this is optional) to highlight the components' names and inputs:
|
||||
|
||||
```python
|
||||
import logging
|
||||
from haystack import tracing
|
||||
from haystack.tracing.logging_tracer import LoggingTracer
|
||||
|
||||
logging.basicConfig(
|
||||
format="%(levelname)s - %(name)s - %(message)s",
|
||||
level=logging.WARNING,
|
||||
)
|
||||
logging.getLogger("haystack").setLevel(logging.DEBUG)
|
||||
|
||||
tracing.tracer.is_content_tracing_enabled = (
|
||||
True # to enable tracing/logging content (inputs/outputs)
|
||||
)
|
||||
tracing.enable_tracing(
|
||||
LoggingTracer(
|
||||
tags_color_strings={
|
||||
"haystack.component.input": "\x1b[1;31m",
|
||||
"haystack.component.name": "\x1b[1;34m",
|
||||
},
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
Here’s what the resulting log would look like when a pipeline is run:
|
||||
<ClickableImage src="/img/55c3d5c84282d726c95fb3350ec36be49a354edca8a6164f5dffdab7121cec58-image_2.png" alt="Console output showing Haystack pipeline execution with DEBUG level tracing logs including component names, types, and input/output specifications" />
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
title: "MLflow"
|
||||
id: mlflow
|
||||
slug: "/tracing-mlflow"
|
||||
description: "Learn how to trace your Haystack pipelines with MLflow."
|
||||
---
|
||||
|
||||
# MLflow
|
||||
|
||||
Learn how to trace your Haystack pipelines with MLflow.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **How to enable** | `mlflow.haystack.autolog()` |
|
||||
| **Content tracing** | Captured automatically, including latencies, token usage, cost, and exceptions |
|
||||
| **Package** | `mlflow` |
|
||||
| **Integration guide** | https://haystack.deepset.ai/integrations/mlflow |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
[MLflow](https://mlflow.org/) is an open-source platform for managing the end-to-end machine learning and AI lifecycle. MLflow provides native tracing support for Haystack, so you can capture traces from all your pipelines and components with a single line of code.
|
||||
|
||||
## Installation
|
||||
|
||||
Install MLflow:
|
||||
|
||||
```shell
|
||||
pip install mlflow
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Enable automatic tracing for all Haystack pipelines and components:
|
||||
|
||||
```python
|
||||
import mlflow
|
||||
|
||||
mlflow.haystack.autolog()
|
||||
# Optionally set an experiment name
|
||||
mlflow.set_experiment("Haystack")
|
||||
```
|
||||
|
||||
This automatically captures traces from all Haystack pipelines and components, including latencies, token usage, cost, and any exceptions.
|
||||
|
||||
:::info
|
||||
Check out the [MLflow Haystack integration guide](https://haystack.deepset.ai/integrations/mlflow) for a full walkthrough with examples.
|
||||
:::
|
||||
@@ -0,0 +1,153 @@
|
||||
---
|
||||
title: "OpenTelemetry"
|
||||
id: opentelemetry
|
||||
slug: "/tracing-opentelemetry"
|
||||
description: "Learn how to trace your Haystack pipelines with OpenTelemetry."
|
||||
---
|
||||
|
||||
import ClickableImage from "@site/src/components/ClickableImage";
|
||||
|
||||
# OpenTelemetry
|
||||
|
||||
Learn how to trace your Haystack pipelines with OpenTelemetry.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Tracer class** | `OpenTelemetryTracer` |
|
||||
| **How to enable** | Configure an OpenTelemetry `TracerProvider`, then enable the tracer with `tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))`, or add the `OpenTelemetryConnector` component to your pipeline |
|
||||
| **Content tracing** | Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` to trace component inputs and outputs |
|
||||
| **Package** | `opentelemetry-haystack` |
|
||||
| **API reference** | [opentelemetry](/reference/integrations-opentelemetry) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/opentelemetry |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
[OpenTelemetry](https://opentelemetry.io/) is an open-source observability framework for collecting traces, metrics, and logs. Haystack integrates with OpenTelemetry, so you can send traces of your pipeline runs to any OpenTelemetry-compatible backend.
|
||||
|
||||
:::info[Moving to an integration]
|
||||
`OpenTelemetryTracer` is deprecated in Haystack core and is moving to the `opentelemetry-haystack` package. Starting with Haystack 3.0, OpenTelemetry tracing is no longer auto-enabled when `opentelemetry-sdk` is installed. Install the integration and either enable the `OpenTelemetryTracer` directly or add the `OpenTelemetryConnector` component to your pipeline.
|
||||
:::
|
||||
|
||||
## Installation
|
||||
|
||||
Install the `opentelemetry-haystack` package:
|
||||
|
||||
```shell
|
||||
pip install opentelemetry-haystack
|
||||
```
|
||||
|
||||
To add traces to even deeper levels of your pipelines, we recommend you check out [OpenTelemetry integrations](https://opentelemetry.io/ecosystem/registry/?s=python), such as:
|
||||
|
||||
- [`urllib3` instrumentation](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-urllib3) for tracing HTTP requests in your pipeline,
|
||||
- [OpenAI instrumentation](https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-openai) for tracing OpenAI requests.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
A configured OpenTelemetry `TracerProvider` with an exporter, for example an OTLP exporter that sends traces to a collector or a backend. Set up the provider before enabling the tracer.
|
||||
|
||||
## Usage
|
||||
|
||||
Enable the `OpenTelemetryTracer` directly to trace any Haystack pipeline, without adding a component to it. Configure your `TracerProvider` and set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable before importing any Haystack components.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||||
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
|
||||
from opentelemetry.sdk.resources import Resource
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import BatchSpanProcessor
|
||||
from opentelemetry.semconv.resource import ResourceAttributes
|
||||
|
||||
# Configure the OpenTelemetry SDK. A service name is required for most backends.
|
||||
resource = Resource(attributes={ResourceAttributes.SERVICE_NAME: "haystack"})
|
||||
tracer_provider = TracerProvider(resource=resource)
|
||||
tracer_provider.add_span_processor(
|
||||
BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")),
|
||||
)
|
||||
trace.set_tracer_provider(tracer_provider)
|
||||
|
||||
from haystack import tracing
|
||||
from haystack_integrations.tracing.opentelemetry import OpenTelemetryTracer
|
||||
|
||||
# Enable the OpenTelemetry tracer
|
||||
tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))
|
||||
```
|
||||
|
||||
Each pipeline run then produces a trace that includes the entire execution context, including prompts, completions, and metadata. You can view the traces in your OpenTelemetry-compatible backend.
|
||||
|
||||
## Alternative: the OpenTelemetryConnector component
|
||||
|
||||
If you prefer to manage tracing as part of your pipeline definition, you can add the `OpenTelemetryConnector` component instead. It enables the same OpenTelemetry tracing as soon as it is initialized.
|
||||
|
||||
:::info
|
||||
See the [`OpenTelemetryConnector` documentation page](../../pipeline-components/connectors/opentelemetryconnector.mdx) for full usage examples, or check out the [integration page](https://haystack.deepset.ai/integrations/opentelemetry).
|
||||
:::
|
||||
|
||||
## Visualizing Traces During Development
|
||||
|
||||
Use [Jaeger](https://www.jaegertracing.io/docs/1.6/getting-started/) as a lightweight tracing backend for local pipeline development. This allows you to experiment with tracing without the need for a complex tracing backend.
|
||||
<ClickableImage src="/img/dd906d7-Screenshot_2024-02-22_at_16.51.01.png" alt="Jaeger UI trace timeline displaying haystack pipeline execution with component spans showing duration and nesting of operations" />
|
||||
|
||||
1. Run the Jaeger container. This creates a tracing backend as well as a UI to visualize the traces:
|
||||
|
||||
```shell
|
||||
docker run --rm -d --name jaeger \
|
||||
-e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
|
||||
-p 6831:6831/udp \
|
||||
-p 6832:6832/udp \
|
||||
-p 5778:5778 \
|
||||
-p 16686:16686 \
|
||||
-p 4317:4317 \
|
||||
-p 4318:4318 \
|
||||
-p 14250:14250 \
|
||||
-p 14268:14268 \
|
||||
-p 14269:14269 \
|
||||
-p 9411:9411 \
|
||||
jaegertracing/all-in-one:latest
|
||||
```
|
||||
2. Install the integration and the OTLP exporter:
|
||||
|
||||
```shell
|
||||
pip install opentelemetry-haystack
|
||||
pip install opentelemetry-exporter-otlp
|
||||
```
|
||||
3. Configure `OpenTelemetry` to use the Jaeger backend and enable the tracer:
|
||||
|
||||
```python
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
|
||||
from opentelemetry.sdk.resources import Resource
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import BatchSpanProcessor
|
||||
from opentelemetry.semconv.resource import ResourceAttributes
|
||||
|
||||
from haystack import tracing
|
||||
from haystack_integrations.tracing.opentelemetry import OpenTelemetryTracer
|
||||
|
||||
# Service name is required for most backends
|
||||
resource = Resource(attributes={
|
||||
ResourceAttributes.SERVICE_NAME: "haystack"
|
||||
})
|
||||
|
||||
tracer_provider = TracerProvider(resource=resource)
|
||||
processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces"))
|
||||
tracer_provider.add_span_processor(processor)
|
||||
trace.set_tracer_provider(tracer_provider)
|
||||
|
||||
tracing.enable_tracing(OpenTelemetryTracer(trace.get_tracer("my_application")))
|
||||
```
|
||||
4. Run your pipeline:
|
||||
|
||||
```python
|
||||
...
|
||||
pipeline.run(...)
|
||||
...
|
||||
```
|
||||
5. Inspect the traces in the UI provided by Jaeger at [http://localhost:16686](http://localhost:16686/search).
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
title: "Weights & Biases Weave"
|
||||
id: weave
|
||||
slug: "/tracing-weave"
|
||||
description: "Learn how to trace your Haystack pipelines with Weights & Biases Weave."
|
||||
---
|
||||
|
||||
# Weights & Biases Weave
|
||||
|
||||
Learn how to trace your Haystack pipelines with Weights & Biases Weave.
|
||||
|
||||
<div className="key-value-table">
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Tracer class** | `WeaveTracer` |
|
||||
| **How to enable** | Enable the tracer with `tracing.enable_tracing(WeaveTracer(project_name="..."))`, or add the `WeaveConnector` component to your pipeline |
|
||||
| **Content tracing** | Required. Set `HAYSTACK_CONTENT_TRACING_ENABLED` to `true` |
|
||||
| **Package** | `weave-haystack` |
|
||||
| **API reference** | [Weave](/reference/integrations-weave) |
|
||||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/weave |
|
||||
|
||||
</div>
|
||||
|
||||
## Overview
|
||||
|
||||
Trace and visualize your pipeline execution in [Weights & Biases](https://wandb.ai/site/). Information captured by the Haystack tracing tool, such as API calls, context data, and prompts, is sent to Weights & Biases, where you can see the complete trace of your pipeline execution.
|
||||
|
||||
## Installation
|
||||
|
||||
Install the `weave-haystack` package:
|
||||
|
||||
```shell
|
||||
pip install weave-haystack
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. A Weave account. You can sign up for free on the [Weights & Biases website](https://wandb.ai/site).
|
||||
2. Set the `WANDB_API_KEY` environment variable with your Weights & Biases API key. Once logged in, you can find your API key on [your home page](https://wandb.ai/home).
|
||||
3. Set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true`.
|
||||
|
||||
## Usage
|
||||
|
||||
Enable the `WeaveTracer` directly to trace any Haystack pipeline, without adding a component to it. The `project_name` is the name that will appear in your Weave project.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||||
|
||||
from haystack import Pipeline, tracing
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
from haystack_integrations.tracing.weave import WeaveTracer
|
||||
|
||||
# Enable the Weave tracer
|
||||
tracing.enable_tracing(WeaveTracer(project_name="test_pipeline"))
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("prompt_builder", ChatPromptBuilder())
|
||||
pipe.add_component("llm", OpenAIChatGenerator())
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
messages = [
|
||||
ChatMessage.from_system(
|
||||
"Always respond in German even if some input data is in other languages.",
|
||||
),
|
||||
ChatMessage.from_user("Tell me about {{location}}"),
|
||||
]
|
||||
|
||||
response = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"location": "Berlin"},
|
||||
"template": messages,
|
||||
},
|
||||
},
|
||||
)
|
||||
print(response["llm"]["replies"][0])
|
||||
```
|
||||
|
||||
You can then see the complete trace for your pipeline at `https://wandb.ai/<user_name>/projects` under the project name you specified.
|
||||
|
||||
## Alternative: the WeaveConnector component
|
||||
|
||||
If you prefer to manage tracing as part of your pipeline definition, you can add the `WeaveConnector` component instead. It enables the same Weave tracing as soon as it runs.
|
||||
|
||||
:::info
|
||||
See the [`WeaveConnector` documentation page](../../pipeline-components/connectors/weaveconnector.mdx) for full usage examples.
|
||||
:::
|
||||
Reference in New Issue
Block a user