c56bef871b
Sync docs with Docusaurus / sync (push) Waiting to run
Tests / Check if changed (push) Waiting to run
Tests / format (push) Blocked by required conditions
Tests / check-imports (push) Blocked by required conditions
Tests / Unit / macos-latest (push) Blocked by required conditions
Tests / Unit / ubuntu-latest (push) Blocked by required conditions
Tests / Unit / windows-latest (push) Blocked by required conditions
Tests / mypy (push) Blocked by required conditions
Tests / Integration / ubuntu-latest (push) Blocked by required conditions
Tests / Integration / macos-latest (push) Blocked by required conditions
Tests / Integration / windows-latest (push) Blocked by required conditions
Tests / notify-slack-on-failure (push) Blocked by required conditions
Tests / Mark tests as completed (push) Blocked by required conditions
Docker image release / Build base image (push) Waiting to run
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
269 lines
6.4 KiB
Markdown
269 lines
6.4 KiB
Markdown
---
|
||
title: "Weave"
|
||
id: integrations-weave
|
||
description: "Weights & Bias integration for Haystack"
|
||
slug: "/integrations-weave"
|
||
---
|
||
|
||
|
||
## haystack_integrations.components.connectors.weave.weave_connector
|
||
|
||
### WeaveConnector
|
||
|
||
Collects traces from your pipeline and sends them to Weights & Biases.
|
||
|
||
Add this component to your pipeline to integrate with the Weights & Biases Weave framework for tracing and
|
||
monitoring your pipeline components.
|
||
|
||
Note that you need to have the `WANDB_API_KEY` environment variable set to your Weights & Biases API key.
|
||
|
||
NOTE: If you don't have a Weights & Biases account it will interactively ask you to set one and your input
|
||
will then be stored in ~/.netrc
|
||
|
||
In addition, you need to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` in order to
|
||
enable Haystack tracing in your pipeline.
|
||
|
||
To use this connector simply add it to your pipeline without any connections, and it will automatically start
|
||
sending traces to Weights & Biases.
|
||
|
||
Example:
|
||
|
||
```python
|
||
import os
|
||
|
||
from haystack import Pipeline
|
||
from haystack.components.builders import ChatPromptBuilder
|
||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||
from haystack.dataclasses import ChatMessage
|
||
|
||
from haystack_integrations.components.connectors import WeaveConnector
|
||
|
||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||
|
||
pipe = Pipeline()
|
||
pipe.add_component("prompt_builder", ChatPromptBuilder())
|
||
pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))
|
||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||
|
||
connector = WeaveConnector(pipeline_name="test_pipeline")
|
||
pipe.add_component("weave", connector)
|
||
|
||
messages = [
|
||
ChatMessage.from_system(
|
||
"Always respond in German even if some input data is in other languages."
|
||
),
|
||
ChatMessage.from_user("Tell me about {{location}}"),
|
||
]
|
||
|
||
response = pipe.run(
|
||
data={
|
||
"prompt_builder": {
|
||
"template_variables": {"location": "Berlin"},
|
||
"template": messages,
|
||
}
|
||
}
|
||
)
|
||
print(response["llm"]["replies"][0])
|
||
```
|
||
|
||
You should then head to `https://wandb.ai/<user_name>/projects` and see the complete trace for your pipeline under
|
||
the pipeline name you specified, when creating the `WeaveConnector`
|
||
|
||
#### __init__
|
||
|
||
```python
|
||
__init__(
|
||
pipeline_name: str, weave_init_kwargs: dict[str, Any] | None = None
|
||
) -> None
|
||
```
|
||
|
||
Initialize WeaveConnector.
|
||
|
||
**Parameters:**
|
||
|
||
- **pipeline_name** (<code>str</code>) – The name of the pipeline you want to trace.
|
||
- **weave_init_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional arguments to pass to the WeaveTracer client.
|
||
|
||
#### warm_up
|
||
|
||
```python
|
||
warm_up() -> None
|
||
```
|
||
|
||
Initialize the WeaveTracer.
|
||
|
||
#### run
|
||
|
||
```python
|
||
run() -> dict[str, str]
|
||
```
|
||
|
||
Run the WeaveConnector, initializing the tracer if needed.
|
||
|
||
#### to_dict
|
||
|
||
```python
|
||
to_dict() -> dict[str, Any]
|
||
```
|
||
|
||
Serializes the component to a dictionary.
|
||
|
||
**Returns:**
|
||
|
||
- <code>dict\[str, Any\]</code> – Dictionary with all the necessary information to recreate this component.
|
||
|
||
#### from_dict
|
||
|
||
```python
|
||
from_dict(data: dict[str, Any]) -> WeaveConnector
|
||
```
|
||
|
||
Deserializes the component from a dictionary.
|
||
|
||
**Parameters:**
|
||
|
||
- **data** (<code>dict\[str, Any\]</code>) – Dictionary to deserialize from.
|
||
|
||
**Returns:**
|
||
|
||
- <code>WeaveConnector</code> – Deserialized component.
|
||
|
||
## haystack_integrations.tracing.weave.tracer
|
||
|
||
### WeaveSpan
|
||
|
||
Bases: <code>Span</code>
|
||
|
||
A bridge between Haystack's Span interface and Weave's Call object.
|
||
|
||
Stores metadata about a component execution and its inputs and outputs, and manages the attributes/tags
|
||
that describe the operation.
|
||
|
||
#### set_tag
|
||
|
||
```python
|
||
set_tag(key: str, value: Any) -> None
|
||
```
|
||
|
||
Set a tag by adding it to the call's inputs.
|
||
|
||
**Parameters:**
|
||
|
||
- **key** (<code>str</code>) – The tag key.
|
||
- **value** (<code>Any</code>) – The tag value.
|
||
|
||
#### set_tags
|
||
|
||
```python
|
||
set_tags(tags: dict[str, Any]) -> None
|
||
```
|
||
|
||
Set multiple tags at once by iterating over the provided dictionary.
|
||
|
||
#### raw_span
|
||
|
||
```python
|
||
raw_span() -> Any
|
||
```
|
||
|
||
Access to the underlying Weave Call object.
|
||
|
||
#### get_correlation_data_for_logs
|
||
|
||
```python
|
||
get_correlation_data_for_logs() -> dict[str, Any]
|
||
```
|
||
|
||
Correlation data for logging.
|
||
|
||
#### set_call
|
||
|
||
```python
|
||
set_call(call: Call) -> None
|
||
```
|
||
|
||
Set the underlying Weave Call object for this span.
|
||
|
||
#### get_attributes
|
||
|
||
```python
|
||
get_attributes() -> dict[str, Any]
|
||
```
|
||
|
||
Return the accumulated attributes dictionary for this span.
|
||
|
||
### WeaveTracer
|
||
|
||
Bases: <code>Tracer</code>
|
||
|
||
Implements a Haystack's Tracer to make an interface with Weights and Bias Weave.
|
||
|
||
It's responsible for creating and managing Weave calls, and for converting Haystack spans
|
||
to Weave spans. It creates spans for each Haystack component run.
|
||
|
||
#### __init__
|
||
|
||
```python
|
||
__init__(project_name: str, **weave_init_kwargs: Any) -> None
|
||
```
|
||
|
||
Initialize the WeaveTracer.
|
||
|
||
**Parameters:**
|
||
|
||
- **project_name** (<code>str</code>) – The name of the project to trace, this is will be the name appearing in Weave project.
|
||
- **weave_init_kwargs** (<code>Any</code>) – Additional arguments to pass to the Weave client.
|
||
|
||
#### create_call
|
||
|
||
```python
|
||
create_call(
|
||
attributes: dict,
|
||
client: WeaveClient,
|
||
parent_span: WeaveSpan | None,
|
||
operation_name: str,
|
||
) -> Call
|
||
```
|
||
|
||
Create and return a Weave Call from the given span attributes and client.
|
||
|
||
#### current_span
|
||
|
||
```python
|
||
current_span() -> Span | None
|
||
```
|
||
|
||
Get the current active span.
|
||
|
||
#### trace
|
||
|
||
```python
|
||
trace(
|
||
operation_name: str,
|
||
tags: dict[str, Any] | None = None,
|
||
parent_span: WeaveSpan | None = None,
|
||
) -> Iterator[WeaveSpan]
|
||
```
|
||
|
||
A context manager that creates and manages spans for tracking operations in Weights & Biases Weave.
|
||
|
||
It has two main workflows:
|
||
|
||
A) For regular operations (operation_name != "haystack.component.run"):
|
||
Creates a Weave Call immediately
|
||
Creates a WeaveSpan with this call
|
||
Sets any provided tags
|
||
Yields the span for use in the with block
|
||
When the block ends, updates the call with pipeline output data
|
||
|
||
B) For component runs (operation_name == "haystack.component.run"):
|
||
Creates a WeaveSpan WITHOUT a call initially (deferred creation)
|
||
Sets any provided tags
|
||
Yields the span for use in the with block
|
||
Creates the actual Weave Call only at the end, when all component information is available
|
||
Updates the call with component output data
|
||
|
||
This distinction is important because Weave's calls can't be updated once created, but the content
|
||
tags are only set on the Span at a later stage. To get the inputs on call creation, we need to create
|
||
the call after we yield the span.
|