chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,268 @@
|
||||
---
|
||||
title: "Weave"
|
||||
id: integrations-weave
|
||||
description: "Weights & Bias integration for Haystack"
|
||||
slug: "/integrations-weave"
|
||||
---
|
||||
|
||||
|
||||
## haystack_integrations.components.connectors.weave.weave_connector
|
||||
|
||||
### WeaveConnector
|
||||
|
||||
Collects traces from your pipeline and sends them to Weights & Biases.
|
||||
|
||||
Add this component to your pipeline to integrate with the Weights & Biases Weave framework for tracing and
|
||||
monitoring your pipeline components.
|
||||
|
||||
Note that you need to have the `WANDB_API_KEY` environment variable set to your Weights & Biases API key.
|
||||
|
||||
NOTE: If you don't have a Weights & Biases account it will interactively ask you to set one and your input
|
||||
will then be stored in ~/.netrc
|
||||
|
||||
In addition, you need to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` in order to
|
||||
enable Haystack tracing in your pipeline.
|
||||
|
||||
To use this connector simply add it to your pipeline without any connections, and it will automatically start
|
||||
sending traces to Weights & Biases.
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from haystack import Pipeline
|
||||
from haystack.components.builders import ChatPromptBuilder
|
||||
from haystack.components.generators.chat import OpenAIChatGenerator
|
||||
from haystack.dataclasses import ChatMessage
|
||||
|
||||
from haystack_integrations.components.connectors import WeaveConnector
|
||||
|
||||
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "true"
|
||||
|
||||
pipe = Pipeline()
|
||||
pipe.add_component("prompt_builder", ChatPromptBuilder())
|
||||
pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))
|
||||
pipe.connect("prompt_builder.prompt", "llm.messages")
|
||||
|
||||
connector = WeaveConnector(pipeline_name="test_pipeline")
|
||||
pipe.add_component("weave", connector)
|
||||
|
||||
messages = [
|
||||
ChatMessage.from_system(
|
||||
"Always respond in German even if some input data is in other languages."
|
||||
),
|
||||
ChatMessage.from_user("Tell me about {{location}}"),
|
||||
]
|
||||
|
||||
response = pipe.run(
|
||||
data={
|
||||
"prompt_builder": {
|
||||
"template_variables": {"location": "Berlin"},
|
||||
"template": messages,
|
||||
}
|
||||
}
|
||||
)
|
||||
print(response["llm"]["replies"][0])
|
||||
```
|
||||
|
||||
You should then head to `https://wandb.ai/<user_name>/projects` and see the complete trace for your pipeline under
|
||||
the pipeline name you specified, when creating the `WeaveConnector`
|
||||
|
||||
#### __init__
|
||||
|
||||
```python
|
||||
__init__(
|
||||
pipeline_name: str, weave_init_kwargs: dict[str, Any] | None = None
|
||||
) -> None
|
||||
```
|
||||
|
||||
Initialize WeaveConnector.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- **pipeline_name** (<code>str</code>) – The name of the pipeline you want to trace.
|
||||
- **weave_init_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional arguments to pass to the WeaveTracer client.
|
||||
|
||||
#### warm_up
|
||||
|
||||
```python
|
||||
warm_up() -> None
|
||||
```
|
||||
|
||||
Initialize the WeaveTracer.
|
||||
|
||||
#### run
|
||||
|
||||
```python
|
||||
run() -> dict[str, str]
|
||||
```
|
||||
|
||||
Run the WeaveConnector, initializing the tracer if needed.
|
||||
|
||||
#### to_dict
|
||||
|
||||
```python
|
||||
to_dict() -> dict[str, Any]
|
||||
```
|
||||
|
||||
Serializes the component to a dictionary.
|
||||
|
||||
**Returns:**
|
||||
|
||||
- <code>dict\[str, Any\]</code> – Dictionary with all the necessary information to recreate this component.
|
||||
|
||||
#### from_dict
|
||||
|
||||
```python
|
||||
from_dict(data: dict[str, Any]) -> WeaveConnector
|
||||
```
|
||||
|
||||
Deserializes the component from a dictionary.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- **data** (<code>dict\[str, Any\]</code>) – Dictionary to deserialize from.
|
||||
|
||||
**Returns:**
|
||||
|
||||
- <code>WeaveConnector</code> – Deserialized component.
|
||||
|
||||
## haystack_integrations.tracing.weave.tracer
|
||||
|
||||
### WeaveSpan
|
||||
|
||||
Bases: <code>Span</code>
|
||||
|
||||
A bridge between Haystack's Span interface and Weave's Call object.
|
||||
|
||||
Stores metadata about a component execution and its inputs and outputs, and manages the attributes/tags
|
||||
that describe the operation.
|
||||
|
||||
#### set_tag
|
||||
|
||||
```python
|
||||
set_tag(key: str, value: Any) -> None
|
||||
```
|
||||
|
||||
Set a tag by adding it to the call's inputs.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- **key** (<code>str</code>) – The tag key.
|
||||
- **value** (<code>Any</code>) – The tag value.
|
||||
|
||||
#### set_tags
|
||||
|
||||
```python
|
||||
set_tags(tags: dict[str, Any]) -> None
|
||||
```
|
||||
|
||||
Set multiple tags at once by iterating over the provided dictionary.
|
||||
|
||||
#### raw_span
|
||||
|
||||
```python
|
||||
raw_span() -> Any
|
||||
```
|
||||
|
||||
Access to the underlying Weave Call object.
|
||||
|
||||
#### get_correlation_data_for_logs
|
||||
|
||||
```python
|
||||
get_correlation_data_for_logs() -> dict[str, Any]
|
||||
```
|
||||
|
||||
Correlation data for logging.
|
||||
|
||||
#### set_call
|
||||
|
||||
```python
|
||||
set_call(call: Call) -> None
|
||||
```
|
||||
|
||||
Set the underlying Weave Call object for this span.
|
||||
|
||||
#### get_attributes
|
||||
|
||||
```python
|
||||
get_attributes() -> dict[str, Any]
|
||||
```
|
||||
|
||||
Return the accumulated attributes dictionary for this span.
|
||||
|
||||
### WeaveTracer
|
||||
|
||||
Bases: <code>Tracer</code>
|
||||
|
||||
Implements a Haystack's Tracer to make an interface with Weights and Bias Weave.
|
||||
|
||||
It's responsible for creating and managing Weave calls, and for converting Haystack spans
|
||||
to Weave spans. It creates spans for each Haystack component run.
|
||||
|
||||
#### __init__
|
||||
|
||||
```python
|
||||
__init__(project_name: str, **weave_init_kwargs: Any) -> None
|
||||
```
|
||||
|
||||
Initialize the WeaveTracer.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
- **project_name** (<code>str</code>) – The name of the project to trace, this is will be the name appearing in Weave project.
|
||||
- **weave_init_kwargs** (<code>Any</code>) – Additional arguments to pass to the Weave client.
|
||||
|
||||
#### create_call
|
||||
|
||||
```python
|
||||
create_call(
|
||||
attributes: dict,
|
||||
client: WeaveClient,
|
||||
parent_span: WeaveSpan | None,
|
||||
operation_name: str,
|
||||
) -> Call
|
||||
```
|
||||
|
||||
Create and return a Weave Call from the given span attributes and client.
|
||||
|
||||
#### current_span
|
||||
|
||||
```python
|
||||
current_span() -> Span | None
|
||||
```
|
||||
|
||||
Get the current active span.
|
||||
|
||||
#### trace
|
||||
|
||||
```python
|
||||
trace(
|
||||
operation_name: str,
|
||||
tags: dict[str, Any] | None = None,
|
||||
parent_span: WeaveSpan | None = None,
|
||||
) -> Iterator[WeaveSpan]
|
||||
```
|
||||
|
||||
A context manager that creates and manages spans for tracking operations in Weights & Biases Weave.
|
||||
|
||||
It has two main workflows:
|
||||
|
||||
A) For regular operations (operation_name != "haystack.component.run"):
|
||||
Creates a Weave Call immediately
|
||||
Creates a WeaveSpan with this call
|
||||
Sets any provided tags
|
||||
Yields the span for use in the with block
|
||||
When the block ends, updates the call with pipeline output data
|
||||
|
||||
B) For component runs (operation_name == "haystack.component.run"):
|
||||
Creates a WeaveSpan WITHOUT a call initially (deferred creation)
|
||||
Sets any provided tags
|
||||
Yields the span for use in the with block
|
||||
Creates the actual Weave Call only at the end, when all component information is available
|
||||
Updates the call with component output data
|
||||
|
||||
This distinction is important because Weave's calls can't be updated once created, but the content
|
||||
tags are only set on the Span at a later stage. To get the inputs on call creation, we need to create
|
||||
the call after we yield the span.
|
||||
Reference in New Issue
Block a user