5a558eb09e
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled
348 lines
10 KiB
Plaintext
348 lines
10 KiB
Plaintext
---
|
|
headline: Log media & attachments | Opik Documentation
|
|
og:description: Capture multimodal traces in Opik by logging images, videos, and audio
|
|
files using the Python SDK's Attachment type.
|
|
og:site_name: Opik Documentation
|
|
og:title: Log Media & Attachments with Opik
|
|
title: Log media & attachments
|
|
canonical-url: https://www.comet.com/docs/opik/tracing/advanced/log_multimodal_traces
|
|
---
|
|
|
|
Opik supports multimodal traces allowing you to track not just the text input
|
|
and output of your LLM, but also images, videos and audio and any other media.
|
|
|
|
<Frame>
|
|
<img src="/img/tracing/attachments.png" />
|
|
</Frame>
|
|
|
|
## Logging Attachments
|
|
|
|
In the Python SDK, you can use the `Attachment` type to add files to your traces.
|
|
Attachements can be images, videos, audio files or any other file that you might
|
|
want to log to Opik.
|
|
|
|
Each attachment is made up of the following fields:
|
|
|
|
- `data`: The path to the file, raw bytes, or a base64 encoded string of the file
|
|
- `file_name`: Optional name for the attachment (required when using raw bytes without a file path)
|
|
- `content_type`: The content type of the file formatted as a MIME type
|
|
|
|
These attachements can then be logged to your traces and spans using The
|
|
`opik_context.update_current_span` and `opik_context.update_current_trace`
|
|
methods:
|
|
|
|
### Using file paths
|
|
|
|
The most common way to log attachments is by providing a file path:
|
|
|
|
```python wordWrap
|
|
from opik import opik_context, track, Attachment
|
|
|
|
@track
|
|
def my_llm_agent(input):
|
|
# LLM chain code
|
|
# ...
|
|
|
|
# Update the trace with a file path
|
|
opik_context.update_current_trace(
|
|
attachments=[
|
|
Attachment(
|
|
data="<path to the image>",
|
|
content_type="image/png",
|
|
)
|
|
]
|
|
)
|
|
|
|
return "World!"
|
|
|
|
print(my_llm_agent("Hello!"))
|
|
```
|
|
|
|
### Using raw bytes (file-like data)
|
|
|
|
You can also pass raw bytes directly to an attachment. This is useful when you have
|
|
file content in memory (e.g., from an API response, generated content, or streaming data)
|
|
and don't want to write it to disk first:
|
|
|
|
```python wordWrap
|
|
from opik import opik_context, track, Attachment
|
|
|
|
@track
|
|
def process_image(image_bytes: bytes):
|
|
# Process the image
|
|
# ...
|
|
|
|
# Log the raw bytes as an attachment
|
|
opik_context.update_current_trace(
|
|
attachments=[
|
|
Attachment(
|
|
data=image_bytes, # Raw bytes
|
|
file_name="processed_image.png", # Required for bytes
|
|
content_type="image/png",
|
|
)
|
|
]
|
|
)
|
|
|
|
return "Image processed!"
|
|
|
|
# Example: Reading a file into memory and logging it
|
|
with open("image.png", "rb") as f:
|
|
image_data = f.read()
|
|
|
|
print(process_image(image_data))
|
|
```
|
|
|
|
<Note>
|
|
When using raw bytes, Opik automatically creates a temporary file for upload
|
|
and cleans it up after the attachment is uploaded. If you don't specify a
|
|
`content_type`, Opik will try to infer it from the `file_name` or default
|
|
to `application/octet-stream`.
|
|
</Note>
|
|
|
|
### Logging images from HTTP responses
|
|
|
|
A common use case is logging images fetched from external APIs or URLs:
|
|
|
|
```python wordWrap
|
|
import httpx
|
|
from opik import opik_context, track, Attachment
|
|
|
|
@track
|
|
def analyze_remote_image(image_url: str):
|
|
# Fetch image from URL
|
|
response = httpx.get(image_url)
|
|
image_bytes = response.content
|
|
content_type = response.headers.get("content-type", "image/jpeg")
|
|
|
|
# Log the fetched image as an attachment
|
|
opik_context.update_current_trace(
|
|
attachments=[
|
|
Attachment(
|
|
data=image_bytes,
|
|
file_name="remote_image.jpg",
|
|
content_type=content_type,
|
|
)
|
|
]
|
|
)
|
|
|
|
# Process the image...
|
|
return "Image analyzed!"
|
|
|
|
# Analyze an image from a URL
|
|
result = analyze_remote_image("https://example.com/image.jpg")
|
|
```
|
|
|
|
### Logging generated content
|
|
|
|
You can also log dynamically generated content like charts or reports:
|
|
|
|
```python wordWrap
|
|
from opik import opik_context, track, Attachment
|
|
import json
|
|
|
|
@track
|
|
def generate_report(data: dict):
|
|
# Generate a JSON report
|
|
report_bytes = json.dumps(data, indent=2).encode("utf-8")
|
|
|
|
opik_context.update_current_trace(
|
|
attachments=[
|
|
Attachment(
|
|
data=report_bytes,
|
|
file_name="report.json",
|
|
content_type="application/json",
|
|
)
|
|
]
|
|
)
|
|
|
|
return "Report generated!"
|
|
```
|
|
|
|
### Using the Opik client directly
|
|
|
|
You can also log attachments using the Opik client directly with both file paths and raw bytes:
|
|
|
|
```python wordWrap
|
|
import opik
|
|
from opik import Attachment
|
|
|
|
client = opik.Opik()
|
|
|
|
# Create a trace
|
|
trace = client.trace(
|
|
name="my-trace",
|
|
input={"query": "Process this data"},
|
|
project_name="my-project",
|
|
)
|
|
|
|
# Log attachment with file path
|
|
span_with_file = client.span(
|
|
trace_id=trace.id,
|
|
name="file-attachment-span",
|
|
attachments=[
|
|
Attachment(
|
|
data="/path/to/document.pdf",
|
|
content_type="application/pdf",
|
|
)
|
|
],
|
|
)
|
|
|
|
# Log attachment with raw bytes
|
|
binary_data = b"Hello, this is binary content!"
|
|
span_with_bytes = client.span(
|
|
trace_id=trace.id,
|
|
name="bytes-attachment-span",
|
|
attachments=[
|
|
Attachment(
|
|
data=binary_data,
|
|
file_name="data.bin",
|
|
content_type="application/octet-stream",
|
|
)
|
|
],
|
|
)
|
|
|
|
client.flush()
|
|
```
|
|
|
|
The attachements will be uploaded to the Opik platform and can be both previewed
|
|
and dowloaded from the UI.
|
|
|
|
<Frame>
|
|
<img src="/img/tracing/attachments.png" />
|
|
</Frame>
|
|
|
|
<Note>
|
|
In order to preview the attachements in the UI, you will need to supply a
|
|
supported content type for the attachment. We support the following content types:
|
|
|
|
- Image: `image/jpeg`, `image/png`, `image/gif` and `image/svg+xml`
|
|
- Video: `video/mp4` and `video/webm`
|
|
- Audio: `audio/wav`, `audio/vorbis` and `audio/x-wav`
|
|
- Text: `text/plain` and `text/markdown`
|
|
- PDF: `application/pdf`
|
|
- Other: `application/json` and `application/octet-stream`
|
|
|
|
</Note>
|
|
|
|
## Managing Attachments Programmatically
|
|
|
|
You can also manage attachments programmatically using the [`AttachmentClient`](https://www.comet.com/docs/opik/python-sdk-reference/Objects/AttachmentClient.html):
|
|
|
|
```python wordWrap
|
|
import opik
|
|
|
|
opik_client = opik.Opik()
|
|
attachment_client = opik_client.get_attachment_client()
|
|
|
|
# Get list of attachments
|
|
attachments_details = attachment_client.get_attachment_list(
|
|
project_name="my-project",
|
|
entity_id="some-trace-uuid-7",
|
|
entity_type="trace"
|
|
)
|
|
|
|
# Download an attachment
|
|
attachment_data = attachment_client.download_attachment(
|
|
project_name="my-project",
|
|
entity_type="trace",
|
|
entity_id="some-trace-uuid-7",
|
|
file_name="report.pdf",
|
|
mime_type="application/pdf"
|
|
)
|
|
|
|
# Upload a new attachment
|
|
attachment_client.upload_attachment(
|
|
project_name="my-project",
|
|
entity_type="trace",
|
|
entity_id="some-trace-uuid-7",
|
|
file_path="/path/to/document.pdf"
|
|
)
|
|
```
|
|
|
|
## Previewing base64 encoded images and image URLs
|
|
|
|
Opik automatically detects base64 encoded images and URLs logged to the platform,
|
|
once an image is detected we will hide the string to make the content more readable
|
|
and display the image in the UI. This is supported in the tracing view, datasets
|
|
view and experiment view.
|
|
|
|
For example if you are using the OpenAI SDK, if you pass an image to the model
|
|
as a URL, Opik will automatically detect it and display
|
|
the image in the UI:
|
|
|
|
```python wordWrap
|
|
from opik.integrations.openai import track_openai
|
|
from openai import OpenAI
|
|
|
|
# Make sure to wrap the OpenAI client to enable Opik tracing
|
|
client = track_openai(OpenAI())
|
|
|
|
response = client.chat.completions.create(
|
|
model="gpt-4o-mini",
|
|
messages=[
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{"type": "text", "text": "What's in this image?"},
|
|
{
|
|
"type": "image_url",
|
|
"image_url": {
|
|
"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
|
|
},
|
|
},
|
|
],
|
|
}
|
|
],
|
|
max_tokens=300,
|
|
)
|
|
|
|
print(response.choices[0])
|
|
```
|
|
|
|
<Frame>
|
|
<img src="/img/tracing/image_trace.png" />
|
|
</Frame>
|
|
|
|
## Embedded Attachments
|
|
|
|
When you embed base64-encoded media directly in your trace/span `input`, `output`, or `metadata` fields, Opik automatically optimizes storage and retrieval for performance.
|
|
|
|
### How It Works
|
|
|
|
For base64-encoded content larger than 250KB, Opik automatically extracts and stores it separately. This happens transparently - you don't need to change your code.
|
|
|
|
When you retrieve your traces or spans later, the attachments are automatically included by default. For faster queries when you don't need the attachment data, use the `strip_attachments=true` parameter.
|
|
|
|
### Size Limits
|
|
|
|
Opik Cloud supports embedded attachments up to **100MB per field**. This limit applies to individual string values in your `input`, `output`, or `metadata` fields.
|
|
|
|
<Note>
|
|
Base64 encoding increases file size by about 33%. For example, a 75MB video becomes ~100MB when base64-encoded.
|
|
</Note>
|
|
|
|
If you need to work with larger files:
|
|
|
|
1. **Use the Attachment API** - Upload files separately using `AttachmentClient` (recommended for files >50MB). See [Managing Attachments Programmatically](#managing-attachments-programmatically)
|
|
|
|
2. **Contact us** - [Get in touch](https://www.comet.com/site/about-us/contact-us/) if you need higher limits
|
|
|
|
3. **Self-host Opik** - Configure your own limits. See the [Self-hosting Guide](/v1/self-host/overview)
|
|
|
|
### Best Practices
|
|
|
|
- Embed smaller files directly - Opik handles them efficiently
|
|
- For files >50MB, use the Attachment API for better performance
|
|
- Use `strip_attachments=true` when querying if you don't need the attachment data
|
|
|
|
## Downloading attachments
|
|
|
|
You can download attachments in two ways:
|
|
|
|
1. **From the UI**: Hover over the attachments and click on the download icon
|
|
2. **Programmatically**: Use the `AttachmentClient` as shown in the examples above
|
|
|
|
<Tip>
|
|
Let's us know on [Github](https://github.com/comet-ml/opik/issues/new/choose) if you would like to us to support
|
|
additional image formats.
|
|
</Tip> |