chore: import upstream snapshot with attribution
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:25:44 +08:00
commit 5a558eb09e
11579 changed files with 1795921 additions and 0 deletions
@@ -0,0 +1,668 @@
import copy
import pytest
from unittest.mock import Mock, patch
from opik import exceptions
from opik.api_objects import opik_client
from opik.api_objects.dashboard import types
from opik.api_objects.dashboard.dashboard import Dashboard
from opik.rest_api.types import dashboard_public as dp
class FakeDashboardsApi:
def __init__(self, store):
self._store = store
self.update_calls = []
self.deleted = False
def update_dashboard(
self, dashboard_id, *, name=None, type=None, description=None, config=None
):
self.update_calls.append(
{"name": name, "type": type, "description": description, "config": config}
)
if config is not None:
self._store["config"] = config
if name is not None:
self._store["name"] = name
if description is not None:
self._store["description"] = description
return self._make()
def get_dashboard_by_id(self, dashboard_id):
return self._make()
def delete_dashboard(self, dashboard_id):
self.deleted = True
def _make(self):
return dp.DashboardPublic(
id=self._store["id"],
name=self._store["name"],
type=self._store.get("type"),
description=self._store.get("description"),
config=self._store["config"],
)
class FakeRest:
def __init__(self, store):
self.dashboards = FakeDashboardsApi(store)
def _make_dashboard(config, dashboard_type="multi_project", project_id=None):
store = {"id": "d1", "name": "My dash", "type": dashboard_type, "config": config}
rest = FakeRest(store)
public = dp.DashboardPublic(
id="d1",
name="My dash",
type=dashboard_type,
config=copy.deepcopy(config),
project_id=project_id,
)
client = Mock(spec=opik_client.Opik)
return (
Dashboard(dashboard_public=public, rest_client=rest, client=client),
rest,
store,
)
def _config_with_legacy_widget():
return {
"version": 4,
"mysteryTopLevel": "keep-me",
"sections": [
{
"id": "s1",
"title": "Overview",
"widgets": [
{
"id": "old",
"type": "future_widget",
"config": {"unknownField": 123},
"futureWidgetProp": "x",
}
],
"layout": [{"i": "old", "x": 0, "y": 0, "w": 2, "h": 2}],
}
],
"lastModified": 111,
}
def test_add_widget__preserves_unknown_fields_round_trip():
dashboard, rest, _ = _make_dashboard(_config_with_legacy_widget())
widget_id = dashboard.add_widget(
types.DashboardWidget(
type=types.WidgetType.TEXT_MARKDOWN,
title="Notes",
config=types.TextMarkdownConfig(content="hello"),
),
)
config = dashboard.config
assert config["mysteryTopLevel"] == "keep-me"
old = next(w for w in config["sections"][0]["widgets"] if w["id"] == "old")
assert old["futureWidgetProp"] == "x"
assert old["config"]["unknownField"] == 123
assert widget_id in [w["id"] for w in config["sections"][0]["widgets"]]
assert widget_id in [i["i"] for i in config["sections"][0]["layout"]]
assert config["version"] == 4
assert config["lastModified"] != 111
def test_add_widget__project_scoped_injects_project_id():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, _, _ = _make_dashboard(config, project_id="proj-xyz")
widget_id = dashboard.add_widget(
types.DashboardWidget(
type=types.WidgetType.PROJECT_STATS_CARD,
config=types.ProjectStatsCardConfig(
metric=types.StatsCardMetric.TRACE_COUNT
),
),
)
widget = dashboard.config["sections"][0]["widgets"][0]
assert widget["id"] == widget_id
assert widget["config"]["projectId"] == "proj-xyz"
def test_add_widget__project_scoped_without_dashboard_project_raises():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, rest, _ = _make_dashboard(config, project_id=None)
with pytest.raises(exceptions.DashboardValidationError, match="project-scoped"):
dashboard.add_widget(
types.DashboardWidget(type=types.WidgetType.PROJECT_STATS_CARD),
)
assert rest.dashboards.update_calls == []
def test_add_widget__incompatible_type_raises_before_write():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, rest, _ = _make_dashboard(config, dashboard_type="multi_project")
with pytest.raises(exceptions.DashboardValidationError, match="not supported"):
dashboard.add_widget(
types.DashboardWidget(type=types.WidgetType.EXPERIMENT_LEADERBOARD)
)
assert rest.dashboards.update_calls == []
def test_add_widget__unknown_section_raises():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, _, _ = _make_dashboard(config)
with pytest.raises(exceptions.DashboardValidationError, match="not found"):
dashboard.add_widget(
types.DashboardWidget(type=types.WidgetType.TEXT_MARKDOWN),
section_id="missing",
)
def test_mutation__unknown_version_refused():
config = {
"version": 5,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, rest, _ = _make_dashboard(config)
with pytest.raises(exceptions.DashboardValidationError, match="schema version 5"):
dashboard.add_widget(types.DashboardWidget(type=types.WidgetType.TEXT_MARKDOWN))
assert rest.dashboards.update_calls == []
def test_reads_never_fail__on_unknown_widget_type():
dashboard, _, _ = _make_dashboard(_config_with_legacy_widget())
state = dashboard.state
assert len(state.sections) == 1
assert state.sections[0].widgets[0].type == "future_widget"
def test_update_widget__merges_config():
config = {
"version": 4,
"sections": [
{
"id": "s1",
"title": "t",
"widgets": [
{"id": "w1", "type": "text_markdown", "config": {"content": "a"}}
],
"layout": [{"i": "w1", "x": 0, "y": 0, "w": 2, "h": 4}],
}
],
"lastModified": 1,
}
dashboard, rest, _ = _make_dashboard(config)
dashboard.update_widget("w1", title="New", config={"content": "b"})
widget = dashboard.config["sections"][0]["widgets"][0]
assert widget["title"] == "New"
assert widget["config"]["content"] == "b"
assert len(rest.dashboards.update_calls) == 1
assert rest.dashboards.update_calls[0]["config"] is not None
def test_remove_widget__removes_widget_and_layout():
config = {
"version": 4,
"sections": [
{
"id": "s1",
"title": "t",
"widgets": [
{"id": "w1", "type": "text_markdown", "config": {}},
{"id": "w2", "type": "text_markdown", "config": {}},
],
"layout": [
{"i": "w1", "x": 0, "y": 0, "w": 2, "h": 4},
{"i": "w2", "x": 2, "y": 0, "w": 2, "h": 4},
],
}
],
"lastModified": 1,
}
dashboard, _, _ = _make_dashboard(config)
dashboard.remove_widget("w1")
section = dashboard.config["sections"][0]
assert [w["id"] for w in section["widgets"]] == ["w2"]
assert [i["i"] for i in section["layout"]] == ["w2"]
def test_remove_widget__missing_raises():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, _, _ = _make_dashboard(config)
with pytest.raises(exceptions.DashboardValidationError, match="not found"):
dashboard.remove_widget("ghost")
def test_rename__sends_name_only():
dashboard, rest, _ = _make_dashboard(_config_with_legacy_widget())
dashboard.rename("Renamed")
assert dashboard.name == "Renamed"
assert rest.dashboards.update_calls[-1]["name"] == "Renamed"
assert rest.dashboards.update_calls[-1]["config"] is None
def test_set_description__sends_description_only():
dashboard, rest, _ = _make_dashboard(_config_with_legacy_widget())
dashboard.set_description("desc")
assert dashboard.description == "desc"
assert rest.dashboards.update_calls[-1]["description"] == "desc"
assert rest.dashboards.update_calls[-1]["config"] is None
def test_add_section__appends_and_returns_id():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, _, _ = _make_dashboard(config)
new_id = dashboard.add_section("Second")
section_ids = [s["id"] for s in dashboard.config["sections"]]
assert new_id in section_ids
assert len(section_ids) == 2
def test_delete__calls_rest():
dashboard, rest, _ = _make_dashboard(_config_with_legacy_widget())
dashboard.delete()
assert rest.dashboards.deleted is True
# --- Opik client-level methods ---
def test_create_dashboard__default_section_and_config():
client = opik_client.Opik()
captured = {}
def fake_create(*, name, config, type, description, project_id, project_name):
captured.update(
name=name,
config=config,
type=type,
description=description,
project_id=project_id,
project_name=project_name,
)
return dp.DashboardPublic(id="d1", name=name, type=type, config=config)
with patch.object(
client._rest_client.dashboards, "create_dashboard", side_effect=fake_create
):
dashboard = client.create_dashboard(
name="Prod", type=types.DashboardType.MULTI_PROJECT
)
assert dashboard.name == "Prod"
assert captured["type"] == "multi_project"
assert captured["config"]["version"] == types.DASHBOARD_VERSION
assert len(captured["config"]["sections"]) == 1
assert captured["config"]["sections"][0]["title"] == "Overview"
assert "lastModified" in captured["config"]
def test_create_dashboard__with_provided_sections():
client = opik_client.Opik()
captured = {}
def fake_create(*, name, config, type, description, project_id, project_name):
captured["config"] = config
return dp.DashboardPublic(id="d1", name=name, type=type, config=config)
section = types.DashboardSection(
title="Custom",
widgets=[types.DashboardWidget(id="w1", type=types.WidgetType.TEXT_MARKDOWN)],
layout=[types.DashboardLayoutItem(id="w1", x=0, y=0, w=2, h=4)],
)
with patch.object(
client._rest_client.dashboards, "create_dashboard", side_effect=fake_create
):
client.create_dashboard(name="Prod", sections=[section])
assert captured["config"]["sections"][0]["title"] == "Custom"
assert captured["config"]["sections"][0]["widgets"][0]["id"] == "w1"
def test_create_dashboard__project_scoped_widget_with_project_id_injects():
client = opik_client.Opik()
def fake_create(*, name, config, type, description, project_id, project_name):
return dp.DashboardPublic(id="d1", name=name, type=type, config=config)
section = types.DashboardSection(
title="S",
widgets=[
types.DashboardWidget(
id="w1",
type=types.WidgetType.PROJECT_STATS_CARD,
config=types.ProjectStatsCardConfig(
metric=types.StatsCardMetric.TRACE_COUNT
),
)
],
layout=[types.DashboardLayoutItem(id="w1", x=0, y=0, w=1, h=2)],
)
with patch.object(
client._rest_client.dashboards, "create_dashboard", side_effect=fake_create
):
dashboard = client.create_dashboard(
name="Prod",
type=types.DashboardType.MULTI_PROJECT,
project_id="proj-123",
sections=[section],
)
widget = dashboard.config["sections"][0]["widgets"][0]
assert widget["config"]["projectId"] == "proj-123"
def test_create_dashboard__project_scoped_widget_with_project_name_only_does_not_raise():
client = opik_client.Opik()
def fake_create(*, name, config, type, description, project_id, project_name):
return dp.DashboardPublic(id="d1", name=name, type=type, config=config)
section = types.DashboardSection(
title="S",
widgets=[
types.DashboardWidget(
id="w1",
type=types.WidgetType.PROJECT_STATS_CARD,
config=types.ProjectStatsCardConfig(
metric=types.StatsCardMetric.TRACE_COUNT
),
)
],
layout=[types.DashboardLayoutItem(id="w1", x=0, y=0, w=1, h=2)],
)
# project_name provided but not project_id — should not raise
with patch.object(
client._rest_client.dashboards, "create_dashboard", side_effect=fake_create
):
client.create_dashboard(
name="Prod",
type=types.DashboardType.MULTI_PROJECT,
project_name="Default Project",
sections=[section],
)
def test_create_dashboard__project_scoped_widget_without_any_project_raises():
client = opik_client.Opik()
section = types.DashboardSection(
title="S",
widgets=[
types.DashboardWidget(id="w1", type=types.WidgetType.PROJECT_STATS_CARD)
],
layout=[types.DashboardLayoutItem(id="w1", x=0, y=0, w=1, h=2)],
)
with patch.object(
client._rest_client.dashboards, "create_dashboard"
) as mock_create:
with pytest.raises(exceptions.DashboardValidationError, match="project-scoped"):
client.create_dashboard(
name="Prod",
type=types.DashboardType.MULTI_PROJECT,
sections=[section],
)
mock_create.assert_not_called()
def test_create_dashboard__incompatible_widget_raises_before_create():
client = opik_client.Opik()
section = types.DashboardSection(
title="Custom",
widgets=[
types.DashboardWidget(id="w1", type=types.WidgetType.EXPERIMENT_LEADERBOARD)
],
layout=[types.DashboardLayoutItem(id="w1", x=0, y=0, w=6, h=6)],
)
with patch.object(
client._rest_client.dashboards, "create_dashboard"
) as mock_create:
with pytest.raises(exceptions.DashboardValidationError, match="not supported"):
client.create_dashboard(
name="Prod",
type=types.DashboardType.MULTI_PROJECT,
sections=[section],
)
mock_create.assert_not_called()
def test_replace_sections__injects_project_id_for_project_scoped_widgets():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, _, _ = _make_dashboard(config, project_id="proj-abc")
new_section = types.DashboardSection(
title="t",
id="s1",
widgets=[
types.DashboardWidget(
id="w1",
type=types.WidgetType.PROJECT_STATS_CARD,
config=types.ProjectStatsCardConfig(
metric=types.StatsCardMetric.TRACE_COUNT
),
)
],
layout=[types.DashboardLayoutItem(id="w1", x=0, y=0, w=1, h=2)],
)
dashboard.replace_sections([new_section])
widget = dashboard.config["sections"][0]["widgets"][0]
assert widget["config"]["projectId"] == "proj-abc"
def test_replace_sections__project_scoped_widget_without_dashboard_project_raises():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, rest, _ = _make_dashboard(config, project_id=None)
new_section = types.DashboardSection(
title="t",
id="s1",
widgets=[
types.DashboardWidget(id="w1", type=types.WidgetType.PROJECT_STATS_CARD)
],
layout=[types.DashboardLayoutItem(id="w1", x=0, y=0, w=1, h=2)],
)
with pytest.raises(exceptions.DashboardValidationError, match="project-scoped"):
dashboard.replace_sections([new_section])
assert rest.dashboards.update_calls == []
def test_replace_sections__incompatible_widget_raises_before_write():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, rest, _ = _make_dashboard(config, dashboard_type="multi_project")
section = types.DashboardSection(
title="Custom",
widgets=[
types.DashboardWidget(id="w1", type=types.WidgetType.EXPERIMENT_LEADERBOARD)
],
layout=[types.DashboardLayoutItem(id="w1", x=0, y=0, w=6, h=6)],
)
with pytest.raises(exceptions.DashboardValidationError, match="not supported"):
dashboard.replace_sections([section])
assert rest.dashboards.update_calls == []
def test_get_dashboard__returns_wrapper():
client = opik_client.Opik()
public = dp.DashboardPublic(
id="d1",
name="My dash",
type="multi_project",
config={"version": 4, "sections": []},
)
with patch.object(
client._rest_client.dashboards, "get_dashboard_by_id", return_value=public
):
dashboard = client.get_dashboard("d1")
assert dashboard.id == "d1"
assert dashboard.name == "My dash"
def test_delete_dashboard__calls_rest():
client = opik_client.Opik()
with patch.object(
client._rest_client.dashboards, "delete_dashboard"
) as mock_delete:
client.delete_dashboard("d1")
mock_delete.assert_called_once_with("d1")
def test_add_widget__null_id_is_replaced():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, _, _ = _make_dashboard(config)
widget_id = dashboard.add_widget(
{"id": None, "type": "text_markdown", "config": {"content": "hi"}}
)
assert widget_id != "None"
assert widget_id is not None
layout_ids = [i["i"] for i in dashboard.config["sections"][0]["layout"]]
assert widget_id in layout_ids
def test_add_widget__partial_size_raises():
config = {
"version": 4,
"sections": [{"id": "s1", "title": "t", "widgets": [], "layout": []}],
"lastModified": 1,
}
dashboard, rest, _ = _make_dashboard(config)
with pytest.raises(exceptions.DashboardValidationError, match="'w' and 'h'"):
dashboard.add_widget(
types.DashboardWidget(type=types.WidgetType.TEXT_MARKDOWN),
size={"w": 2},
)
assert rest.dashboards.update_calls == []
def test_update_widget__null_config_from_api_does_not_raise():
config = {
"version": 4,
"sections": [
{
"id": "s1",
"title": "t",
"widgets": [{"id": "w1", "type": "text_markdown", "config": None}],
"layout": [{"i": "w1", "x": 0, "y": 0, "w": 2, "h": 4}],
}
],
"lastModified": 1,
}
dashboard, _, _ = _make_dashboard(config)
dashboard.update_widget("w1", config={"content": "new"})
widget = dashboard.config["sections"][0]["widgets"][0]
assert widget["config"]["content"] == "new"
def test_find_dashboards__paginates_and_respects_max_results():
from opik.api_objects.dashboard import rest_operations
from opik.rest_api.types import dashboard_page_public as dpp
pages = {
1: [
dp.DashboardPublic(
id=f"d{i}", name=f"d{i}", config={"version": 4, "sections": []}
)
for i in range(100)
],
2: [
dp.DashboardPublic(
id="d100", name="d100", config={"version": 4, "sections": []}
)
],
}
class PaginatedApi:
def __init__(self):
self.calls = []
def find_dashboards(self, *, page, size, name, project_id, sorting, filters):
self.calls.append(page)
return dpp.DashboardPagePublic(content=pages.get(page, []))
class Rest:
def __init__(self):
self.dashboards = PaginatedApi()
rest = Rest()
result = rest_operations.find_dashboards(
rest_client=rest, client=Mock(spec=opik_client.Opik), max_results=100
)
assert len(result) == 100
# only the first page is needed to satisfy max_results
assert rest.dashboards.calls == [1]
assert all(isinstance(d, Dashboard) for d in result)
@@ -0,0 +1,191 @@
"""Ported from apps/opik-frontend/src/lib/dashboard/layout.test.ts.
Keeps the SDK auto-layout behavior identical to the frontend so widgets added
via the SDK are positioned the same way the UI would position them.
"""
import pytest
from opik.api_objects.dashboard import layout
from opik.api_objects.dashboard.types import DashboardLayoutItem, WidgetType
def _item(id: str, x: int, y: int, w: int, h: int) -> DashboardLayoutItem:
return DashboardLayoutItem(id=id, x=x, y=y, w=w, h=h)
def test_grid_constants__match_frontend():
assert layout.GRID_COLUMNS == 6
assert layout.MAX_WIDGET_HEIGHT == 12
assert layout.MIN_WIDGET_WIDTH == 1
assert layout.MIN_WIDGET_HEIGHT == 1
@pytest.mark.parametrize(
"widget_type,expected",
[
(WidgetType.PROJECT_METRICS.value, {"w": 2, "h": 4, "minW": 2, "minH": 4}),
(WidgetType.PROJECT_STATS_CARD.value, {"w": 1, "h": 2, "minW": 1, "minH": 2}),
(WidgetType.TEXT_MARKDOWN.value, {"w": 2, "h": 4, "minW": 1, "minH": 4}),
("unknown-type", {"w": 2, "h": 2, "minW": 1, "minH": 1}),
],
)
def test_get_widget_size_config__per_type(widget_type, expected):
assert layout.get_widget_size_config(widget_type) == expected
@pytest.mark.parametrize(
"layout_items,expected",
[
([], [0, 0, 0, 0, 0, 0]),
([_item("w1", 0, 0, 2, 3)], [3, 3, 0, 0, 0, 0]),
(
[_item("w1", 0, 0, 2, 3), _item("w2", 2, 0, 2, 5)],
[3, 3, 5, 5, 0, 0],
),
(
[_item("w1", 0, 0, 2, 3), _item("w2", 1, 3, 2, 2)],
[3, 5, 5, 0, 0, 0],
),
([_item("w1", 0, 0, 6, 4)], [4, 4, 4, 4, 4, 4]),
(
[_item("w1", 0, 2, 2, 3), _item("w2", 2, 0, 2, 2)],
[5, 5, 2, 2, 0, 0],
),
],
)
def test_get_column_heights(layout_items, expected):
assert layout.get_column_heights(layout_items) == expected
@pytest.mark.parametrize(
"w,h,heights,expected",
[
(2, 3, [0, 0, 0, 0, 0, 0], {"x": 0, "y": 0}),
(2, 2, [3, 3, 0, 0, 0, 0], {"x": 2, "y": 0}),
(2, 2, [3, 3, 5, 5, 4, 4], {"x": 0, "y": 3}),
(6, 1, [2, 2, 3, 3, 2, 2], {"x": 0, "y": 3}),
(2, 3, [5, 5, 2, 2, 8, 8], {"x": 2, "y": 2}),
(1, 2, [3, 0, 2, 4, 1, 5], {"x": 1, "y": 0}),
],
)
def test_find_first_available_position(w, h, heights, expected):
assert layout.find_first_available_position(w, h, heights) == expected
def test_calculate_layout_for_adding_widget__first_widget_to_empty_layout():
result = layout.calculate_layout_for_adding_widget(
[], WidgetType.TEXT_MARKDOWN.value, "widget-1"
)
assert len(result) == 1
assert result[0].id == "widget-1"
assert (result[0].x, result[0].y, result[0].w, result[0].h) == (0, 0, 2, 4)
def test_calculate_layout_for_adding_widget__custom_size():
result = layout.calculate_layout_for_adding_widget(
[], WidgetType.TEXT_MARKDOWN.value, "widget-1", size={"w": 3, "h": 5}
)
assert result[0].w == 3
assert result[0].h == 5
def test_calculate_layout_for_adding_widget__clamps_oversized_custom_size():
result = layout.calculate_layout_for_adding_widget(
[],
WidgetType.TEXT_MARKDOWN.value,
"widget-1",
size={"w": 100, "h": 100},
)
assert result[0].w == layout.GRID_COLUMNS
assert result[0].h == layout.MAX_WIDGET_HEIGHT
def test_calculate_layout_for_adding_widget__clamps_undersized_custom_size():
result = layout.calculate_layout_for_adding_widget(
[],
WidgetType.PROJECT_METRICS.value,
"widget-1",
size={"w": 1, "h": 1},
)
size_config = layout.get_widget_size_config(WidgetType.PROJECT_METRICS.value)
assert result[0].w == size_config["minW"]
assert result[0].h == size_config["minH"]
def test_calculate_layout_for_adding_widget__second_widget_placed_next_to_first():
existing = [_item("widget-1", 0, 0, 2, 4)]
result = layout.calculate_layout_for_adding_widget(
existing, WidgetType.TEXT_MARKDOWN.value, "widget-2"
)
assert len(result) == 2
assert result[1].id == "widget-2"
assert (result[1].x, result[1].y) == (2, 0)
def test_calculate_layout_for_adding_widget__sets_min_and_max_constraints():
result = layout.calculate_layout_for_adding_widget(
[], WidgetType.PROJECT_METRICS.value, "widget-1"
)
assert result[0].min_w == 2
assert result[0].min_h == 4
assert result[0].max_w == layout.GRID_COLUMNS
assert result[0].max_h == layout.MAX_WIDGET_HEIGHT
def test_calculate_layout_for_adding_widget__optimal_position_with_complex_layout():
existing = [_item("widget-1", 0, 0, 3, 4), _item("widget-2", 3, 0, 3, 2)]
result = layout.calculate_layout_for_adding_widget(
existing, WidgetType.PROJECT_STATS_CARD.value, "widget-3"
)
assert result[2].y == 2
def test_normalize_layout__empty():
assert layout.normalize_layout([]) == []
def test_normalize_layout__clamps_position_to_grid_bounds():
normalized = layout.normalize_layout([_item("w1", 7, -1, 2, 3)])
assert normalized[0].x == 4
assert normalized[0].y == 0
def test_normalize_layout__clamps_width_to_grid_columns():
normalized = layout.normalize_layout([_item("w1", 0, 0, 10, 3)])
assert normalized[0].w == layout.GRID_COLUMNS
def test_normalize_layout__clamps_height_to_max():
normalized = layout.normalize_layout([_item("w1", 0, 0, 2, 20)])
assert normalized[0].h == layout.MAX_WIDGET_HEIGHT
def test_normalize_layout__enforces_minimum_dimensions():
normalized = layout.normalize_layout([_item("w1", 0, 0, 0, 0)])
assert normalized[0].w == layout.MIN_WIDGET_WIDTH
assert normalized[0].h == layout.MIN_WIDGET_HEIGHT
def test_normalize_layout__applies_widget_specific_constraints():
widgets = [{"id": "w1", "type": WidgetType.PROJECT_METRICS.value}]
normalized = layout.normalize_layout([_item("w1", 0, 0, 1, 2)], widgets)
assert normalized[0].min_w == 2
assert normalized[0].min_h == 4
assert normalized[0].w == 2
assert normalized[0].h == 4
def test_remove_widget_from_layout():
items = [_item("w1", 0, 0, 2, 3), _item("w2", 2, 0, 2, 3), _item("w3", 4, 0, 2, 3)]
result = layout.remove_widget_from_layout(items, "w2")
assert [item.id for item in result] == ["w1", "w3"]
# original not mutated
assert len(items) == 3
def test_remove_widget_from_layout__non_existent_id_is_noop():
items = [_item("w1", 0, 0, 2, 3)]
result = layout.remove_widget_from_layout(items, "ghost")
assert len(result) == 1
assert result[0].id == "w1"
@@ -0,0 +1,104 @@
import pytest
from opik import exceptions
from opik.api_objects.dashboard import types
from ....testlib import assert_equal
def test_widget_serialization__uses_camelcase_and_enum_values():
widget = types.DashboardWidget(
type=types.WidgetType.PROJECT_STATS_CARD,
title="Traces",
config=types.ProjectStatsCardConfig(metric=types.StatsCardMetric.TRACE_COUNT),
)
result = widget.to_jsonable()
assert result["type"] == "project_stats_card"
assert result["title"] == "Traces"
assert "id" in result
assert_equal(
{
"source": "traces",
"metric": "trace_count",
},
result["config"],
)
def test_project_metrics_config__defaults_and_breakdown_camelcase():
config = types.ProjectMetricsConfig(
metric_type=types.ProjectMetricType.DURATION,
breakdown=types.BreakdownConfig(
field=types.BreakdownField.METADATA, metadata_key="provider"
),
)
result = config.to_jsonable()
assert result["metricType"] == "DURATION"
assert result["chartType"] == "line"
assert_equal({"field": "metadata", "metadataKey": "provider"}, result["breakdown"])
def test_breakdown_config__metadata_field_requires_metadata_key():
with pytest.raises(exceptions.DashboardValidationError):
types.BreakdownConfig(field=types.BreakdownField.METADATA)
def test_breakdown_config__non_metadata_field_does_not_require_key():
config = types.BreakdownConfig(field=types.BreakdownField.TAGS)
assert config.to_jsonable() == {"field": "tags"}
def test_leaderboard_config__enable_ranking_requires_ranking_metric():
with pytest.raises(exceptions.DashboardValidationError):
types.ExperimentLeaderboardConfig(enable_ranking=True)
def test_leaderboard_config__enable_ranking_with_metric_ok():
config = types.ExperimentLeaderboardConfig(
enable_ranking=True, ranking_metric="pass_rate"
)
assert config.to_jsonable()["rankingMetric"] == "pass_rate"
@pytest.mark.parametrize("value", [0, 101, 500, "0", "200"])
def test_leaderboard_config__max_rows_out_of_range_raises(value):
with pytest.raises(exceptions.DashboardValidationError):
types.ExperimentLeaderboardConfig(max_rows=value)
@pytest.mark.parametrize("value", [1, 100, "10", 50])
def test_leaderboard_config__max_rows_in_range_ok(value):
config = types.ExperimentLeaderboardConfig(max_rows=value)
assert config.to_jsonable()["maxRows"] == value
@pytest.mark.parametrize("value", [0, 101])
def test_feedback_scores_config__max_experiments_out_of_range_raises(value):
with pytest.raises(exceptions.DashboardValidationError):
types.ExperimentsFeedbackScoresConfig(max_experiments_count=value)
def test_dashboard_state__defaults_to_known_version():
state = types.DashboardState()
result = state.to_jsonable()
assert result["version"] == types.DASHBOARD_VERSION
assert "lastModified" in result
assert result["sections"] == []
def test_widget_accepts_raw_dict_config():
widget = types.DashboardWidget(type="future_widget", config={"someCamelField": 1})
assert widget.to_jsonable()["config"] == {"someCamelField": 1}
def test_models_preserve_unknown_fields_on_parse():
state = types.DashboardState.model_validate(
{
"version": 4,
"lastModified": 1,
"sections": [],
"mysteryField": "keep",
}
)
assert state.to_jsonable()["mysteryField"] == "keep"
@@ -0,0 +1,169 @@
from unittest import mock
import pytest
from opik import exceptions
from opik.api_objects.dashboard import types, validation
def _section(section_id, widgets, layout_ids):
return {
"id": section_id,
"title": "t",
"widgets": [{"id": w, "type": "text_markdown", "config": {}} for w in widgets],
"layout": [{"i": i, "x": 0, "y": 0, "w": 1, "h": 1} for i in layout_ids],
}
def test_validate_structure__valid_passes():
state = {"version": 4, "sections": [_section("s1", ["w1"], ["w1"])]}
validation.validate_structure(state)
def test_validate_structure__widget_without_layout_raises():
state = {"version": 4, "sections": [_section("s1", ["w1"], [])]}
with pytest.raises(exceptions.DashboardValidationError, match="without a layout"):
validation.validate_structure(state)
def test_validate_structure__orphan_layout_item_raises():
state = {"version": 4, "sections": [_section("s1", [], ["ghost"])]}
with pytest.raises(exceptions.DashboardValidationError, match="missing widgets"):
validation.validate_structure(state)
def test_validate_structure__duplicate_widget_ids_across_sections_raises():
state = {
"version": 4,
"sections": [
_section("s1", ["dup"], ["dup"]),
_section("s2", ["dup"], ["dup"]),
],
}
with pytest.raises(exceptions.DashboardValidationError, match="Duplicate widget"):
validation.validate_structure(state)
def test_validate_structure__duplicate_section_ids_raises():
state = {
"version": 4,
"sections": [_section("s1", ["w1"], ["w1"]), _section("s1", ["w2"], ["w2"])],
}
with pytest.raises(exceptions.DashboardValidationError, match="Duplicate section"):
validation.validate_structure(state)
def test_validate_widget_for_dashboard__incompatible_type_raises():
widget = {"type": "experiment_leaderboard", "config": {}}
with pytest.raises(exceptions.DashboardValidationError, match="not supported"):
validation.validate_widget_for_dashboard(widget, "multi_project")
def test_validate_widget_for_dashboard__compatible_type_ok():
widget = {"type": "project_metrics", "config": {"metricType": "TRACE_COUNT"}}
validation.validate_widget_for_dashboard(widget, "multi_project")
def test_validate_widget_for_dashboard__no_dashboard_type_skips_compatibility():
widget = {"type": "experiment_leaderboard", "config": {}}
validation.validate_widget_for_dashboard(widget, None)
def test_validate_widget_for_dashboard__unknown_metric_warns_not_raises():
widget = {"type": "project_metrics", "config": {"metricType": "not_a_metric"}}
with mock.patch.object(validation.LOGGER, "warning") as mock_warning:
validation.validate_widget_for_dashboard(widget, "multi_project")
mock_warning.assert_called_once()
assert "Unknown project_metrics metricType" in mock_warning.call_args[0][0]
def test_validate_widget_for_dashboard__dynamic_feedback_metric_no_warning():
widget = {
"type": "project_stats_card",
"config": {"metric": "feedback_scores.helpfulness"},
}
with mock.patch.object(validation.LOGGER, "warning") as mock_warning:
validation.validate_widget_for_dashboard(widget, "multi_project")
mock_warning.assert_not_called()
def test_validate_writable_version__known_version_ok():
validation.validate_writable_version(4)
validation.validate_writable_version(None)
def test_validate_writable_version__unknown_version_raises():
with pytest.raises(exceptions.DashboardValidationError, match="schema version 5"):
validation.validate_writable_version(5)
def test_as_widget_dict__from_model__returns_jsonable():
widget = types.DashboardWidget(
id="w1", type=types.WidgetType.TEXT_MARKDOWN, config=types.TextMarkdownConfig()
)
result = validation.as_widget_dict(widget)
assert result["id"] == "w1"
assert result["type"] == "text_markdown"
def test_as_widget_dict__from_dict__returns_same_dict():
d = {"id": "w1", "type": "text_markdown", "config": {}}
result = validation.as_widget_dict(d)
assert result is d
def test_as_widget_dict__invalid_type__raises():
with pytest.raises(
exceptions.DashboardValidationError, match="Expected a DashboardWidget"
):
validation.as_widget_dict(42)
def test_as_section_dicts__from_models__returns_jsonable():
section = types.DashboardSection(title="S1")
result = validation.as_section_dicts([section])
assert len(result) == 1
assert result[0]["title"] == "S1"
assert "id" in result[0]
def test_as_section_dicts__from_dicts__returns_same_dicts():
d = {"id": "s1", "title": "S1", "widgets": [], "layout": []}
result = validation.as_section_dicts([d])
assert result[0] is d
def test_as_section_dicts__invalid_type__raises():
with pytest.raises(
exceptions.DashboardValidationError, match="Expected a DashboardSection"
):
validation.as_section_dicts(["not-a-section"])
def test_inject_project_id__project_scoped_widget_injects():
widget = {"type": "project_stats_card", "config": {"metric": "trace_count"}}
validation.inject_project_id(widget, "proj-123")
assert widget["config"]["projectId"] == "proj-123"
def test_inject_project_id__project_metrics_injects():
widget = {"type": "project_metrics", "config": {"metricType": "DURATION"}}
validation.inject_project_id(widget, "proj-abc")
assert widget["config"]["projectId"] == "proj-abc"
def test_inject_project_id__non_project_widget_untouched():
widget = {"type": "text_markdown", "config": {"content": "hi"}}
validation.inject_project_id(widget, "proj-123")
assert "projectId" not in widget["config"]
def test_inject_project_id__no_project_id_raises():
widget = {"type": "project_stats_card", "config": {}}
with pytest.raises(exceptions.DashboardValidationError, match="project-scoped"):
validation.inject_project_id(widget, None)
def test_inject_project_id__no_project_id_for_non_project_widget_ok():
widget = {"type": "text_markdown", "config": {}}
validation.inject_project_id(widget, None)