chore: import upstream snapshot with attribution
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:25:44 +08:00
commit 5a558eb09e
11579 changed files with 1795921 additions and 0 deletions
@@ -0,0 +1,62 @@
from unittest import mock
import pytest
from opik.api_objects import opik_client
from opik.rest_api import core as rest_api_core
from opik.rest_api.types.agent_blueprint_public import AgentBlueprintPublic
from opik.decorator.context_manager import start_as_current_trace
def make_raw_blueprint(blueprint_id="bp-1", name=None, values=None, description=None):
if values is None:
values = []
return AgentBlueprintPublic(
id=blueprint_id,
name=name,
type="blueprint",
values=values,
description=description,
)
@pytest.fixture
def mock_rest_client():
client = mock.Mock()
client.agent_configs = mock.Mock()
client.agent_configs.create_agent_config.return_value = None
client.agent_configs.get_latest_blueprint.side_effect = rest_api_core.ApiError(
status_code=404, body="not found"
)
client.agent_configs.get_blueprint_by_env.side_effect = rest_api_core.ApiError(
status_code=404, body="not found"
)
client.agent_configs.get_blueprint_by_id.return_value = make_raw_blueprint()
client.projects.retrieve_project.return_value = mock.Mock(id="proj-test")
return client
@pytest.fixture
def mock_opik_client(mock_rest_client):
client = opik_client.Opik.__new__(opik_client.Opik)
client._rest_client = mock_rest_client
client._project_name = "test-project"
return client
@pytest.fixture(autouse=True)
def clear_caches():
yield
from opik.api_objects.agent_config.cache import get_global_registry
get_global_registry().clear()
@pytest.fixture(autouse=True)
def fake_track_context():
"""Push a fake trace so _get_or_create_from_backend's @track guard passes in all unit tests."""
with mock.patch.object(
opik_client, "get_global_client", return_value=mock.Mock(spec=opik_client.Opik)
):
with start_as_current_trace(name="test-trace"):
yield
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,470 @@
from typing import List
from unittest import mock
import pytest
from opik.api_objects.agent_config.blueprint import Blueprint
from opik.api_objects.prompt.text.prompt import Prompt
from opik.api_objects.prompt.chat.chat_prompt import ChatPrompt
from opik.api_objects.prompt.base_prompt import BasePrompt
from opik.rest_api.types.agent_blueprint_public import AgentBlueprintPublic
from opik.rest_api.types.agent_config_value_public import AgentConfigValuePublic
from opik.rest_api.types.prompt_version_detail import PromptVersionDetail
def _make_raw_blueprint(
blueprint_id="bp-1",
values=None,
description=None,
bp_type="blueprint",
envs=None,
created_by=None,
created_at=None,
):
if values is None:
values = [
AgentConfigValuePublic(key="temperature", type="float", value="0.6"),
AgentConfigValuePublic(key="name", type="string", value="agent"),
]
return AgentBlueprintPublic(
id=blueprint_id,
type=bp_type,
values=values,
description=description,
envs=envs,
created_by=created_by,
created_at=created_at,
)
class TestBlueprintProperties:
@pytest.mark.parametrize(
"kwargs,attr,expected",
[
({"blueprint_id": "bp-42"}, "id", "bp-42"),
({"description": "test desc"}, "description", "test desc"),
({"bp_type": "mask"}, "type", "mask"),
({"envs": ["prod", "staging"]}, "envs", ["prod", "staging"]),
({"created_by": "user-1"}, "created_by", "user-1"),
],
)
def test_property__returns_value(self, kwargs, attr, expected):
bp = Blueprint(_make_raw_blueprint(**kwargs))
assert getattr(bp, attr) == expected
class TestBlueprintValueResolution:
def test_without_field_types__values_are_inferred_from_backend_type(self):
bp = Blueprint(_make_raw_blueprint())
assert bp["temperature"] == 0.6
assert isinstance(bp["temperature"], float)
assert bp["name"] == "agent"
assert isinstance(bp["name"], str)
def test_with_field_types__deserializes_values(self):
bp = Blueprint(
_make_raw_blueprint(),
field_types={"temperature": float, "name": str},
)
assert bp["temperature"] == 0.6
assert bp["name"] == "agent"
@pytest.mark.parametrize(
"backend_type,value_str,py_type,expected",
[
("string", "true", bool, True),
("integer", "42", int, 42),
],
)
def test_with_field_types__type_deserialization(
self, backend_type, value_str, py_type, expected
):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(key="val", type=backend_type, value=value_str)
]
)
bp = Blueprint(raw, field_types={"val": py_type})
assert bp["val"] == expected
class TestBlueprintDictLikeAccess:
def test_get__existing_key(self):
bp = Blueprint(_make_raw_blueprint())
assert bp.get("temperature") == 0.6
def test_get__missing_key__returns_default(self):
bp = Blueprint(_make_raw_blueprint())
assert bp.get("missing", 42) == 42
def test_get__missing_key_no_default__returns_none(self):
bp = Blueprint(_make_raw_blueprint())
assert bp.get("missing") is None
def test_getitem__existing_key(self):
bp = Blueprint(_make_raw_blueprint())
assert bp["name"] == "agent"
def test_getitem__missing_key__raises_key_error(self):
bp = Blueprint(_make_raw_blueprint())
with pytest.raises(KeyError):
_ = bp["missing"]
def test_keys__returns_all_keys(self):
bp = Blueprint(_make_raw_blueprint())
assert set(bp.keys()) == {"temperature", "name"}
def test_values__returns_deep_copy(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(key="items", type="string", value="[1, 2, 3]"),
]
)
bp = Blueprint(raw, field_types={"items": List[int]})
vals = bp.values
vals["items"].append(4)
assert bp.values["items"] == [1, 2, 3]
class TestBlueprintPromptResolution:
def test_prompt_field__resolves_to_prompt_object(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(
key="system_prompt", type="prompt", value="abc12345"
),
]
)
mock_rest = mock.Mock()
version_detail = mock.Mock()
version_detail.template_structure = "text"
prompt_detail = mock.Mock()
prompt_detail.name = "my-prompt"
prompt_detail.requested_version = version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
fake_prompt = mock.Mock(spec=Prompt)
with mock.patch(
"opik.api_objects.prompt.text.prompt.Prompt.from_fern_prompt_version",
return_value=fake_prompt,
):
bp = Blueprint(
raw,
field_types={"system_prompt": Prompt},
rest_client_=mock_rest,
)
assert bp["system_prompt"] is fake_prompt
mock_rest.prompts.get_prompt_by_commit.assert_called_once_with("abc12345")
def test_chat_prompt_field__resolves_to_chat_prompt_object(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(key="messages", type="prompt", value="bcd23456"),
]
)
mock_rest = mock.Mock()
version_detail = mock.Mock()
version_detail.type = "mustache"
prompt_detail = mock.Mock()
prompt_detail.name = "chat-prompt"
prompt_detail.template_structure = "chat"
prompt_detail.requested_version = version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
fake_chat_prompt = mock.Mock(spec=ChatPrompt)
with mock.patch(
"opik.api_objects.prompt.chat.chat_prompt.ChatPrompt.from_fern_prompt_version",
return_value=fake_chat_prompt,
):
bp = Blueprint(
raw,
field_types={"messages": ChatPrompt},
rest_client_=mock_rest,
)
assert bp["messages"] is fake_chat_prompt
def test_base_prompt_annotation__chat_structure__resolves_to_chat_prompt(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(key="p", type="prompt", value="cde34567"),
]
)
mock_rest = mock.Mock()
version_detail = mock.Mock()
version_detail.type = "mustache"
prompt_detail = mock.Mock()
prompt_detail.name = "any-prompt"
prompt_detail.template_structure = "chat"
prompt_detail.requested_version = version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
fake_chat_prompt = mock.Mock(spec=ChatPrompt)
with mock.patch(
"opik.api_objects.prompt.chat.chat_prompt.ChatPrompt.from_fern_prompt_version",
return_value=fake_chat_prompt,
):
bp = Blueprint(
raw,
field_types={"p": BasePrompt},
rest_client_=mock_rest,
)
assert bp["p"] is fake_chat_prompt
def test_base_prompt_annotation__text_structure__resolves_to_prompt(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(key="p", type="prompt", value="def45678"),
]
)
mock_rest = mock.Mock()
version_detail = mock.Mock()
version_detail.template_structure = "text"
prompt_detail = mock.Mock()
prompt_detail.name = "any-prompt"
prompt_detail.requested_version = version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
fake_prompt = mock.Mock(spec=Prompt)
with mock.patch(
"opik.api_objects.prompt.text.prompt.Prompt.from_fern_prompt_version",
return_value=fake_prompt,
):
bp = Blueprint(
raw,
field_types={"p": BasePrompt},
rest_client_=mock_rest,
)
assert bp["p"] is fake_prompt
def test_prompt_resolution_fails__raises(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(
key="system_prompt", type="prompt", value="badbad00"
),
]
)
mock_rest = mock.Mock()
mock_rest.prompts.get_prompt_by_commit.side_effect = Exception("network error")
with pytest.raises(Exception, match="network error"):
Blueprint(
raw,
field_types={"system_prompt": Prompt},
rest_client_=mock_rest,
)
def test_prompt_version_field__resolves_to_prompt_version_detail(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(
key="version", type="prompt_commit", value="pv111111"
),
]
)
mock_rest = mock.Mock()
fake_version_detail = mock.Mock(spec=PromptVersionDetail)
prompt_detail = mock.Mock()
prompt_detail.requested_version = fake_version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
bp = Blueprint(
raw,
field_types={"version": PromptVersionDetail},
rest_client_=mock_rest,
)
assert bp["version"] is fake_version_detail
mock_rest.prompts.get_prompt_by_commit.assert_called_once_with("pv111111")
def test_prompt_version_field__resolution_fails__raises(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(
key="version", type="prompt_commit", value="badbad00"
),
]
)
mock_rest = mock.Mock()
mock_rest.prompts.get_prompt_by_commit.side_effect = Exception("not found")
with pytest.raises(Exception, match="not found"):
Blueprint(
raw,
field_types={"version": PromptVersionDetail},
rest_client_=mock_rest,
)
def test_prompt_field_type_declared_as_chatprompt__chat_template_structure__resolves_to_chat_prompt(
self,
):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(key="p", type="prompt", value="aaa11111"),
]
)
mock_rest = mock.Mock()
version_detail = mock.Mock()
prompt_detail = mock.Mock()
prompt_detail.name = "my-prompt"
prompt_detail.template_structure = "chat"
prompt_detail.requested_version = version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
fake_chat_prompt = mock.Mock(spec=ChatPrompt)
with mock.patch(
"opik.api_objects.prompt.chat.chat_prompt.ChatPrompt.from_fern_prompt_version",
return_value=fake_chat_prompt,
):
bp = Blueprint(
raw,
field_types={"p": ChatPrompt},
rest_client_=mock_rest,
)
assert bp["p"] is fake_chat_prompt
assert isinstance(bp["p"], ChatPrompt)
def test_two_prompt_fields__makes_exactly_one_api_call_per_prompt(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(key="p1", type="prompt", value="aaaaaaaa"),
AgentConfigValuePublic(key="p2", type="prompt", value="bbbbbbbb"),
]
)
mock_rest = mock.Mock()
def _commit_side_effect(commit):
v = mock.Mock()
d = mock.Mock()
d.name = f"name-{commit}"
d.template_structure = "text"
d.requested_version = v
return d
mock_rest.prompts.get_prompt_by_commit.side_effect = _commit_side_effect
with mock.patch(
"opik.api_objects.prompt.text.prompt.Prompt.from_fern_prompt_version",
return_value=mock.Mock(spec=Prompt),
):
Blueprint(
raw,
field_types={"p1": Prompt, "p2": Prompt},
rest_client_=mock_rest,
)
assert mock_rest.prompts.get_prompt_by_commit.call_count == 2
class TestBlueprintPromptResolutionWithoutFieldTypes:
def test_prompt_type__resolves_to_prompt_object(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(
key="system_prompt", type="prompt", value="abc12345"
),
]
)
mock_rest = mock.Mock()
version_detail = mock.Mock()
prompt_detail = mock.Mock()
prompt_detail.name = "my-prompt"
prompt_detail.template_structure = "text"
prompt_detail.requested_version = version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
fake_prompt = mock.Mock(spec=Prompt)
with mock.patch(
"opik.api_objects.prompt.text.prompt.Prompt.from_fern_prompt_version",
return_value=fake_prompt,
):
bp = Blueprint(raw, rest_client_=mock_rest)
assert bp["system_prompt"] is fake_prompt
def test_chat_prompt_type__resolves_to_chat_prompt_object(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(key="messages", type="prompt", value="bcd23456"),
]
)
mock_rest = mock.Mock()
version_detail = mock.Mock()
prompt_detail = mock.Mock()
prompt_detail.name = "chat-prompt"
prompt_detail.template_structure = "chat"
prompt_detail.requested_version = version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
fake_chat_prompt = mock.Mock(spec=ChatPrompt)
with mock.patch(
"opik.api_objects.prompt.chat.chat_prompt.ChatPrompt.from_fern_prompt_version",
return_value=fake_chat_prompt,
):
bp = Blueprint(raw, rest_client_=mock_rest)
assert bp["messages"] is fake_chat_prompt
def test_prompt_commit_type__resolves_to_prompt_version_detail(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(
key="version", type="prompt_commit", value="pv111111"
),
]
)
mock_rest = mock.Mock()
fake_version_detail = mock.Mock(spec=PromptVersionDetail)
prompt_detail = mock.Mock()
prompt_detail.requested_version = fake_version_detail
mock_rest.prompts.get_prompt_by_commit.return_value = prompt_detail
bp = Blueprint(raw, rest_client_=mock_rest)
assert bp["version"] is fake_version_detail
mock_rest.prompts.get_prompt_by_commit.assert_called_once_with("pv111111")
def test_prompt_resolution_fails__raises(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(
key="system_prompt", type="prompt", value="badbad00"
),
]
)
mock_rest = mock.Mock()
mock_rest.prompts.get_prompt_by_commit.side_effect = Exception("network error")
with pytest.raises(Exception, match="network error"):
Blueprint(raw, rest_client_=mock_rest)
def test_without_rest_client__prompt_stays_raw_string(self):
raw = _make_raw_blueprint(
values=[
AgentConfigValuePublic(
key="system_prompt", type="prompt", value="ver-111"
),
]
)
bp = Blueprint(raw)
assert bp["system_prompt"] == "ver-111"
@@ -0,0 +1,357 @@
import threading
import time
from unittest import mock
import pytest
from opik.api_objects.agent_config.cache import (
CacheRefreshThread,
SharedCacheRegistry,
SharedConfigCache,
)
@pytest.fixture
def registry():
r = SharedCacheRegistry()
yield r
r.clear()
class TestSharedCacheRegistry:
def test_get__same_key__returns_same_instance(self, registry):
a = registry.get("proj", None, None)
b = registry.get("proj", None, None)
assert a is b
def test_get__different_key__returns_different_instance(self, registry):
a = registry.get("proj-a", None, None)
b = registry.get("proj-b", None, None)
assert a is not b
def test_clear__empties_registry(self, registry):
registry.get("proj", None, None)
registry.clear()
# After clear, a new call returns a fresh instance
fresh = registry.get("proj", None, None)
assert fresh.blueprint_id is None
def test_clear__stops_thread(self, registry):
registry.ensure_refresh_thread_started()
assert registry._thread is not None and registry._thread.is_alive()
registry.clear()
assert registry._thread is None
def test_ensure_refresh_thread_started__starts_thread(self, registry):
assert registry._thread is None
registry.ensure_refresh_thread_started()
assert registry._thread is not None
assert registry._thread.is_alive()
def test_ensure_refresh_thread_started__second_call_noop(self, registry):
registry.ensure_refresh_thread_started()
first_thread = registry._thread
registry.ensure_refresh_thread_started()
assert registry._thread is first_thread
def test_stop_refresh_thread__stops_and_nulls(self, registry):
registry.ensure_refresh_thread_started()
thread = registry._thread
registry.stop_refresh_thread()
thread.join(timeout=2)
assert not thread.is_alive()
assert registry._thread is None
def test_concurrent_get__returns_same_instance(self, registry):
results = [None] * 10
barrier = threading.Barrier(10)
def fetch(idx):
barrier.wait()
results[idx] = registry.get("proj", None, None)
threads = [threading.Thread(target=fetch, args=(i,)) for i in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
assert all(r is results[0] for r in results)
class TestSharedConfigCacheThreadSafety:
def test_apply__concurrent_reads_see_consistent_dict(self):
cache = SharedConfigCache(ttl_seconds=300)
errors = []
stop = threading.Event()
def writer():
for i in range(100):
bp = mock.Mock()
bp.id = f"bp-{i}"
bp._values = {f"key-{j}": f"val-{i}" for j in range(5)}
cache.update(bp)
def reader():
while not stop.is_set():
vals = cache.values
unique_vals = set(vals.values())
if len(unique_vals) > 1:
errors.append(f"Inconsistent values: {vals}")
break
writer_thread = threading.Thread(target=writer)
reader_thread = threading.Thread(target=reader)
reader_thread.start()
writer_thread.start()
writer_thread.join()
stop.set()
reader_thread.join()
assert errors == [], f"Found inconsistencies: {errors}"
def test_register_fields__concurrent__all_fields_present(self):
cache = SharedConfigCache()
barrier = threading.Barrier(4)
def register(prefix: str):
barrier.wait()
cache.register_fields({f"{prefix}.f1": str, f"{prefix}.f2": int})
threads = [
threading.Thread(target=register, args=(f"Class{i}",)) for i in range(4)
]
for t in threads:
t.start()
for t in threads:
t.join()
all_types = cache.all_field_types
assert len(all_types) == 8
for i in range(4):
assert f"Class{i}.f1" in all_types
assert f"Class{i}.f2" in all_types
class TestRefreshCallback:
def test_set_refresh_callback__first_writer_wins(self):
cache = SharedConfigCache()
cb1 = mock.Mock()
cb2 = mock.Mock()
cache.set_refresh_callback(cb1)
cache.set_refresh_callback(cb2)
cache.try_background_refresh()
cb1.assert_called_once()
cb2.assert_not_called()
def test_try_background_refresh__no_callback__noop(self):
cache = SharedConfigCache()
cache.try_background_refresh()
def test_try_background_refresh__callback_returns_blueprint__applies(self):
cache = SharedConfigCache(ttl_seconds=300)
bp = mock.Mock()
bp.id = "bp-new"
bp._values = {"A.x": 42}
cache.set_refresh_callback(lambda: bp)
cache.try_background_refresh()
assert cache.blueprint_id == "bp-new"
assert cache.values == {"A.x": 42}
assert not cache.is_stale()
def test_try_background_refresh__callback_returns_none__no_change(self):
cache = SharedConfigCache()
cache.set_refresh_callback(lambda: None)
cache.try_background_refresh()
assert cache.blueprint_id is None
assert cache.values == {}
def test_try_background_refresh__callback_raises__no_crash(self):
cache = SharedConfigCache()
cache.set_refresh_callback(mock.Mock(side_effect=RuntimeError("boom")))
cache.try_background_refresh()
assert cache.blueprint_id is None
class TestRefreshPolicy:
"""Verify which cache lookups get a background refresh callback and which do not.
Tests use a local SharedCacheRegistry to avoid interference with the global
singleton used by init_cache_entry.
"""
def _make_bp(self, bp_id: str, values: dict) -> mock.Mock:
bp = mock.Mock()
bp.id = bp_id
bp._values = values
return bp
def test_latest_lookup__refresh_callback_registered(self, registry):
manager = mock.Mock()
bp = self._make_bp("bp-latest", {"K.v": "v1"})
cache = registry.get("proj", None, None, None)
cache.update(bp)
cache.set_refresh_callback(
lambda: manager.get_blueprint(env=None, mask_id=None, field_types={})
)
registry.ensure_refresh_thread_started()
assert cache._refresh_callback is not None
def test_env_lookup__refresh_callback_registered(self, registry):
manager = mock.Mock()
bp = self._make_bp("bp-env", {"K.v": "v1"})
cache = registry.get("proj", "prod", None, None)
cache.update(bp)
cache.set_refresh_callback(
lambda: manager.get_blueprint(env="prod", mask_id=None, field_types={})
)
registry.ensure_refresh_thread_started()
assert cache._refresh_callback is not None
def test_version_lookup__no_refresh_callback(self, registry):
bp = self._make_bp("bp-v1", {"K.v": "v1"})
cache = registry.get("proj", None, None, "v1")
cache.update(bp)
# version-pinned: no refresh callback registered
assert cache._refresh_callback is None
def test_masked_lookup__no_refresh_callback(self, registry):
bp = self._make_bp("bp-masked", {"K.v": "v1"})
cache = registry.get("proj", None, "mask-abc", None)
cache.update(bp)
# masked: no refresh callback registered
assert cache._refresh_callback is None
def test_latest_and_version__separate_cache_entries(self, registry):
bp_latest = self._make_bp("bp-latest", {"K.v": "latest"})
bp_v1 = self._make_bp("bp-v1", {"K.v": "v1"})
latest_cache = registry.get("proj", None, None, None)
latest_cache.update(bp_latest)
version_cache = registry.get("proj", None, None, "v1")
version_cache.update(bp_v1)
assert latest_cache is not version_cache
assert latest_cache.blueprint_id == "bp-latest"
assert version_cache.blueprint_id == "bp-v1"
def test_latest__background_refresh_updates_cache(self, registry):
bp_new = self._make_bp("bp-refreshed", {"K.v": "refreshed"})
cache = registry.get("proj", None, None, None)
cache._ttl_seconds = 0
cache.set_refresh_callback(lambda: bp_new)
cache.try_background_refresh()
assert cache.blueprint_id == "bp-refreshed"
assert cache.values == {"K.v": "refreshed"}
def test_version__no_background_refresh_even_when_stale(self):
bp = self._make_bp("bp-v1", {"K.v": "v1"})
cache = SharedConfigCache(ttl_seconds=0)
cache.update(bp)
# No refresh callback registered for version-pinned cache
assert cache._refresh_callback is None
cache.try_background_refresh()
assert cache.blueprint_id == "bp-v1"
def test_init_cache_entry__latest__registers_refresh(self, registry):
"""init_cache_entry with version=None, mask_id=None must register a refresh callback."""
import opik.api_objects.agent_config.cache as cache_mod
manager = mock.Mock()
bp = self._make_bp("bp-latest", {"K.v": "v1"})
with mock.patch.object(cache_mod, "_registry", registry):
cache_mod.init_cache_entry(
"proj", None, None, {}, manager, blueprint=bp, version=None
)
cache = registry.get("proj", None, None, None)
assert cache._refresh_callback is not None
def test_init_cache_entry__version__no_refresh(self, registry):
"""init_cache_entry with version set must NOT register a refresh callback."""
import opik.api_objects.agent_config.cache as cache_mod
manager = mock.Mock()
bp = self._make_bp("bp-v1", {"K.v": "v1"})
with mock.patch.object(cache_mod, "_registry", registry):
cache_mod.init_cache_entry(
"proj", None, None, {}, manager, blueprint=bp, version="v1"
)
cache = registry.get("proj", None, None, "v1")
assert cache._refresh_callback is None
def test_init_cache_entry__masked__no_refresh(self, registry):
"""init_cache_entry with mask_id set must NOT register a refresh callback."""
import opik.api_objects.agent_config.cache as cache_mod
manager = mock.Mock()
bp = self._make_bp("bp-masked", {"K.v": "v1"})
with mock.patch.object(cache_mod, "_registry", registry):
cache_mod.init_cache_entry(
"proj", None, "mask-abc", {}, manager, blueprint=bp, version=None
)
cache = registry.get("proj", None, "mask-abc", None)
assert cache._refresh_callback is None
class TestCacheRefreshThread:
def test_stops_on_close(self):
thread = CacheRefreshThread(get_caches=list, interval_seconds=0.01)
thread.start()
assert thread.is_alive()
thread.close()
thread.join(timeout=2)
assert not thread.is_alive()
def test_refreshes_stale_cache(self, registry):
bp = mock.Mock()
bp.id = "bp-bg"
bp._values = {"K.v": "refreshed"}
callback = mock.Mock(return_value=bp)
cache = registry.get("bg-proj", None, None)
cache._ttl_seconds = 0
cache.set_refresh_callback(callback)
thread = CacheRefreshThread(
get_caches=lambda: list(registry._caches.values()),
interval_seconds=0.05,
)
thread.start()
try:
time.sleep(0.3)
assert callback.call_count >= 1
assert cache.values == {"K.v": "refreshed"}
finally:
thread.close()
thread.join(timeout=2)
def test_skips_non_stale_caches(self, registry):
bp = mock.Mock()
bp.id = "bp-init"
bp._values = {"K.v": "initial"}
cache = registry.get("fresh-proj", None, None)
cache.update(bp)
callback = mock.Mock()
cache.set_refresh_callback(callback)
thread = CacheRefreshThread(
get_caches=lambda: list(registry._caches.values()),
interval_seconds=0.05,
)
thread.start()
try:
time.sleep(0.2)
callback.assert_not_called()
finally:
thread.close()
thread.join(timeout=2)
@@ -0,0 +1,406 @@
from unittest import mock
import pytest
from opik.api_objects.agent_config import types
from opik.api_objects.agent_config.config import ConfigManager
from opik.api_objects.agent_config.blueprint import Blueprint
from opik.rest_api import core as rest_api_core
from opik.rest_api.types.agent_blueprint_public import AgentBlueprintPublic
from opik.rest_api.types.agent_config_value_public import AgentConfigValuePublic
def _make_raw_blueprint(blueprint_id="bp-1", values=None, description=None):
if values is None:
values = [
AgentConfigValuePublic(key="temp", type="float", value="0.6"),
AgentConfigValuePublic(key="name", type="string", value="agent"),
]
return AgentBlueprintPublic(
id=blueprint_id, type="blueprint", values=values, description=description
)
@pytest.fixture
def mock_rest_client():
client = mock.Mock()
client.agent_configs = mock.Mock()
client.agent_configs.create_agent_config.return_value = None
client.agent_configs.get_latest_blueprint.return_value = _make_raw_blueprint()
client.projects.retrieve_project.return_value = mock.Mock(id="proj-default")
return client
@pytest.fixture
def agent_config(mock_rest_client):
return ConfigManager(
project_name="my-project",
rest_client_=mock_rest_client,
)
class TestConfigManagerProperties:
def test_project_name(self, agent_config):
assert agent_config.project_name == "my-project"
class TestConfigManagerGetBlueprint:
def test_get_blueprint__returns_blueprint(self, agent_config, mock_rest_client):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_latest_blueprint.return_value = (
_make_raw_blueprint()
)
result = agent_config.get_blueprint()
assert isinstance(result, Blueprint)
assert result.id == "bp-1"
def test_get_blueprint__with_env__routes_to_get_blueprint_by_env(
self, agent_config, mock_rest_client
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_blueprint_by_env.return_value = (
_make_raw_blueprint()
)
agent_config.get_blueprint(env="prod")
mock_rest_client.agent_configs.get_blueprint_by_env.assert_called_once_with(
env_name="prod",
project_id="proj-1",
mask_id=None,
request_options=None,
)
mock_rest_client.agent_configs.get_latest_blueprint.assert_not_called()
def test_get_blueprint__with_mask_id__passes_mask_id(
self, agent_config, mock_rest_client
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_latest_blueprint.return_value = (
_make_raw_blueprint()
)
agent_config.get_blueprint(mask_id="mask-1")
mock_rest_client.agent_configs.get_latest_blueprint.assert_called_once_with(
project_id="proj-1",
mask_id="mask-1",
request_options=None,
)
def test_get_blueprint__with_field_types__resolves_values(
self, agent_config, mock_rest_client
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_latest_blueprint.return_value = (
_make_raw_blueprint()
)
result = agent_config.get_blueprint(field_types={"temp": float, "name": str})
assert result["temp"] == 0.6
assert result["name"] == "agent"
def test_get_blueprint__without_field_types__infers_types_from_backend(
self, agent_config, mock_rest_client
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_latest_blueprint.return_value = (
_make_raw_blueprint()
)
result = agent_config.get_blueprint()
assert result["temp"] == 0.6
assert isinstance(result["temp"], float)
assert result["name"] == "agent"
assert isinstance(result["name"], str)
def test_get_blueprint__not_found__returns_none(
self, agent_config, mock_rest_client
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_latest_blueprint.side_effect = (
rest_api_core.ApiError(status_code=404, body="not found")
)
result = agent_config.get_blueprint()
assert result is None
@pytest.mark.parametrize(
"mask_id",
["mask-1", "mask-2", None],
ids=["mask_1", "mask_2", "no_mask"],
)
def test_get_blueprint__mask_id__passed_to_backend(
self, agent_config, mock_rest_client, mask_id
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_latest_blueprint.return_value = (
_make_raw_blueprint()
)
agent_config.get_blueprint(mask_id=mask_id)
mock_rest_client.agent_configs.get_latest_blueprint.assert_called_once_with(
project_id="proj-1",
mask_id=mask_id,
request_options=None,
)
def test_get_blueprint__env_with_mask_id__routes_to_get_blueprint_by_env(
self, agent_config, mock_rest_client
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_blueprint_by_env.return_value = (
_make_raw_blueprint()
)
agent_config.get_blueprint(env="prod", mask_id="mask-1")
mock_rest_client.agent_configs.get_blueprint_by_env.assert_called_once_with(
env_name="prod",
project_id="proj-1",
mask_id="mask-1",
request_options=None,
)
class TestConfigManagerGetBlueprintByName:
def test_get_blueprint_by_name__returns_blueprint(
self, agent_config, mock_rest_client
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_blueprint_by_name.return_value = (
_make_raw_blueprint(blueprint_id="bp-specific")
)
result = agent_config.get_blueprint(name="v1")
assert isinstance(result, Blueprint)
assert result.id == "bp-specific"
mock_rest_client.agent_configs.get_blueprint_by_name.assert_called_once_with(
project_id="proj-1", name="v1", mask_id=None, request_options=None
)
def test_get_blueprint_by_name__not_found__returns_none(
self, agent_config, mock_rest_client
):
mock_rest_client.projects.retrieve_project.return_value = mock.Mock(id="proj-1")
mock_rest_client.agent_configs.get_blueprint_by_name.side_effect = (
rest_api_core.ApiError(status_code=404, body="not found")
)
result = agent_config.get_blueprint(name="nonexistent")
assert result is None
class TestConfigManagerCreateBlueprint:
def test_create_blueprint__happy_path__calls_backend_and_returns_blueprint(
self, agent_config, mock_rest_client
):
mock_rest_client.agent_configs.get_blueprint_by_id.return_value = (
_make_raw_blueprint(blueprint_id="bp-new")
)
result = agent_config.create_blueprint(
fields_with_values={
"temperature": types.FieldValueSpec(float, 0.6),
"name": types.FieldValueSpec(str, "agent"),
}
)
mock_rest_client.agent_configs.create_agent_config.assert_called_once()
call_kwargs = mock_rest_client.agent_configs.create_agent_config.call_args[1]
blueprint = call_kwargs["blueprint"]
assert blueprint.type == "blueprint"
assert blueprint.values is not None
assert blueprint.id is not None
assert isinstance(result, Blueprint)
def test_create_blueprint__bool_field__serialized_as_boolean_type(
self, agent_config, mock_rest_client
):
mock_rest_client.agent_configs.get_blueprint_by_id.return_value = (
_make_raw_blueprint()
)
agent_config.create_blueprint(
fields_with_values={"flag": types.FieldValueSpec(bool, False)}
)
call_kwargs = mock_rest_client.agent_configs.create_agent_config.call_args[1]
blueprint = call_kwargs["blueprint"]
flag_param = [v for v in blueprint.values if v.key == "flag"][0]
assert flag_param.type == "boolean"
assert flag_param.value == "false"
def test_create_blueprint__with_project_name__passes_project_to_backend(
self, agent_config, mock_rest_client
):
mock_rest_client.agent_configs.get_blueprint_by_id.return_value = (
_make_raw_blueprint()
)
agent_config.create_blueprint(
fields_with_values={"temp": types.FieldValueSpec(float, 0.5)}
)
call_kwargs = mock_rest_client.agent_configs.create_agent_config.call_args[1]
assert call_kwargs["project_name"] == "my-project"
def test_create_blueprint__with_parameters__returns_blueprint(
self, agent_config, mock_rest_client
):
mock_rest_client.agent_configs.get_blueprint_by_id.return_value = (
_make_raw_blueprint(blueprint_id="bp-new")
)
result = agent_config.create_blueprint(
parameters={"temp": 0.6, "name": "agent"}
)
assert isinstance(result, Blueprint)
assert result.id == "bp-new"
call_kwargs = mock_rest_client.agent_configs.create_agent_config.call_args[1]
bp = call_kwargs["blueprint"]
keys = {v.key for v in bp.values}
assert "temp" in keys
assert "name" in keys
def test_create_blueprint__with_fields_with_values__returns_blueprint(
self, agent_config, mock_rest_client
):
mock_rest_client.agent_configs.get_blueprint_by_id.return_value = (
_make_raw_blueprint()
)
result = agent_config.create_blueprint(
fields_with_values={"temp": types.FieldValueSpec(float, 0.6)}
)
assert isinstance(result, Blueprint)
call_kwargs = mock_rest_client.agent_configs.create_agent_config.call_args[1]
keys = {v.key for v in call_kwargs["blueprint"].values}
assert "temp" in keys
def test_create_blueprint__with_description__passes_description(
self, agent_config, mock_rest_client
):
mock_rest_client.agent_configs.get_blueprint_by_id.return_value = (
_make_raw_blueprint()
)
agent_config.create_blueprint(parameters={"temp": 0.6}, description="v1")
call_kwargs = mock_rest_client.agent_configs.create_agent_config.call_args[1]
assert call_kwargs["blueprint"].description == "v1"
class TestConfigManagerCreateMask:
def test_create_mask__happy_path__calls_backend_with_mask_type(
self, agent_config, mock_rest_client
):
agent_config.create_mask(
fields_with_values={"temperature": types.FieldValueSpec(float, 0.3)}
)
call_kwargs = mock_rest_client.agent_configs.update_agent_config.call_args[1]
blueprint = call_kwargs["blueprint"]
assert blueprint.type == "mask"
assert blueprint.values is not None
def test_create_mask__returns_mask_id(self, agent_config, mock_rest_client):
result = agent_config.create_mask(
fields_with_values={"temperature": types.FieldValueSpec(float, 0.3)}
)
assert isinstance(result, str)
mock_rest_client.agent_configs.get_blueprint_by_id.assert_not_called()
def test_create_mask__sends_under_blueprint_key(
self, agent_config, mock_rest_client
):
agent_config.create_mask(
fields_with_values={"temperature": types.FieldValueSpec(float, 0.3)}
)
call_kwargs = mock_rest_client.agent_configs.update_agent_config.call_args[1]
assert "blueprint" in call_kwargs
assert call_kwargs["blueprint"].id is not None
def test_create_mask__with_parameters__returns_mask_id(
self, agent_config, mock_rest_client
):
result = agent_config.create_mask(parameters={"temp": 0.3})
assert isinstance(result, str)
def test_create_mask__with_description__passes_description(
self, agent_config, mock_rest_client
):
agent_config.create_mask(
fields_with_values={"temperature": types.FieldValueSpec(float, 0.3)},
description="variant-A",
)
call_kwargs = mock_rest_client.agent_configs.update_agent_config.call_args[1]
assert call_kwargs["blueprint"].description == "variant-A"
def test_create_mask__with_project_name__passes_project_to_backend(
self, agent_config, mock_rest_client
):
agent_config.create_mask(
fields_with_values={"temp": types.FieldValueSpec(float, 0.5)}
)
call_kwargs = mock_rest_client.agent_configs.update_agent_config.call_args[1]
assert call_kwargs["project_name"] == "my-project"
class TestResolveFieldsWithValues:
def test_none_value__included_with_str_type(self):
result = ConfigManager._resolve_fields_with_values(
parameters={"temp": 0.5, "name": None},
fields_with_values=None,
)
assert result["temp"] == types.FieldValueSpec(float, 0.5)
assert result["name"] == types.FieldValueSpec(str, None)
def test_all_none_parameters__included_with_str_type(self):
result = ConfigManager._resolve_fields_with_values(
parameters={"a": None, "b": None},
fields_with_values=None,
)
assert result == {
"a": types.FieldValueSpec(str, None),
"b": types.FieldValueSpec(str, None),
}
def test_fields_with_values_takes_precedence_over_parameters(self):
explicit = {"x": types.FieldValueSpec(int, 1)}
result = ConfigManager._resolve_fields_with_values(
parameters={"x": 99},
fields_with_values=explicit,
)
assert result is explicit
def test_create_blueprint__none_parameter__included_in_payload_with_string_type(
self, agent_config, mock_rest_client
):
mock_rest_client.agent_configs.get_blueprint_by_id.return_value = (
_make_raw_blueprint()
)
agent_config.create_blueprint(parameters={"temp": 0.6, "name": None})
call_kwargs = mock_rest_client.agent_configs.create_agent_config.call_args[1]
values_by_key = {v.key: v for v in call_kwargs["blueprint"].values}
assert "temp" in values_by_key
assert "name" in values_by_key
assert values_by_key["name"].type == "string"
assert values_by_key["name"].value is None
@@ -0,0 +1,35 @@
from opik.api_objects.agent_config.context import (
agent_config_context,
get_active_config_mask,
)
class TestConfigContext:
def test_no_context__returns_none(self):
assert get_active_config_mask() is None
def test_inside_context__returns_active_mask(self):
with agent_config_context("mask-1"):
assert get_active_config_mask() == "mask-1"
def test_after_context_exit__returns_none(self):
with agent_config_context("mask-1"):
pass
assert get_active_config_mask() is None
def test_nested_contexts__inner_wins_then_restores_outer(self):
with agent_config_context("outer"):
assert get_active_config_mask() == "outer"
with agent_config_context("inner"):
assert get_active_config_mask() == "inner"
assert get_active_config_mask() == "outer"
assert get_active_config_mask() is None
def test_exception_inside_context__mask_still_resets(self):
try:
with agent_config_context("mask-err"):
assert get_active_config_mask() == "mask-err"
raise ValueError("test error")
except ValueError:
pass
assert get_active_config_mask() is None
@@ -0,0 +1,445 @@
import json
from typing import List, Dict, Callable, Optional
from unittest import mock
import pytest
from opik.api_objects import type_helpers
from opik.api_objects.prompt.base_prompt import BasePrompt
from opik.api_objects.prompt.text.prompt import Prompt
from opik.api_objects.prompt.chat.chat_prompt import ChatPrompt
from opik.rest_api.types.prompt_version_detail import PromptVersionDetail
class TestIsPromptType:
@pytest.mark.parametrize(
"py_type",
[BasePrompt, Prompt, ChatPrompt],
ids=["BasePrompt", "Prompt", "ChatPrompt"],
)
def test_prompt_classes__returns_true(self, py_type):
assert type_helpers.is_prompt_type(py_type) is True
@pytest.mark.parametrize(
"py_type",
[str, int, float, bool, object, list],
ids=["str", "int", "float", "bool", "object", "list"],
)
def test_non_prompt_types__returns_false(self, py_type):
assert type_helpers.is_prompt_type(py_type) is False
def test_custom_subclass_of_base_prompt__returns_true(self):
class MyPrompt(BasePrompt):
@property
def name(self):
return ""
@property
def commit(self):
return None
@property
def version_id(self):
return ""
@property
def metadata(self):
return None
@property
def type(self):
return None
@property
def id(self):
return None
@property
def description(self):
return None
@property
def change_description(self):
return None
@property
def tags(self):
return None
def format(self, *args, **kwargs):
return ""
def __internal_api__to_info_dict__(self):
return {}
assert type_helpers.is_prompt_type(MyPrompt) is True
class TestBackendTypeToPythonType:
@pytest.mark.parametrize(
"backend_type, expected",
[
("string", str),
("integer", int),
("float", float),
("boolean", bool),
],
ids=["string", "integer", "float", "boolean"],
)
def test_primitive_types__returns_python_type(self, backend_type, expected):
assert type_helpers.backend_type_to_python_type(backend_type) is expected
@pytest.mark.parametrize(
"backend_type",
["prompt", "prompt_commit", "unknown", ""],
ids=["prompt", "prompt_commit", "unknown", "empty"],
)
def test_non_primitive_types__returns_none(self, backend_type):
assert type_helpers.backend_type_to_python_type(backend_type) is None
class TestIsSupportedType:
@pytest.mark.parametrize(
"py_type, expected",
[
(str, True),
(int, True),
(float, True),
(bool, True),
(List[str], True),
(List[int], True),
(List[float], True),
(List[bool], True),
(Dict[str, float], True),
(Dict[str, bool], True),
(Dict[str, int], True),
(Dict[str, str], True),
(Prompt, True),
(ChatPrompt, True),
(BasePrompt, True),
(PromptVersionDetail, True),
],
ids=[
"str",
"int",
"float",
"bool",
"List[str]",
"List[int]",
"List[float]",
"List[bool]",
"Dict[str,float]",
"Dict[str,bool]",
"Dict[str,int]",
"Dict[str,str]",
"Prompt",
"ChatPrompt",
"BasePrompt",
"PromptVersionDetail",
],
)
def test_supported_types__returns_true(self, py_type, expected):
assert type_helpers.is_supported_type(py_type) is expected
@pytest.mark.parametrize(
"py_type",
[
list,
dict,
Callable,
object,
Dict[int, str],
],
ids=["bare_list", "bare_dict", "Callable", "object", "Dict[int,str]"],
)
def test_unsupported_types__returns_false(self, py_type):
assert type_helpers.is_supported_type(py_type) is False
def test_custom_class__returns_false(self):
class Foo:
pass
assert type_helpers.is_supported_type(Foo) is False
def test_list_of_custom_class__returns_false(self):
class Bar:
pass
assert type_helpers.is_supported_type(List[Bar]) is False
class TestPythonTypeToBackendType:
@pytest.mark.parametrize(
"py_type, expected_backend_type",
[
(str, "string"),
(int, "integer"),
(float, "float"),
(bool, "boolean"),
(List[str], "string"),
(Dict[str, int], "string"),
],
ids=["str", "int", "float", "bool", "List", "Dict"],
)
def test_known_types__returns_correct_backend_type(
self, py_type, expected_backend_type
):
assert (
type_helpers.python_type_to_backend_type(py_type) == expected_backend_type
)
def test_unsupported_type__raises_type_error(self):
with pytest.raises(TypeError):
type_helpers.python_type_to_backend_type(object)
@pytest.mark.parametrize(
"py_type",
[Prompt, ChatPrompt, BasePrompt],
ids=["Prompt", "ChatPrompt", "BasePrompt"],
)
def test_prompt_types__return_prompt(self, py_type):
assert type_helpers.python_type_to_backend_type(py_type) == "prompt"
def test_prompt_version_type__returns_prompt_commit(self):
assert (
type_helpers.python_type_to_backend_type(PromptVersionDetail)
== "prompt_commit"
)
class TestPythonValueToBackendValue:
@pytest.mark.parametrize(
"value, py_type, expected",
[
("hello", str, "hello"),
(42, int, "42"),
(0, int, "0"),
(0.6, float, "0.6"),
(0.0, float, "0.0"),
(True, bool, "true"),
(False, bool, "false"),
],
ids=[
"str",
"int",
"int_zero",
"float",
"float_zero",
"bool_true",
"bool_false",
],
)
def test_primitives__serialized_correctly(self, value, py_type, expected):
assert type_helpers.python_value_to_backend_value(value, py_type) == expected
@pytest.mark.parametrize(
"value, py_type, expected_parsed",
[
([1, 2, 3], List[int], [1, 2, 3]),
([], List[str], []),
({"a": 1, "b": 2}, Dict[str, int], {"a": 1, "b": 2}),
({}, Dict[str, str], {}),
],
ids=["list", "empty_list", "dict", "empty_dict"],
)
def test_collections__serialized_as_json(self, value, py_type, expected_parsed):
result = type_helpers.python_value_to_backend_value(value, py_type)
assert json.loads(result) == expected_parsed
@pytest.mark.parametrize(
"py_type",
[Prompt, ChatPrompt, BasePrompt],
ids=["Prompt", "ChatPrompt", "BasePrompt"],
)
def test_prompt_value__returns_commit(self, py_type):
prompt = mock.Mock()
prompt.commit = "abc12345"
assert type_helpers.python_value_to_backend_value(prompt, py_type) == "abc12345"
def test_prompt_version_value__returns_commit(self):
version = mock.Mock(spec=PromptVersionDetail)
version.commit = "pv123456"
assert (
type_helpers.python_value_to_backend_value(version, PromptVersionDetail)
== "pv123456"
)
@pytest.mark.parametrize(
"py_type",
[str, int, float, bool, List[str], Dict[str, int]],
ids=["str", "int", "float", "bool", "List[str]", "Dict[str,int]"],
)
def test_none_value__returns_none(self, py_type):
assert type_helpers.python_value_to_backend_value(None, py_type) is None
class TestBackendValueToPythonValue:
@pytest.mark.parametrize(
"value, py_type, expected",
[
("hello", str, "hello"),
("42", int, 42),
("42.0", int, 42),
(42, int, 42),
("0.6", float, 0.6),
(0.6, float, 0.6),
("true", bool, True),
("1", bool, True),
("yes", bool, True),
("false", bool, False),
("0", bool, False),
(True, bool, True),
],
ids=[
"str",
"int_from_str",
"int_from_float_str",
"int_native",
"float_from_str",
"float_native",
"bool_true_str",
"bool_one_str",
"bool_yes_str",
"bool_false_str",
"bool_zero_str",
"bool_native",
],
)
def test_primitives__deserialized_correctly(self, value, py_type, expected):
assert type_helpers.backend_value_to_python_value(value, py_type) == expected
@pytest.mark.parametrize(
"value, py_type, expected",
[
("[1, 2, 3]", List[int], [1, 2, 3]),
([1, 2, 3], List[int], [1, 2, 3]),
('{"a": 1}', Dict[str, int], {"a": 1}),
({"a": 1}, Dict[str, int], {"a": 1}),
],
ids=["list_from_json", "list_native", "dict_from_json", "dict_native"],
)
def test_collections__deserialized_correctly(self, value, py_type, expected):
assert type_helpers.backend_value_to_python_value(value, py_type) == expected
def test_none__returns_none(self):
assert type_helpers.backend_value_to_python_value(None, str) is None
@pytest.mark.parametrize(
"py_type",
[Prompt, ChatPrompt, BasePrompt],
ids=["Prompt", "ChatPrompt", "BasePrompt"],
)
def test_prompt_type__returns_raw_version_id_string(self, py_type):
result = type_helpers.backend_value_to_python_value("ver-xyz", py_type)
assert result == "ver-xyz"
@pytest.mark.parametrize(
"py_type",
[Prompt, ChatPrompt, BasePrompt],
ids=["Prompt", "ChatPrompt", "BasePrompt"],
)
def test_prompt_type__none_value__returns_none(self, py_type):
assert type_helpers.backend_value_to_python_value(None, py_type) is None
def test_prompt_version_type__returns_raw_version_id_string(self):
result = type_helpers.backend_value_to_python_value(
"ver-pv-xyz", PromptVersionDetail
)
assert result == "ver-pv-xyz"
def test_prompt_version_type__none_value__returns_none(self):
assert (
type_helpers.backend_value_to_python_value(None, PromptVersionDetail)
is None
)
class TestRoundTrip:
@pytest.mark.parametrize(
"value, py_type",
[
("hello", str),
(42, int),
(0, int),
(0.6, float),
(0.0, float),
(True, bool),
(False, bool),
],
ids=[
"str",
"int",
"int_zero",
"float",
"float_zero",
"bool_true",
"bool_false",
],
)
def test_serialize_then_deserialize__recovers_original(self, value, py_type):
backend_value = type_helpers.python_value_to_backend_value(value, py_type)
restored = type_helpers.backend_value_to_python_value(backend_value, py_type)
assert restored == value
assert isinstance(restored, py_type)
class TestUnwrapOptional:
@pytest.mark.parametrize(
"py_type, expected",
[
(Optional[str], str),
(Optional[int], int),
(Optional[float], float),
(Optional[bool], bool),
],
ids=["Optional[str]", "Optional[int]", "Optional[float]", "Optional[bool]"],
)
def test_optional_primitives__returns_inner_type(self, py_type, expected):
assert type_helpers.unwrap_optional(py_type) is expected
@pytest.mark.parametrize(
"py_type",
[str, int, float, bool, List[str], Dict[str, int]],
ids=["str", "int", "float", "bool", "List[str]", "Dict[str,int]"],
)
def test_non_optional__returns_none(self, py_type):
assert type_helpers.unwrap_optional(py_type) is None
def test_union_with_multiple_types__returns_none(self):
# Union[str, int] is not Optional so must not be unwrapped
import typing
assert type_helpers.unwrap_optional(typing.Union[str, int]) is None
class TestIsSupportedTypeOptional:
@pytest.mark.parametrize(
"py_type",
[Optional[str], Optional[int], Optional[float], Optional[bool]],
ids=["Optional[str]", "Optional[int]", "Optional[float]", "Optional[bool]"],
)
def test_optional_primitives__returns_true(self, py_type):
assert type_helpers.is_supported_type(py_type) is True
def test_optional_unsupported__returns_false(self):
assert type_helpers.is_supported_type(Optional[object]) is False
def test_union_with_multiple_types__returns_false(self):
import typing
assert type_helpers.is_supported_type(typing.Union[str, int]) is False
class TestPythonTypeToBackendTypeOptional:
@pytest.mark.parametrize(
"py_type, expected",
[
(Optional[str], "string"),
(Optional[int], "integer"),
(Optional[float], "float"),
(Optional[bool], "boolean"),
],
ids=["Optional[str]", "Optional[int]", "Optional[float]", "Optional[bool]"],
)
def test_optional_primitives__returns_inner_backend_type(self, py_type, expected):
assert type_helpers.python_type_to_backend_type(py_type) == expected