chore: import upstream snapshot with attribution
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,51 @@
|
||||
Annotation Queues Client
|
||||
========================
|
||||
|
||||
The Annotation Queues client provides methods for managing annotation queues in the Opik platform.
|
||||
Annotation queues enable human-in-the-loop workflows for reviewing and annotating traces or threads.
|
||||
|
||||
.. autoclass:: opik.rest_api.annotation_queues.client.AnnotationQueuesClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Create an annotation queue for traces
|
||||
queue = client.create_annotation_queue(
|
||||
name="Review Queue",
|
||||
scope="trace",
|
||||
description="Queue for reviewing model outputs",
|
||||
instructions="Check for accuracy and relevance"
|
||||
)
|
||||
|
||||
# Get traces and add them to the queue
|
||||
traces = client.search_traces(project_name="my-project")
|
||||
queue.add_traces(traces[:10])
|
||||
|
||||
# Fetch all the traces currently in the queue
|
||||
items = queue.get_items()
|
||||
|
||||
# Get an existing queue by ID
|
||||
existing_queue = client.get_annotation_queue("queue-id")
|
||||
|
||||
# List all annotation queues
|
||||
queues = client.get_annotation_queues()
|
||||
|
||||
# Update queue properties
|
||||
queue.update(description="Updated description")
|
||||
|
||||
# Remove traces from the queue
|
||||
queue.remove_traces(traces[:5])
|
||||
|
||||
# Delete the queue
|
||||
queue.delete()
|
||||
@@ -0,0 +1,40 @@
|
||||
Attachments Client
|
||||
==================
|
||||
|
||||
The Attachments client provides methods for managing file attachments in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.attachments.client.AttachmentsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Upload an attachment
|
||||
client.rest_client.attachments.upload_attachment(
|
||||
entity_type="trace",
|
||||
entity_id="trace-id",
|
||||
name="results.json",
|
||||
content=b"{'result': 'success'}"
|
||||
)
|
||||
|
||||
# List attachments for an entity
|
||||
attachments = client.rest_client.attachments.list_attachments(
|
||||
entity_type="trace",
|
||||
entity_id="trace-id"
|
||||
)
|
||||
|
||||
# Download an attachment
|
||||
content = client.rest_client.attachments.download_attachment(
|
||||
entity_type="trace",
|
||||
attachment_id="attachment-id"
|
||||
)
|
||||
+38
@@ -0,0 +1,38 @@
|
||||
Automation Rule Evaluators Client
|
||||
=================================
|
||||
|
||||
The Automation Rule Evaluators client provides methods for managing automated evaluation rules in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.automation_rule_evaluators.client.AutomationRuleEvaluatorsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# List automation rule evaluators
|
||||
evaluators = client.rest_client.automation_rule_evaluators.find_automation_rule_evaluators(
|
||||
page=0,
|
||||
size=10
|
||||
)
|
||||
|
||||
# Get an evaluator by ID
|
||||
evaluator = client.rest_client.automation_rule_evaluators.get_automation_rule_evaluator_by_id(
|
||||
"evaluator-id"
|
||||
)
|
||||
|
||||
# Create a new evaluator
|
||||
client.rest_client.automation_rule_evaluators.create_automation_rule_evaluator(
|
||||
name="my-evaluator",
|
||||
project_id="project-id",
|
||||
code="def evaluate(trace): return {'score': 0.8}"
|
||||
)
|
||||
@@ -0,0 +1,29 @@
|
||||
Chat Completions Client
|
||||
=======================
|
||||
|
||||
The Chat Completions client provides methods for interacting with chat completion endpoints in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.chat_completions.client.ChatCompletionsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Create a chat completion
|
||||
response = client.rest_client.chat_completions.create(
|
||||
model="gpt-3.5-turbo",
|
||||
messages=[
|
||||
{"role": "user", "content": "Hello, how are you?"}
|
||||
],
|
||||
temperature=0.7
|
||||
)
|
||||
@@ -0,0 +1,29 @@
|
||||
Check Client
|
||||
============
|
||||
|
||||
The Check client provides methods for checking system status and access in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.check.client.CheckClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Check access to the workspace
|
||||
client.rest_client.check.access(request={})
|
||||
|
||||
# Get workspace name
|
||||
workspace_info = client.rest_client.check.get_workspace_name()
|
||||
|
||||
# Get bootstrap info
|
||||
bootstrap_info = client.rest_client.check.bootstrap()
|
||||
@@ -0,0 +1,42 @@
|
||||
Datasets Client
|
||||
===============
|
||||
|
||||
The Datasets client provides methods for managing datasets in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.datasets.client.DatasetsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Find datasets
|
||||
datasets = client.rest_client.datasets.find_datasets(
|
||||
page=0,
|
||||
size=10
|
||||
)
|
||||
|
||||
# Get a dataset by ID
|
||||
dataset = client.rest_client.datasets.get_dataset_by_id("dataset-id")
|
||||
|
||||
# Create a new dataset
|
||||
client.rest_client.datasets.create_dataset(
|
||||
name="my-dataset",
|
||||
description="A test dataset"
|
||||
)
|
||||
|
||||
# Get dataset items
|
||||
items = client.rest_client.datasets.get_dataset_items(
|
||||
dataset_id="dataset-id",
|
||||
page=0,
|
||||
size=100
|
||||
)
|
||||
@@ -0,0 +1,40 @@
|
||||
Experiments Client
|
||||
==================
|
||||
|
||||
The Experiments client provides methods for managing experiments in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.experiments.client.ExperimentsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Find experiments
|
||||
experiments = client.rest_client.experiments.find_experiments(
|
||||
page=0,
|
||||
size=10
|
||||
)
|
||||
|
||||
# Get an experiment by ID
|
||||
experiment = client.rest_client.experiments.get_experiment_by_id("experiment-id")
|
||||
|
||||
# Create a new experiment
|
||||
client.rest_client.experiments.create_experiment(
|
||||
name="my-experiment",
|
||||
dataset_name="my-dataset"
|
||||
)
|
||||
|
||||
# Stream experiment items
|
||||
items_generator = client.rest_client.experiments.stream_experiment_items(
|
||||
experiment_id="experiment-id"
|
||||
)
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
Feedback Definitions Client
|
||||
===========================
|
||||
|
||||
The Feedback Definitions client provides methods for managing feedback score definitions in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.feedback_definitions.client.FeedbackDefinitionsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Find feedback definitions
|
||||
definitions = client.rest_client.feedback_definitions.find_feedback_definitions(
|
||||
type="numerical",
|
||||
page=0,
|
||||
size=10
|
||||
)
|
||||
|
||||
# Create a feedback definition
|
||||
client.rest_client.feedback_definitions.create_feedback_definition(
|
||||
name="accuracy",
|
||||
type="numerical",
|
||||
min_value=0.0,
|
||||
max_value=1.0
|
||||
)
|
||||
|
||||
# Get a feedback definition by ID
|
||||
definition = client.rest_client.feedback_definitions.get_feedback_definition_by_id(
|
||||
"definition-id"
|
||||
)
|
||||
@@ -0,0 +1,35 @@
|
||||
Guardrails Client
|
||||
=================
|
||||
|
||||
The Guardrails client provides methods for managing guardrails in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.guardrails.client.GuardrailsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Validate content with guardrails
|
||||
result = client.rest_client.guardrails.validate(
|
||||
checks=[
|
||||
{
|
||||
"name": "pii_check",
|
||||
"enabled": True
|
||||
},
|
||||
{
|
||||
"name": "toxicity_check",
|
||||
"enabled": True
|
||||
}
|
||||
],
|
||||
input="This is the text to validate"
|
||||
)
|
||||
@@ -0,0 +1,96 @@
|
||||
REST API Clients
|
||||
================
|
||||
|
||||
This section documents all the REST API client modules available through ``opik.rest_client``.
|
||||
Each client provides methods for interacting with specific resources in the Opik platform.
|
||||
|
||||
Core Resource Clients
|
||||
---------------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
traces
|
||||
spans
|
||||
datasets
|
||||
experiments
|
||||
projects
|
||||
|
||||
These clients handle the main resources you'll work with in Opik: traces for observability,
|
||||
spans for detailed execution tracking, datasets for evaluation data, experiments for testing,
|
||||
and projects for organization.
|
||||
|
||||
Feedback & Evaluation Clients
|
||||
-----------------------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
annotation_queues
|
||||
feedback_definitions
|
||||
automation_rule_evaluators
|
||||
optimizations
|
||||
|
||||
These clients manage evaluation and feedback systems: annotation queues for human review,
|
||||
defining feedback score types, setting up automated evaluation rules, and running optimization experiments.
|
||||
|
||||
Content & Asset Clients
|
||||
-----------------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
prompts
|
||||
attachments
|
||||
|
||||
These clients handle content management: prompt templates and versions, and file attachments
|
||||
for traces and spans.
|
||||
|
||||
System & Configuration Clients
|
||||
------------------------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
check
|
||||
workspaces
|
||||
llm_provider_key
|
||||
service_toggles
|
||||
system_usage
|
||||
|
||||
These clients provide system-level functionality: health checks, workspace management,
|
||||
API key configuration, feature toggles, and usage monitoring.
|
||||
|
||||
Integration Clients
|
||||
-------------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
chat_completions
|
||||
open_telemetry_ingestion
|
||||
guardrails
|
||||
redirect
|
||||
|
||||
These clients support integrations with external systems: chat completion APIs,
|
||||
OpenTelemetry data ingestion, content validation, and URL redirection.
|
||||
|
||||
Usage Examples
|
||||
--------------
|
||||
|
||||
Each client page includes specific usage examples. Here's how to access any client:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Access any client through the rest_client property
|
||||
traces_client = client.rest_client.traces
|
||||
datasets_client = client.rest_client.datasets
|
||||
experiments_client = client.rest_client.experiments
|
||||
|
||||
# Use the client methods
|
||||
trace = traces_client.get_trace_by_id("trace-id")
|
||||
datasets = datasets_client.find_datasets(page=0, size=10)
|
||||
@@ -0,0 +1,37 @@
|
||||
LLM Provider Key Client
|
||||
=======================
|
||||
|
||||
The LLM Provider Key client provides methods for managing LLM provider API keys in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.llm_provider_key.client.LlmProviderKeyClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Create or update a provider API key
|
||||
client.rest_client.llm_provider_key.create_or_update_provider_api_key(
|
||||
provider="openai",
|
||||
api_key="your-api-key"
|
||||
)
|
||||
|
||||
# List provider API keys
|
||||
keys = client.rest_client.llm_provider_key.get_provider_api_keys(
|
||||
page=0,
|
||||
size=10
|
||||
)
|
||||
|
||||
# Delete a provider API key
|
||||
client.rest_client.llm_provider_key.delete_provider_api_key(
|
||||
provider="openai"
|
||||
)
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
OpenTelemetry Ingestion Client
|
||||
==============================
|
||||
|
||||
The OpenTelemetry Ingestion client provides methods for ingesting OpenTelemetry data into the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.open_telemetry_ingestion.client.OpenTelemetryIngestionClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Ingest OpenTelemetry traces data
|
||||
client.rest_client.open_telemetry_ingestion.ingest_traces(
|
||||
traces_data=traces_payload
|
||||
)
|
||||
|
||||
# Ingest OpenTelemetry logs data
|
||||
client.rest_client.open_telemetry_ingestion.ingest_logs(
|
||||
logs_data=logs_payload
|
||||
)
|
||||
@@ -0,0 +1,44 @@
|
||||
Optimizations Client
|
||||
====================
|
||||
|
||||
The Optimizations client provides methods for managing optimization experiments in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.optimizations.client.OptimizationsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Create an optimization
|
||||
client.rest_client.optimizations.create_optimization(
|
||||
name="my-optimization",
|
||||
dataset_name="my-dataset",
|
||||
objective_name="accuracy",
|
||||
status="running"
|
||||
)
|
||||
|
||||
# Get an optimization by ID
|
||||
optimization = client.rest_client.optimizations.get_optimization_by_id(
|
||||
"optimization-id"
|
||||
)
|
||||
|
||||
# Update optimization status
|
||||
client.rest_client.optimizations.update_optimization(
|
||||
id="optimization-id",
|
||||
status="completed"
|
||||
)
|
||||
|
||||
# Delete optimizations
|
||||
client.rest_client.optimizations.delete_optimizations_by_id(
|
||||
ids=["optimization-id-1", "optimization-id-2"]
|
||||
)
|
||||
@@ -0,0 +1,42 @@
|
||||
Projects Client
|
||||
===============
|
||||
|
||||
The Projects client provides methods for managing projects in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.projects.client.ProjectsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Find projects
|
||||
projects = client.rest_client.projects.find_projects(
|
||||
page=0,
|
||||
size=10
|
||||
)
|
||||
|
||||
# Get a project by ID
|
||||
project = client.rest_client.projects.get_project_by_id("project-id")
|
||||
|
||||
# Create a new project
|
||||
client.rest_client.projects.create_project(
|
||||
name="my-project",
|
||||
description="A test project"
|
||||
)
|
||||
|
||||
# Get project metrics
|
||||
metrics = client.rest_client.projects.get_project_metrics(
|
||||
project_id="project-id",
|
||||
metric_type="trace_count",
|
||||
interval="1h"
|
||||
)
|
||||
@@ -0,0 +1,41 @@
|
||||
Prompts Client
|
||||
==============
|
||||
|
||||
The Prompts client provides methods for managing prompts in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.prompts.client.PromptsClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Create a prompt
|
||||
client.rest_client.prompts.create_prompt(
|
||||
name="my-prompt",
|
||||
prompt="Tell me about {{topic}}",
|
||||
type="mustache"
|
||||
)
|
||||
|
||||
# Get a prompt by name
|
||||
prompt = client.rest_client.prompts.get_prompt_by_name("my-prompt")
|
||||
|
||||
# List all prompts
|
||||
prompts = client.rest_client.prompts.find_prompts(
|
||||
page=0,
|
||||
size=10
|
||||
)
|
||||
|
||||
# Get prompt versions
|
||||
versions = client.rest_client.prompts.get_prompt_versions(
|
||||
prompt_name="my-prompt"
|
||||
)
|
||||
@@ -0,0 +1,25 @@
|
||||
Redirect Client
|
||||
===============
|
||||
|
||||
The Redirect client provides methods for handling URL redirects in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.redirect.client.RedirectClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Handle redirect operations
|
||||
result = client.rest_client.redirect.redirect(
|
||||
target_url="https://example.com/target"
|
||||
)
|
||||
@@ -0,0 +1,28 @@
|
||||
Service Toggles Client
|
||||
======================
|
||||
|
||||
The Service Toggles client provides methods for managing service feature toggles in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.service_toggles.client.ServiceTogglesClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Get service toggles configuration
|
||||
config = client.rest_client.service_toggles.get_service_toggles()
|
||||
|
||||
# Check if a specific feature is enabled
|
||||
if config.feature_enabled:
|
||||
# Use the feature
|
||||
pass
|
||||
@@ -0,0 +1,37 @@
|
||||
Spans Client
|
||||
============
|
||||
|
||||
The Spans client provides methods for managing spans in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.spans.client.SpansClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Get a span by ID
|
||||
span = client.rest_client.spans.get_span_by_id("span-id")
|
||||
|
||||
# Search for spans
|
||||
spans = client.rest_client.spans.search_spans(
|
||||
project_name="my-project",
|
||||
trace_id="trace-id",
|
||||
max_results=100
|
||||
)
|
||||
|
||||
# Add feedback score to a span
|
||||
client.rest_client.spans.add_span_feedback_score(
|
||||
id="span-id",
|
||||
name="relevance",
|
||||
value=0.85
|
||||
)
|
||||
@@ -0,0 +1,26 @@
|
||||
System Usage Client
|
||||
===================
|
||||
|
||||
The System Usage client provides methods for retrieving system usage metrics in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.system_usage.client.SystemUsageClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Get system usage metrics
|
||||
usage = client.rest_client.system_usage.get_system_usage()
|
||||
|
||||
# Get workspace usage summary
|
||||
workspace_usage = client.rest_client.system_usage.get_workspace_usage_summary()
|
||||
@@ -0,0 +1,36 @@
|
||||
Traces Client
|
||||
=============
|
||||
|
||||
The Traces client provides methods for managing traces in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.traces.client.TracesClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Get a trace by ID
|
||||
trace = client.rest_client.traces.get_trace_by_id("trace-id")
|
||||
|
||||
# Search for traces
|
||||
traces = client.rest_client.traces.search_traces(
|
||||
project_name="my-project",
|
||||
max_results=100
|
||||
)
|
||||
|
||||
# Add feedback score to a trace
|
||||
client.rest_client.traces.add_trace_feedback_score(
|
||||
id="trace-id",
|
||||
name="accuracy",
|
||||
value=0.95
|
||||
)
|
||||
@@ -0,0 +1,26 @@
|
||||
Workspaces Client
|
||||
=================
|
||||
|
||||
The Workspaces client provides methods for managing workspaces in the Opik platform.
|
||||
|
||||
.. autoclass:: opik.rest_api.workspaces.client.WorkspacesClient
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:inherited-members:
|
||||
:exclude-members: with_raw_response
|
||||
|
||||
Usage Example
|
||||
-------------
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
client = opik.Opik()
|
||||
|
||||
# Get workspace information
|
||||
workspace = client.rest_client.workspaces.get_workspace()
|
||||
|
||||
# Get workspace statistics
|
||||
stats = client.rest_client.workspaces.get_workspace_stats()
|
||||
@@ -0,0 +1,19 @@
|
||||
REST API Objects
|
||||
================
|
||||
|
||||
This page documents all the public data types and response models used by the REST API methods.
|
||||
These objects represent the data structures returned by REST API calls and used as parameters.
|
||||
|
||||
The types are automatically generated from the REST API definition and include classes for resources,
|
||||
page objects for pagination, filter objects for queries, and various utility types.
|
||||
|
||||
.. note::
|
||||
These are the raw REST API data types. For the main SDK objects like ``Dataset``, ``Experiment``, etc.,
|
||||
see the main :doc:`../Opik` documentation and the Objects section in the main documentation.
|
||||
|
||||
.. automodule:: opik.rest_api.types
|
||||
:members:
|
||||
:undoc-members:
|
||||
:show-inheritance:
|
||||
:imported-members:
|
||||
:exclude-members: Dataset, Trace, Span, Experiment, Project, Prompt, SpanPublic, TracePublic
|
||||
@@ -0,0 +1,147 @@
|
||||
REST API Overview
|
||||
=================
|
||||
|
||||
The Opik SDK provides direct access to the underlying REST API client through the ``rest_client`` property.
|
||||
This allows advanced users to make direct API calls when needed, providing full access to all Opik platform functionality.
|
||||
|
||||
.. warning::
|
||||
The REST client is not guaranteed to be backward compatible with future SDK versions.
|
||||
While it provides a convenient way to use the current REST API of Opik,
|
||||
it's not considered safe to heavily rely on its API as Opik's REST API contracts may change.
|
||||
|
||||
When to Use the REST API
|
||||
------------------------
|
||||
|
||||
The REST API is useful when you need to:
|
||||
|
||||
* Perform operations not available in the high-level SDK
|
||||
* Build custom integrations or tools
|
||||
* Access advanced filtering or querying capabilities
|
||||
* Implement batch operations for performance
|
||||
* Work with raw API responses for specific use cases
|
||||
|
||||
Getting Started
|
||||
---------------
|
||||
|
||||
To access the REST client, first create an Opik instance and then use the ``rest_client`` property:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import opik
|
||||
|
||||
# Initialize Opik client
|
||||
client = opik.Opik()
|
||||
|
||||
# Access REST API through the rest_client property
|
||||
rest_client = client.rest_client
|
||||
|
||||
Basic Examples
|
||||
--------------
|
||||
|
||||
Here are some common patterns for using the REST API:
|
||||
|
||||
**Working with Traces**
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# Get a specific trace
|
||||
trace = client.rest_client.traces.get_trace_by_id("trace-id")
|
||||
|
||||
# Search for traces with filters
|
||||
traces = client.rest_client.traces.search_traces(
|
||||
project_name="my-project",
|
||||
filters=[{
|
||||
"field": "name",
|
||||
"operator": "contains",
|
||||
"value": "important"
|
||||
}],
|
||||
max_results=100
|
||||
)
|
||||
|
||||
**Managing Datasets**
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# List all datasets
|
||||
datasets = client.rest_client.datasets.find_datasets(
|
||||
page=0,
|
||||
size=20
|
||||
)
|
||||
|
||||
# Create a new dataset
|
||||
dataset = client.rest_client.datasets.create_dataset(
|
||||
name="my-dataset",
|
||||
description="A test dataset"
|
||||
)
|
||||
|
||||
# Add items to the dataset
|
||||
items = [
|
||||
{
|
||||
"input": {"question": "What is AI?"},
|
||||
"expected_output": {"answer": "Artificial Intelligence"}
|
||||
}
|
||||
]
|
||||
client.rest_client.datasets.create_or_update_dataset_items(
|
||||
dataset_id=dataset.id,
|
||||
items=items
|
||||
)
|
||||
|
||||
**Running Experiments**
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# Create an experiment
|
||||
experiment = client.rest_client.experiments.create_experiment(
|
||||
name="my-experiment",
|
||||
dataset_name="my-dataset"
|
||||
)
|
||||
|
||||
# Add experiment results
|
||||
client.rest_client.experiments.create_experiment_items(
|
||||
experiment_id=experiment.id,
|
||||
items=[{
|
||||
"dataset_item_id": "item-id",
|
||||
"trace_id": "trace-id",
|
||||
"output": {"result": "success"}
|
||||
}]
|
||||
)
|
||||
|
||||
Response Types and Pagination
|
||||
------------------------------
|
||||
|
||||
Most list operations return paginated results with a consistent structure:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# Example paginated response structure
|
||||
response = client.rest_client.datasets.find_datasets(page=0, size=10)
|
||||
|
||||
# Access the data
|
||||
datasets = response.content # List of dataset objects
|
||||
total_count = response.total # Total number of items
|
||||
current_page = response.page # Current page number
|
||||
page_size = response.size # Items per page
|
||||
|
||||
Error Handling
|
||||
--------------
|
||||
|
||||
The REST API raises specific exceptions for different error conditions:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from opik.rest_api.core.api_error import ApiError
|
||||
|
||||
try:
|
||||
trace = client.rest_client.traces.get_trace_by_id("invalid-id")
|
||||
except ApiError as e:
|
||||
if e.status_code == 404:
|
||||
print("Trace not found")
|
||||
else:
|
||||
print(f"API error: {e.status_code} - {e.body}")
|
||||
|
||||
Next Steps
|
||||
----------
|
||||
|
||||
* Browse the :doc:`clients/index` for detailed API reference
|
||||
* See :doc:`objects` for data type documentation
|
||||
* Check the main SDK documentation for higher-level operations
|
||||
Reference in New Issue
Block a user