5a558eb09e
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled
218 lines
6.6 KiB
Plaintext
218 lines
6.6 KiB
Plaintext
---
|
|
headline: Re-running an existing experiment | Opik Documentation
|
|
og:description: Learn how to update existing experiments in Opik by changing names,
|
|
configurations, and scores effectively.
|
|
og:site_name: Opik Documentation
|
|
og:title: 'Re-running Experiments in Opik: Update Names & Scores'
|
|
subtitle: Guides you through the process of updating an existing experiment
|
|
title: Re-running an existing experiment
|
|
canonical-url: https://www.comet.com/docs/opik/evaluation/advanced/evaluate_your_llm
|
|
---
|
|
|
|
<Note>
|
|
In Opik 2.0, datasets and experiments are project-scoped. Make sure to specify a `project_name` when creating datasets and running experiments so they are associated with the correct project.
|
|
</Note>
|
|
|
|
You can update existing experiments in several ways:
|
|
|
|
1. **Update experiment name and configuration** - Change the experiment name or update its configuration metadata
|
|
2. **Update experiment scores** - Add new scores or re-compute existing scores for experiment items
|
|
|
|
## Update Experiment Name and Configuration
|
|
|
|
You can update an experiment's name and configuration from both the Opik UI and the SDKs.
|
|
|
|
### From the Opik UI
|
|
|
|
To update an experiment from the UI:
|
|
|
|
1. Navigate to the **Experiments** page
|
|
2. Find the experiment you want to update
|
|
3. Click the **...** menu button on the experiment row
|
|
4. Select **Edit** from the dropdown menu
|
|
|
|
<Frame>
|
|
<img src="/img/evaluation/edit_experiment.png" />
|
|
</Frame>
|
|
|
|
5. Update the experiment name and/or configuration (JSON format)
|
|
6. Click **Update Experiment** to save your changes
|
|
|
|
<Frame>
|
|
<img src="/img/evaluation/edit_experiment_dialog.png" />
|
|
</Frame>
|
|
|
|
The configuration is stored as JSON and is useful for tracking parameters like model names, temperatures, prompt templates, or any other metadata relevant to your experiment.
|
|
|
|
### From the Python SDK
|
|
|
|
Use the `update_experiment` method to update an experiment's name and configuration:
|
|
|
|
```python
|
|
import opik
|
|
|
|
client = opik.Opik()
|
|
|
|
# Update experiment name
|
|
client.update_experiment(
|
|
id="experiment-id",
|
|
name="Updated Experiment Name"
|
|
)
|
|
|
|
# Update experiment configuration
|
|
client.update_experiment(
|
|
id="experiment-id",
|
|
experiment_config={
|
|
"model": "gpt-4",
|
|
"temperature": 0.7,
|
|
"prompt_template": "Answer the following question: {question}"
|
|
}
|
|
)
|
|
|
|
# Update both name and configuration
|
|
client.update_experiment(
|
|
id="experiment-id",
|
|
name="Updated Experiment Name",
|
|
experiment_config={
|
|
"model": "gpt-4",
|
|
"temperature": 0.7
|
|
}
|
|
)
|
|
```
|
|
|
|
### From the TypeScript SDK
|
|
|
|
Use the `updateExperiment` method to update an experiment's name and configuration:
|
|
|
|
```typescript
|
|
import { Opik } from "opik";
|
|
|
|
const opik = new Opik();
|
|
|
|
// Update experiment name
|
|
await opik.updateExperiment("experiment-id", {
|
|
name: "Updated Experiment Name"
|
|
});
|
|
|
|
// Update experiment configuration
|
|
await opik.updateExperiment("experiment-id", {
|
|
experimentConfig: {
|
|
model: "gpt-4",
|
|
temperature: 0.7,
|
|
promptTemplate: "Answer the following question: {question}"
|
|
}
|
|
});
|
|
|
|
// Update both name and configuration
|
|
await opik.updateExperiment("experiment-id", {
|
|
name: "Updated Experiment Name",
|
|
experimentConfig: {
|
|
model: "gpt-4",
|
|
temperature: 0.7
|
|
}
|
|
});
|
|
```
|
|
|
|
## Update Experiment Scores
|
|
|
|
Sometimes you may want to update an existing experiment with new scores, or update existing scores for an experiment. You can do this using the [`evaluate_experiment` function](https://www.comet.com/docs/opik/python-sdk-reference/evaluation/evaluate_existing.html).
|
|
|
|
This function will re-run the scoring metrics on the existing experiment items and update the scores:
|
|
|
|
```python
|
|
from opik.evaluation import evaluate_experiment
|
|
from opik.evaluation.metrics import Hallucination
|
|
|
|
hallucination_metric = Hallucination()
|
|
|
|
# Replace "my-experiment" with the name of your experiment which can be found in the Opik UI
|
|
evaluate_experiment(experiment_name="my-experiment", scoring_metrics=[hallucination_metric])
|
|
```
|
|
|
|
<Tip>
|
|
The `evaluate_experiment` function can be used to update existing scores for an experiment. If you use a scoring
|
|
metric with the same name as an existing score, the scores will be updated with the new values.
|
|
</Tip>
|
|
|
|
<Note>
|
|
You can also compute experiment-level aggregate metrics when updating experiments using the `experiment_scoring_functions` parameter.
|
|
Learn more about [experiment-level metrics](/v1/evaluation/evaluate_your_llm#computing-experiment-level-metrics).
|
|
</Note>
|
|
|
|
## Example
|
|
|
|
### Create an experiment
|
|
|
|
Suppose you are building a chatbot and want to compute the hallucination scores for a set of example conversations. For this you would create a first experiment with the `evaluate` function:
|
|
|
|
```python
|
|
import opik
|
|
from opik import Opik, track
|
|
from opik.evaluation import evaluate
|
|
from opik.evaluation.metrics import Hallucination
|
|
from opik.integrations.openai import track_openai
|
|
import openai
|
|
|
|
opik.configure(project_name="my-project")
|
|
|
|
# Define the task to evaluate
|
|
openai_client = track_openai(openai.OpenAI())
|
|
|
|
MODEL = "gpt-3.5-turbo"
|
|
|
|
@track
|
|
def your_llm_application(input: str) -> str:
|
|
response = openai_client.chat.completions.create(
|
|
model=MODEL,
|
|
messages=[{"role": "user", "content": input}],
|
|
)
|
|
|
|
return response.choices[0].message.content
|
|
|
|
# Define the evaluation task
|
|
def evaluation_task(x):
|
|
return {
|
|
"output": your_llm_application(x['input'])
|
|
}
|
|
|
|
# Create a simple dataset
|
|
client = Opik()
|
|
dataset = client.get_or_create_dataset(name="Existing experiment dataset", project_name="my-project")
|
|
dataset.insert([
|
|
{"input": "What is the capital of France?"},
|
|
{"input": "What is the capital of Germany?"},
|
|
])
|
|
|
|
# Define the metrics
|
|
hallucination_metric = Hallucination()
|
|
|
|
|
|
evaluation = evaluate(
|
|
experiment_name="Existing experiment example",
|
|
dataset=dataset,
|
|
task=evaluation_task,
|
|
scoring_metrics=[hallucination_metric],
|
|
project_name="my-project",
|
|
experiment_config={
|
|
"model": MODEL
|
|
}
|
|
)
|
|
|
|
experiment_name = evaluation.experiment_name
|
|
print(f"Experiment name: {experiment_name}")
|
|
```
|
|
|
|
<Tip>Learn more about the `evaluate` function in our [LLM evaluation guide](/v1/evaluation/evaluate_your_llm).</Tip>
|
|
|
|
### Update the experiment
|
|
|
|
Once the first experiment is created, you realise that you also want to compute a moderation score for each example. You could re-run the experiment with new scoring metrics but this means re-running the output. Instead, you can simply update the experiment with the new scoring metrics:
|
|
|
|
```python
|
|
from opik.evaluation import evaluate_experiment
|
|
from opik.evaluation.metrics import Moderation
|
|
|
|
moderation_metric = Moderation()
|
|
|
|
evaluate_experiment(experiment_name="already_existing_experiment", scoring_metrics=[moderation_metric])
|
|
``` |