Files
wehub-resource-sync 5a558eb09e
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:25:44 +08:00

148 lines
9.2 KiB
Plaintext

---
headline: Migrate data between projects | Opik Documentation
og:description: Move datasets and prompts — with their full version history and
attached data — between projects with the opik migrate CLI command.
og:site_name: Opik Documentation
og:title: Migrate data between projects with Opik
title: Migrate data
---
`opik migrate` moves an entity — and everything attached to it — from one project to another **in the same workspace**. It copies the entity's full version history and its related data into the destination project.
It has two subcommands:
- **`opik migrate dataset`** — a dataset (or test suite) with its full version history, plus the experiments, traces, spans, feedback scores, assertion results, comments, and optimizations attached to it.
- **`opik migrate prompt`** — a prompt and its full version history.
Use it to consolidate or re-home an entity and its history under a different project.
<Note>
These commands move data **between projects in a single workspace**. To move data between
separate Opik installations or environments, use [`opik export` / `opik import`](/tracing/advanced/export-data#command-line-tools) instead.
</Note>
<Warning>
The migration **renames the source** to `<name>_v1` and gives the destination the original
name. Preview with `--dry-run` first to see exactly what will change.
</Warning>
## Options
The `--workspace` and `--api-key` flags go on the `opik migrate` group, **before** the subcommand. The rest are shared by both `dataset` and `prompt`.
| Flag | Default | Description |
| -------------------------- | ----------------------------------------------- | --------------------------------------------------------------------------------------------- |
| `NAME` (argument) | — | Exact name of the source dataset or prompt to migrate. |
| `--to-project` | — (**required**) | Destination project. It must already exist — create it first if needed. |
| `--from-project` | whole workspace | Optional hint for which project the source lives in. Omit to search the whole workspace. |
| `--dry-run` | `false` | Preview what would happen without making any changes. See [Previewing a migration](#previewing-a-migration). |
| `--workspace` (group flag) | `OPIK_WORKSPACE` → `~/.opik.config` → `default` | Workspace to operate in. |
| `--api-key` (group flag) | `OPIK_API_KEY` → `~/.opik.config` | Opik API key. |
## Previewing a migration
Add `--dry-run` to any migration to see exactly what it would do **without changing anything** — nothing is renamed and nothing is copied. The command resolves the source, prints the plan it would run step by step, then exits.
```bash
opik migrate dataset "MyDataset" --to-project="production" --dry-run
```
Use the preview to confirm the source resolved to the right entity and that the destination project is correct. When the plan looks right, run the same command again without `--dry-run` to apply it.
## `opik migrate dataset`
```bash
opik migrate dataset NAME --to-project=DESTINATION_PROJECT [OPTIONS]
```
`NAME` is the exact name of the source dataset. Both plain datasets and test suites are supported. [Preview with `--dry-run`](#previewing-a-migration) before applying.
### What gets copied
| Entity | What comes across |
| --------------------- | -------------------------------------------------------------------------------------------------- |
| **Dataset** | Name, description, visibility, tags, and type (dataset or test suite). |
| **Version history** | Every version, in order, with the same items and ordering as the source. |
| **Items** | Each item's data, description, tags, evaluators, execution policy, and source. |
| **Experiments** | Name, type, evaluation method, tags, and metadata. |
| **Traces** | Input, output, metadata, tags, timing, thread, errors, and environment. |
| **Spans** | The full span tree, inputs and outputs, metadata, model, provider, usage, cost, tags, and errors. |
| **Feedback scores** | On traces and spans. |
| **Assertion results** | For test suites. |
| **Comments** | On traces and spans. |
| **Optimizations** | Optimizations linked to the dataset, with their experiments re-linked on the destination. |
### What happens when you run it
The same steps appear in the `--dry-run` plan:
| # | Step | What it does |
| --- | --------------------- | -------------------------------------------------------------------- |
| 1 | Rename source | Renames the source to `<name>_v1` to free up its name. |
| 2 | Create destination | Creates the dataset under `--to-project` with the original name. |
| 3 | Replay versions | Replays every version onto the destination, in order. |
| 4 | Copy optimizations | Recreates any optimizations linked to the dataset. |
| 5 | Copy experiments | Recreates the experiments, along with their traces and spans. |
### What is not copied
- **Prompt snapshots on experiments** — migrate prompts separately with `opik migrate prompt`.
- **Attachments on traces and spans** — files like images and audio are not copied.
- **Thread tags, status, feedback scores, and comments** — the traces themselves (including their environment) do come across, but these thread-level fields don't yet.
### Examples
```bash
# Migrate a dataset (with its experiments, traces, and spans)
opik migrate dataset "MyDataset" --to-project="production"
# Preview without making any changes
opik migrate dataset "MyDataset" --to-project="production" --dry-run
# Tell it which project the source is in
opik migrate dataset "MyDataset" --to-project="production" --from-project="staging"
```
### Resuming an interrupted migration
If a migration is interrupted — a crash, a dropped connection, or the process being killed (for example, out-of-memory on a very large dataset) — just re-run the exact same command from the same machine. It resumes instead of starting over:
- **Already-migrated experiments are skipped.** Progress is checkpointed after each experiment completes, so a re-run picks up from the last completed one. The progress bar reflects this — a migration that was ~53% done resumes near 53%, not 0%.
- **An experiment interrupted mid-flight is re-migrated cleanly.** Its partial traces, spans, and experiment record at the destination are deleted first, so the re-run doesn't leave duplicates behind.
The checkpoint is stored locally under `~/.opik/migrate-checkpoints/`, keyed by workspace + destination project + dataset name, so a re-run resumes regardless of which directory you run it from. It is removed automatically once the migration finishes successfully. Resume is local to the machine that ran the migration — moving to a different machine starts fresh. (Experiments are the checkpoint granularity; there is no finer per-trace or per-span resume.)
## `opik migrate prompt`
```bash
opik migrate prompt NAME --to-project=DESTINATION_PROJECT [OPTIONS]
```
`NAME` is the exact name of the source prompt. This subcommand migrates the prompt and its version history only — it does not copy experiments, traces, or spans. [Preview with `--dry-run`](#previewing-a-migration) before applying.
### What gets copied
- **Prompt** — name, description, tags, and template structure.
- **Version history** — every version, oldest first, with its template, metadata, type, change description, and tags.
- **Commit hashes** — each version's commit hash is preserved, so the history matches the source exactly.
It renames the source to `<name>_v1`, creates the destination prompt under `--to-project`, and replays every version.
### Examples
```bash
# Migrate a prompt and its full version history
opik migrate prompt "MyPrompt" --to-project="production"
# Preview without making any changes
opik migrate prompt "MyPrompt" --to-project="production" --dry-run
```
## Troubleshooting
- **Name already used** — the rename target `<name>_v1` already exists. Rename or delete the conflicting entity and re-run:
```
Cannot rename source to 'MyDataset_v1' — that name is already used by a dataset in project 'staging'. Rename or delete the conflicting dataset and re-run.
```