Files
comet-ml--opik/apps/opik-documentation/documentation/fern/docs-v2/prompt_engineering/mcp-server.mdx
T
wehub-resource-sync 5a558eb09e
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:25:44 +08:00

424 lines
13 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
headline: Opik's MCP server | Opik Documentation
og:description: Configure Opik's Python MCP server with Claude Code, Cursor, and VS Code Copilot to read traces, log scores, and manage prompts from your AI host.
og:site_name: Opik Documentation
og:title: Integrate with Opik's MCP server
title: Opik's MCP server
---
Opik's [MCP server](https://github.com/comet-ml/opik-mcp) connects your AI host
(Claude Code, Cursor, VS Code Copilot, MCP Inspector) directly to your Opik
workspace — read traces, log scores, save prompt versions, and ask Ollie
investigative questions, all from the chat.
## Quick setup with the Opik CLI
The fastest way to connect the MCP server is the Opik CLI. It detects your AI
host (Claude Code, Cursor, VS Code Copilot), picks the right server for your
Opik deployment, and configures it for you.
<Tip>
Prefer not to use the CLI? You can wire up any host by hand — skip to
[Manual setup](#manual-setup).
</Tip>
<Steps>
<Step title="Install the Opik CLI">
The CLI ships with the `opik` Python package. The `opik mcp` commands require
**version 2.1.3 or later**:
```bash
pip install --upgrade "opik>=2.1.3"
```
</Step>
<Step title="Configure the MCP server">
```bash
opik mcp configure
```
This reuses your existing Opik configuration (`~/.opik.config`). If you
haven't configured Opik yet, the wizard offers to do it for you first.
`opik configure` also offers to run this for you at the end of its setup:
> Set up the Opik MCP server for an AI assistant (Claude Code, Cursor, VS Code)? (y/N)
</Step>
<Step title="Restart your AI host and verify">
Restart your AI host, then ask **"list my Opik projects"** in the chat to
confirm it works.
<Note>
If your host isn't detected, use [Manual setup](#manual-setup) below.
</Note>
</Step>
</Steps>
## Check your setup
Each AI host keeps its own copy of the MCP configuration, which isn't updated
automatically when your Opik configuration changes. To see what every detected
host points at — and whether it still matches your current Opik configuration —
run:
```bash
opik mcp status
```
It prints your active Opik configuration, then each assistant that has the Opik
MCP server configured: the config file it lives in, the server it reports to
(hosted or local), its workspace, and whether it has drifted from your Opik
configuration.
```text
Your Opik configuration
File ~/.opik.config
Environment https://www.comet.com/opik/api
Workspace my-workspace
Opik MCP server — configured for 1 AI assistant:
Claude Code
Config ~/.claude.json
Connection Hosted (HTTP + OAuth)
Reports to https://www.comet.com/opik/api/v1/mcp
Status ✓ in sync with your Opik configuration
```
A host that has drifted is flagged `✗ OUT OF SYNC` — re-run
`opik mcp configure` to fix it.
<Warning>
A host keeps its MCP connection for the lifetime of its process. After changing
your Opik configuration or re-running `opik mcp configure`, **restart your AI
host** so it reconnects with the updated settings.
</Warning>
To view just your active Opik configuration (file path, environment, workspace):
```bash
opik configure status
```
## Opik Cloud and self-hosted deployments
`opik mcp configure` works the same whether you're on Opik Cloud, self-hosted,
or a local install — it sets up the right server for your deployment
automatically.
### Opik Cloud (hosted server)
On [Opik Cloud](https://www.comet.com/opik), the CLI registers the **hosted MCP
server** over HTTP. Your AI host signs in with a browser-based OAuth flow on
first connect, so:
- **No API key is stored** in the host's config — you authenticate through OAuth
in the browser.
- **`uv` is not required** — there is no local process to run.
- Your workspace is selected during the OAuth sign-in, so a hosted server shows
no workspace in `opik mcp status`.
### Self-hosted and local (local server)
If no hosted server is available for your environment, the CLI sets up the
**local server**, which runs on demand via `uvx opik-mcp`. This requires
[`uv`](https://docs.astral.sh/uv/) — install it with `brew install uv` (macOS)
or `curl -LsSf https://astral.sh/uv/install.sh | sh`.
## Manual setup
Prefer to wire it up yourself, or your host wasn't detected? Configure any host
by hand below.
<Note>
There are two servers you can add by hand. [`opik mcp configure`](#quick-setup-with-the-opik-cli)
picks the right one for you, but you can also add either directly in your AI
host's MCP settings:
- **Hosted server** (HTTP + OAuth) — available on Opik Cloud and any deployment
that provides it. No API key is stored; your host signs in through the browser.
- **Local server** (`uvx opik-mcp`, stdio) — runs on your machine with your
credentials in the host's `env` block.
</Note>
### Hosted server (Opik Cloud)
The hosted server connects over HTTP and signs in with a browser-based OAuth
flow on first connect — no API key is stored in the host config. Point your host
at your deployment's MCP endpoint, which is your Opik API base plus `/v1/mcp`. On
Opik Cloud that is `https://www.comet.com/opik/api/v1/mcp`.
<Tabs>
<Tab title="Claude Code">
Add the server with one command:
```bash
claude mcp add --transport http opik-mcp https://www.comet.com/opik/api/v1/mcp
```
Or edit `~/.claude.json` directly:
```json
{
"mcpServers": {
"opik-mcp": {
"type": "http",
"url": "https://www.comet.com/opik/api/v1/mcp"
}
}
}
```
Restart Claude Code and complete the browser sign-in when prompted, then ask
in the chat: **"list my Opik projects"**.
</Tab>
<Tab title="Cursor">
Edit `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project):
```json
{
"mcpServers": {
"opik-mcp": {
"type": "http",
"url": "https://www.comet.com/opik/api/v1/mcp"
}
}
}
```
Reload Cursor and complete the browser sign-in when prompted.
</Tab>
<Tab title="VS Code Copilot">
Create or open `.vscode/mcp.json` in your workspace:
```json
{
"servers": {
"opik-mcp": {
"type": "http",
"url": "https://www.comet.com/opik/api/v1/mcp"
}
}
}
```
Reload the window and complete the browser sign-in when prompted.
</Tab>
</Tabs>
### Local server (uvx)
The local server runs on demand via `uvx opik-mcp` (requires
[`uv`](https://docs.astral.sh/uv/)), with your credentials passed through the
host's `env` block.
<Note>
`opik-mcp` is now a Python package. If you previously ran the npx-based
JavaScript server, use the `uvx opik-mcp` commands below in place of
`npx -y opik-mcp`.
</Note>
<Tip>
`OPIK_WORKSPACE` is **optional** — you can omit the `OPIK_WORKSPACE` line/key
entirely and the server uses the `default` workspace (correct for local/OSS
installs). The snippets below include it for completeness; set it only if you
connect to a named cloud workspace.
</Tip>
<Tabs>
<Tab title="Claude Code">
Add the server with one command:
```bash
claude mcp add --transport stdio opik-mcp \
--env OPIK_API_KEY=<your-key> \
--env OPIK_WORKSPACE=<your-workspace> \
-- uvx opik-mcp
```
Or edit `~/.claude.json` directly:
```json
{
"mcpServers": {
"opik-mcp": {
"type": "stdio",
"command": "uvx",
"args": ["opik-mcp"],
"env": {
"OPIK_API_KEY": "<your-key>",
"OPIK_WORKSPACE": "<your-workspace>"
}
}
}
}
```
Restart Claude Code, verify with `/mcp` (`opik-mcp` should appear as
connected), and then ask in the chat: **"list my Opik projects"**.
</Tab>
<Tab title="Cursor">
Edit `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project), or open
**Cmd+Shift+J → Features → Model Context Protocol**:
```json
{
"mcpServers": {
"opik-mcp": {
"type": "stdio",
"command": "uvx",
"args": ["opik-mcp"],
"env": {
"OPIK_API_KEY": "<your-key>",
"OPIK_WORKSPACE": "<your-workspace>"
}
}
}
}
```
Reload Cursor; the green dot next to `opik-mcp` in the MCP panel confirms
the connection. Ask in chat: **"list my Opik projects"**.
<Tip>
**Cursor 60s timeout.** Cursor enforces a hard tool-call timeout that does
not reset on progress notifications. Long `ask_ollie` turns will fail on
Cursor — see [Known host limits](#known-host-limits).
</Tip>
</Tab>
<Tab title="VS Code Copilot">
Create or open `.vscode/mcp.json` in your workspace (or run the
**MCP: Open User Configuration** command to add it globally):
```json
{
"servers": {
"opik-mcp": {
"type": "stdio",
"command": "uvx",
"args": ["opik-mcp"],
"env": {
"OPIK_API_KEY": "<your-key>",
"OPIK_WORKSPACE": "<your-workspace>"
}
}
}
}
```
Reload the window. The Copilot Chat **MCP** indicator shows `opik-mcp` once
the server is reachable. Ask in chat: **"list my Opik projects"**.
</Tab>
<Tab title="MCP Inspector">
For manual testing or debugging, run the inspector against `opik-mcp`:
```bash
OPIK_API_KEY=<your-key> OPIK_WORKSPACE=<your-workspace> \
npx @modelcontextprotocol/inspector uvx opik-mcp
```
The inspector opens in your browser and lets you call each tool directly.
</Tab>
</Tabs>
<Tip>
**Self-hosted Opik.** Add `COMET_URL_OVERRIDE` to the `env` block (and `OPIK_URL`
if Opik lives at a non-default path). `ask_ollie` and `run_experiment` are
available on Comet Cloud only — on self-hosted those calls fail at dispatch;
use `read` / `list` / `write` directly.
</Tip>
## Using the MCP server
### The tools at a glance
| Tool | Purpose |
|---|---|
| `read` | Universal read by id / name / `opik://` URI. |
| `list` | Universal list with optional name filter and pagination. |
| `ask_ollie` | Investigate or synthesize via the Opik in-product assistant. |
| `write` | Universal write — log traces/spans, score, comment, save prompts, manage test suites and experiments. |
| `schema` | Introspect write-operation schemas (used by the LLM to construct valid payloads). |
| `run_experiment` | Run an evaluation experiment end-to-end via Ollie. |
### Browsing your workspace
> list my Opik projects
> what was the most recent trace logged to the "demo" project?
> show me trace `<trace-id>`
### Scoring, commenting, saving prompts
> score trace `<trace-id>` 0.9 on helpfulness with reason "great recovery"
> comment "retry with temperature=0" on span `<span-id>`
> save the following text as a new version of the "rerank-system" prompt: ...
For the full set of write operations and their payload shapes, ask the host
**"show me the schema for trace.create"** (calls the `schema` tool) or see the
[README](https://github.com/comet-ml/opik-mcp#tools).
### Asking Ollie
For investigative or cross-entity questions:
> why are spans in the "demo" project slower this week than last?
> compare experiments "rerank-v2" and "rerank-v3" on factuality
`ask_ollie` returns a `thread_id` you can pass back on follow-ups to preserve
context. For more about Ollie itself, see [Ollie](/ollie).
See [Ollie & auto-approve](#ollie--auto-approve) below before running
write-style prompts in shared workspaces.
## Ollie & auto-approve
By default, writes that Ollie performs mid-stream (scores, comments, prompt
versions, test-suite items) execute without a per-action confirmation step.
Each auto-approved write is logged as a JSON audit row on the `opik_mcp.audit`
Python logger.
To require manual confirmation instead, set `OPIK_MCP_AUTO_APPROVE=disabled` in
the server's `env` block. Ollie's confirmation requests then surface as typed
errors that you can re-issue manually.
`ask_ollie` and `run_experiment` are available on Comet Cloud only — on
self-hosted those calls fail at dispatch; use `read` / `list` / `write`
directly.
## Known host limits
- **Cursor enforces a 60-second hard tool-call timeout** that does not reset on
progress notifications. Long `ask_ollie` turns will fail on Cursor. For
long-running investigations, use Claude Code or VS Code Copilot.
## Example conversation
A typical investigative loop using Claude Code:
> **You:** Why did the experiment "gpt-4o-rerank-v3" regress on factuality?
>
> **Claude:** *(calls `ask_ollie`)* Three traces failed because the reranker
> dropped the system message. The remaining 12 traces scored above 0.8…
>
> **You:** Score the bottom 3 traces 0.2 with reason "dropped system message".
>
> **Claude:** *(calls `write` with `score.create` ×3)* Done — three scores
> recorded on traces `<id-1>`, `<id-2>`, `<id-3>`.