5a558eb09e
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled
106 lines
4.4 KiB
Plaintext
106 lines
4.4 KiB
Plaintext
---
|
|
headline: Prompt Playground | Opik Documentation
|
|
og:description: Experiment with LLM prompts in Opik's playground to evaluate performance
|
|
and enhance your prompt engineering skills.
|
|
og:site_name: Opik Documentation
|
|
og:title: Explore Prompt Playground - Opik
|
|
title: Prompt Playground
|
|
canonical-url: https://www.comet.com/docs/opik/development/prompt-playground
|
|
---
|
|
|
|
<Note>
|
|
In Opik 2.0, prompts are project-scoped. Playground conversations are logged to the `playground` project by default.
|
|
</Note>
|
|
|
|
<Tip>
|
|
The Opik prompt playground is currently in public preview, if you have any feedback or suggestions, please [let us
|
|
know](https://github.com/comet-ml/opik/pulls).
|
|
</Tip>
|
|
|
|
When working with LLMs, there are times when you want to quickly try out different prompts and see how they perform. Opik's
|
|
prompt playground is a great way to do just that.
|
|
|
|
## Using the prompt playground
|
|
|
|
The prompt playground is a simple interface that allows you to enter prompts and see the output of the LLM. It allows you
|
|
to enter system, user and assistant messages and see the output of the LLM in real time.
|
|
|
|
You can also easily evaluate how different models impact the prompt by duplicating a prompt and changing either the
|
|
model or the model parameters.
|
|
|
|
<Frame>
|
|
<img src="/img/prompt_engineering/playground.png" />
|
|
</Frame>
|
|
|
|
_All of the conversations from the playground are logged to the `playground` project so that you can easily refer back to them later._
|
|
|
|
## Configuring the prompt playground
|
|
|
|
The playground supports the following LLM providers:
|
|
|
|
- OpenAI
|
|
- Anthropic
|
|
- OpenRouter
|
|
- Gemini
|
|
- Vertex AI
|
|
- Azure OpenAI
|
|
- Amazon Bedrock
|
|
- LM Studio (coming soon)
|
|
- vLLM / Ollama / any other OpenAI API-compliant provider
|
|
|
|
<Tip>
|
|
If you would like us to support additional LLM providers, please let us know by opening an issue on
|
|
[GitHub](https://github.com/comet-ml/opik/issues).
|
|
</Tip>
|
|
|
|
Go to [configuring AI Providers](/v1/administration/workspace-settings/ai_providers) to learn how to configure the prompt playground.
|
|
|
|
## Running experiments in the playground
|
|
|
|
You can evaluate prompts in the playground by using variables in the prompts using the `{{variable}}` syntax. You can then connect a dataset and run the prompts on each dataset item. This allows both technical and non-technical users to evaluate prompts quickly and easily.
|
|
|
|

|
|
|
|
When using datasets in the playground, you need to ensure the prompt contains variables in the mustache syntax (`{{variable}}`) that align with the columns in the dataset. For example if the dataset contains a column named `user_question` you need to ensure the prompt contains `{{user_question}}`.
|
|
|
|
Once you are ready to run the experiment, simply select a dataset next to the run button and click on the `Run` button. You will then be able to see the LLM outputs for each sample in the dataset.
|
|
|
|
### Accessing nested JSON dataset fields
|
|
|
|
If a dataset column contains JSON-formatted content, you can use dot notation to reference nested values directly when querying or filtering. Dot notation lets you specify the path to a nested field (e.g., `{{input.user.name}}`) to extract only the value you need from within a structured object. This makes it easier to work with deeply nested data without manual flattening or custom parsing.
|
|
|
|
## Using images in the playground
|
|
|
|
The playground supports multimodal prompts with images when using vision-capable models. You can add images in two ways:
|
|
|
|
### Adding images directly in messages
|
|
|
|
You can add images directly to your prompt messages through the playground UI:
|
|
|
|
<Frame>
|
|
<img src="/img/prompt_engineering/playground_add_image.png" />
|
|
</Frame>
|
|
|
|
<Note>
|
|
**Internal representation**: When you add an image through the UI, Opik internally stores it with `<<<image>>><</image>>>` wrapper tags on a new line in the prompt. This internal format is not visible in the UI but ensures proper serialization and processing of multimodal content.
|
|
</Note>
|
|
|
|
### Using images from datasets
|
|
|
|
When evaluating prompts with datasets, you can reference image columns using the standard Mustache syntax:
|
|
|
|
```
|
|
Analyze this product image: {{product_image}}
|
|
```
|
|
|
|
Where `product_image` is a column in your dataset containing image data.
|
|
|
|
**Supported image formats:**
|
|
- Image URL
|
|
- Base64 encoded image
|
|
|
|
<Tip>
|
|
When using images in prompts, ensure you select a vision-capable model.
|
|
Opik automatically detects which models support vision based on provider capabilities.
|
|
</Tip>
|