5a558eb09e
TypeScript SDK Compatibility V1.x E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / TypeScript SDK Compatibility V1.x E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
TypeScript SDK E2E Tests / TypeScript SDK E2E Tests Node ${{matrix.node_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
Python SDK E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK E2E Tests / Python SDK E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK E2E Tests / build-opik (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Select Python version matrix (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / Python SDK Compatibility V1.x E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Python SDK Compatibility V1.x E2E Tests / build-opik (push) Has been cancelled
TypeScript SDK E2E Tests / Select Node version matrix (push) Has been cancelled
TypeScript SDK E2E Tests / build-opik (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer E2E Tests Python ${{matrix.python_version}} (push) Has been cancelled
Opik Optimizer - E2E Tests / Opik Optimizer Integration Smoke Tests (push) Has been cancelled
🐙 Code Quality / detect (push) Has been cancelled
🐙 Code Quality / lint (${{ matrix.leg.name }}) (push) Has been cancelled
🐙 Code Quality / summary (push) Has been cancelled
TypeScript SDK Library Integration Tests / Check Secrets (push) Has been cancelled
TypeScript SDK Library Integration Tests / opik-vercel (Vercel AI SDK / eve) (push) Has been cancelled
SDK Library Integration Tests Runner / Check Secrets (push) Has been cancelled
SDK Library Integration Tests Runner / Missed OpenAI API Key Warning (push) Has been cancelled
SDK Library Integration Tests Runner / Build (push) Has been cancelled
SDK Library Integration Tests Runner / openai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_tests (push) Has been cancelled
SDK Library Integration Tests Runner / langchain_legacy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / llama_index_tests (push) Has been cancelled
SDK Library Integration Tests Runner / anthropic_tests (push) Has been cancelled
SDK Library Integration Tests Runner / mistral_tests (push) Has been cancelled
SDK Library Integration Tests Runner / groq_tests (push) Has been cancelled
SDK Library Integration Tests Runner / aisuite_tests (push) Has been cancelled
SDK Library Integration Tests Runner / haystack_tests (push) Has been cancelled
SDK Library Integration Tests Runner / dspy_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / crewai_v1_tests (push) Has been cancelled
SDK Library Integration Tests Runner / genai_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_tests (push) Has been cancelled
SDK Library Integration Tests Runner / adk_legacy_1_3_0_tests (push) Has been cancelled
SDK Library Integration Tests Runner / evaluation_metrics_tests (push) Has been cancelled
SDK Library Integration Tests Runner / bedrock_tests (push) Has been cancelled
SDK Library Integration Tests Runner / litellm_tests (push) Has been cancelled
SDK Library Integration Tests Runner / harbor_tests (push) Has been cancelled
SDK Library Integration Tests Runner / Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / render-equality (push) Has been cancelled
Opik Optimizer - Unit Tests / Opik Optimizer Unit Tests Python ${{matrix.python_version}} (push) Has been cancelled
Python BE E2E Tests / Python BE E2E (push) Has been cancelled
Python Backend Tests / run-python-backend-tests (push) Has been cancelled
Python SDK Unit Tests / Python SDK Unit Tests ${{matrix.python_version}} (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
SDK E2E Libraries Integration Tests / Check Secrets (push) Has been cancelled
SDK E2E Libraries Integration Tests / Missed OpenAI API Key Warning (push) Has been cancelled
SDK E2E Libraries Integration Tests / build-opik (push) Has been cancelled
SDK E2E Libraries Integration Tests / E2E Lib Integration Python ${{matrix.python_version}} (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-gemini) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-langchain) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-openai) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-otel) (push) Has been cancelled
TypeScript SDK Integration Build & Publish / build-and-publish (opik-vercel) (push) Has been cancelled
TypeScript SDK Build & Publish / build-and-publish (push) Has been cancelled
TypeScript SDK Unit Tests / Test on Node ${{ matrix.node-version }} (push) Has been cancelled
Backend Tests / discover-tests (push) Has been cancelled
Backend Tests / ${{ matrix.name }} (push) Has been cancelled
Build and Publish SDK / build-and-publish (push) Has been cancelled
Build Opik Docker Images / set-version (push) Has been cancelled
Build Opik Docker Images / build-backend (push) Has been cancelled
Build Opik Docker Images / build-sandbox-executor-python (push) Has been cancelled
Build Opik Docker Images / build-python-backend (push) Has been cancelled
Build Opik Docker Images / build-frontend (push) Has been cancelled
Build Opik Docker Images / create-git-tag (push) Has been cancelled
ClickHouse Migration Cluster Check / validate-clickhouse-migrations (push) Has been cancelled
Docs - Publish / run (push) Has been cancelled
E2E Tests - Post Merge (v2) / 🧪 E2E v2 Tests (${{ github.event.inputs.tier || 't1' }}) (push) Has been cancelled
E2E Tests - Post Merge (v2) / 📢 Slack Notification (push) Has been cancelled
Frontend Unit Tests / Test on Node 20 (push) Has been cancelled
Guardrails E2E Tests / Select Python version matrix (push) Has been cancelled
Guardrails E2E Tests / Guardrails E2E Tests ${{matrix.python_version}} (push) Has been cancelled
Guardrails E2E Tests / 📢 Slack Notification (push) Has been cancelled
Guardrails Backend Unit Tests / Guardrails Backend Unit Tests (push) Has been cancelled
Guardrails Backend Unit Tests / 📢 Slack Notification (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v3.21.0) (push) Has been cancelled
Lint Opik Helm Chart / lint-helm-chart (Helm v4.2.0) (push) Has been cancelled
Lint Opik Helm Chart / unittest-helm-chart (push) Has been cancelled
296 lines
8.1 KiB
Plaintext
296 lines
8.1 KiB
Plaintext
---
|
|
headline: Kubernetes deployment | Opik Documentation
|
|
og:description: Learn to configure and deploy Opik on Kubernetes using our Helm chart
|
|
for robust, production-ready environments.
|
|
og:site_name: Opik Documentation
|
|
og:title: 'Deploying with Opik: Kubernetes Made Easy'
|
|
subtitle: Describes how to run Opik on a Kubernetes cluster
|
|
title: Kubernetes deployment
|
|
canonical-url: https://www.comet.com/docs/opik/self-host/kubernetes
|
|
---
|
|
|
|
> **Important:** If you're using or looking to use Opik or Comet enterprise version please reach out to Sales@comet.com to gain access to the correct deployment documentation.
|
|
|
|
For production deployments, we recommend using our Kubernetes Helm chart. This chart is designed to be highly configurable and has been battle-tested in Comet's managed cloud offering.
|
|
|
|
# Prerequisites
|
|
|
|
In order to install Opik on a Kubernetes cluster, you will need to have the following tools installed:
|
|
|
|
- [Helm](https://helm.sh/)
|
|
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
|
|
- Optional: [kubectx](https://github.com/ahmetb/kubectx) and [kubens](https://github.com/ahmetb/kubectx) to switch between Kubernetes clusters and namespaces.
|
|
|
|
# Installation
|
|
|
|
You can install Opik using the helm chart maintained by the Opik team by running the following commands:
|
|
|
|
Add Opik Helm repo
|
|
|
|
```bash
|
|
helm repo add opik https://comet-ml.github.io/opik/
|
|
helm repo update
|
|
```
|
|
|
|
You can set VERSION to the specific Opik version or leave it as 'latest'
|
|
|
|
```bash
|
|
VERSION=latest
|
|
helm upgrade --install opik -n opik --create-namespace opik/opik \
|
|
--set component.backend.image.tag=$VERSION \
|
|
--set component.python-backend.image.tag=$VERSION \
|
|
--set component.python-backend.env.PYTHON_CODE_EXECUTOR_IMAGE_TAG="$VERSION" \
|
|
--set component.frontend.image.tag=$VERSION
|
|
```
|
|
|
|
You can port-forward any service you need to your local machine:
|
|
|
|
```bash
|
|
kubectl port-forward -n opik svc/opik-frontend 5173
|
|
```
|
|
|
|
Opik will be available at `http://localhost:5173`.
|
|
|
|
# Configuration
|
|
|
|
You can find a full list of the configuration options in the [helm chart documentation](https://comet-ml.github.io/opik/).
|
|
|
|
# Advanced deployment options
|
|
|
|
## Configure external access
|
|
|
|
### Configure ingress for opik-frontend
|
|
|
|
```yaml
|
|
component:
|
|
frontend:
|
|
ingress:
|
|
enabled: true
|
|
ingressClassName: <your ingress class>
|
|
annotations:
|
|
<your annotations>
|
|
hosts:
|
|
- host: opik.example.com
|
|
paths:
|
|
- path: /
|
|
port: 5173
|
|
pathType: Prefix
|
|
# For TLS configuration (optional)
|
|
tls:
|
|
enabled: true
|
|
hosts: # Optional - defaults to hosts from rules if not specified
|
|
- opik.example.com
|
|
secretName: <your-tls-secret> # Optional - omit if using cert-manager or similar
|
|
```
|
|
|
|
### Configure LoadBalancer service for clickhouse
|
|
|
|
```yaml
|
|
clickhouse:
|
|
service:
|
|
serviceTemplate: clickhouse-cluster-svc-lb-template
|
|
annotations: <your clickhouse LB service annotations>
|
|
```
|
|
|
|
## Configure Clickhouse backup
|
|
|
|
[Clickhouse Backup](./backup)
|
|
|
|
## Configure replication for Clickhouse
|
|
|
|
<Warning>
|
|
|
|
**Important Limitation:**
|
|
You must have Opik running _before_ you enable replication for ClickHouse.
|
|
Attempting to set up replication before Opik is running may result in errors or misconfiguration.
|
|
|
|
</Warning>
|
|
|
|
```yaml
|
|
clickhouse:
|
|
replicasCount: 2
|
|
```
|
|
|
|
## Configure additional ClickHouse users and profiles
|
|
|
|
You can create read-only ClickHouse users with custom settings profiles.
|
|
|
|
### Using inline passwords
|
|
|
|
```yaml
|
|
clickhouse:
|
|
additionalProfiles:
|
|
- name: readonly_profile
|
|
settings:
|
|
readonly: 1
|
|
max_execution_time: 60
|
|
max_memory_usage: 10000000000
|
|
max_rows_to_read: 20000000
|
|
max_concurrent_queries_for_user: 2
|
|
additionalUsers:
|
|
- username: myuser
|
|
password: my_secure_password
|
|
profile: readonly_profile
|
|
```
|
|
|
|
### Using Kubernetes secrets
|
|
|
|
When `adminUser.useSecret.enabled: true`, user passwords are read from a Kubernetes secret. By default, it uses the admin secret (`adminUser.secretname`) with the key `<username>_pass`:
|
|
|
|
```yaml
|
|
clickhouse:
|
|
adminUser:
|
|
useSecret:
|
|
enabled: true
|
|
secretname: clickhouse-admin-pass
|
|
additionalProfiles:
|
|
- name: readonly_profile
|
|
settings:
|
|
readonly: 1
|
|
max_execution_time: 60
|
|
max_memory_usage: 10000000000
|
|
max_rows_to_read: 20000000
|
|
max_concurrent_queries_for_user: 2
|
|
additionalUsers:
|
|
- username: myuser
|
|
profile: readonly_profile
|
|
# password read from secret "clickhouse-admin-pass", key "myuser_pass"
|
|
- username: anotheruser
|
|
profile: readonly_profile
|
|
secretname: my-custom-secret # override secret name
|
|
password_key: custom_key # override key name
|
|
```
|
|
|
|
## Use S3 bucket for Opik
|
|
|
|
### Using AWS key and secret keys
|
|
|
|
```yaml
|
|
component:
|
|
backend:
|
|
env:
|
|
S3_BUCKET: <your_bucket_name>
|
|
S3_REGION: <aws_region>
|
|
AWS_ACCESS_KEY_ID: <your AWS Key>
|
|
AWS_SECRET_ACCESS_KEY: <your AWS Secret>
|
|
```
|
|
|
|
## Use IAM Role
|
|
|
|
If your IAM role is configured for the k8s nodes, the only things you will need is to set for opik-backend:
|
|
|
|
```yaml
|
|
component:
|
|
backend:
|
|
env:
|
|
S3_BUCKET: <your_bucket_name>
|
|
S3_REGION: <aws_region>
|
|
```
|
|
|
|
If your role should be used by opik-backend serviceAccount, in addition you need to set:
|
|
|
|
```yaml
|
|
component:
|
|
backend:
|
|
serviceAccount:
|
|
enabled: true
|
|
annotations:
|
|
eks.amazonaws.com/role-arn: <your IAM Role arn>
|
|
```
|
|
|
|
## Use external Clickhouse installation
|
|
|
|
Supported from Opik chart version 1.4.2
|
|
|
|
Configuration snippet for using external Clickhouse:
|
|
|
|
```yaml
|
|
component:
|
|
backend:
|
|
...
|
|
waitForClickhouse:
|
|
clickhouse:
|
|
host: <YOUR CLICKHOUSE HOST>
|
|
port: 8123
|
|
protocol: http
|
|
env:
|
|
ANALYTICS_DB_MIGRATIONS_URL: "jdbc:clickhouse://<YOUR CLICKHOUSE HOST>:8123"
|
|
ANALYTICS_DB_HOST: "<YOUR CLICKHOUSE HOST>"
|
|
ANALYTICS_DB_DATABASE_NAME: "opik"
|
|
ANALYTICS_DB_MIGRATIONS_USER: "opik"
|
|
ANALYTICS_DB_USERNAME: "opik"
|
|
ANALYTICS_DB_MIGRATIONS_PASS: "xxx"
|
|
ANALYTICS_DB_PASS: "xxx"
|
|
...
|
|
clickhouse:
|
|
enabled: false
|
|
```
|
|
|
|
The passwords can be handled in the secret, and then you should configure it as following
|
|
|
|
```yaml
|
|
component:
|
|
backend:
|
|
...
|
|
envFrom:
|
|
- configMapRef:
|
|
name: opik-backend
|
|
- secretRef:
|
|
name: <your secret name>
|
|
env:
|
|
ANALYTICS_DB_MIGRATIONS_URL: "jdbc:clickhouse://<YOUR CLICKHOUSE HOST>:8123"
|
|
ANALYTICS_DB_HOST: "<YOUR CLICKHOUSE HOST>"
|
|
ANALYTICS_DB_DATABASE_NAME: "opik"
|
|
ANALYTICS_DB_MIGRATIONS_USER: "opik"
|
|
ANALYTICS_DB_USERNAME: "opik"
|
|
...
|
|
clickhouse:
|
|
enabled: false
|
|
```
|
|
|
|
# Delete your installation
|
|
|
|
Before deleting opik installation with helm, make sure to remove finalizer on the clickhouse resource:
|
|
|
|
```bash
|
|
kubectl patch -n opik chi opik-clickhouse --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'
|
|
```
|
|
|
|
Then, uninstall the opik:
|
|
|
|
```bash
|
|
helm uninstall opik -n opik
|
|
```
|
|
|
|
# Version Compatibility
|
|
|
|
It's important to ensure that your Python SDK version matches your Kubernetes deployment version to avoid compatibility issues.
|
|
|
|
## Check your current versions
|
|
|
|
### Check Opik UI version
|
|
You can check your current Opik deployment version in the UI by clicking on the user menu in the top right corner.
|
|
|
|
### Check Python SDK version
|
|
You can check your installed Python SDK version by running:
|
|
|
|
```bash
|
|
pip show opik
|
|
```
|
|
|
|
## Ensure version compatibility
|
|
|
|
Make sure both versions match. If they don't match:
|
|
|
|
1. **To update your Python SDK**: Run `pip install --upgrade opik==<VERSION>` where `<VERSION>` matches your Kubernetes deployment
|
|
2. **To update your Kubernetes deployment**: Update the VERSION variable in the helm installation command to match your Python SDK version
|
|
|
|
# Troubleshooting
|
|
|
|
If you get this error when running helm
|
|
|
|
```bash
|
|
ERROR: Exception Primary Reason: Code: 225. DB::Exception: Can't create replicated table without ZooKeeper. (NO_ZOOKEEPER) (version 24.3.5.47.altinitystable (altinity build))
|
|
```
|
|
|
|
Please make sure you use the latest Opik helm chart version that runs zookeeper by default |