comet-ml--opik/scripts/README.md

Scripts

General guidelines

Scripts in this folder are meant to be run from the repository base folder. Example:

./scripts/generate_openapi.sh

Scripts list

generate_openapi.sh

Use this script to generate an updated OpenAPI specification file for the documentation application and the SDKs, and also to build the SDKs autogenerated code, for any supported language by Fern.

You require to install Fern to run this script.

See:

• https://buildwithfern.com/

start_openapi_server.sh

Use this script to start a local server with an updated OpenAPI specification file, to be able to test the specification quickly.

Open the server in your browser at http://localhost:3003/

See:

• https://github.com/Redocly/redoc
• https://docs.oracle.com/en/java/javase/23/docs/specs/man/jwebserver.html

sync-codex.sh

Synchronize .agents/rules/*.mdc into Codex-friendly markdown and generate a local AGENTS.override.md for Codex sessions.

Usage:

./scripts/sync-codex.sh .agents AGENTS.md AGENTS.override.md

This script is executed by make codex.

dev-runner.sh

Development environment runner script for local Opik development. This script manages Docker infrastructure, backend, and frontend services for development workflows.

Quick Start

# Full restart with rebuild (default)
./scripts/dev-runner.sh

# Or explicitly
./scripts/dev-runner.sh --restart

Available Commands

Standard Mode (Backend and Frontend as local processes):

Command	Description
--start	Start services without rebuilding
--stop	Stop all services
--restart	Stop, rebuild, and start all services (default)
--quick-restart	Quick restart: rebuild backend only, keep infrastructure running
--verify	Check status of all services

BE-Only Mode (Backend as local process, Frontend in Docker):

Command	Description
--be-only-start	Start services without rebuilding
--be-only-stop	Stop all services
--be-only-restart	Stop, rebuild, and start services
--be-only-verify	Check status of services

EM / Platform Mode (Opik-team only — opt-in via PLATFORM_ENABLED=true):

Runs the Comet EM/Platform stack (comet-backend + comet-react, auto-detected sibling checkouts) alongside Opik behind a single-origin proxy, with Opik in comet mode. Off by default; Standard/BE-only dev is unaffected. See --help for env vars (COMET_BACKEND_PATH, COMET_REACT_PATH, EM_JAVA_HOME, EM_*_PORT).

Command	Description
PLATFORM_ENABLED=true ./scripts/dev-runner.sh --restart	Bring up Opik + EM stack; integrated UI at http://localhost:9100 (Opik at /opik). Also works with --start/--stop/--verify.
--platform-build	Build the EM stack only (comet-backend jar + comet-react deps)

Other Commands:

Command	Description
--build-be	Build backend only
--build-fe	Build frontend only
--migrate	Run database migrations
--lint-be	Lint backend code
--lint-fe	Lint frontend code
--logs	Show recent logs
--debug	Enable debug mode (combine with other flags)
--help	Show help message

---

Multi-Worktree Support

The dev-runner.sh and opik.sh scripts support running multiple Opik development environments simultaneously from different git worktrees. Each worktree automatically gets isolated ports and Docker containers.

How It Works

1. Worktree Detection: The script identifies your worktree by its directory name
2. Port Offset Calculation: A deterministic offset (0-99) is calculated from an MD5 hash of your project path
3. Port Assignment: All service ports are offset from their base values
4. Docker Isolation: Each worktree gets a unique Docker Compose project name (opik-<worktree-id>)

Port Assignments

Service	Base Port	Formula
Backend	8080	8080 + offset
Frontend	5174	5174 + offset
MySQL	3306	3306 + offset
Redis	6379	6379 + offset
ClickHouse HTTP	8123	8123 + offset
ClickHouse Native	9000	9000 + offset
Python Backend	8000	8000 + offset
Zookeeper	2181	2181 + offset
MinIO API	9001	9001 + offset
MinIO Console	9090	9090 + offset

Manual Port Override

If you need to use a specific port offset (e.g., to avoid conflicts or use standard ports):

# Use standard ports (offset 0)
OPIK_PORT_OFFSET=0 ./scripts/dev-runner.sh --restart

# Use a specific offset
OPIK_PORT_OFFSET=10 ./scripts/dev-runner.sh --restart

Running Multiple Worktrees

# Terminal 1: Main branch
cd ~/opik
./scripts/dev-runner.sh --restart
# Services running on ports based on hash of ~/opik

# Terminal 2: Feature branch worktree
cd ~/opik-worktrees/feature-xyz
./scripts/dev-runner.sh --restart
# Services running on different ports based on hash of ~/opik-worktrees/feature-xyz

Port Collision Detection

The script automatically checks for port conflicts before starting services. If a collision is detected, you'll see an error message with suggestions:

[ERROR] Port 8122 (Backend) is already in use

Port collision detected! Another process is using one or more required ports.
This might be caused by:
  - Another Opik instance running from a different worktree
  - Stale containers from a previous run
  - Other services using the same ports

To resolve:
  1. Stop other Opik instances: ./scripts/dev-runner.sh --stop
  2. Use a different port offset: export OPIK_PORT_OFFSET=<0-99>
  3. Check running processes: lsof -i :8122

File Isolation

Each worktree also gets isolated log and PID files:

File	Path
Backend PID	/tmp/opik-<worktree-id>-backend.pid
Frontend PID	/tmp/opik-<worktree-id>-frontend.pid
Backend Log	/tmp/opik-<worktree-id>-backend.log
Frontend Log	/tmp/opik-<worktree-id>-frontend.log

Docker Container Naming

Docker containers are prefixed with the worktree project name:

• Main repo (opik): opik-opik-mysql-1, opik-opik-backend-1, etc.
• Worktree (feature-xyz): opik-feature-xyz-mysql-1, opik-feature-xyz-backend-1, etc.

Environment Variables

Variable	Description
OPIK_PORT_OFFSET	Override automatic port offset (0-99)
DEBUG_MODE=true	Enable verbose debug output

SDK Configuration

When using the Opik SDK with a worktree-based development environment, configure it to use your worktree's backend port (shown when you start the environment):

# Configure SDK (use the backend port shown at startup)
export OPIK_URL_OVERRIDE='http://localhost:8080'  # or your worktree's port
export OPIK_WORKSPACE='default'

Or edit ~/.opik.config:

[opik]
url_override = http://localhost:8122
workspace = default