--- title: "Cluster Decisions" version: 3.8.40 lastUpdated: 2026-06-28 --- # Cluster Decisions — Optional Sidecar Profiles **Status:** proposal (awaiting @diegosouzapw review) **Date:** 2026-06-20 **Refs:** [#3932](https://github.com/diegosouzapw/OmniRoute/issues/3932), PR #4381 ## TL;DR Two opt-in compose profiles (`memory`, `bifrost`) for the existing 8-service deployment in [`docker-compose.yml`](../../docker-compose.yml). Default-up behaviour is **unchanged**: 3 × `omniroute` replicas + Caddy + Redis + CliproxyAPI. The two new profiles add Qdrant and Bifrost as optional sidecars, gated by `docker compose --profile up`. **No existing service is removed or replaced.** ## Why this is conservative OmniRoute's existing deployment shape is already lean and proven: - **`redis:7-alpine`** handles the rate-limit/cache workload at production scale. - **SQLite + sqlite-vec + FTS5** cover local memory + vector + text-search (see [`src/lib/memory/vectorStore.ts:108`](../../src/lib/memory/vectorStore.ts)). - **Caddy** is already the LB + TLS terminator ([`docker-compose.yml`](../../docker-compose.yml)). - **Bifrost** is already integrated as the Tier-1 router in [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (sidecar proxy with kill switch via `BIFROST_ENABLED` env var — set `=0` to bypass the sidecar and fall through to the TS path). The two profiles here are **scale-out options for deployments that hit the SQLite ceiling** — not migrations. Both are default-off. ## The two profiles ### `memory` — Qdrant Vector Memory Sidecar **When to flip on:** - > 1M embeddings per deployment (sqlite-vec starts to slow at scale). - Multi-replica deployment that needs shared vector state across `omniroute-1/2/3`. - You already have an external Qdrant cluster (Qdrant Cloud, on-prem). **What it adds:** | Service | Image | Ports | Notes | | -------- | ----------------------- | ----------- | ----------------------------------------------------- | | `qdrant` | `qdrant/qdrant:v1.12.4` | `6333` HTTP | HNSW index; persistent volume `omniroute_qdrant_data` | **Activation:** flip `qdrantEnabled = true` in the Settings UI **or** set `QDRANT_HOST=qdrant` env. See [`src/lib/memory/qdrant.ts:60`](../../src/lib/memory/qdrant.ts) for the precedence rules (settings table → env var → default). **Env vars:** `QDRANT_HOST`, `QDRANT_PORT`, `QDRANT_API_KEY`, `QDRANT_COLLECTION`, `QDRANT_VECTOR_SIZE`, `QDRANT_HNSW_EF_CONSTRUCT` (see `.env.example` lines 1672-1683). ### `bifrost` — Bifrost Tier-1 Router Sidecar **When to flip on:** - You run ≥3 `omniroute` replicas and want provider rotation centralised in a single Go process. - You want a single audit/logging surface for upstream-provider requests across all replicas. - You want horizontal scaling of the Tier-1 routing layer independent of the OmniRoute replicas. **What it adds:** | Service | Image | Ports | Notes | | --------- | -------------------------------- | ------ | ----------------------------------------------------------------------- | | `bifrost` | `ghcr.io/maximhq/bifrost:1.5.21` | `8080` | Go-based Tier-1 router; persistent logs volume `omniroute_bifrost_logs` | **Activation:** set `BIFROST_BASE_URL=http://bifrost:8080` in `.env.example`. The existing sidecar proxy route at [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (added in PR #4381) will pick this up automatically. **Env vars:** `BIFROST_BASE_URL`, `BIFROST_API_KEY`, `BIFROST_STREAMING_ENABLED`, `BIFROST_TIMEOUT_MS` (see `.env.example` lines 1685-1695). ## What this PR explicitly does NOT do The original issue thread floated a larger cluster rewrite. After auditing the actual workload shape, the following are **rejected** for the reasons given: | Component | Verdict | Reason | | ------------------------------------ | -------- | ------------------------------------------------------------------------------------------------------ | | **Dragonfly** | **DROP** | `redis:7-alpine` is already fine for the rate-limit workload at production scale; no ceiling to break. | | **NATS** | **DROP** | Each `omniroute` replica is a single Node.js process; no multi-process pub/sub workload exists. | | **PostgreSQL** | **DROP** | SQLite + sqlite-vec + FTS5 cover all 3 use cases; 97 migrations + Electron packaging block migration. | | **Neo4j** | **DROP** | Routing is a 5-table join; recursive CTE on SQLite is sufficient. | | **MinIO** | **DROP** | No multi-MB blob workload; images/audio are passthrough proxies. | | **pgvector / pg_ai / pg_textsearch** | **DROP** | Same SQLite-ceiling reason as PostgreSQL; pgvector ecosystem fragmented. | | **HAProxy / Envoy** | **DROP** | Caddy already does LB + TLS; both were explicitly rejected as Tier-1 routers (see `AGENTS.md`). | If a future use case proves out one of these, this doc is the place to amend. ## 4-week rollout (if approved) 1. **Wk 1** — Land this PR + verification of opt-in profiles with a 3-replica compose stack. 2. **Wk 2** — Bifrost full activation for OpenAI/Claude/Gemini/Ollama (4 of 14+ providers) using the sidecar proxy route at [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (gated by `BIFROST_ENABLED`, kill-switchable at runtime). 3. **Wk 3** — Qdrant memory profile enabled in a single test deployment; measure latency delta vs sqlite-vec. 4. **Wk 4** — Observability healthchecks (`docker compose ps` exit codes + `wget` smoke tests); 71-pillar refresh per ADR-041. ## Files changed in this PR | File | Change | | ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `docker-compose.yml` | +30 lines: `memory` profile (Qdrant), `bifrost` profile (Bifrost), persistent volumes, healthchecks. | | `.env.example` | +24 lines: `QDRANT_*` (6 vars), `BIFROST_*` (4 vars). | | `docs/reference/ENVIRONMENT.md` | +6 rows in section 25 for the `QDRANT_*` env vars. | | `src/lib/memory/qdrant.ts` | +33 lines: env-var fallback chain (settings → env → default) for `QDRANT_HOST`/`QDRANT_PORT`/`QDRANT_API_KEY`/`QDRANT_COLLECTION`/`QDRANT_VECTOR_SIZE`/`QDRANT_HNSW_EF_CONSTRUCT`/`QDRANT_EMBEDDING_MODEL`. | | `src/lib/memory/__tests__/qdrant-wiring.test.ts` | +88 lines: 9 new test cases pinning the env-var fallback precedence. | | `docs/architecture/cluster-decisions.md` (this file) | NEW — decision record for the opt-in profiles. | | `AGENTS.md` | +1 line: pointer to this doc in the reference documentation table. | **Net touched code:** 4 production files (`docker-compose.yml`, `qdrant.ts`, `.env.example`, `ENVIRONMENT.md`), 1 test file (`qdrant-wiring.test.ts`), 2 doc files (`cluster-decisions.md`, `AGENTS.md`).