wehub/diegosouzapw--omniroute
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:12 +08:00
commit d8dcd5f6d1
8604 changed files with 2479390 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/README.md
+191
View File
@@ -0,0 +1,191 @@
---
title: "OmniRoute Documentation"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute Documentation
Navigable index of the OmniRoute documentation set. Topics are grouped by intent so you can find what you need quickly.
> Looking for the project overview, install steps, or release notes? See the root [README.md](../README.md), [CHANGELOG.md](../CHANGELOG.md), and [CONTRIBUTING.md](../CONTRIBUTING.md).
---
## For Non-Tech Users
Simple guides for using OmniRoute — no technical background needed.
### getting-started/
- [QUICK-START.md](getting-started/QUICK-START.md) — install and run OmniRoute in 3 minutes.
- [AUTO-COMBO-GUIDE.md](getting-started/AUTO-COMBO-GUIDE.md) — let OmniRoute pick the best AI for you.
- [PROVIDERS-GUIDE.md](getting-started/PROVIDERS-GUIDE.md) — how to connect AI providers.
- [FREE-TIERS-GUIDE.md](getting-started/FREE-TIERS-GUIDE.md) — get free AI with no credit card.
- [TROUBLESHOOTING.md](getting-started/TROUBLESHOOTING.md) — fix common issues.
### guides/
- [SETUP_GUIDE.md](guides/SETUP_GUIDE.md) — first-time setup of OmniRoute.
- [USER_GUIDE.md](guides/USER_GUIDE.md) — daily usage of the dashboard and API.
- [FEATURES.md](guides/FEATURES.md) — dashboard feature gallery.
- [TIERS.md](guides/TIERS.md) — OmniRoute tiers explained (user guide).
- [USAGE_QUOTA_GUIDE.md](guides/USAGE_QUOTA_GUIDE.md) — usage, quota & spend tracking.
- [COST_TRACKING.md](guides/COST_TRACKING.md) — cost and spend tracking.
- [FREE_PROVIDER_RANKINGS.md](guides/FREE_PROVIDER_RANKINGS.md) — free provider rankings (Arena ELO).
- [DOCKER_GUIDE.md](guides/DOCKER_GUIDE.md) — running OmniRoute under Docker.
- [ELECTRON_GUIDE.md](guides/ELECTRON_GUIDE.md) — desktop (Electron) builds.
- [TERMUX_GUIDE.md](guides/TERMUX_GUIDE.md) — running on Android via Termux.
- [PWA_GUIDE.md](guides/PWA_GUIDE.md) — installing the dashboard as a PWA.
- [REMOTE-MODE.md](guides/REMOTE-MODE.md) — exposing OmniRoute remotely + scoped tokens.
- [CLI-INTEGRATIONS.md](guides/CLI-INTEGRATIONS.md) — master table of `setup-*` CLI integrations.
- [CLAUDE-CODE-CONFIGURATION.md](guides/CLAUDE-CODE-CONFIGURATION.md) — Claude Code CLI with OmniRoute.
- [CODEX-CLI-CONFIGURATION.md](guides/CODEX-CLI-CONFIGURATION.md) — Codex CLI with OmniRoute.
- [KIRO_SETUP.md](guides/KIRO_SETUP.md) — Kiro setup.
- [I18N.md](guides/I18N.md) — translation and locale workflow.
- [TROUBLESHOOTING.md](guides/TROUBLESHOOTING.md) — detailed troubleshooting reference.
- [UNINSTALL.md](guides/UNINSTALL.md) — clean removal steps.
---
## For Tech Users
Technical documentation for developers and contributors.
## architecture/
How the system is put together — read these to understand the runtime, code layout, and resilience model.
- [ARCHITECTURE.md](architecture/ARCHITECTURE.md) — high-level system architecture (request pipeline, layers, modules).
- [CODEBASE_DOCUMENTATION.md](architecture/CODEBASE_DOCUMENTATION.md) — engineering reference for the codebase.
- [REPOSITORY_MAP.md](architecture/REPOSITORY_MAP.md) — directory-by-directory navigation guide.
- [AUTHZ_GUIDE.md](architecture/AUTHZ_GUIDE.md) — authorization pipeline (route classifier + policy engine).
- [RESILIENCE_GUIDE.md](architecture/RESILIENCE_GUIDE.md) — provider circuit breaker, connection cooldown, and model lockout.
- [QUALITY_GATES.md](architecture/QUALITY_GATES.md) — quality-gate scripts and CI jobs inventory.
- [MONITORING_SECTIONS.md](architecture/MONITORING_SECTIONS.md) — monitoring/costs dashboard navigation.
- [cluster-decisions.md](architecture/cluster-decisions.md) — optional sidecar/cluster profile decisions.
## reference/
Lookup material — API surface, environment variables, CLI flags, provider catalog.
- [API_REFERENCE.md](reference/API_REFERENCE.md) — REST API endpoints and shapes.
- [PROVIDER_REFERENCE.md](reference/PROVIDER_REFERENCE.md) — auto-generated provider catalog (do not edit by hand).
- [PROVIDER_PLUGIN_MANIFEST.md](reference/PROVIDER_PLUGIN_MANIFEST.md) — sidecar-safe provider plugin contract for Bifrost and CLIProxyAPI migration.
- [openapi.yaml](openapi.yaml) — OpenAPI spec for the public API.
- [ENVIRONMENT.md](reference/ENVIRONMENT.md) — environment variables reference.
- [FEATURE_FLAGS.md](reference/FEATURE_FLAGS.md) — feature flags and their defaults.
- [CLI-TOOLS.md](reference/CLI-TOOLS.md) — bundled CLI commands.
- [FREE_TIERS.md](reference/FREE_TIERS.md) — free-tier LLM provider directory.
## frameworks/
Pluggable subsystems exposed to clients, agents, and operators.
- [MCP-SERVER.md](frameworks/MCP-SERVER.md) — Model Context Protocol server.
- [A2A-SERVER.md](frameworks/A2A-SERVER.md) — Agent-to-Agent (A2A) JSON-RPC server.
- [ACP.md](frameworks/ACP.md) — Agent Client Protocol.
- [AGENT_PROTOCOLS_GUIDE.md](frameworks/AGENT_PROTOCOLS_GUIDE.md) — A2A / ACP / Cloud agent overview.
- [AGENTBRIDGE.md](frameworks/AGENTBRIDGE.md) — IDE agent bridge.
- [AGENT-SKILLS.md](frameworks/AGENT-SKILLS.md) — agent skills catalog.
- [CLOUD_AGENT.md](frameworks/CLOUD_AGENT.md) — cloud agent runtime and providers.
- [SKILLS.md](frameworks/SKILLS.md) — Skills framework (sandboxed extension).
- [MEMORY.md](frameworks/MEMORY.md) — persistent memory (FTS5 + Qdrant).
- [WEBHOOKS.md](frameworks/WEBHOOKS.md) — webhook events and dispatch.
- [EVALS.md](frameworks/EVALS.md) — eval suites.
- [GAMIFICATION.md](frameworks/GAMIFICATION.md) — gamification & leaderboard system.
- [EMBEDDED-SERVICES.md](frameworks/EMBEDDED-SERVICES.md) — embedded sidecar services (9Router, CLIProxyAPI).
- [NOTION_CONTEXT.md](frameworks/NOTION_CONTEXT.md) — Notion context source.
- [OBSIDIAN_CONTEXT.md](frameworks/OBSIDIAN_CONTEXT.md) — Obsidian context source.
- [OPENCODE.md](frameworks/OPENCODE.md) — OpenCode integration.
- [OPEN_SSE_ARCHITECTURE.md](frameworks/OPEN_SSE_ARCHITECTURE.md) — open-sse streaming engine internals.
- [PLAYGROUND_STUDIO.md](frameworks/PLAYGROUND_STUDIO.md) — Playground Studio UI.
- [SEARCH_TOOLS_STUDIO.md](frameworks/SEARCH_TOOLS_STUDIO.md) — Search Tools Studio UI.
- [TRAFFIC_INSPECTOR.md](frameworks/TRAFFIC_INSPECTOR.md) — traffic inspector (MITM).
- [PLUGINS.md](frameworks/PLUGINS.md) — CLI plugin system overview.
- [PLUGIN_SDK.md](frameworks/PLUGIN_SDK.md) — plugin SDK reference.
- [PLUGIN_MARKETPLACE.md](frameworks/PLUGIN_MARKETPLACE.md) — plugin marketplace.
## routing/
Combo routing, scoring, and replay.
- [AUTO-COMBO.md](routing/AUTO-COMBO.md) — Auto-Combo (multi-factor scoring, 17 strategies).
- [QUOTA_SHARE.md](routing/QUOTA_SHARE.md) — quota sharing engine.
- [REASONING_REPLAY.md](routing/REASONING_REPLAY.md) — reasoning replay cache.
## security/
Guardrails, compliance, stealth, and the mandatory patterns for handling public credentials and error messages.
- [GUARDRAILS.md](security/GUARDRAILS.md) — PII, prompt injection, vision guardrails.
- [COMPLIANCE.md](security/COMPLIANCE.md) — audit trails and compliance.
- [STEALTH_GUIDE.md](security/STEALTH_GUIDE.md) — TLS / fingerprint stealth.
- [PUBLIC_CREDS.md](security/PUBLIC_CREDS.md) — **mandatory** pattern for embedding public upstream OAuth client_id/secret + Firebase Web keys without tripping secret scanners.
- [ERROR_SANITIZATION.md](security/ERROR_SANITIZATION.md) — **mandatory** pattern for routing every error response through `sanitizeErrorMessage` to prevent stack-trace exposure.
- [ROUTE_GUARD_TIERS.md](security/ROUTE_GUARD_TIERS.md) — route-guard classification tiers.
- [CLI_TOKEN.md](security/CLI_TOKEN.md) — CLI machine-ID token (HMAC + legacy SHA-256) auth.
- [EGRESS_POLICY.md](security/EGRESS_POLICY.md) — egress IP family (IPv4/IPv6) policy.
- [MITM-TPROXY-DECRYPT.md](security/MITM-TPROXY-DECRYPT.md) — transparent MITM decrypt.
- [SUPPLY_CHAIN.md](security/SUPPLY_CHAIN.md) — supply-chain gates (SLSA, SBOM, Trivy, osv-scanner, Scorecard).
- [SOCKET_DEV_FINDINGS.md](security/SOCKET_DEV_FINDINGS.md) — supply-chain finding attestations.
## compression/
Prompt compression engines, rules, and language packs.
- [COMPRESSION_GUIDE.md](compression/COMPRESSION_GUIDE.md) — top-level compression overview.
- [COMPRESSION_ENGINES.md](compression/COMPRESSION_ENGINES.md) — available compression engines.
- [COMPRESSION_RULES_FORMAT.md](compression/COMPRESSION_RULES_FORMAT.md) — rule file format.
- [COMPRESSION_LANGUAGE_PACKS.md](compression/COMPRESSION_LANGUAGE_PACKS.md) — language packs.
- [RTK_COMPRESSION.md](compression/RTK_COMPRESSION.md) — RTK engine deep dive.
- [CONTEXT_EDITING.md](compression/CONTEXT_EDITING.md) — delegated context editing (Anthropic).
- [EXTENDING_COMPRESSION.md](compression/EXTENDING_COMPRESSION.md) — adding a custom compression engine.
## providers/
Provider-specific integration guides.
- [CLAUDE_WEB.md](providers/CLAUDE_WEB.md) — Claude Web (cookie-auth) provider.
- [AGENTROUTER.md](providers/AGENTROUTER.md) — AgentRouter setup.
- [ZED-DOCKER.md](providers/ZED-DOCKER.md) — Zed IDE integration under Docker.
## comparison/
- [OMNIROUTE_VS_ALTERNATIVES.md](comparison/OMNIROUTE_VS_ALTERNATIVES.md) — how OmniRoute compares to alternatives.
## ops/
Release, deployment, proxies, tunnels, coverage, database, monitoring.
- [RELEASE_CHECKLIST.md](ops/RELEASE_CHECKLIST.md) — release flow checklist.
- [RELEASE_GREEN.md](ops/RELEASE_GREEN.md) — keeping the PR queue and release branch green.
- [QUALITY_GATE_PLAYBOOK.md](ops/QUALITY_GATE_PLAYBOOK.md) — quality-gate playbook.
- [BRANCH_PROTECTION_MAIN.md](ops/BRANCH_PROTECTION_MAIN.md) — `main` branch protection.
- [COVERAGE_PLAN.md](ops/COVERAGE_PLAN.md) — test coverage plan.
- [DATABASE_GUIDE.md](ops/DATABASE_GUIDE.md) — DB schema and operations.
- [SQLITE_RUNTIME.md](ops/SQLITE_RUNTIME.md) — SQLite driver resolution chain.
- [MONITORING_GUIDE.md](ops/MONITORING_GUIDE.md) — monitoring & observability.
- [FLY_IO_DEPLOYMENT_GUIDE.md](ops/FLY_IO_DEPLOYMENT_GUIDE.md) — Fly.io deployment.
- [VM_DEPLOYMENT_GUIDE.md](ops/VM_DEPLOYMENT_GUIDE.md) — generic VM deployment.
- [PROXY_GUIDE.md](ops/PROXY_GUIDE.md) — upstream proxy configuration.
- [TUNNELS_GUIDE.md](ops/TUNNELS_GUIDE.md) — Cloudflare tunnel and friends.
## diagrams/
Mermaid sources and exported SVG/PNG diagrams referenced from the docs above. See [diagrams/README.md](diagrams/README.md).
## i18n/
Translated mirrors of the documentation in 42 locales. See [i18n/README.md](i18n/README.md) for the supported language list.
## screenshots/
Static screenshots used by the dashboard and the README. Not part of the doc body.
---
## Auto-generated artifacts
- [reference/PROVIDER_REFERENCE.md](reference/PROVIDER_REFERENCE.md) is generated by `scripts/docs/gen-provider-reference.ts` from `src/shared/constants/providers.ts`. Do not edit by hand.
- The `/docs` UI is backed by Fumadocs MDX source generation from the subfolders above.
docs/architecture/ARCHITECTURE.md
+1167
View File
File diff suppressed because it is too large Load Diff
docs/architecture/AUTHZ_GUIDE.md
+219
View File
@@ -0,0 +1,219 @@
---
title: "Authorization Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Authorization Guide
> **Source of truth:** `src/server/authz/`, `src/shared/constants/publicApiRoutes.ts`, `src/lib/api/requireManagementAuth.ts`, `src/shared/utils/apiAuth.ts`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute has a route-aware authorization pipeline that gates every API request. Classification is **deterministic** and **fail-closed** — anything that cannot be classified ends up as `MANAGEMENT` and demands a session or management-grade token. This page explains the model for engineers maintaining routes or designing new endpoints.
![AuthZ pipeline (3 route classes + policy evaluation)](../diagrams/exported/authz-pipeline.svg)
> Source: [diagrams/authz-pipeline.mmd](../diagrams/authz-pipeline.mmd)
## Two Auth Modes
### 1. API Key (Bearer)
Used for the OpenAI/Anthropic/Gemini-compatible client APIs and a few management routes when the key has the `manage` scope.
```
Authorization: Bearer <api-key>
```
Validated by `isValidApiKey()` / `extractApiKey()` in `src/sse/services/auth.ts` and re-exported through `src/shared/utils/apiAuth.ts`. The validator also accepts the `OMNIROUTE_API_KEY` / `ROUTER_API_KEY` env vars as persistent passthrough keys (issue #1350).
### 2. Dashboard Session (auth_token cookie)
For dashboard pages and admin operations.
```
Cookie: auth_token=<JWT signed with JWT_SECRET>
```
Verified by `isDashboardSessionAuthenticated()` in `src/shared/utils/apiAuth.ts`. The pipeline auto-refreshes the JWT when it has fewer than 7 days left in its 30-day lifetime.
Some management routes accept **either** mode: cookie OR `Bearer <key>` when the API key has the `manage` (or `admin`) scope. This is what enables the "configurable via API calls" workflow added in v3.8.
## Route Classes
`src/server/authz/types.ts` defines three classes; any route that cannot be classified deterministically falls back to `MANAGEMENT`.
| Class | Description | Auth required |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| `PUBLIC` | Explicitly safe routes — login, logout, status, init, health, onboarding bootstrap. | None |
| `CLIENT_API` | Model-serving endpoints — `/api/v1/*`, `/api/v1beta/*`, plus aliases `/v1/*`, `/v1beta/*`, `/chat/completions`, `/responses`, `/models`, `/codex/*`. | Bearer key when the effective `REQUIRE_API_KEY` feature flag is enabled |
| `MANAGEMENT` | Dashboard pages, settings, providers, keys, admin and diagnostics endpoints. | Dashboard session OR Bearer with `manage` scope |
## Pipeline
```
Incoming request → src/proxy.ts
→ runAuthzPipeline() in src/server/authz/pipeline.ts
1. Strip trusted internal headers (x-omniroute-auth-*, x-omniroute-route-class)
2. Generate request id, classify route via classifyRoute()
3. If pathname == "/" → redirect /dashboard
4. If draining (graceful shutdown) and /api/* → 503
5. If non-GET /api/* → checkBodySize() guard
6. If OPTIONS → CORS preflight 204
7. If options.enforce == false → pass-through with route-class headers
8. Otherwise: POLICIES[routeClass].evaluate(ctx)
- allow → stamp x-omniroute-auth-{kind,id,label,scopes} → NextResponse.next()
- reject → JSON error w/ correlation_id (dashboard pages → 302 /login)
```
Trusted internal headers (defined in `src/server/authz/headers.ts`) are **stripped from incoming requests** before classification — clients cannot pre-populate `x-omniroute-auth-*` to impersonate a subject.
### Policy contracts
Each route class has a policy in `src/server/authz/policies/`:
- **`publicPolicy`** (`policies/public.ts`) — always returns `allow({ kind: "anonymous", id: "anonymous" })`.
- **`clientApiPolicy`** (`policies/clientApi.ts`) — extracts Bearer, validates via `validateApiKey()`. Falls through to anonymous only when the effective `REQUIRE_API_KEY` feature flag is disabled. The effective flag is resolved through `isRequireApiKeyEnabled()` (`DB feature flag override > process.env.REQUIRE_API_KEY > default`) so Dashboard Feature Flags and environment variables govern `/api/v1/*`, `/api/v1beta/*`, and aliases consistently; resolver failures fail closed. Allows dashboard-session requests on client API routes (including `/api/v1/models`, used by the dashboard model catalog).
- **`managementPolicy`** (`policies/management.ts`) — accepts dashboard session, internal model-sync requests (matched against `/api/providers/[name]/(sync-models|models)`), or skips entirely if `isAuthRequired()` returns false. Returns 403 (`AUTH_001`) when a Bearer token is present but invalid, 401 otherwise. Also enforces the route-guard tiers (LOCAL_ONLY / ALWAYS_PROTECTED) before any auth branch — see [Route Guard Tiers](../security/ROUTE_GUARD_TIERS.md). LOCAL_ONLY paths in `LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES` (today: `/api/mcp/`) may be accessed from non-loopback when the Bearer key carries the `manage` scope; all other LOCAL_ONLY paths remain strict-loopback regardless of scope.
A successful policy returns `AuthSubject` with `kind ∈ { client_api_key, dashboard_session, management_key, anonymous }`. Downstream handlers can read it via `assertAuth(request, "CLIENT_API")` in `src/server/authz/assertAuth.ts` instead of re-running auth logic.
## Public Routes List
`src/shared/constants/publicApiRoutes.ts` is the explicit allowlist:
```ts
PUBLIC_API_ROUTE_PREFIXES = [
"/api/auth/login",
"/api/auth/logout",
"/api/auth/status",
"/api/init",
"/api/v1/", // treated as CLIENT_API in classify, not as "no-auth public"
"/api/cloud/",
"/api/sync/bundle",
"/api/oauth/",
];
PUBLIC_READONLY_API_ROUTE_PREFIXES = ["/api/monitoring/health", "/api/settings/require-login"];
PUBLIC_READONLY_METHODS = new Set(["GET", "HEAD", "OPTIONS"]);
```
Read-only prefixes are public **only** for safe methods. Note: `classifyRoute()` excludes `/api/v1/*` and `/api/v1beta/*` from the PUBLIC fall-through — those are always `CLIENT_API` so the Bearer-key policy still applies.
## Adding a New Route
### Pattern 1 — Public client API endpoint (Bearer-auth)
Routes under `/api/v1/` and `/api/v1beta/` are classified `CLIENT_API` automatically. The middleware enforces the Bearer check; route handlers don't need to redo it but can read the subject if useful.
```typescript
// src/app/api/v1/your-route/route.ts
import { NextRequest, NextResponse } from "next/server";
import { assertAuth } from "@/server/authz/assertAuth";
export async function POST(req: NextRequest) {
const subject = assertAuth(req, "CLIENT_API");
// subject.kind === "client_api_key" | "anonymous" | "dashboard_session"
// ... handler logic
}
```
### Pattern 2 — Management endpoint (session or Bearer + manage)
Use `requireManagementAuth()` from `src/lib/api/requireManagementAuth.ts`:
```typescript
import { requireManagementAuth } from "@/lib/api/requireManagementAuth";
export async function POST(request: Request) {
const rejection = await requireManagementAuth(request);
if (rejection) return rejection;
// ... handler logic
}
```
`requireManagementAuth()` returns `null` on success or a JSON error `Response`:
- 401 `AUTH_001` "Authentication required" — no credentials at all
- 403 — invalid Bearer **or** Bearer present but key lacks the `manage` / `admin` scope
`hasManageScope(scopes)` returns true for `"manage"` or `"admin"`.
### Pattern 3 — Adding to the public allowlist
Add the prefix to `PUBLIC_API_ROUTE_PREFIXES` (or `PUBLIC_READONLY_API_ROUTE_PREFIXES` for GET-only). Update unit tests at `tests/unit/public-api-routes.test.ts` and `tests/unit/authz/classify.test.ts`.
## Scopes
API keys carry a `scopes` array (stored as JSON in `api_keys.scopes`, see `src/lib/db/apiKeys.ts`).
### Management scope
- `manage` / `admin` — grants the key access to management API endpoints when sent as Bearer.
### MCP scopes (`src/shared/constants/mcpScopes.ts`)
Each MCP tool requires specific scopes via `MCP_TOOL_SCOPES`. Full list (`MCP_SCOPE_LIST`):
```
read:health, read:combos, write:combos, read:quota, read:usage,
read:models, execute:completions, execute:search, write:budget,
write:resilience, pricing:write, read:cache, write:cache,
read:compression, write:compression, read:proxies
```
Scope enforcement in `open-sse/mcp-server/server.ts` passes each tool's scope list into
`evaluateToolScopes()` after `resolveCallerScopeContext()` resolves scopes from MCP auth info,
request metadata, or `OMNIROUTE_MCP_SCOPES`.
## Auth Required Toggle
`isAuthRequired()` in `src/shared/utils/apiAuth.ts` decides whether **any** auth is enforced for a request:
- `settings.requireLogin === false` → auth is globally disabled.
- No password configured **and** no `INITIAL_PASSWORD` env var → bootstrap mode allows the onboarding wizard and loopback requests, but exposed network requests still need credentials.
- Any DB error → fails closed (secure-by-default).
Client API key enforcement uses `isRequireApiKeyEnabled()` in `src/shared/utils/featureFlags.ts`, not a direct `process.env.REQUIRE_API_KEY` read. This matters for deployed instances: toggling `REQUIRE_API_KEY` in Dashboard → Feature Flags stores a DB override and immediately affects `/v1/*`, `/v1beta/*`, `/models`, `/responses`, `/chat/completions`, `/codex/*`, and other client-API auth checks that share this helper. If the feature flag store cannot be read, client API auth fails closed and requires a key.
## Breaking Change — v3.8.0
The `/api/v1/agents/tasks/*` and `/api/resilience/model-cooldowns` endpoints **now require management auth** (commit `588a0333`). Clients previously sending a normal API key without the `manage` scope receive `403`. Migration: either issue the key the `manage` scope in the API Keys dashboard, or use a logged-in dashboard session.
## Behaviour Change — v3.8.2
`/api/mcp/*` (the remote MCP server) is still LOCAL_ONLY by default but now accepts non-loopback requests when the `Authorization: Bearer <api-key>` header carries the `manage` scope. The carve-out is gated explicitly per-path via `LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES` in `src/server/authz/routeGuard.ts`; the sibling LOCAL_ONLY prefix `/api/cli-tools/runtime/*` is intentionally NOT bypassable because it can spawn arbitrary subprocesses. Anonymous requests to `/api/mcp/*` from non-loopback continue to return `403 LOCAL_ONLY` — the default for any new LOCAL_ONLY path remains strict-loopback. See [Route Guard Tiers](../security/ROUTE_GUARD_TIERS.md#manage-scope-carve-out).
## Testing
- Unit tests: `tests/unit/authz/` — `classify.test.ts`, `pipeline.test.ts`, `client-api-policy.test.ts`, `management-policy.test.ts`, `public-policy.test.ts`.
- Public allowlist: `tests/unit/public-api-routes.test.ts`.
- Run focused: `node --import tsx/esm --test tests/unit/authz/classify.test.ts`.
## Debugging
The pipeline always stamps responses with:
```
x-request-id: <correlation id, echoed in error bodies>
x-omniroute-route-class: PUBLIC | CLIENT_API | MANAGEMENT
```
For authenticated requests the upstream (handler-side) request headers also include:
```
x-omniroute-auth-kind: client_api_key | dashboard_session | management_key | anonymous
x-omniroute-auth-id: key_<last-4> | "dashboard" | "anonymous"
x-omniroute-auth-label: (optional)
x-omniroute-auth-scopes: comma-separated list
```
Use `assertAuth(req, expectedClass)` inside handlers — it throws `AuthzAssertionError` with code `AUTHZ_NOT_INITIALIZED` if the middleware was bypassed (helpful for catching configuration regressions in tests).
## See Also
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — auth marker per endpoint
- [COMPLIANCE.md](../security/COMPLIANCE.md) — audit log for auth events
- [MCP-SERVER.md](../frameworks/MCP-SERVER.md) — MCP scope enforcement details
- Source: `src/server/authz/`, `src/lib/api/requireManagementAuth.ts`
docs/architecture/CODEBASE_DOCUMENTATION.md
+844
View File
@@ -0,0 +1,844 @@
---
title: "OmniRoute Codebase Documentation"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute Codebase Documentation
> **Version:** v3.8.0
> **Last updated:** 2026-06-28
> **Audience:** Engineers contributing to OmniRoute or building integrations on top of it.
>
> For high-level architecture diagrams and the reasoning behind each subsystem, read
> [ARCHITECTURE.md](./ARCHITECTURE.md). For deep dives on individual subsystems
> (Auto Combo, MCP server, A2A server, Skills, Memory, Cloud Agents, Resilience,
> Compression, etc.) see their dedicated files in this `docs/` directory.
This file describes **what exists in the repository today** so that a new engineer
can navigate the tree, understand the runtime layering, and know where to add code
without inventing new modules.
---
## 1. Tech Stack
| Concern | Choice |
| ------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Web framework | **Next.js 16** (App Router, standalone output, no global middleware) |
| Language | **TypeScript 6.0+** — target `ES2022`, `module: esnext`, `moduleResolution: bundler`, `strict: false` |
| Runtime | **Node.js** `>=22.22.2 <23` or `>=24.0.0 <27` (enforced via `engines` + `SUPPORTED_NODE_RANGE`) |
| Database | **SQLite** via `better-sqlite3` (singleton, WAL journaling) |
| Desktop | **Electron 41** + `electron-builder` 26.10 (separate workspace at `electron/`) |
| Tests | **Node native test runner** (unit/integration), **Vitest** (MCP, autoCombo, cache), **Playwright** (e2e + protocols-e2e) |
| Build | Next.js standalone via `scripts/build/build-next-isolated.mjs` |
| Lint/format | ESLint flat config + Prettier (`lint-staged` via Husky pre-commit) |
| Module system | ESM everywhere (`"type": "module"`) |
| Workspaces | npm workspace — `open-sse` is the only sub-workspace |
Path aliases (`tsconfig.json`):
- `@/*` → `src/*`
- `@omniroute/open-sse` → `open-sse/index.ts`
- `@omniroute/open-sse/*` → `open-sse/*`
Default HTTP port: **`20128`** (API and dashboard share the same process). Data
directory is `DATA_DIR` env var, defaulting to `~/.omniroute/`.
---
## 2. Repository Layout
```
OmniRoute/
├── src/ Next.js application (App Router, libs, domain, server, shared)
├── open-sse/ Streaming engine workspace (@omniroute/open-sse)
├── electron/ Desktop wrapper (Electron 41 main + preload)
├── bin/ CLI entry points (omniroute, reset-password)
├── tests/ Unit, integration, e2e, protocols-e2e, translator, security, fixtures
├── scripts/ Build, sync, check, migration, and runtime helper scripts
├── docs/ Public documentation (this directory)
├── public/ Static assets, PWA manifest, service worker
├── config/ Runtime config samples
├── images/ Marketing/screenshot assets
├── _ideia/, _references/, _mono_repo/, _tasks/ Internal scratch / planning (not shipped)
├── CLAUDE.md Repo rules for Claude Code
├── AGENTS.md Deeper architecture reference for agents
├── package.json v3.8.0, workspace root
└── tsconfig.json Path aliases + core compiler options
```
---
## 3. `src/` — Next.js Application
```
src/
├── app/ App Router pages + API routes
├── lib/ Core libraries (DB, auth, OAuth, skills, memory, …)
├── domain/ Pure domain layer (policy, fallback, cost, lockout, …)
├── server/ Server-only modules (authz, cors, auth)
├── shared/ Types, constants, validation, contracts, utils (cross-boundary safe)
├── mitm/ Man-in-the-middle proxy helpers for CLI integration
├── models/ Local model metadata / aliasing
├── sse/ Legacy SSE handlers that still live under src/ (not open-sse/)
├── store/ Client-side state stores
├── middleware/ Route-level middleware utilities (not Next.js global middleware)
├── scripts/ In-tree scripts importable by app code
├── types/ Ambient and shared TS types
├── i18n/ Locale bundles
├── instrumentation.ts Next.js instrumentation hook
├── instrumentation-node.ts
├── server-init.ts Process-level bootstrap (env, DB, jobs, sync)
└── proxy.ts Top-level proxy bootstrap helper
```
### 3.1 `src/app/` — App Router
The App Router exposes both the dashboard UI and the public/management HTTP API.
There is **no global middleware** — interception is done per-route.
Top-level segments under `src/app/`:
| Path | Purpose |
| ----------------------------------------------------------------------------- | ----------------------------------------- |
| `api/` | All HTTP API routes (see breakdown below) |
| `a2a/` | A2A JSON-RPC 2.0 endpoint (`POST /a2a`) |
| `.well-known/agent.json/` | A2A Agent Card discovery document |
| `(dashboard)/` | Dashboard UI (route group, no URL prefix) |
| `auth/`, `login/`, `forgot-password/`, `callback/` | Auth flows |
| `landing/` | Marketing/landing page |
| `docs/` | Embedded API docs viewer |
| `status/`, `maintenance/`, `offline/` | Operational pages |
| `privacy/`, `terms/` | Legal pages |
| `400/`, `401/`, `403/`, `408/`, `429/`, `500/`, `502/`, `503/` | Static error pages |
| `error.tsx`, `global-error.tsx`, `not-found.tsx`, `forbidden/`, `loading.tsx` | Framework error/loading boundaries |
| `layout.tsx`, `page.tsx`, `globals.css`, `manifest.ts` | Root shell |
#### 3.1.1 `src/app/(dashboard)/dashboard/` — UI pages
`agents`, `analytics`, `api-manager`, `audit`, `auto-combo`, `batch`, `cache`,
`changelog`, `cli-tools`, `cloud-agents`, `combos`, `compression`, `context`,
`costs`, `endpoint`, `health`, `limits`, `logs`, `memory`, `onboarding`,
`playground`, `providers`, `search-tools`, `settings`, `skills`, `system`,
`translator`, `usage`, `webhooks`, plus root `page.tsx`, `HomePageClient.tsx`,
`BootstrapBanner.tsx`.
#### 3.1.2 `src/app/api/` — Top-level API groups
```
src/app/api/
├── a2a/{status, tasks}
├── acp/
├── admin/
├── analytics/
├── assess/
├── auth/
├── batches/
├── cache/
├── cli-tools/
├── cloud/{codex-responses-ws}
├── combos/
├── compliance/
├── compression/
├── context/
├── db/, db-backups/
├── evals/
├── fallback/
├── files/
├── health/
├── init/
├── internal/{concurrency}
├── keys/
├── logs/
├── mcp/{audit, sse, status, stream, tools}
├── memory/{health, [id]/, route.ts}
├── model-combo-mappings/
├── models/
├── monitoring/
├── oauth/
├── openapi/
├── policies/
├── pricing/
├── provider-metrics/, provider-models/, provider-nodes/
├── providers/
├── rate-limit/, rate-limits/
├── resilience/
├── restart/, shutdown/
├── search/
├── sessions/
├── settings/
├── skills/{executions, [id], install, marketplace, route.ts, skillssh}
├── storage/
├── sync/, synced-available-models/
├── system/
├── tags/
├── telemetry/
├── token-health/
├── translator/
├── tunnels/
├── services/ Embedded service management (9router, cliproxy) — LOCAL_ONLY
├── upstream-proxy/
├── usage/
├── v1/ OpenAI-compatible public API
├── v1beta/ Gemini-style compat
├── version-manager/
└── webhooks/
```
#### 3.1.2a `src/app/api/services/` — Embedded Services management
Routes for installing, starting, stopping, and monitoring 9Router and CLIProxyAPI.
All paths are classified **LOCAL_ONLY** (loopback only, hard rule #17) because they
can invoke `npm install` and spawn child processes.
```
src/app/api/services/
├── 9router/
│ ├── _lib.ts getOrInitSupervisor() helper
│ ├── install/route.ts POST — npm install via execFile
│ ├── start/route.ts POST — supervisor.start()
│ ├── stop/route.ts POST — supervisor.stop()
│ ├── restart/route.ts POST — supervisor.restart()
│ ├── update/route.ts POST — npm install newer version
│ ├── rotate-key/route.ts POST — generate new API key + restart
│ ├── status/route.ts GET — live + DB status + version metadata
│ └── auto-start/route.ts POST — toggle auto_start flag
├── cliproxy/
│ ├── _lib.ts getOrInitSupervisor() helper
│ ├── install/route.ts POST — npm install
│ ├── start/route.ts POST — supervisor.start()
│ ├── stop/route.ts POST — supervisor.stop()
│ ├── restart/route.ts POST — supervisor.restart()
│ ├── update/route.ts POST — npm install newer version
│ ├── status/route.ts GET — live + DB status + version metadata
│ └── auto-start/route.ts POST — toggle auto_start flag
└── [name]/
└── logs/route.ts GET — SSE log tail (shared by all services)
```
Corresponding dashboard UI:
`src/app/(dashboard)/dashboard/providers/services/` — two-tab page (CLIProxyAPI + 9Router).
Reverse proxy for 9Router embedded UI:
`src/app/(dashboard)/dashboard/providers/services/[name]/embed/[...path]/route.ts`
Deep-dive: `docs/frameworks/EMBEDDED-SERVICES.md`
#### 3.1.3 `src/app/api/v1/` — OpenAI-compatible public API
```
v1/
├── accounts/[id]/ account lookup
├── agents/tasks/[id]/, agents/tasks/ A2A-flavored task endpoints
├── api/ internal API helpers exposed under v1/api
├── audio/{speech, transcriptions}/ TTS + STT
├── batches/[id]/{cancel}, batches/ OpenAI Batches API
├── chat/completions/ Chat Completions (the main endpoint)
├── chatgpt-web/ ChatGPT-Web compat
├── completions/ Legacy text completions
├── embeddings/ Embeddings
├── files/[id]/, files/ Files API
├── _helpers/ Shared route helpers (no public URL)
├── images/{edits, generations}/ Image gen + edit
├── issues/ Triage helper endpoints
├── management/{proxies}/ Management-scoped routes inside v1
├── messages/{count_tokens}/ Anthropic-style messages compat
├── models/ Model listing (`route.ts`, `catalog.ts`)
├── moderations/ Moderation
├── music/ Music gen
├── providers/[provider]/ Per-provider operations
├── quotas/{check} Quota probes
├── registered-keys/ Registered key admin
├── rerank/ Reranking
├── responses/[...path]/ OpenAI Responses API (catch-all)
├── search/ Web search
├── videos/ Video gen
├── ws/ WebSocket bridge
└── route.ts Index handler
```
Every route file follows the same pattern:
```
Route → CORS preflight → Zod body validation → optional auth
→ API key policy enforcement → handler delegation (open-sse)
```
`v1beta/` is the Gemini-style compat surface (a thin wrapper that translates into
the same `open-sse/handlers/` pipeline).
### 3.2 `src/lib/` — Core libraries
Always import data, sync, OAuth, skill, memory, etc. through these modules. The
table groups the actual directories and notable top-level files.
| Module | Purpose |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `a2a/` | A2A protocol server: `taskManager.ts`, `streaming.ts`, `taskExecution.ts`, `routingLogger.ts`, `skills/` (6 skills: cost analysis, health report, provider discovery, quota management, smart routing, list-capabilities) |
| `acp/` | Agent-Control-Protocol: `index.ts`, `manager.ts`, `registry.ts` |
| `api/` | Internal API helpers: `requireManagementAuth.ts`, `requireCliToolsAuth.ts`, `errorResponse.ts` |
| `auth/` | `managementPassword.ts` (password reset / hashing) |
| `batches/` | OpenAI Batches API service (`service.ts`) |
| `catalog/` | OpenRouter catalog sync (`openrouterCatalog.ts`) |
| `cloudAgent/` | Cloud agent registry: `api.ts`, `baseAgent.ts`, `db.ts`, `index.ts`, `registry.ts`, `types.ts`, `agents/{codex, devin, jules}.ts` |
| `combos/` | Combo resolution helpers |
| `compliance/` | Audit + provider audit: `index.ts`, `providerAudit.ts` |
| `config/` | Runtime config glue |
| `db/` | SQLite domain modules (see §3.2.1) |
| `display/` | UI/display helpers used by API responses |
| `embeddings/` | Embedding service registry |
| `env/` | Env loading + introspection |
| `evals/` | Eval runtime |
| `guardrails/` | `piiMasker.ts`, `promptInjection.ts`, `visionBridge.ts`, `visionBridgeHelpers.ts`, `registry.ts`, `base.ts` |
| `jobs/` | Background jobs (`autoUpdate.ts`, …) |
| `memory/` | Persistent memory: `store.ts`, `cache.ts`, `retrieval.ts`, `summarization.ts`, `extraction.ts`, `injection.ts`, `qdrant.ts`, `settings.ts`, `verify.ts`, `schemas.ts`, `types.ts` |
| `monitoring/` | `observability.ts` |
| `oauth/` | OAuth providers (14): `antigravity`, `claude`, `cline`, `codex`, `cursor`, `gemini`, `github`, `gitlab-duo`, `kilocode`, `kimi-coding`, `kiro`, `qoder`, `qwen`, `windsurf` plus `services/`, `utils/{pkce, server, banner, codexAuthFile, ui}`, `constants/oauth.ts` |
| `plugins/` | Plugin loader (`index.ts`) |
| `promptCache/` | `prefixAnalyzer.ts`, `index.ts` |
| `providerModels/` | Managed model lifecycle: `modelDiscovery.ts`, `managedModelImport.ts`, `managedAvailableModels.ts`, `cursorAgent.ts` |
| `providers/` | Provider helpers: `catalog.ts`, `validation.ts`, `imageValidation.ts`, `claudeExtraUsage.ts`, `codexConnectionDefaults.ts`, `codexFastTier.ts`, `webCookieAuth.ts`, `managedAvailableModels.ts`, `requestDefaults.ts` |
| `resilience/` | `settings.ts` — settings for circuit breaker, cooldown, lockout |
| `runtime/` | Runtime feature detection |
| `search/` | `executeWebSearch.ts` |
| `services/` | Embedded services framework: `ServiceSupervisor.ts` (generic child-process supervisor with operation lock, ring buffer, health checker), `bootstrap.ts` (process-level registration and auto-start), `registry.ts` (tool → supervisor map), `apiKey.ts` (AES-256-GCM key store), `modelSync.ts` (periodic model sync), `ringBuffer.ts` (5 MB circular log buffer), `healthCheck.ts` (HTTP health probe), `types.ts`, `embedWsProxy.ts` (WebSocket proxy), `installers/{ninerouter,cliproxy}.ts`. See `docs/frameworks/EMBEDDED-SERVICES.md` |
| `agentSkills/` | Agent Skills catalog + generator: `catalog.ts` (getCatalog/getSkillById/filterCatalog/computeCoverage), `generator.ts` (generateAgentSkills → writes `skills/{id}/SKILL.md`), `openapiParser.ts` (extracts REST endpoints from OpenAPI spec), `cliRegistryParser.ts` (extracts CLI subcommands from bin/cli-registry), `schemas.ts` (Zod: AgentSkillSchema, SkillCoverageSchema, ListQuerySchema, GenerateBodySchema), `types.ts` (AgentSkill, SkillCoverage, SkillMarkdown, GeneratorReport). Consumed by REST routes (`/api/agent-skills/*`), MCP tools (`omniroute_agent_skills_*`), and A2A skill `list-capabilities`. See [AGENT-SKILLS.md](../frameworks/AGENT-SKILLS.md). |
| `skills/` | Skill framework: `registry.ts`, `executor.ts`, `interception.ts`, `injection.ts`, `sandbox.ts`, `custom.ts`, `hybrid.ts`, `builtins.ts`, `a2a.ts`, `providerSettings.ts`, `schemas.ts`, `skillssh.ts`, `types.ts`, plus `builtin/browser.ts` |
| `spend/` | `batchWriter.ts` (write-behind buffer) |
| `sync/` | `bundle.ts`, `tokens.ts` (Cloud Sync) |
| `system/` | System-level helpers |
| `translator/` | Top-level translator glue (delegates into `open-sse/translator/`) |
| `usage/` | Usage accounting: `costCalculator.ts`, `tokenAccounting.ts`, `usageHistory.ts`, `aggregateHistory.ts`, `usageStats.ts`, `callLogs.ts`, `callLogArtifacts.ts`, `fetcher.ts`, `providerLimits.ts`, `migrations.ts` |
| `versionManager/` | Auto-update + version manifest |
| `ws/` | WebSocket bridge |
| `zed-oauth/` | Zed editor OAuth flow |
Top-level files in `src/lib/`:
- `localDb.ts` — re-export layer only. **Never** add logic here.
- `proxyHealth.ts`, `proxyLogger.ts`, `tokenHealthCheck.ts`, `localHealthCheck.ts`
- `oneproxyRotator.ts`, `oneproxySync.ts`
- `apiBridgeServer.ts`, `cacheLayer.ts`, `semanticCache.ts`, `settingsCache.ts`
- `cloudSync.ts`, `initCloudSync.ts`
- `cloudflaredTunnel.ts`, `ngrokTunnel.ts`, `tailscaleTunnel.ts`
- `consoleInterceptor.ts`, `container.ts`, `gracefulShutdown.ts`, `idempotencyLayer.ts`
- `ipUtils.ts`, `logEnv.ts`, `logPayloads.ts`, `logRotation.ts`
- `modelAliasSeed.ts`, `modelCapabilities.ts`, `modelMetadataRegistry.ts`, `modelsDevSync.ts`
- `piiSanitizer.ts`, `pricingSync.ts`
- `apiKeyExposure.ts`, `cacheControlSettings.ts`, `dataPaths.ts`, `toolPolicy.ts`
- `translatorEvents.ts`, `usageDb.ts`, `usageAnalytics.ts`, `webhookDispatcher.ts`
#### 3.2.1 `src/lib/db/`
Singleton SQLite database (`getDbInstance()` in `core.ts`, WAL journaling).
**Never write raw SQL in routes or handlers** — go through these modules.
![Database schema overview (selected core tables)](../diagrams/exported/db-schema-overview.svg)
> Source: [diagrams/db-schema-overview.mmd](../diagrams/db-schema-overview.mmd)
Domain modules (each owns one or more tables): `apiKeys.ts`, `backup.ts`,
`batches.ts`, `cleanup.ts`, `cliToolState.ts`, `combos.ts`,
`commandCodeAuth.ts`, `compression.ts`, `compressionAnalytics.ts`,
`compressionCacheStats.ts`, `compressionCombos.ts`, `compressionScheduler.ts`,
`contextHandoffs.ts`, `core.ts`, `creditBalance.ts`, `databaseSettings.ts`,
`detailedLogs.ts`, `domainState.ts`, `encryption.ts`, `evals.ts`, `files.ts`,
`healthCheck.ts`, `jsonMigration.ts`, `migrationRunner.ts`,
`modelComboMappings.ts`, `models.ts`, `oneproxy.ts`, `prompts.ts`,
`providers.ts`, `providerLimits.ts`, `proxies.ts`, `quotaSnapshots.ts`,
`readCache.ts`, `reasoningCache.ts`, `registeredKeys.ts`, `secrets.ts`,
`sessionAccountAffinity.ts`, `settings.ts`, `stateReset.ts`, `stats.ts`,
`syncTokens.ts`, `tierConfig.ts`, `upstreamProxy.ts`, `versionManager.ts`,
`webhooks.ts`.
`migrations/` holds 55 versioned `.sql` files (idempotent, transactional) and is
executed by `migrationRunner.ts` at boot.
Tables created across the migrations (52 total):
`a`, `account_key_limits`, `api_keys`, `batches`, `call_logs`,
`combo_adaptation_state`, `combos`, `command_code_auth_sessions`,
`compression_analytics`, `compression_cache_stats`,
`compression_combo_assignments`, `compression_combos`, `context_handoffs`,
`daily_usage_summary`, `db_meta`, `domain_budgets`, `domain_circuit_breakers`,
`domain_cost_history`, `domain_fallback_chains`, `domain_lockout_state`,
`eval_cases`, `eval_runs`, `eval_suites`, `files`, `hourly_usage_summary`,
`key_value`, `mcp_tool_audit`, `memories`, `model_combo_mappings`,
`provider_connections`, `provider_key_limits`, `provider_nodes`,
`proxy_assignments`, `proxy_logs`, `proxy_registry`, `quota_snapshots`,
`reasoning_cache`, `registered_keys`, `request_detail_logs`,
`routing_decisions`, `semantic_cache`, `session_account_affinity`,
`skill_executions`, `skills`, `sync_tokens`, `tier_assignments`,
`tier_config`, `upstream_proxy_config`, `usage_history`, `version_manager`,
`webhooks` (plus FTS5 virtual tables for memory search).
### 3.3 `src/domain/` — Domain layer
Pure business logic, no I/O. Imported by routes and handlers.
| File | Purpose |
| ------------------------------------------ | ------------------------------------------------- |
| `policyEngine.ts` | Top-level policy resolver |
| `fallbackPolicy.ts` | Fallback decision tree |
| `costRules.ts` | Cost calculation rules |
| `lockoutPolicy.ts` | Model lockout decisions |
| `tagRouter.ts` | Tag-based routing |
| `comboResolver.ts` | Combo resolution from request → target list |
| `connectionModelRules.ts` | Per-connection model filters |
| `modelAvailability.ts` | Model availability check |
| `degradation.ts` | Degraded-mode transitions |
| `providerExpiration.ts` | Expired account/key detection |
| `quotaCache.ts` | Cached quota decisions |
| `responses.ts`, `omnirouteResponseMeta.ts` | Response shape helpers |
| `configAudit.ts` | Config change audit |
| `assessment/` | Model assessment (per RFC, partially implemented) |
| `types.ts` | Shared domain types |
### 3.4 `src/server/` — Server-only
Cannot be imported from client components.
```
server/
├── auth/loginGuard.ts
├── authz/
│ ├── classify.ts Classifies routes as public vs management
│ ├── assertAuth.ts Assertion helper
│ ├── context.ts Per-request authz context
│ ├── headers.ts
│ ├── pipeline.ts Authz pipeline
│ ├── policies/ Concrete policies
│ └── types.ts
└── cors/origins.ts CORS origin allowlist
```
### 3.5 `src/shared/` — Safe-to-share
Split into focused subdirectories:
- `constants/` — `providers.ts` (Zod-validated provider catalog), `models.ts`,
`modelSpecs.ts`, `modelCompat.ts`, `pricing.ts`, `cliTools.ts`,
`cliCompatProviders.ts`, `routingStrategies.ts`, `comboConfigMode.ts`,
`headers.ts`, `upstreamHeaders.ts` (denylist), `mcpScopes.ts`,
`errorCodes.ts`, `publicApiRoutes.ts`, `batch.ts`, `batchEndpoints.ts`,
`bodySize.ts`, `colors.ts`, `appConfig.ts`, `config.ts`,
`sidebarVisibility.ts`, `visionBridgeDefaults.ts`.
- `validation/` — `schemas.ts` (~80 Zod schemas), `compressionConfigSchemas.ts`,
`oneproxySchemas.ts`, `providerSchema.ts`, `settingsSchemas.ts`, `helpers.ts`.
- `contracts/` — public API contracts shipped to npm.
- `types/` — shared TS types.
- `utils/` — `circuitBreaker.ts`, `apiAuth.ts`, `apiKey.ts`, `apiKeyPolicy.ts`,
`apiResponse.ts`, `api.ts`, `classify429.ts`, `cliCompat.ts`, `clipboard.ts`,
`cloud.ts`, `cn.ts`, `cors.ts`, `costEstimator.ts`, `featureFlags.ts`,
`fetchTimeout.ts`, `formatting.ts`, `inputSanitizer.ts`, `logger.ts`,
`machine.ts`, `machineId.ts`, `maskEmail.ts`, `modelCatalogSearch.ts`,
`nodeRuntimeSupport.ts`, `parseApiKeys.ts`, `providerHints.ts`,
`providerModelAliases.ts`, `rateLimiter.ts`, `releaseNotes.ts`,
`a11yAudit.ts`, plus dashboard hooks/components under `services/`, `network/`,
`middleware/`, `schemas/`, `hooks/`, `components/`.
---
## 4. `open-sse/` — Streaming engine workspace
Separate npm workspace published as `@omniroute/open-sse`. Owns request
processing, executors, translators, services, transformer, and the MCP server.
```
open-sse/
├── index.ts Public exports
├── package.json Workspace manifest
├── tsconfig.json
├── types.d.ts
├── config/ Provider registries, header profiles, identity, …
├── handlers/ Request handlers (chat, embeddings, audio, image, …)
├── executors/ 75 provider-specific HTTP executors
├── translator/ Format conversion (OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro)
├── transformer/ Responses API ↔ Chat Completions stream transformer
├── services/ 80+ service modules (combos, fallback, quotas, identity, …)
├── utils/ Streaming helpers, TLS client, AWS SigV4, proxy fetch, …
└── mcp-server/ MCP server (3 transports, 30 scopes, 94 tools)
```
### 4.1 `open-sse/handlers/`
| Handler | Purpose |
| ----------------------- | ------------------------------------------------------------------------ |
| `chatCore.ts` | Main chat pipeline (cache, rate limit, combo routing, executor dispatch) |
| `responsesHandler.ts` | OpenAI Responses API entry point |
| `embeddings.ts` | Embeddings |
| `imageGeneration.ts` | Image generation |
| `audioSpeech.ts` | Text-to-speech |
| `audioTranscription.ts` | Speech-to-text |
| `videoGeneration.ts` | Video generation |
| `musicGeneration.ts` | Music generation |
| `rerank.ts` | Reranking |
| `moderations.ts` | Moderation |
| `search.ts` | Web search |
| `sseParser.ts` | SSE event parser |
| `usageExtractor.ts` | Pull token counts out of upstream streams |
| `responseSanitizer.ts` | Strip provider-specific noise |
| `responseTranslator.ts` | Glue between provider response and translator layer |
### 4.2 `open-sse/executors/`
75 provider executors, each extending `BaseExecutor` (`base.ts`):
`antigravity`, `azure-openai`, `blackbox-web`, `chatgpt-web`, `cliproxyapi`,
`cloudflare-ai`, `codex`, `commandCode`, `cursor`, `default`, `devin-cli`,
`muse-spark-web`, `nlpcloud`, `opencode`, `perplexity-web`, `petals`,
`pollinations`, `puter`, `qoder`, `vertex`, `windsurf`, plus `claudeIdentity.ts`
(shared identity helper) and `index.ts` (registry).
> Note: providers not listed here are served by `default.ts` using the generic
> OpenAI-compatible executor. The full provider catalog (237 entries) lives in
> `src/shared/constants/providers.ts`.
### 4.3 `open-sse/translator/`
Hub-and-spoke translation (OpenAI is the hub).
- **9 request translators** (`translator/request/`):
`antigravity-to-openai`, `claude-to-gemini`, `claude-to-openai`,
`gemini-to-openai`, `openai-responses`, `openai-to-claude`,
`openai-to-cursor`, `openai-to-gemini`, `openai-to-kiro`.
- **9 response translators** (`translator/response/`):
`claude-to-openai`, `cursor-to-openai`, `gemini-to-claude`, `gemini-to-openai`,
`kiro-to-openai`, `openai-responses`, `openai-to-antigravity`,
`openai-to-claude`.
- **9 helpers** (`translator/helpers/`):
`claudeHelper`, `geminiHelper`, `geminiToolsSanitizer`, `maxTokensHelper`,
`openaiHelper`, `responsesApiHelper`, `schemaCoercion`, `toolCallHelper`, plus
helper tests.
- **Image helpers** (`translator/image/sizeMapper.ts`).
- Top-level: `bootstrap.ts`, `formats.ts`, `registry.ts`, `index.ts`.
### 4.4 `open-sse/transformer/`
- `responsesTransformer.ts` — `TransformStream`-based Responses API ↔ Chat
Completions converter (used by the `responses/` route catch-all).
### 4.5 `open-sse/services/`
Highlights (full list under `open-sse/services/`):
| Concern | Files |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Combo routing | `combo.ts` (17 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` |
| Auto Combo engine | `autoCombo/` — `engine.ts`, `scoring.ts`, `taskFitness.ts`, `virtualFactory.ts`, `modePacks.ts`, `autoPrefix.ts`, `persistence.ts`, `providerDiversity.ts`, `providerRegistryAccessor.ts`, `routerStrategy.ts`, `selfHealing.ts`, `index.ts` |
| Resilience | `accountFallback.ts` (cooldown + lockout), `errorClassifier.ts`, `emergencyFallback.ts`, `rateLimitManager.ts`, `rateLimitSemaphore.ts`, `accountSemaphore.ts`, `accountSelector.ts` |
| Quotas | `quotaMonitor.ts`, `quotaPreflight.ts`, `bailianQuotaFetcher.ts`, `codexQuotaFetcher.ts`, `deepseekQuotaFetcher.ts`, `crofUsageFetcher.ts`, `antigravityCredits.ts` |
| Caching | `reasoningCache.ts`, `searchCache.ts`, `signatureCache.ts`, `requestDedup.ts` |
| Routing intelligence | `intentClassifier.ts`, `taskAwareRouter.ts`, `backgroundTaskDetector.ts`, `volumeDetector.ts`, `wildcardRouter.ts`, `workflowFSM.ts`, `specificityDetector.ts`, `specificityRules.ts`, `specificityTypes.ts` |
| Model handling | `modelCapabilities.ts`, `modelDeprecation.ts`, `modelFamilyFallback.ts`, `modelStrip.ts`, `model.ts`, `provider.ts`, `providerRequestDefaults.ts`, `providerCostData.ts`, `payloadRules.ts` |
| Compression | `compression/` — full compression engine wiring |
| Token + session | `tokenRefresh.ts`, `sessionManager.ts`, `apiKeyRotator.ts`, `contextManager.ts`, `contextHandoff.ts`, `systemPrompt.ts`, `roleNormalizer.ts`, `responsesInputSanitizer.ts`, `toolSchemaSanitizer.ts`, `toolLimitDetector.ts`, `thinkingBudget.ts` |
| Tier / manifest | `tierResolver.ts`, `tierConfig.ts`, `tierDefaults.json`, `tierTypes.ts`, `manifestAdapter.ts` |
| IP / network | `ipFilter.ts`, `webSearchFallback.ts` |
| Batches | `batchProcessor.ts` |
| Usage | `usage.ts` |
### 4.6 `open-sse/mcp-server/`
- **31 registered tools** wired in `server.ts` (12 scoped under `schemas/tools.ts`,
5 compression tools, 3 memory tools, 4 skills tools, plus advanced tools added
through `advancedTools.ts`).
- **3 transports**: stdio, HTTP Streamable, SSE.
- **13 scopes** declared in `src/shared/constants/mcpScopes.ts`.
- Audit table: `mcp_tool_audit` (populated by `audit.ts`).
- Files: `server.ts`, `index.ts`, `httpTransport.ts`, `audit.ts`, `scopeEnforcement.ts`,
`runtimeHeartbeat.ts`, `descriptionCompressor.ts`, `schemas/{tools, a2a, audit, index}.ts`,
`tools/{advancedTools, compressionTools, memoryTools, skillTools}.ts`,
plus tests under `__tests__/`.
- See [MCP-SERVER.md](../frameworks/MCP-SERVER.md) for the full tool catalog.
### 4.7 `open-sse/config/`
Provider registries (`providerRegistry.ts`, `providerModels.ts`,
`providerHeaderProfiles.ts`), per-format model registries (`audioRegistry.ts`,
`embeddingRegistry.ts`, `imageRegistry.ts`, `moderationRegistry.ts`,
`musicRegistry.ts`, `rerankRegistry.ts`, `searchRegistry.ts`, `videoRegistry.ts`),
identity helpers (`codexIdentity.ts`, `codexInstructions.ts`,
`anthropicHeaders.ts`, `antigravityUpstream.ts`, `antigravityModelAliases.ts`,
`cliFingerprints.ts`, `toolCloaking.ts`, `defaultThinkingSignature.ts`),
credential helpers (`credentialLoader.ts`, `codexClient.ts`), and cloud
adapters (`azureAi.ts`, `bedrock.ts`, `datarobot.ts`, `glmProvider.ts`,
`maritalk.ts`, `oci.ts`, `petals.ts`, `runway.ts`, `sap.ts`, `watsonx.ts`,
`ollamaModels.ts`, `errorConfig.ts`, `constants.ts`, `registryUtils.ts`).
### 4.8 `open-sse/utils/`
Streaming primitives and provider helpers: `stream.ts`, `streamHandler.ts`,
`streamHelpers.ts`, `streamPayloadCollector.ts`, `streamReadiness.ts`,
`sseHeartbeat.ts`, `proxyFetch.ts`, `proxyDispatcher.ts`, `tlsClient.ts`,
`networkProxy.ts`, `awsSigV4.ts`, `cacheControlPolicy.ts`,
`cursorChecksum.ts`, `cursorAgentProtobuf.ts`, `cursorVersionDetector.ts`,
`comfyuiClient.ts`, `kieTask.ts`, `bypassHandler.ts`, `aiSdkCompat.ts`,
`thinkTagParser.ts`, `urlSanitize.ts`, `usageTracking.ts`, `requestLogger.ts`,
`progressTracker.ts`, `cors.ts`, `error.ts`, `logger.ts`, `sleep.ts`,
`ollamaTransform.ts`.
---
## 5. `electron/` — Desktop wrapper
```
electron/
├── main.js Electron main process
├── preload.js Preload bridge (contextIsolation enabled)
├── types.d.ts
├── package.json electron-builder config, version 3.8.0
├── README.md
├── assets/ Build resources (icons, entitlements, …)
├── node_modules/ Dedicated node_modules (better-sqlite3, electron-updater)
└── dist-electron/ Build output (not committed)
```
Five npm scripts at the workspace root: `electron:dev`, `electron:build`,
`electron:build:{win,mac,linux}`, `electron:smoke:packaged`. Auto-update is via
`electron-updater` pointing at the GitHub release feed.
---
## 6. `bin/` — CLI
```
bin/
├── omniroute.mjs Main CLI entry (Node ESM)
├── reset-password.mjs Reset the management password from CLI
├── mcp-server.mjs MCP server launcher (stdio)
├── nodeRuntimeSupport.mjs Node version guard
└── cli/
├── program.mjs Commander program builder
├── runtime.mjs withRuntime helper (server-first/db-fallback)
├── output.mjs Output formatters (json/jsonl/table/csv)
├── i18n.mjs t() helper with locales
├── api.mjs API fetch helper
├── data-dir.mjs
├── encryption.mjs
├── sqlite.mjs
└── commands/
├── registry.mjs Command registration
├── setup.mjs
├── doctor.mjs
├── providers.mjs
└── ... (one file per command/group)
```
Two binaries are exposed in `package.json` → `bin`:
- `omniroute` → `bin/omniroute.mjs`
- `omniroute-reset-password` → `bin/reset-password.mjs`
---
## 7. `tests/`
| Directory | Type |
| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| `tests/unit/` | Unit tests via Node native test runner (1821 files, plus `api/`, `auth/`, `authz/` subdirs) |
| `tests/integration/` | Cross-module + DB-state tests |
| `tests/e2e/` | Playwright UI tests |
| `tests/protocols-e2e/` | MCP/A2A protocol e2e |
| `tests/translator/` | Translator-specific tests |
| `tests/security/` | Security regressions |
| `tests/load/` | Load / stress tests |
| `tests/golden-set/` | Reference outputs for translator regressions |
| `tests/helpers/`, `tests/fixtures/`, `tests/manual/`, `tests/scratch_test.mjs` | Support |
Common commands:
| Command | What it runs |
| -------------------------------------------------------- | ---------------------------------------------------------------- |
| `npm run test:unit` | All `tests/unit/*.test.ts` via Node test runner (concurrency 10) |
| `npm run test:vitest` | Vitest suite (MCP, autoCombo, cache) |
| `npm run test:e2e` | Playwright UI suite |
| `npm run test:protocols:e2e` | MCP + A2A protocol e2e |
| `npm run test:coverage` | Coverage gate (≥60% lines/statements/functions/branches) |
| `node --import tsx/esm --test tests/unit/<file>.test.ts` | Single file run |
---
## 8. `scripts/`
Organized into 6 subfolders by purpose.
- **`scripts/build/`** — `build-next-isolated.mjs`, `prepublish.ts`,
`prepare-electron-standalone.mjs`, `pack-artifact-policy.ts`,
`validate-pack-artifact.ts`, `postinstall.mjs`, `postinstallSupport.mjs`,
`uninstall.mjs`, `bootstrap-env.mjs`, `runtime-env.mjs`,
`native-binary-compat.mjs`.
- **`scripts/dev/`** — `run-next.mjs`, `run-next-playwright.mjs`,
`run-standalone.mjs`, `standalone-server-ws.mjs`, `responses-ws-proxy.mjs`,
`v1-ws-bridge.mjs`, `smoke-electron-packaged.mjs`,
`run-playwright-tests.mjs`, `run-ecosystem-tests.mjs`,
`run-protocol-clients-tests.mjs`, `sync-env.mjs`, `healthcheck.mjs`,
`system-info.mjs`.
- **`scripts/check/`** — `check-cycles.mjs`, `check-docs-sync.mjs`,
`check-docs-counts-sync.mjs`, `check-env-doc-sync.mjs`,
`check-deprecated-versions.mjs`, `check-route-validation.mjs`,
`check-t11-any-budget.mjs`, `check-pr-test-policy.mjs`,
`check-supported-node-runtime.ts`, `test-report-summary.mjs`.
- **`scripts/docs/`** — `generate-docs-index.mjs`, `gen-provider-reference.ts`.
- **`scripts/i18n/`** — `generate-multilang.mjs`, `run-visual-qa.mjs`,
`generate-qa-checklist.mjs`, `apply-priority-overrides.mjs`,
`validate_translation.py`, `check_translations.py`, `i18n_autotranslate.py`,
`untranslatable-keys.json`.
- **`scripts/ad-hoc/`** — `cursor-tap.cjs`, `sync-cursor-models.mjs`,
`migrate-env.mjs`, `dbsetup.js`.
---
## 9. Request Pipeline (Summary)
![Request pipeline (/v1/chat/completions)](../diagrams/exported/request-pipeline.svg)
> Source: [diagrams/request-pipeline.mmd](../diagrams/request-pipeline.mmd)
```
Client request
→ /v1/chat/completions (route.ts)
CORS preflight check
Zod validation (chatCompletionsSchema in shared/validation/schemas.ts)
Auth (extractApiKey + isValidApiKey OR requireManagementAuth)
Policy engine (src/server/authz/pipeline.ts)
Guardrails (PII masker, prompt injection, vision bridge)
→ handleChatCore() (open-sse/handlers/chatCore.ts)
Cache check (semantic + read cache)
Rate limit (rateLimitManager, accountSemaphore)
Combo routing (if model resolves to a combo)
comboResolver → loop per target → handleSingleModel()
translateRequest() (open-sse/translator/request/*)
getExecutor(providerId).execute() (open-sse/executors/*)
fetch upstream → retry/backoff via accountFallback
translateResponse() (open-sse/translator/response/*)
SSE stream OR JSON response
If Responses API: TransformStream via open-sse/transformer/responsesTransformer.ts
→ Compliance audit (src/lib/compliance/)
→ Response to client
```
### Resilience runtime state (three mechanisms)
| Mechanism | Scope | Where |
| ------------------------ | ----------------------------- | ------------------------------------------------------------------------------------------------------------ |
| Provider circuit breaker | Whole provider | `src/shared/utils/circuitBreaker.ts`, persisted in `domain_circuit_breakers` |
| Connection cooldown | One account/key | `markAccountUnavailable()` in `src/sse/services/auth.ts`; consumed by `accountFallback.checkFallbackError()` |
| Model lockout | Provider + connection + model | `open-sse/services/accountFallback.ts`, persisted in `domain_lockout_state` |
See [RESILIENCE_GUIDE.md](./RESILIENCE_GUIDE.md) and the dedicated section in
[CLAUDE.md](../../CLAUDE.md).
---
## 10. How to Contribute
### Add a new provider
1. Register in `src/shared/constants/providers.ts` (Zod-validated at load).
2. Add an executor in `open-sse/executors/` if custom logic is required
(extend `BaseExecutor`).
3. Add a translator in `open-sse/translator/` if it does not speak OpenAI format.
4. If OAuth-based, add config under `src/lib/oauth/providers/` and
`src/lib/oauth/services/`.
5. Register models in `open-sse/config/providerRegistry.ts` (or the format-specific
registry under `open-sse/config/`).
6. Write tests under `tests/unit/`.
### Add a new API route
1. Create `src/app/api/your-route/route.ts`.
2. Follow the pattern: CORS → Zod body validation → auth → handler delegation.
3. If new request shape: add the Zod schema in `src/shared/validation/schemas.ts`.
4. If management-only: add the path to `src/shared/constants/publicApiRoutes.ts`
(denylist for the public API surface).
5. Add tests under `tests/unit/`.
6. Update `docs/reference/API_REFERENCE.md` and `docs/openapi.yaml`.
### Add a new DB module
1. Create `src/lib/db/yourModule.ts` and import `getDbInstance()` from `./core.ts`.
2. Export CRUD functions for your domain.
3. If new tables: add a migration under `src/lib/db/migrations/`, numbered
sequentially, idempotent, transactional.
4. Re-export from `src/lib/localDb.ts` (re-export only — **no logic**).
5. Add tests under `tests/unit/`.
### Add a new MCP tool
1. Add the tool definition under `open-sse/mcp-server/tools/` (or extend
`open-sse/mcp-server/schemas/tools.ts`).
2. Assign the appropriate scope(s) in `src/shared/constants/mcpScopes.ts`.
3. Register the tool in `open-sse/mcp-server/server.ts`.
4. Add tests under `open-sse/mcp-server/__tests__/`.
5. Update [MCP-SERVER.md](../frameworks/MCP-SERVER.md).
### Add a new A2A skill
See [A2A-SERVER.md § Adding a New Skill](../frameworks/A2A-SERVER.md). Skills live in
`src/lib/a2a/skills/` and are registered through the A2A task manager.
---
## 11. Conventions
- **Code style**: 2-space indent, double quotes, 100 char width, semicolons,
`es5` trailing commas — enforced by Prettier via `lint-staged`.
- **Imports**: external → internal (`@/`, `@omniroute/open-sse`) → relative.
- **Naming**: files `camelCase` or `kebab-case`, components `PascalCase`,
constants `UPPER_SNAKE`.
- **ESLint**: `no-eval`, `no-implied-eval`, `no-new-func` = `error` everywhere;
`no-explicit-any` = `warn` in `open-sse/` and `tests/`, error elsewhere.
- **TypeScript**: `strict: false` (legacy posture). Prefer explicit types over
inference for cross-module boundaries.
- **Database**: never write raw SQL in routes or handlers — always go through
`src/lib/db/` modules. Never add logic to `src/lib/localDb.ts`.
- **Errors**: try/catch with specific error types, log with pino context. Never
silently swallow errors in SSE streams; use abort signals for cleanup.
- **Security**: never use `eval()` / `new Function()` / implied eval. Validate
all inputs with Zod. Encrypt credentials at rest (AES-256-GCM). Keep
`src/shared/constants/upstreamHeaders.ts` denylist aligned with the
sanitize/validation layer.
- **Commits**: Conventional Commits — `feat(scope): subject`. Allowed scopes:
`db`, `sse`, `oauth`, `dashboard`, `api`, `cli`, `docker`, `ci`, `mcp`,
`a2a`, `memory`, `skills`.
- **Branches**: prefixes `feat/`, `fix/`, `refactor/`, `docs/`, `test/`,
`chore/`. Never commit directly to `main`.
- **Husky**: pre-commit runs `lint-staged` + `check:docs-sync` +
`check:any-budget:t11`; pre-push runs `check:any-budget:t11` + `check:tracked-artifacts` (fast gates; excludes `test:unit`).
---
## 12. Hard Rules (from CLAUDE.md)
1. Never commit secrets or credentials.
2. Never add logic to `src/lib/localDb.ts`.
3. Never use `eval()` / `new Function()` / implied eval.
4. Never commit directly to `main`.
5. Never write raw SQL in routes — always go through `src/lib/db/` modules.
6. Never silently swallow errors in SSE streams.
7. Always validate inputs with Zod schemas.
8. Always include tests when changing production code.
9. Coverage must stay ≥ 60% (statements, lines, functions, branches).
---
## 13. See Also
- [ARCHITECTURE.md](./ARCHITECTURE.md) — high-level architecture and module
responsibilities.
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — public + management API reference.
- [FEATURES.md](../guides/FEATURES.md) — feature matrix and version highlights.
- [RESILIENCE_GUIDE.md](./RESILIENCE_GUIDE.md) — circuit breaker, cooldown,
lockout deep dive.
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — Auto Combo scoring and strategies.
- [MCP-SERVER.md](../frameworks/MCP-SERVER.md) — full MCP tool catalog + transports.
- [A2A-SERVER.md](../frameworks/A2A-SERVER.md) — A2A protocol skills and discovery.
- [COMPRESSION_GUIDE.md](../compression/COMPRESSION_GUIDE.md) — RTK + Caveman compression.
- [CLI-TOOLS.md](../reference/CLI-TOOLS.md) — CLI integrations.
- [ELECTRON_GUIDE.md](../guides/ELECTRON_GUIDE.md) (if present), [DOCKER_GUIDE.md](../guides/DOCKER_GUIDE.md), [FLY_IO_DEPLOYMENT_GUIDE.md](../ops/FLY_IO_DEPLOYMENT_GUIDE.md), [VM_DEPLOYMENT_GUIDE.md](../ops/VM_DEPLOYMENT_GUIDE.md), [TERMUX_GUIDE.md](../guides/TERMUX_GUIDE.md), [PWA_GUIDE.md](../guides/PWA_GUIDE.md) — deployment targets.
- [TROUBLESHOOTING.md](../guides/TROUBLESHOOTING.md) — common operational issues.
- [CONTRIBUTING.md](../../CONTRIBUTING.md) — contributor workflow.
- [CLAUDE.md](../../CLAUDE.md) — repo rules for Claude Code (the source of truth
for many of the conventions above).
- [AGENTS.md](../../AGENTS.md) — deeper architecture reference used by agents.
docs/architecture/MONITORING_SECTIONS.md
+146
View File
@@ -0,0 +1,146 @@
---
title: "Monitoring & Costs — Navigation Structure"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Monitoring & Costs — Navigation Structure
> Implemented in Group B (plan 16). See `src/shared/constants/sidebarVisibility.ts`.
---
## High-Level Navigation
The dashboard sidebar (after Group B) has these top-level sections in order:
```
Home
Providers
Combos
API Keys
Settings
Analytics
Costs ← NEW (Group B, plan 16)
Monitoring ← REORGANIZED (Group B, plan 16)
...
```
---
## Costs section (new, level 1)
Path prefix: `/dashboard/costs/`
| Item | URL | Description |
| ------------- | ------------------------------------ | ------------------------------------------------ |
| Overview | `/dashboard/costs` | Aggregated cost dashboard (moved from Analytics) |
| Pricing | `/dashboard/costs/pricing` | Per-model pricing table |
| Budget | `/dashboard/costs/budget` | Budget thresholds + alerts |
| Quota Sharing | `/dashboard/costs/quota-share` | Quota Share pools + usage |
| Plan Config | `/dashboard/costs/quota-share/plans` | Per-provider plan overrides |
**Rationale**: Pricing, Budget, and Quota Sharing were previously under
`Monitoring > Costs Parameters`. Moving them to a dedicated top-level section
makes them discoverable without navigating through observability tooling.
---
## Monitoring section (reorganized)
The Monitoring section now has **Activity at the top** followed by **3 subgroups**:
```
Monitoring
├── Activity ← Timeline feed (top-level item)
├── Logs group
│ ├── Logs (all)
│ ├── Proxy Logs
│ └── Console Logs
├── Audit group
│ ├── Audit Log
│ ├── MCP Audit
│ └── A2A Audit
└── System group
├── Health
└── Runtime
```
### What changed from the old structure
| Before | After |
| -------------------------------------------------------------------------------- | ------------------------------------------------- |
| Activity = tab inside Logs that rendered the Audit Log | Activity = dedicated feed (`/dashboard/activity`) |
| Costs Parameters group in Monitoring | Moved to Costs section |
| Flat list: Logs, Activity (logs), Audit, Health, Runtime, Pricing, Budget, Quota | Structured 3-group + dedicated Costs section |
---
## Activity vs Audit Log
These two are now distinct:
| Dimension | Activity (`/dashboard/activity`) | Audit Log (`/dashboard/audit`) |
| ---------------- | ------------------------------------------------------ | ----------------------------------------- |
| **Purpose** | User-facing event feed ("what happened recently") | Compliance / security log |
| **Data source** | `GET /api/compliance/audit-log?level=high` | `GET /api/compliance/audit-log?level=all` |
| **Format** | Timeline, grouped by day, human-readable verbs + icons | Dense paginaged table, 50/page |
| **Filters** | Event type category | Action, severity, actor, date range |
| **Export** | Not available | JSON export |
| **Actor filter** | Not applicable | Filterable by actor |
| **Events shown** | High-level actions only (allowlist) | All audit events |
### High-Level Actions allowlist
Defined in `src/lib/audit/highLevelActions.ts`. Controls which events appear in
the Activity feed. The allowlist includes:
- Provider add/remove/test events
- Combo create/update/delete
- API key lifecycle (create, revoke, rotate)
- Budget threshold reached
- Auth login/logout
- Cloud agent session creation
- MCP tool registration
- Webhook create/delete
- Quota pool/plan changes (`quota.*` actions, Group B)
- Platform events (update, deploy)
- Skill install/remove
Events not in this list appear only in the Audit Log.
### Adding a new high-level action
Edit `src/lib/audit/highLevelActions.ts` and add the action string to
`HIGH_LEVEL_ACTIONS`. This requires a PR (the list is code, not DB-configurable).
The corresponding icon can be added to `src/lib/audit/activityIcons.ts`.
---
## Redirect: `/dashboard/logs/activity`
The old path `/dashboard/logs/activity` is permanently redirected (HTTP 308) to
`/dashboard/activity` via `permanentRedirect()` in
`src/app/(dashboard)/dashboard/logs/activity/page.tsx`.
The legacy sidebar ID `logs-activity` is preserved in `HIDEABLE_SIDEBAR_ITEM_IDS`
(but removed from `SIDEBAR_DEFINITIONS`) to avoid breaking user presets that
reference the old ID.
---
## i18n
Namespaces added by Group B:
| Namespace key | Covers |
| ----------------------- | -------------------------------------------------------------- |
| `sidebar.costsSection` | Costs section label |
| `sidebar.activity` | Activity sidebar item |
| `sidebar.logsGroup` | Logs subgroup label |
| `sidebar.systemGroup` | System subgroup label |
| `sidebar.costsOverview` | Costs overview item |
| `activity.*` | All Activity page strings (title, verbs, filters, empty state) |
Source-of-truth locales: `pt-BR` and `en`. All other 39 locales fall back to
English via the `next-intl` fallback mechanism (configured in `src/i18n/config.ts`).
docs/architecture/QUALITY_GATES.md
+279
View File
@@ -0,0 +1,279 @@
---
title: Quality Gates Reference
---
# Quality Gates Reference
This document is the authoritative reference for all CI quality gates in OmniRoute.
It describes each gate, what it validates, which CI job it runs in, whether it uses
a ratchet baseline or a pass/fail policy, and whether it blocks the build or is advisory.
For a short summary and the allowlist policy, see the "Quality Gates & Ratchets" section
in `CLAUDE.md`.
---
## Gate Inventory (~50 scripts)
Scripts live under `scripts/check/` (policy gates) and `scripts/quality/` (ratchet engine).
The CI source of truth is `.github/workflows/ci.yml`.
### Job: `lint`
Runs on every PR to `main`. Blocks merge on failure.
| Script (`npm run ...`) | Validates | Blocking |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------- |
| `check:node-runtime` | Node.js version is within the supported range | Yes |
| `check:cycles` | Circular imports — all `src/` + `open-sse/` modules | Yes |
| `check:route-validation:t06` | Zod schemas present on all routes (Tier 6 policy) | Yes |
| `check:any-budget:t11` | `@ts-expect-error // any` count does not exceed budget (Tier 11 catraca) | Yes |
| `check:provider-consistency` | Every provider in `providers.ts` has a matching entry in `providerRegistry.ts` (and vice-versa, within the allowlist) | Yes |
| `check:fetch-targets` | Every `fetch("/api/...")` in client-side `src/` resolves to a real `route.ts` | Yes |
| `check:deps` | All `npm install`-able deps across every `package.json` in the repo are in `dependency-allowlist.json`; new unpinned or slopsquatted packages flagged | Yes |
| `audit:deps` | `npm audit` (root + electron) — no high/critical advisories (overlaps osv `check:vuln-ratchet`; see Rationalization Backlog) | Yes |
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-push) | Yes |
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
| `check:public-creds` | No literal OAuth `client_id`/`client_secret` or Firebase Web keys outside `publicCreds.ts` (Hard Rule #11) | Yes |
| `check:db-rules` | No raw SQL outside `src/lib/db/` modules; no barrel-imports from `localDb.ts` (Hard Rules #2/#5) | Yes |
| `check:known-symbols` | Provider executors, routing strategies, and translators registered in their dispatch tables match the files on disk — no orphaned or undeclared symbols | Yes |
| `check:route-guard-membership` | Every route that spawns a child process is classified by `isLocalOnlyPath()` (Hard Rules #15/#17) | Yes |
| `check:test-discovery` | Every `*.test.ts` / `*.spec.ts` file in the repo is collected by at least one test runner (ratchet: orphan list in `test-discovery-baseline.json` can only shrink) | Yes |
| `check:docs-sync` | CHANGELOG version, OpenAPI version, and `llm.txt` are in sync | Yes |
| `typecheck:core` | TypeScript compilation without errors (advisory warnings only) | Yes |
| `typecheck:noimplicit:core` | Strict `noImplicitAny` — forward-looking; many pre-existing call sites still need annotations | **Advisory** (`continue-on-error: true`) |
### Job: `quality-gate`
Runs after `test-coverage`. Blocks merge on failure.
| Script | Validates | Blocking |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------------- |
| `quality:collect` | Emits `quality-metrics.json` (ESLint warning count, coverage from merged shard report) | Yes (upstream of ratchet) |
| `quality:ratchet` | Each metric in `quality-baseline.json` has not regressed (ESLint warnings ≤ baseline; coverage ≥ baseline) | Yes |
| `check:duplication` | Code duplication (jscpd@4) does not exceed baseline in `quality-baseline.json` | Yes |
| `check:complexity` | File-level cyclomatic complexity does not exceed the cap (core ESLint `complexity` + `max-lines-per-function`) | Yes |
| `check:cognitive-complexity` | Cognitive complexity ratchet (`eslint-plugin-sonarjs`) — separate ESLint pass; mergeable with `check:complexity` (see Backlog) | Yes |
| `check:dead-code` | Unused exports / files ratchet (knip) does not regress vs baseline | Yes |
| `check:type-coverage` | Percent-typed ratchet (`type-coverage`) does not regress; largely subsumes `typecheck:noimplicit:core` | Yes |
| `check:codeql-ratchet` | Open CodeQL alert count does not regress (reads via `gh api`; graceful-skip without token) | Yes |
### Job: `quality-extended`
Entire job is advisory (`continue-on-error: true`). The npm-based ratchets run for
real; the external scanners install via `gh release download` and self-skip (exit 0)
when a binary is still absent.
| Script | Validates | Blocking |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------ |
| `check:circular-deps` | No circular dependencies (dpdm) | **Advisory** |
| `check:bundle-size` | Bundle size does not exceed the cap | **Advisory** |
| `check:secrets` | Secret scanning (gitleaks) — skips if binary absent | **Advisory** |
| `check:vuln-ratchet` | Dependency vulnerabilities (osv-scanner) do not regress — skips if binary absent | **Advisory** |
| `check:workflows` | Workflow lint (actionlint + zizmor) — skips if binaries absent | **Advisory** |
| `check:openapi-breaking` | Breaking changes to the public API contract (`openapi.yaml`) vs the base branch (oasdiff) — emits `openapiBreaking=N`; skips if oasdiff absent or base spec unresolvable | **Advisory** |
### Job: `docs-sync-strict`
Runs on every PR to `main`. Blocks merge on failure.
| Script | Validates | Blocking |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- |
| `check:docs-all` | Meta-gate that runs the 6 sub-gates below sequentially | Yes |
| ↳ `check:docs-sync` | CHANGELOG / OpenAPI / llm.txt version consistency | Yes |
| ↳ `check:docs-counts` | Counts in prose (provider count, migration count, etc.) are within the ratchet window of the real counts | Yes |
| ↳ `check:env-doc-sync` | Every env var in `.env.example` is documented in a docs table, and vice versa | Yes |
| ↳ `check:deprecated-versions` | No deprecated version strings in docs | Yes |
| ↳ `check:doc-links` | Internal markdown links in docs resolve to real files (`[text]`/`(path)` form) | Yes |
| ↳ `check:fabricated-docs` | Routes, env vars, CLI commands, hook names, and file paths cited in docs exist in the codebase. Hard gate via `--strict`; soft-fail without flag. | Yes (via `--strict` in CI) |
| `check:cli-i18n` | CLI command strings are present in all i18n locale files | Yes |
| `check:openapi-coverage` | OpenAPI spec covers at least a ratcheted floor of real routes | Yes |
| `check:openapi-security-tiers` | Security tier annotations in `openapi.yaml` are consistent with `routeGuard.ts` classifications | **Advisory** |
| `check:openapi-routes` | Every path in `openapi.yaml` resolves to a real `route.ts` (anti-hallucination) | Yes |
| `check:docs-symbols` | Every `/api/...` reference in `docs/**/*.md` resolves to a real `route.ts` (anti-hallucination) | Yes |
| `i18n translation drift` | Untranslated keys in i18n locale files — warn only | **Advisory** |
### Job: `i18n-ui-coverage`
| Script | Validates | Blocking |
| --------------------------------- | ----------------------------- | -------- |
| `check-ui-keys-coverage` (inline) | UI i18n key coverage is ≥ 65% | Yes |
### Job: `i18n`
Full i18n validation matrix (one job per locale). Entire job is advisory.
| Script | Validates | Blocking |
| ------------------------------- | ----------------------------------- | ----------------------------------------------------- |
| `validate_translation.py quick` | Translation completeness per locale | **Advisory** (`continue-on-error: true` on whole job) |
### Job: `pr-test-policy`
Runs on pull requests only.
| Script | Validates | Blocking |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------- |
| `check:pr-test-policy` | PRs that change production code in `src/`, `open-sse/`, `electron/`, or `bin/` must include or update tests (Hard Rule #8) | Yes |
| `check:test-masking` | Changed test files do not reduce net assert count or add `assert.ok(true)` tautologies | Yes |
| `check:pr-evidence` | PR body cites test/VPS evidence for the change (mechanizes Hard Rule #18 by grepping PR prose — fragile, see Backlog) | Yes |
### Job: `test-vitest`
Runs after `build`. Blocks merge on failure.
| Suite | Validates | Blocking |
| ---------------- | ------------------------------------------------------- | -------------------------------------------------------------------------- |
| `test:vitest` | MCP server (94 tools), autoCombo, cache — vitest runner | Yes |
| `test:vitest:ui` | UI component tests — vitest runner | **Advisory** (`continue-on-error: true`) — failing until Fase 6A UI triage |
### Nightly workflows (scheduled, advisory)
These run on a cron schedule (and `workflow_dispatch`), never on PRs. All are advisory.
| Workflow | Validates | Blocking |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
| `nightly-property` | fast-check property tests with a random seed + high run count | **Advisory** |
| `nightly-resilience` | heap-growth gate, chaos fault-injection, k6 load/soak | **Advisory** |
| `nightly-llm-security` | promptfoo injection guard (block mode) + garak probes (skipped without a provider secret) | **Advisory** |
| `nightly-schemathesis` | OpenAPI contract fuzzing (schemathesis) against a live OmniRoute using `docs/openapi.yaml` — surfaces spec violations / unhandled 500s (Fase 8 B.4) | **Advisory** |
---
## Ratchet Baseline (`quality-baseline.json`)
The ratchet engine (`scripts/quality/check-quality-ratchet.mjs`) reads `quality-baseline.json`
and compares it against the freshly collected `quality-metrics.json`. Any metric that regresses
beyond its epsilon fails the build.
Current tracked metrics:
| Metric | Direction | Meaning |
| --------------------- | --------- | ---------------------------------- |
| `eslintWarnings` | `down` | ESLint warning count must not grow |
| `coverage.statements` | `up` | Statement coverage must not fall |
| `coverage.lines` | `up` | Line coverage must not fall |
| `coverage.functions` | `up` | Function coverage must not fall |
| `coverage.branches` | `up` | Branch coverage must not fall |
To update the baseline after a genuine improvement:
```bash
npm run quality:ratchet -- --update
git add quality-baseline.json
```
The `--update` flag writes the current measured values into `quality-baseline.json`.
Commit this file alongside the change that improved the metric. A PR that improves a
metric without updating the baseline will be caught by `--require-tighten` (Fase 6A.5,
pending implementation).
---
## Allowlist Policy
Every gate that cannot fail on pre-existing violations uses a frozen allowlist
(e.g., `KNOWN_STALE_DOC_REFS`, `KNOWN_MISSING`, `KNOWN_RAW_SQL`). The policy is:
**Fix the root cause; use the allowlist only when the violation is pre-existing and
cannot be fixed in the same PR.**
When adding an entry to an allowlist:
1. Include a comment with the justification.
2. Reference the tracking issue (e.g., `// #3498 — Phase 2 feature, not yet implemented`).
3. Remove the entry in the same PR that fixes the violation — a stale entry that no longer
suppresses an active violation is itself a defect (6A.3 stale-enforcement will
fail the gate on an orphaned allowlist entry once implemented).
Do **not** add allowlist entries to make tests pass faster. A green gate with a growing
allowlist is a false sense of quality.
### When a gate fails on your PR
1. **Read the gate output carefully** — it tells you exactly which file or symbol violated
the rule.
2. **Fix the violation** — most gates are deterministic filesystem checks that pass as soon
as the code is correct.
3. **If the violation is pre-existing** (i.e., you did not introduce it but the gate now
covers it): add an allowlist entry with a justification comment and a tracking issue.
4. **If the gate is a ratchet** (coverage, ESLint warnings, duplication, complexity):
your change made the metric worse. Fix the underlying issue, or (rarely) run
`npm run quality:ratchet -- --update` if the change is intentional and the metric
degradation is acceptable — but document why in the PR description.
5. **Advisory gates** (`continue-on-error: true`) are informational — they do not block
merge but appear in the CI summary. Fix them anyway.
---
## Adding a New Gate
1. Create `scripts/check/check-<name>.mjs` (or `.ts`). Policy gates exit 0/1.
Ratchet-style gates emit a metric to `quality-metrics.json` via `collect-metrics.mjs`.
2. Add `"check:<name>": "node scripts/check/check-<name>.mjs"` to `package.json`.
3. Wire it in `.github/workflows/ci.yml` under the appropriate job
(policy → `lint` or `docs-sync-strict`; ratchet → `quality-gate`).
4. If it has an allowlist, apply `reportStaleEntries()` from
`scripts/check/lib/allowlist.mjs` so stale entries are detected automatically.
5. Write a test in `tests/unit/build/` covering the gate's detection logic.
6. Update this document (add a row to the relevant job table).
---
## Agent tooling: LSP-in-the-loop (opt-in)
Beyond the CI gates, OmniRoute ships an **opt-in** `agent-lsp` scaffold
(a project-level `.mcp.json`, Fase 7 Task 15). Create `.mcp.json`
to expose a TypeScript language server to coding agents, so they resolve symbols /
diagnostics **before** writing code — a compile-before-claim companion to
`typecheck:core` that cuts "invented symbol" errors at the source. It is intentionally
not auto-loaded (you pick and verify the MCP↔LSP bridge); a broken entry only logs a
connection error and never breaks sessions.
---
## Rationalization Backlog (ROI review — Fase 9 Onda 3)
This inventory was reconciled against `ci.yml` on 2026-06-17 (the prior version omitted
`audit:deps`, `check:tracked-artifacts`, `check:lockfile`, `check:licenses`,
`check:dead-code`, `check:cognitive-complexity`, `check:type-coverage`,
`check:codeql-ratchet`, `check:pr-evidence`). An ROI review of the reconciled set
identified the following rationalization candidates. **The merges are mechanical CI
changes; the flips/drops are policy decisions reserved for the operator.** Nothing below
is applied yet.
**Also undocumented above** (advisory, low signal): the `docs-lint` job
(markdownlint + Vale, whole job `continue-on-error`) and the standalone scanner workflows
`semgrep.yml` / `codeql.yml` / `scorecard.yml`. `semgrepFindings: 0` is in
`quality-baseline.json` but is not wired to a blocking ratchet in `ci.yml` — the metric is
currently orphaned.
### Merge / dedup (mechanical, lower risk)
Each candidate was validated against the live gate state on 2026-06-17 (trust-but-verify);
several "obvious" merges turned out to hide debt and are **not** clean drop-ins.
- **`check:docs-sync` runs twice** — standalone in the `lint` job and again inside `check:docs-all` (`docs-sync-strict`) and the husky pre-commit hook. ✅ **DONE** — standalone `lint` invocation removed.
- **CVE scanning** — ❌ **NOT a clean merge.** `audit:deps` hard-fails on any high/critical CVE; `check:vuln-ratchet` (osv) only fails on a _regression_ vs baseline (currently 1 MODERATE). Different semantics — dropping `audit:deps` would lose the absolute high/critical gate. Keep both.
- **Cycle detection** — ❌ **NOT a clean merge.** `check:circular-deps` (dpdm) reports **91 cycles** (that is why it is advisory); it cannot be promoted to blocking without first resolving them, and it has a broader scope than the green, curated `check:cycles`. Keep `check:cycles` blocking; resolving the 91 dpdm cycles is its own backlog.
- **Complexity** — ⏳ valid but real surgery. `check:complexity` (core ESLint) + `check:cognitive-complexity` (sonarjs) are two ESLint passes over `src` + `open-sse`; merging into one config emitting both metrics needs careful ratchet re-wiring. Deferred.
- **`/api` anti-hallucination** — ⏳ valid but script surgery. `check:openapi-routes` (spec→route) + `check:docs-symbols` (prose→route) share resolution logic; collapsing them is a non-trivial script change. Deferred.
- **`check:node-runtime` runs in 11 jobs** — ⚠️ **low ROI.** Each is a separate runner and the check is <1s; total savings ~10s, against losing a cheap per-job guard. Not worth the churn.
### Flip / decide (operator policy)
- `check:openapi-security-tiers` (advisory) — ❌ **NOT cleanly flippable.** It exits 0 but warns that several `traffic-inspector` routes under `LOCAL_ONLY_API_PREFIXES` lack the `x-loopback-only: true` annotation. Enforcing it requires adding those annotations to `openapi.yaml` first.
- `typecheck:noimplicit:core` (advisory) — largely subsumed by the blocking `check:type-coverage` ratchet. Flip to a ratchet or drop the redundant second `tsc` pass.
- `test:vitest:ui` (advisory, 14 parked fails) — fix-and-block or delete; don't leave rotting.
- `check:secrets` (gitleaks, blocking ratchet frozen at 3 documented false-positives) — allowlist the 3 to reach 0, or demote to advisory. Overlaps GitHub native secret-scanning + `check:public-creds`.
- `check:pr-evidence` (blocking, greps PR-body prose) — high false-positive risk; weakens Hard Rule #18 enforcement if dropped, so this is a genuine policy call.
- `semgrep` (advisory standalone) — overlaps CodeQL for the OWASP families; wire its baseline to a ratchet or drop.
---
## Related Documentation
- Supply-chain (provenance, SBOM, Trivy, Scorecard): [`docs/security/SUPPLY_CHAIN.md`](../security/SUPPLY_CHAIN.md)
docs/architecture/REPOSITORY_MAP.md
+583
View File
@@ -0,0 +1,583 @@
---
title: "Repository Map"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Repository Map
> **One-line description for every directory and root file.**
> Last updated: 2026-06-28 — OmniRoute v3.8.40
>
> Use this map to navigate the codebase quickly. For deep dives, follow links to dedicated docs.
## Top-level tree
```
OmniRoute/
├── src/ # Next.js 16 application (UI + API routes + libs + domain + server)
├── open-sse/ # Streaming engine workspace (handlers, executors, translator, MCP server)
├── electron/ # Desktop wrapper (Electron 41 + electron-builder 26.10)
├── bin/ # CLI entry point and command handlers
├── scripts/ # Build, check, sync, and one-off scripts
├── docs/ # Public documentation (you are here)
├── tests/ # All test suites (unit, integration, e2e, protocols-e2e)
├── public/ # Next.js static assets, PWA manifest, service worker, icons
├── config/ # Static config + quality-gate state (i18n, payloadRules, quality/)
├── images/ # Marketing / README image assets
├── @omniroute/ # Publishable companion packages (opencode-plugin, opencode-provider)
├── skills/ # CLI/agent skill packs (cli-* + omni-* + config-codex-cli)
├── examples/ # Sample plugins + omniroute-cmd-hello starter
├── contrib/ # Community contributions (podman/)
├── .source/ # Fumadocs source config (source.config.mjs + server/browser/dynamic)
├── .github/ # GitHub Actions workflows + issue templates + PR template
├── .husky/ # Git hooks (pre-commit, pre-push)
├── .claude/ # Claude Code slash commands (project-scoped)
├── .agents/ # Codex / generic agent workflows + skills (mirror of .claude/)
├── .vscode/ # VS Code workspace settings
├── _ideia/ # Planning notes (informal; not shipped)
├── _mono_repo/ # Historic subprojects (cloud, site, vscode-extension)
├── _references/ # Read-only reference clones from related OSS projects
├── _tasks/ # Per-release task tracking files (informal)
├── .build/ .worktrees/ dist/ # local build / git-worktree / build-output scratch (gitignored)
├── .issues/ # Local issue cache (gitignored)
├── .playwright-mcp/ # Playwright MCP test artifacts
├── coverage/ # c8 coverage output (gitignored)
├── logs/ # Runtime logs (gitignored)
├── node_modules/ # Dependencies (gitignored)
├── package/ # npm pack staging area (build artifact)
├── .next/ # Next.js build output (gitignored)
└── (root files — see below)
```
---
## Root files
| File | Purpose |
| ------------------------------------------- | ---------------------------------------------------------------------------------------- |
| **README.md** | Marketing landing page + quick start + feature matrix (see also `llm.txt`) |
| **CHANGELOG.md** | Per-release changelog (auto-generated by `/version-bump-cc` skill) |
| **LICENSE** | MIT license text |
| **CLAUDE.md** | Project rules for Claude Code agents (hard rules, conventions, scenarios) |
| **AGENTS.md** | Same as CLAUDE.md but for non-Claude AI agents (Codex, Cursor, etc.) |
| **GEMINI.md** | Concise rules for Gemini-based agents (subset of CLAUDE.md) |
| **CONTRIBUTING.md** | Contributor guide: setup, conventional commits, testing, PR flow |
| **SECURITY.md** | Vulnerability reporting policy, supported versions, threat model |
| **CODE_OF_CONDUCT.md** | Contributor Covenant — community behavior expectations |
| **llm.txt** | Plain-text landing optimized for LLM crawlers (SEO for AI assistants) |
| **package.json** | npm manifest, scripts, dependencies, engines, c8 coverage gate |
| **package-lock.json** | Locked dependency tree |
| **tsconfig.json** | Root TypeScript config |
| **tsconfig.typecheck-core.json** | Typecheck config for `src/` core |
| **tsconfig.typecheck-noimplicit-core.json** | Strict (`noImplicitAny`) typecheck |
| **tsconfig.tsbuildinfo** | TS incremental build cache (gitignored) |
| **next.config.mjs** | Next.js 16 build configuration (standalone output) |
| **next-env.d.ts** | Next.js auto-generated env types |
| **eslint.config.mjs** | ESLint flat config (rules per project area) |
| **prettier.config.mjs** | Prettier formatting rules |
| **postcss.config.mjs** | PostCSS config for Tailwind/CSS pipeline |
| **playwright.config.ts** | Playwright E2E test config |
| **vitest.config.ts** | Vitest config (default suite) |
| **vitest.mcp.config.ts** | Vitest config for MCP server / autoCombo / cache suites |
| **sonar-project.properties** | SonarQube/SonarCloud config (code quality) |
| **Dockerfile** | Multi-stage Docker build (builder → runner-base → runner-cli) |
| **docker-compose.yml** | Dev compose with 4 profiles (base, cli, host, cliproxyapi) + redis sidecar |
| **docker-compose.prod.yml** | Production compose (port 20130, redis, named volumes) |
| **.dockerignore** | Files excluded from Docker context |
| **fly.toml** | Fly.io deployment config (region `sin`, port 20128, /data volume) |
| **.env.example** | Template env file (auto-copied to `.env` on first install) |
| **.gitignore** | Git ignore patterns |
| **.npmignore** | npm publish exclusion list |
| **.npmrc** | npm config (registry, lockfile policy) |
| **.node-version** | Node version pin (used by nvm-compatible tools) |
| **.nvmrc** | Node version pin for nvm |
| **eslint.complexity.config.mjs** | ESLint config for the complexity ratchet (`scripts/check/check-complexity.mjs --config`) |
| **eslint.sonarjs.config.mjs** | ESLint config for SonarJS rules (cognitive complexity / duplication) |
| **source.config.ts** | Fumadocs `defineDocs` source config (feeds `.source/`) |
| **knip.json** | Knip config — unused files/exports/deps (feeds the dead-code gate) |
| **stryker.conf.json** | Stryker mutation-testing config |
| **.size-limit.json** | size-limit bundle budget config |
| **promptfooconfig.yaml** | promptfoo eval config |
| **.gitleaks.toml** | gitleaks secret-scan ruleset |
| **.zizmor.yml** | zizmor GitHub-Actions security-lint config |
| **socket.yml** | Socket.dev supply-chain config |
| **news.json** | In-app release-notes feed (read by `src/shared/utils/releaseNotes.ts`) |
| **flake.nix** / **flake.lock** | Nix dev-shell definition + lock |
| **.env** | Local secrets (gitignored — generated from `.env.example`) |
> **Moved out of the root in v3.8.26 (declutter):**
>
> - **→ `config/quality/`:** `quality-baseline.json`, `complexity-baseline.json`, `duplication-baseline.json`, `file-size-baseline.json`, `test-discovery-baseline.json`, `dependency-allowlist.json`, `.license-allowlist.json`, and the generated `quality-metrics.json` (gitignored). See [`## config/`](#config--static-configs--quality-gate-state).
---
## `src/` — Next.js application
```
src/
├── app/ # App Router (pages + API routes + status pages + landing)
├── lib/ # Core libraries / domain modules (~50 subdirs + ~30 top-level files)
├── domain/ # Pure domain logic (policy engine, fallback, cost, lockout, comboResolver, assessment)
├── server/ # Server-only modules (authz pipeline, cors, auth middleware) — cannot import from client
├── shared/ # Shared between server and client where safe (constants, types, validation, contracts, utils)
├── i18n/ # next-intl config + per-locale message JSON (30+ locales)
├── middleware/ # Next.js middleware (request enrichment, locale detection)
├── mitm/ # MITM proxy core: cert gen/install, handlers, targets, inspector, masks, passthrough
│ ├── handlers/ # 9 IDE-agent handler classes extending MitmHandlerBase (antigravity, kiro, copilot, codex, cursor, zed, claudeCode, openCode, trae)
│ └── inspector/ # Traffic capture layer: buffer (in-memory ring), sseMerger, conversationNormalizer, kindDetector, contextKey, httpProxyServer, systemProxyConfig
├── models/ # Model adapter glue (legacy shim)
├── scripts/ # In-tree maintenance scripts (e.g., backfillAggregation)
├── sse/ # Legacy SSE handlers/services (chat.ts, chatHelpers.ts, services/auth.ts)
├── store/ # Legacy in-memory store (being phased out for src/lib/db)
├── types/ # Shared TS type files
├── instrumentation.ts # Next.js telemetry hook (browser + edge)
├── instrumentation-node.ts # Node-only instrumentation
├── server-init.ts # Server bootstrap (DB migrations, jobs, cleanup)
└── proxy.ts # HTTP-proxy entry shim
```
### `src/app/` — App Router (Next.js 16)
| Path | Purpose |
| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `app/api/v1/` | Public OpenAI-compat API (~25 sub-routes: chat, completions, embeddings, files, batches, audio, images, videos, music, rerank, moderations, search, ws, agents, accounts, providers, etc.) |
| `app/api/v1beta/` | Gemini-style API endpoints |
| `app/api/playground/` | Playground Studio routes: `improve-prompt/` (POST — LLM prompt rewriter), `presets/` (GET list / POST create), `presets/[id]/` (GET / PUT / DELETE) — see `docs/frameworks/PLAYGROUND_STUDIO.md` |
| `app/api/` (non-v1) | Management/admin routes (~60 directories: providers, combos, settings, mcp, a2a, evals, memory, skills, webhooks, compliance, resilience, monitoring, tunnels, cli-tools, etc.) |
| `app/api/tools/agent-bridge/` | AgentBridge REST API — 12 routes (server control, agent state/DNS/mappings, bypass, cert, upstream-CA). LOCAL_ONLY + SPAWN_CAPABLE. See `docs/frameworks/AGENTBRIDGE.md §7`. |
| `app/api/tools/traffic-inspector/` | Traffic Inspector REST + WS API — 16+ routes (requests, sessions, hosts, capture-modes, export, ws). LOCAL_ONLY + SPAWN_CAPABLE. See `docs/frameworks/TRAFFIC_INSPECTOR.md §8`. |
| `app/a2a/` | A2A JSON-RPC 2.0 entry point (`POST /a2a`) |
| `app/.well-known/agent.json/` | A2A Agent Card (discovery) |
| `app/(dashboard)/dashboard/` | Dashboard UI pages (~35 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, activity, etc.) |
| `app/(dashboard)/dashboard/search-tools/` | Search Tools Studio UI (3 tabs: Search/Scrape/Compare + SearchConceptCard + ProviderCatalog) — see `docs/frameworks/SEARCH_TOOLS_STUDIO.md` |
| `app/(dashboard)/dashboard/` | Dashboard UI pages (~30 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, etc.) |
| `app/(dashboard)/dashboard/memory/` | Memory Studio (plan 21): `page.tsx` (3-tab shell), `components/` (MemoryConceptCard, MemoryEngineStatus, EmbeddingSourceSelector, EditMemoryModal, RetrievePreview, QdrantConfigCard, RerankConfigCard), `components/tabs/` (MemoriesTab, PlaygroundTab, EngineTab), `hooks/` (useEngineStatus, useMemorySettings) |
| `app/(dashboard)/dashboard/tools/agent-bridge/` | AgentBridge dashboard page — server card, 9 agent cards, setup wizard, model mapping, bypass list. i18n PT-BR + EN. See `docs/frameworks/AGENTBRIDGE.md`. |
| `app/(dashboard)/dashboard/tools/traffic-inspector/` | Traffic Inspector dashboard page — DevTools split, 7 detail tabs, 4 capture mode toggles, session recorder, context colorization. i18n PT-BR + EN. See `docs/frameworks/TRAFFIC_INSPECTOR.md`. |
| `app/(dashboard)/dashboard/activity/` | Activity feed page (Group B): `page.tsx` (server) + `ActivityFeedClient.tsx` + `components/{ActivityFeed,ActivityItem,DayHeader,EventTypeFilter}.tsx` — see `docs/architecture/MONITORING_SECTIONS.md` |
| `app/(dashboard)/dashboard/costs/quota-share/` | Quota Sharing page (Group B): `QuotaSharePageClient.tsx` + `components/{PoolCard,DimensionBar,AllocationTable,BurnRateChart,QuotaConceptCard,CreatePoolModal,EditAllocationsModal}.tsx` + `hooks/{usePools,usePoolUsage,useLocalStoragePoolMigration}.ts` |
| `app/(dashboard)/dashboard/costs/quota-share/plans/` | Provider plan config page (Group B): `page.tsx` + `ProviderPlanConfigClient.tsx` — quota dimensions per connection override |
| `app/docs/` | Embedded documentation viewer (renders `docs/*.md`) |
| `app/landing/` | Marketing landing page |
| `app/login/`, `forgot-password/`, `forbidden/` | Auth-related pages |
| `app/{400,401,403,408,429,500,502,503}/` | HTTP error pages |
| `app/maintenance/`, `offline/`, `status/`, `privacy/`, `terms/`, `callback/` | Static/status pages |
| `app/layout.tsx`, `page.tsx`, `manifest.ts`, `globals.css` | Root layout, home, PWA manifest, global CSS |
| `app/error.tsx`, `global-error.tsx`, `not-found.tsx`, `loading.tsx` | Error boundaries |
### `src/lib/` — Core libraries (~50 modules)
| Module | Purpose |
| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `a2a/` | A2A protocol task manager, skills (5), streaming |
| `acp/` | CLI Agent Registry (local CLI discovery — see `docs/frameworks/AGENT_PROTOCOLS_GUIDE.md`) |
| `api/` | Shared API helpers (`requireManagementAuth`, validation) |
| `auth/` | Session, password hashing, token validation |
| `batches/` | OpenAI Batches API handlers |
| `catalog/` | Provider catalog Zod validation + capability resolution |
| `cloudAgent/` | Cloud Agents (Codex Cloud, Devin, Jules) — see `docs/frameworks/CLOUD_AGENT.md` |
| `combos/` | Combo resolution + reorder helpers |
| `audit/` | Activity feed helpers: `highLevelActions.ts` (allowlist + `isHighLevelAction()`), `activityIcons.ts` (action → icon/verb map), `timeline.ts` (groupByDay/relativeTime) — see `docs/architecture/MONITORING_SECTIONS.md` |
| `compliance/` | Audit log + provider audit — see `docs/security/COMPLIANCE.md` |
| `compression/` | Compression engine glue (engines live in `open-sse/services/compression/`) |
| `config/` | Runtime config helpers |
| `db/` | 95+ domain DB modules + 110+ migrations (always go through here for SQLite) |
| `quota/` | Quota Sharing Engine: `dimensions.ts` (types/Zod), `types.ts` (QuotaStore interface), `sqliteQuotaStore.ts`, `redisQuotaStore.ts`, `storeFactory.ts`, `fairShare.ts`, `burnRate.ts`, `planResolver.ts`, `planRegistry.ts`, `saturationSignals.ts`, `enforce.ts`, `spendRecorder.ts` — see `docs/routing/QUOTA_SHARE.md` |
| `display/` | UI formatting helpers (cost, latency, etc.) |
| `embeddings/` | Embeddings service helpers |
| `env/` | Env variable parsing + validation |
| `evals/` | Eval framework (suites, runner, runtime) — see `docs/frameworks/EVALS.md` |
| `guardrails/` | PII masker, prompt injection, vision bridge — see `docs/security/GUARDRAILS.md` |
| `jobs/` | Background jobs (cron-like) |
| `memory/` | Conversational memory (SQLite FTS5 + sqlite-vec hybrid RRF + Qdrant tier 2) — see `docs/frameworks/MEMORY.md` |
| `memory/embedding/` | Multi-source embedding layer: `index.ts` (resolver), `remote.ts`, `staticPotion.ts`, `transformersLocal.ts`, `cache.ts`, `types.ts` (plan 21) |
| `memory/vectorStore.ts` | sqlite-vec v0.1.9 wrapper — KNN brute-force + hybrid RRF (FTS5 + vector, k=60). Lazy-init, degrades gracefully when sqlite-vec unavailable. (plan 21) |
| `memory/reindex.ts` | `runReindexBatch()` — processes memories with `needs_reindex=1` in background; called by `POST /api/memory/reindex` and lazy-backfill path. (plan 21) |
| `monitoring/` | Health checks, metrics emission |
| `oauth/` | OAuth flows for 14 providers (claude, codex, antigravity, cursor, github, gemini, kimi-coding, kilocode, cline, qwen, kiro, qoder, gitlab-duo, windsurf) |
| `plugins/` | Plugin registry |
| `promptCache/` | Anthropic-style prompt cache breakpoints |
| `skills/` | Skills framework (built-in + marketplace + SkillsSH) — see `docs/frameworks/SKILLS.md` |
| `playground/` | Playground Studio shared helpers: `codeExport.ts` (curl/Python/TS generator), `promptImprover.ts` (meta-prompt builder), `streamMetrics.ts` (pure TTFT/TPS), `types.ts` (pricing table) — see `docs/frameworks/PLAYGROUND_STUDIO.md` |
| `webhookDispatcher.ts` | HMAC webhook delivery — see `docs/frameworks/WEBHOOKS.md` |
| `cloudflaredTunnel.ts`, `ngrokTunnel.ts` | Tunnel managers — see `docs/ops/TUNNELS_GUIDE.md` |
| `oneproxySync.ts`, `oneproxyRotator.ts` | 1proxy free proxy marketplace — see `docs/ops/PROXY_GUIDE.md` |
| `cloudSync.ts`, `initCloudSync.ts` | Optional cloud sync of state |
| `localDb.ts` | Re-export barrel for db modules (no logic — re-exports only) |
| `cacheLayer.ts`, `idempotencyLayer.ts` | Request caching + idempotency |
| (~30 more top-level files) | Specialized helpers (logEnv, modelsDevSync, piiSanitizer, etc.) |
### `src/db/` — Database (94 modules + 106 migrations)
| Subdir | Purpose |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `db/core.ts` | `getDbInstance()` singleton with WAL journaling |
| `db/migrations/` | Versioned SQL files (idempotent, transactional). `073_memory_vec.sql` adds `memory_vec_meta` + `needs_reindex` column (plan 21). |
| `db/playgroundPresets.ts` | CRUD module for Playground Studio presets (`listPlaygroundPresets`, `getPlaygroundPreset`, `createPlaygroundPreset`, `updatePlaygroundPreset`, `deletePlaygroundPreset`) |
| `db/memoryVec.ts` | CRUD for `memory_vec_meta` (active_dim, embedding_signature, last_reset_at, vec_loaded) + `markMemoryNeedsReindex`, `getMemoryReindexQueue`, etc. (plan 21) |
| `db/<domain>.ts` | One module per domain: providers, combos, apiKeys, users, sessions, usage, audit*log, webhooks, skills, memory_entries, cloud_agent_tasks, evals*\*, reasoning_cache, etc. |
### `src/domain/`
| Module | Purpose |
| ---------------------- | ----------------------------------------------------------------------- |
| `policy.ts` | Policy engine |
| `fallbackPolicy.ts` | Fallback decision tree |
| `costRules.ts` | Cost calculation rules |
| `lockoutPolicy.ts` | Model/connection lockout policy |
| `tagRouter.ts` | Tag-based routing |
| `comboResolver.ts` | Combo resolution (used by combo engine) |
| `modelAvailability.ts` | Per-model availability check |
| `assessment/` | Model assessment (Phase 1 of RFC-AUTO-ASSESSMENT — see `docs/archive/`) |
### `src/server/`
| Module | Purpose |
| -------- | ---------------------------------------------------------------------------------------------------- |
| `authz/` | Authorization pipeline: `classify` → `policies` → `enforce` — see `docs/architecture/AUTHZ_GUIDE.md` |
| `cors/` | CORS configuration |
| `auth/` | Session middleware |
### `src/shared/`
| Module | Purpose |
| -------------------------------- | ---------------------------------------------------------------------- |
| `constants/providers.ts` | **236 providers** with Zod validation (source of truth) |
| `constants/cliTools.ts` | External CLI tool registry |
| `constants/routingStrategies.ts` | **17 routing strategies** with priorities |
| `constants/publicApiRoutes.ts` | Routes that require Bearer (vs management) auth |
| `constants/upstreamHeaders.ts` | Header denylist for upstream requests |
| `validation/schemas.ts` | ~80 Zod schemas (single source of truth for API contracts) |
| `validation/helpers.ts` | Zod validation helpers (`validateBody`, etc.) |
| `types/` | Shared TS types |
| `contracts/` | Public API contracts (consumed by `files:` in `package.json`) |
| `utils/circuitBreaker.ts` | Provider circuit breaker (see `docs/architecture/RESILIENCE_GUIDE.md`) |
| `utils/apiAuth.ts` | API key validation, scope checking |
| `utils/fetchTimeout.ts` | Timeout/abort wrappers for upstream fetch |
---
## `open-sse/` — Streaming Engine Workspace
Separate npm workspace (`@omniroute/open-sse`). Handles request processing + provider execution.
```
open-sse/
├── handlers/ # 16 files (12 handlers + 4 helpers): chatCore, responsesHandler, embeddings, audio, image, video, music, rerank, moderations, search, etc.
├── executors/ # 67 provider-specific executors (extend BaseExecutor)
├── translator/ # Format converters (9 request, 9 response, 9 helpers)
├── transformer/ # Responses API ↔ Chat Completions (TransformStream)
├── services/ # ~80+ service modules (combo, accountFallback, autoCombo, reasoningCache, claude code/chatgpt stealth, modelDeprecation, taskAwareRouter, workflowFSM, etc.)
├── mcp-server/ # MCP server (94 tools, 3 transports, 30 scopes)
├── config/ # Provider/model registries, header config, model aliases
├── utils/ # TLS client, proxy fetch/dispatcher, network helpers
├── index.ts # Workspace entry
├── package.json # Workspace manifest
├── tsconfig.json # Workspace TS config
└── types.d.ts # Workspace type declarations
```
### `open-sse/mcp-server/`
| Path | Purpose |
| --------------------------- | ------------------------------------------------------------------------------ |
| `server.ts` | MCP server lifecycle (stdio + HTTP transports) |
| `httpTransport.ts` | HTTP Streamable + SSE transports (`/api/mcp/sse`, `/api/mcp/stream`) |
| `audit.ts` | Audit logging to `mcp_tool_audit` table |
| `scopeEnforcement.ts` | Per-tool scope validation |
| `runtimeHeartbeat.ts` | Health heartbeat to `DATA_DIR/runtime/mcp-heartbeat.json` |
| `descriptionCompressor.ts` | Compress tool description metadata to save context |
| `schemas/tools.ts` | 34 base tool definitions + scopes |
| `tools/advancedTools.ts` | Advanced tool implementations |
| `tools/memoryTools.ts` | 3 memory tools (search/add/clear) |
| `tools/skillTools.ts` | 4 skill tools (list/enable/execute/executions) |
| `tools/compressionTools.ts` | 5 compression tools |
| `README.md` | Internal MCP server README (cross-linked from `docs/frameworks/MCP-SERVER.md`) |
---
## `electron/` — Desktop Wrapper
| File | Purpose |
| ---------------- | --------------------------------------------------------------------------------- |
| `main.js` | Electron main process (BrowserWindow, embedded Next.js server, tray, auto-update) |
| `preload.js` | IPC bridge (contextBridge → `window.omniroute`) |
| `package.json` | electron-builder config + Electron 41 + electron-builder 26.10 deps |
| `assets/` | App icons (Windows .ico, macOS .icns, Linux .png) |
| `dist-electron/` | Build output (gitignored) |
| `types.d.ts` | Type declarations for renderer bridge |
| `README.md` | Internal Electron README (see also `docs/guides/ELECTRON_GUIDE.md`) |
---
## `bin/` — CLI
| File | Purpose |
| ----------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `omniroute.mjs` | Main CLI entry — `omniroute serve`, `omniroute setup`, `omniroute doctor`, `omniroute providers`, `omniroute combos`, etc. |
| `reset-password.mjs` | Standalone password reset CLI |
| `cli/commands/setup.mjs` | Interactive + non-interactive setup wizard |
| `cli/commands/doctor.mjs` | System health diagnostics (8+ checks) |
| `cli/commands/providers.mjs` | Provider list/test/validate |
| `cli/{args,data-dir,encryption,io,provider-catalog,provider-store,provider-test,settings-store,sqlite}.mjs` | CLI helper modules |
| `cli/tray/tray.ts` | System tray integration (cross-platform: NotifyIcon on Windows, systray2 on macOS/Linux) |
| `cli/tray/tray.ps1` | PowerShell NotifyIcon backend (Windows, zero new binaries) |
| `cli/tray/autostart.ts` | Cross-platform autostart (LaunchAgent / .desktop / registry) |
| `cli/runtime/sqliteRuntime.mjs` | 5-step SQLite driver resolution chain (bundled → runtime → lazy-install → node:sqlite → sql.js) |
| `cli/runtime/magicBytes.mjs` | Binary magic-byte validation (ELF / Mach-O / Mach-O fat / PE) |
| `cli/runtime/index.mjs` | `warmUpRuntimes()` — pre-resolves drivers at postinstall / first startup |
| `nodeRuntimeSupport.mjs` | Validate supported Node.js version on install |
---
## `skills/` — Public Agent Skills
| File | Purpose |
| ---------------------------- | ---------------------------------------------------------------------------------- |
| `skills/omniroute*/SKILL.md` | 10 skill manifests for external AI agents (Claude Desktop, ChatGPT, Cursor, Cline) |
---
## `scripts/` — Build & Check Scripts
| Script | Purpose |
| ----------------------------------- | -------------------------------------------------------------------------- |
| `run-next.mjs` | Dev/start runner with env hydration |
| `build-next-isolated.mjs` | Standalone build (Next.js 16 standalone) |
| `prepublish.ts` | Package preparation before `npm pack` |
| `postinstall.mjs` | Auto-create `.env` from `.env.example` on first install |
| `sync-env.mjs` | Re-sync `.env` keys with `.env.example` |
| `check-cycles.mjs` | Detect circular dependencies |
| `check-route-validation.mjs` | Validate all API routes have Zod validation |
| `check-t11-any-budget.mjs` | Enforce explicit `any` budget per file |
| `check-docs-sync.mjs` | Validate docs version sync (existing pre-commit) |
| **`check-env-doc-sync.mjs`** | NEW: cross-check env vars in code vs `.env.example` vs `ENVIRONMENT.md` |
| **`check-docs-counts-sync.mjs`** | NEW: validate counts (executors, strategies, OAuth, A2A skills) match docs |
| **`check-deprecated-versions.mjs`** | NEW: flag stale versions/dates in docs |
| `check-supported-node-runtime.ts` | Validate current Node version is supported |
| `check-pr-test-policy.mjs` | Enforce "tests required" rule on production code changes |
| **`gen-provider-reference.ts`** | NEW: auto-generate `docs/reference/PROVIDER_REFERENCE.md` from catalog |
| `i18n/generate-multilang.mjs` | Translate UI strings + docs via Google Translate |
| `i18n_autotranslate.py` | LLM-based doc translation pipeline |
| `validate_translation.py` | Per-locale translation validation |
| `check_translations.py` | Code-side i18n key check |
| `run-playwright-tests.mjs` | Playwright E2E runner |
| `run-protocol-clients-tests.mjs` | MCP/A2A E2E runner |
| `run-ecosystem-tests.mjs` | Ecosystem (provider integration) tests |
| `test-report-summary.mjs` | Generate coverage summary markdown |
| `smoke-electron-packaged.mjs` | Smoke-test packaged Electron build |
| `native-binary-compat.mjs` | Validate native deps (`better-sqlite3`) match Electron's Node |
| `validate-pack-artifact.ts` | Validate npm pack output |
| `responses-ws-proxy.mjs` | WebSocket bridge for Codex Responses API |
| `v1-ws-bridge.mjs` | WebSocket bridge for `/api/v1/ws` endpoint |
| `standalone-server-ws.mjs` | Standalone WS server runner |
| `system-info.mjs` | Print system/runtime info for support |
| `healthcheck.mjs` | One-shot health check (used by Docker HEALTHCHECK) |
| `uninstall.mjs` | Clean uninstall script |
---
## `docs/` — Public Documentation (44 files + 4 subdirs)
### Top-level guides
| Doc | Purpose |
| --------------------------- | ------------------------------------------------------------------------------------- |
| `ARCHITECTURE.md` | High-level architecture, subsystem map, dashboard surface |
| `CODEBASE_DOCUMENTATION.md` | Engineering reference: directories, modules, conventions |
| `FEATURES.md` | Feature matrix with v3.8 highlights |
| `USER_GUIDE.md` | End-user manual (setup, models, combos, CLIs, audio, etc.) |
| `API_REFERENCE.md` | API endpoint reference with auth model |
| `openapi.yaml` | OpenAPI 3.0 spec (121 paths) |
| `SETUP_GUIDE.md` | Install methods (npm, npx, Docker, Electron, Termux, source) |
| `ENVIRONMENT.md` | All env vars (~219 used in code, ~810 lines `.env.example`) |
| `TROUBLESHOOTING.md` | Common errors + v3.8.0 known issues |
| `RELEASE_CHECKLIST.md` | Full release flow (skills, husky, conventional commits, deploy) |
| `COVERAGE_PLAN.md` | Coverage goals and current state |
| `FREE_TIERS.md` | Curated free-tier providers (48+ free + 11 OAuth) |
| `CLI-TOOLS.md` | External CLI integrations + Internal OmniRoute CLI |
| `I18N.md` | i18n architecture, adding a language, 30 locales |
| `UNINSTALL.md` | Clean uninstall steps |
| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 236 providers (regen: `npm run gen:provider-reference`) |
### Subsystem deep-dives
| Doc | Purpose |
| -------------------------- | ------------------------------------------------------------------- |
| `MCP-SERVER.md` | MCP server: 94 tools, 3 transports, 30 scopes, REST endpoints |
| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 5 skills, REST helpers, agent card |
| `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents |
| `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration |
| `SKILLS.md` | Skills framework (built-in + marketplace + SkillsSH + sandbox) |
| `MEMORY.md` | Memory system (SQLite FTS5 + Qdrant) |
| `EVALS.md` | Eval framework (suites, runs, rubrics) |
| `GUARDRAILS.md` | PII masker, prompt injection, vision bridge |
| `COMPLIANCE.md` | Audit log, retention, noLog opt-out |
| `WEBHOOKS.md` | HMAC-signed webhook delivery |
| `REASONING_REPLAY.md` | Hybrid memory/SQLite cache for `reasoning_content` |
| `AUTHZ_GUIDE.md` | Authorization pipeline (`classify` → `policies` → `enforce`) |
| `RESILIENCE_GUIDE.md` | Circuit breaker + cooldown + model lockout |
| `STEALTH_GUIDE.md` | TLS fingerprinting (JA3/JA4), Claude Code CCH, MITM cert |
| `AUTO-COMBO.md` | Auto Combo engine (9-factor scoring, 4 mode packs, virtual factory) |
### Compression
| Doc | Purpose |
| ------------------------------- | ---------------------------------------- |
| `COMPRESSION_GUIDE.md` | Overview of compression modes + roadmap |
| `COMPRESSION_ENGINES.md` | Caveman + RTK engines, registry contract |
| `COMPRESSION_RULES_FORMAT.md` | Caveman rule pack JSON schema |
| `COMPRESSION_LANGUAGE_PACKS.md` | Per-language rule pack inventory |
| `RTK_COMPRESSION.md` | RTK declarative pipeline (49 filters) |
### Deployment
| Doc | Purpose |
| ---------------------------- | ----------------------------------------------------------------- |
| `DOCKER_GUIDE.md` | Docker build, profiles (base/cli/host/cliproxyapi), Redis sidecar |
| `VM_DEPLOYMENT_GUIDE.md` | Generic VM/VPS deployment (Ubuntu/Debian + nginx + systemd) |
| `FLY_IO_DEPLOYMENT_GUIDE.md` | Fly.io deployment (currently Chinese-only) |
| `TERMUX_GUIDE.md` | Android headless via Termux |
| `PWA_GUIDE.md` | Progressive Web App install + service worker |
| `ELECTRON_GUIDE.md` | Desktop app build + sign + distribute |
| `TUNNELS_GUIDE.md` | Cloudflared + ngrok + Tailscale Funnel |
| `PROXY_GUIDE.md` | 4-level outbound proxy + 1proxy marketplace |
### Subdirectories
| Subdir | Purpose |
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `docs/archive/` | Archived/historical docs (e.g., `RFC-AUTO-ASSESSMENT-DRAFT.md` — superseded by EVALS) |
| `docs/i18n/` | Localized doc translations (~42 locales) |
| `docs/screenshots/` | Image assets for guides |
| `_tasks/superpowers/` | Plans/specs from superpowers (`writing-plans`/`brainstorming`) + research — isolated, separately-versioned repo, gitignored by the main tree. See CLAUDE.md → "Planning & Research Artifacts". |
---
## `tests/` — Test Suites
| Subdir | Type | Runner |
| ---------------------- | --------------------------------------- | --------------------------------------- |
| `tests/unit/` | Unit tests (~500 files, fastest) | Node native test runner |
| `tests/integration/` | Multi-module + DB integration tests | Node native test runner (concurrency 1) |
| `tests/e2e/` | UI + workflow E2E | Playwright |
| `tests/protocols-e2e/` | MCP + A2A real-client E2E | Custom protocol clients |
| `tests/ecosystem/` | Provider integration (network-touching) | Node native test runner |
---
## `public/` — Static Assets
| Path | Purpose |
| ------------------- | ---------------------------------------------------------------- |
| `public/` (root) | Favicons, robots.txt, manifest, service worker, marketing images |
| `public/providers/` | Provider logo PNG/SVG (used in dashboard) |
---
## `config/` — Static Configs + Quality-Gate State
Shipped configuration templates plus the committed quality-gate baselines
(moved here from the repo root in v3.8.26 to keep the root lean).
| Path | Purpose |
| --------------------------------------------- | -------------------------------------------------------------------------------- |
| `config/i18n.json` | Locale list + metadata (canonical source for the 42-locale count) |
| `config/i18n-schema.json` | JSON schema validating `i18n.json` |
| `config/payloadRules.json` | Upstream payload sanitization rules |
| `config/quality/quality-baseline.json` | Multi-metric ratchet baseline (`scripts/quality/check-quality-ratchet.mjs`) |
| `config/quality/complexity-baseline.json` | Frozen ESLint-complexity baseline (`check-complexity.mjs`) |
| `config/quality/duplication-baseline.json` | Frozen jscpd duplication baseline (`check-duplication.mjs`) |
| `config/quality/file-size-baseline.json` | Frozen per-file size baseline (`check-file-size.mjs`) |
| `config/quality/test-discovery-baseline.json` | Frozen orphan-test baseline (`check-test-discovery.mjs`) |
| `config/quality/dependency-allowlist.json` | Approved dependencies allowlist (`check-deps.mjs`) |
| `config/quality/.license-allowlist.json` | SPDX license allowlist (`check-licenses.mjs`) |
| `config/quality/quality-metrics.json` | Ephemeral collected metrics (generated by `collect-metrics.mjs`; **gitignored**) |
---
## `.github/` — GitHub Integration
| Path | Purpose |
| ---------------------------------- | -------------------------------------------------------------- |
| `.github/workflows/` | GitHub Actions CI/CD workflows (lint, test, coverage, release) |
| `.github/ISSUE_TEMPLATE/` | Bug/feature issue templates |
| `.github/PULL_REQUEST_TEMPLATE.md` | PR template |
| `.github/dependabot.yml` | Dependency update config |
---
## `.husky/` — Git Hooks
| File | Purpose |
| ------------ | ----------------------------------------------------------------- |
| `pre-commit` | Runs `lint-staged + check-docs-sync + check:any-budget:t11` |
| `pre-push` | Currently disabled (commented). Run `npm run test:unit` manually. |
| `_/` | Husky internals |
---
## `.claude/` — Claude Code Slash Commands
| File | Purpose |
| --------------------------------------------------- | -------------------------------------------------- |
| `commands/version-bump-cc.md` | `/version-bump-cc` — bump version + auto-changelog |
| `commands/generate-release-cc.md` | `/generate-release-cc` — full release workflow |
| `commands/deploy-vps-{local,akamai,both}-cc.md` | Deploy to VPS |
| `commands/capture-release-evidences-cc.md` | Browser-record new features as WebP |
| `commands/review-{prs,discussions}-cc.md` | Triage GitHub PRs/discussions |
| `commands/{review-issues,implement-features}-cc.md` | Issue workflows |
| `settings.local.json` | Per-project Claude Code settings |
---
## `.agents/` — Generic Agent Workflows (Codex / Cursor / etc.)
| Path | Purpose |
| ------------------------ | ------------------------------------------------------- |
| `workflows/*-ag.md` | 11 workflow definitions (mirror of `.claude/commands/`) |
| `skills/<name>/SKILL.md` | 9 skill definitions with Codex Execution Notes |
> **Note:** Workflows and commands are currently identical byte-by-byte. If `.agents/` is meant to target a different agent runtime (Codex), the variants need to diverge meaningfully.
---
## `_ideia/`, `_mono_repo/`, `_references/`, `_tasks/` — Out-of-tree
These underscore-prefixed directories hold non-shipping content:
- **`_ideia/`** — design notes (defer / notfit / viable categories)
- **`_mono_repo/`** — historic subprojects (omnirouteCloud, omnirouteSite, vscode-extension)
- **`_references/`** — read-only clones of related OSS projects (LiteLLM, 9router, ClawRouter, CLIProxyAPI, modelrelay, new-api, etc.) for cross-reference during development
- **`_tasks/`** — per-release task tracking files (informal)
Not included in `npm pack` output. See `.npmignore`.
---
## Generated / Gitignored
| Path | Purpose |
| ---------------------- | ----------------------------- |
| `node_modules/` | npm dependencies |
| `.next/` | Next.js build output |
| `coverage/` | c8 coverage reports |
| `logs/` | Runtime logs |
| `package/` | npm pack staging |
| `.playwright-mcp/` | Playwright MCP test artifacts |
| `.issues/` | Local issue cache |
| `tsconfig.tsbuildinfo` | TS incremental cache |
---
## Navigation tips
- **New contributor?** Read `CONTRIBUTING.md` → `CLAUDE.md` → `docs/architecture/ARCHITECTURE.md` → `docs/architecture/CODEBASE_DOCUMENTATION.md`.
- **Adding a provider?** Follow `docs/architecture/ARCHITECTURE.md § Adding a New Provider` + cross-check `docs/reference/PROVIDER_REFERENCE.md`.
- **Adding a route?** `docs/architecture/ARCHITECTURE.md § Adding a New API Route` + `src/shared/validation/schemas.ts`.
- **Adding an MCP tool?** `docs/frameworks/MCP-SERVER.md § Adding a Tool`.
- **Adding an A2A skill?** `docs/frameworks/A2A-SERVER.md § Adding a New Skill`.
- **Running locally?** `docs/guides/SETUP_GUIDE.md`.
- **Deploying?** `docs/guides/DOCKER_GUIDE.md` / `docs/ops/VM_DEPLOYMENT_GUIDE.md` / `docs/ops/FLY_IO_DEPLOYMENT_GUIDE.md`.
- **Releasing?** `docs/ops/RELEASE_CHECKLIST.md` (and `/generate-release-cc` Claude Code skill).
docs/architecture/RESILIENCE_GUIDE.md
+248
View File
@@ -0,0 +1,248 @@
---
title: "Resilience Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Resilience Guide
OmniRoute has three distinct but related resilience mechanisms. Each has a different scope and purpose. Keep them separate when debugging routing behavior.
![3-layer resilience model](../diagrams/exported/resilience-3layers.svg)
> Source: [diagrams/resilience-3layers.mmd](../diagrams/resilience-3layers.mmd)
## 1. Provider Circuit Breaker
**Scope:** entire provider (e.g., `glm`, `openai`, `anthropic`).
**Purpose:** stop sending traffic to a provider that is repeatedly failing at the upstream/service level.
**Implementation:**
- Core class: `src/shared/utils/circuitBreaker.ts`
- Wiring: `src/sse/handlers/chatHelpers.ts`, `src/sse/handlers/chat.ts`
- Status API: `GET /api/monitoring/health`
- Reset API: `POST /api/resilience/reset`
- Wrappers: `open-sse/services/accountFallback.ts`
- DB table: `domain_circuit_breakers`
**States:**
- `CLOSED` — normal traffic allowed
- `DEGRADED` — traffic still allowed, but elevated provider failures are being tracked
- `OPEN` — provider temporarily blocked; combo routing skips it
- `HALF_OPEN` — reset timeout elapsed; probe request allowed
**Configurable defaults (`open-sse/config/constants.ts`, exposed in Dashboard → Settings → Resilience):**
| Class | Degraded at | Opens at | Reset timeout |
| ------- | ----------- | ----------- | ------------- |
| OAuth | 5 failures | 8 failures | 60s |
| API-key | 7 failures | 12 failures | 30s |
| Local | derived | 2 failures | 15s |
`degradationThreshold` controls when a provider enters `DEGRADED`; `failureThreshold` controls when it opens and is skipped. Local provider profiles are not exposed on the Resilience settings page yet.
**Trip codes:** only provider-level statuses `[408, 500, 502, 503, 504]`. Do NOT trip for account-level errors (most 401/403/429 — those belong to cooldown or lockout).
**Lazy recovery:** when `OPEN` expires, `getStatus()`, `canExecute()`, `getRetryAfterMs()` refresh state to `HALF_OPEN`. No background timer needed.
---
## 2. Connection Cooldown
**Scope:** single provider connection/account/key.
**Purpose:** skip one bad key while other connections for the same provider keep serving.
**Implementation:**
- Mark unavailable: `src/sse/services/auth.ts::markAccountUnavailable()`
- Selection: `getProviderCredentials*` in same file
- Cooldown calc: `open-sse/services/accountFallback.ts::checkFallbackError()`
- Settings: `src/lib/resilience/settings.ts`
**Fields per connection:**
- `rateLimitedUntil` — timestamp until cooldown expires
- `testStatus: "unavailable"`
- `lastError`, `lastErrorType`, `errorCode`
- `backoffLevel` — exponential backoff counter
**Default cooldowns:**
- OAuth base: 5s
- API-key base: 3s
- API-key 429: prefers upstream `Retry-After`/reset headers/parseable reset text
- Backoff: `baseCooldownMs * 2 ** failureIndex`
**Anti-thundering-herd guard:** prevents concurrent failures from over-extending cooldown or double-incrementing `backoffLevel`.
**Terminal states (NOT cooldowns):**
- `banned` — set by banned-keyword / account-ban detection (see [BAN_DETECTION](../security/BAN_DETECTION.md))
- `expired`
- `credits_exhausted`
These persist until credentials change or an operator resets them. Do not overwrite terminal states with transient cooldown state.
**Lazy recovery:** when `rateLimitedUntil` is past, connection becomes eligible again. On successful use, `clearAccountError()` clears all error fields.
---
## 3. Model Lockout
**Scope:** provider + connection + model triple.
**Purpose:** avoid disabling a whole connection when only one model is unavailable or quota-limited.
**Examples:**
- Per-model quota providers returning 429
- Local providers returning 404 for one missing model
- Provider-specific mode/model permission failures (e.g., Grok modes)
**Implementation:** `open-sse/services/accountFallback.ts` — `lockModel()`, `clearModelLock()`, `getAllModelLockouts()`.
### Model Cooldowns Dashboard (v3.8.0)
UI: Settings → Model Cooldowns (`src/app/(dashboard)/dashboard/settings/components/ModelCooldownsCard.tsx`)
Lists active lockouts with: provider, connection, model, reason, expiresAt. Operators can manually re-enable a model from the card.
**REST API:**
- `GET /api/resilience/model-cooldowns` — list active lockouts
- `DELETE /api/resilience/model-cooldowns` — manual re-enable. Body: `{provider, connection, model}`. Auth: management.
### Lockout settings UI + success-decay recovery (v3.8.23)
Model lockout went from always-on hardcoded behavior to a fully configurable,
opt-in feature with its own settings card and a self-healing recovery path.
**Settings card:** Settings → Model Lockout
(`src/app/(dashboard)/dashboard/settings/components/ModelLockoutCard.tsx`).
This is **distinct** from the read-only `ModelCooldownsCard` above (which only
_lists_ active lockouts) — the new card _configures the parameters_. Defaults
live in `DEFAULT_MODEL_LOCKOUT_SETTINGS`
(`src/lib/resilience/modelLockoutSettings.ts`):
| Setting | Default | Meaning |
| ----------------------- | -------------------------------- | -------------------------------------------------------------- |
| `enabled` | `false` | Master toggle — model lockout is **off by default**. |
| `errorCodes` | `[403, 404, 429, 502, 503, 504]` | Upstream statuses that count as a model-scoped failure. |
| `baseCooldownMs` | `120_000` (120 s) | Initial lockout duration for the first failure. |
| `maxCooldownMs` | `1_800_000` (30 min) | Cap on the escalated cooldown. |
| `maxBackoffSteps` | `10` | Max exponential-backoff escalation steps. |
| `useExponentialBackoff` | `true` | Whether repeated failures escalate the cooldown exponentially. |
Settings persist through the normal settings store and validate via the
resilience settings schema; the card clamps `baseCooldownMs`/`maxCooldownMs`
(with `maxCooldownMs ≥ baseCooldownMs`) and `maxBackoffSteps`.
**Success-decay recovery:** recovery is **not** purely timer expiry. A healthy
response walks the model's failure count back down so a model that recovered
mid-window stops escalating (and clears) before its timer would. On a successful
combo target, `open-sse/services/combo.ts` calls `decayModelFailureCount()`
(`open-sse/services/accountFallback.ts`), which **halves** the stored
`failureCount` (`Math.floor(failureCount / 2)`); when it reaches `0` the lockout
entry is deleted entirely. The counterpart `recordModelLockoutFailure()`
increments the count (and escalates the cooldown) on failures within the
escalation window. This success-decay is in addition to plain timer expiry —
either path can re-enable a model.
**State:** lockouts are held **in-memory** (per-process `Map`s of
`ModelLockoutEntry` keyed by `provider:connectionId:model`), not persisted to
the DB — they are lost on restart. The _settings_ are persisted; the active
lockout _state_ is ephemeral.
---
## 4. Quota-Share Concurrency Control (v3.8.36)
Subscription accounts (GLM, MiniMax, etc.) often accept only ~1–3 concurrent
requests; exceeding that triggers 429s and cooldowns. This is acute under
**quota-share** (`qtSd/…`) combos, where several API keys share one upstream
account. Three layers keep a shared account from being flooded.
### Per-connection concurrency cap (`max_concurrent`)
Each provider connection can declare a `max_concurrent` ceiling
(`provider_connections.max_concurrent`, set in the connection modal / API / DB).
Leave it empty for no limit. This is the single knob that drives the serialization
layer below — set it to the account's real concurrency (e.g. GLM ~1, MiniMax ~2).
### Quota-share request serialization
When a quota-share dispatch targets a connection that declares a positive
`max_concurrent`, concurrent requests to that **account** are serialized through a
per-connection semaphore (key `qsconn:<connectionId>`): excess requests **wait in
the queue** instead of flooding the account. It is **fail-open** — a saturated
queue or timeout proceeds without a slot rather than ever rejecting a dispatchable
request. Toggle in **Settings → Resilience → Quota-share per-connection
concurrency** (`resilienceSettings.quotaShareConcurrencyLimit.enabled`, default
on). Without a `max_concurrent` cap the behavior is unchanged.
> The quota-share routing gate (`selectQuotaShareTarget`, DRR + P2C) is itself
> fail-open and only _deprioritizes_ an at-cap connection — with a
> single-connection pool it cannot hard-limit, so this semaphore is what actually
> contains the flood.
### Combo cooldown-aware retry
For quota-share combos only, a request that would crystallize a 429 for a SHORT
transient cooldown waits it out and re-dispatches instead of returning the 429.
Bounded by `comboCooldownWait` (`enabled`, `maxWaitMs` 5s, `maxAttempts` 2,
`budgetMs` 8s) in **Settings → Resilience**. It never waits on `quota_exhausted`
(locked until midnight) or auth/not-found reasons.
---
## Other Resilience Features
- **18 routing strategies** (priority, weighted, round-robin, context-relay, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, fusion, pipeline) — see [AUTO-COMBO.md](../routing/AUTO-COMBO.md).
- **Reset-aware routing** (v3.8.0) — prioritizes connections by quota reset time.
- **Background mode degradation** — Responses API `background: true` degraded to sync with warning.
- **Dynamic tool limit detection** — backs off providers when tool count limits hit.
- **Emergency fallback** — controlled by `OMNIROUTE_EMERGENCY_FALLBACK`; operators can override it from the Feature Flags page without a restart.
---
## Debugging
- All keys for a provider skipped → check both circuit breaker state AND each connection's `rateLimitedUntil`/`testStatus`.
- Provider permanently excluded after reset window → code reading raw `state` instead of `getStatus()`/`canExecute()`.
- One key fails, others should work → prefer connection cooldown over circuit breaker.
- Only one model fails → prefer model lockout over connection cooldown.
- State should self-recover but doesn't → check for future timestamp + read path that refreshes expired state. Permanent statuses require manual changes.
---
## TLS Fingerprinting & Stealth
Provider-specific stealth (JA3/JA4, CCH, obfuscation) is separately documented — see [STEALTH_GUIDE.md](../security/STEALTH_GUIDE.md).
---
## Resilience testing (Phase 8 · Block C)
Beyond unit tests for resilience logic, three tests exercise the runtime under
real stress/failure conditions (all integration/nightly — none block PRs):
| Test | What | Run |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
| Chaos | Fake-upstream node injects real latency/reset/timeout/503; validates that the circuit breaker opens/recovers and `checkFallbackError` classifies 503 as recoverable fallback. | `RUN_CHAOS_INT=1 npm run test:chaos` |
| Heap-growth | ~500 streams per `createSSEStream` under `--expose-gc`; fails if the heap grows beyond the ceiling (OOM guard #3069). | `npm run test:heap` |
| k6 soak | Sustained load against `/api/monitoring/health`; p95/error thresholds. | `k6 run tests/load/k6-soak.js` (nightly) |
Orchestrated by `.github/workflows/nightly-resilience.yml` (cron + dispatch). In the
default `test:integration`, chaos and heap self-skip (without `RUN_CHAOS_INT`/`--expose-gc`).
---
## See Also
- [Architecture Guide](./ARCHITECTURE.md) — System architecture and internals
- [User Guide](../guides/USER_GUIDE.md) — Providers, combos, CLI integration
- [Auto-Combo Engine](../routing/AUTO-COMBO.md) — 12-factor scoring, mode packs
docs/architecture/ROUTER_BACKENDS.md
+149
View File
@@ -0,0 +1,149 @@
---
title: "Router Backends & Embedded Services (ADR)"
version: 3.8.43
lastUpdated: 2026-07-02
---
# Router Backends & Embedded Services — architecture contract (ADR)
> **Status:** Accepted · **Context:** [#5670](https://github.com/diegosouzapw/OmniRoute/issues/5670),
> [#5603](https://github.com/diegosouzapw/OmniRoute/issues/5603) · **Contract:** `domain/routing/routerBackends.ts`
> (typed registry — code lands with [#5868](https://github.com/diegosouzapw/OmniRoute/pull/5868))
This ADR pins down how `ts` (native), `bifrost`, `cliproxy`, `9router`, and
VibeProxy-compatible engines relate to each other, so contributors stop
conflating two things that are architecturally distinct. It documents the typed
registry introduced by the router-backend-registry work as the single source of
truth for that model.
## The core distinction — two orthogonal axes
An engine's role is described by **two independent axes**, encoded together in the
registry's `RouterBackendDefinition`:
1. **Lifecycle** (`RouterBackendLifecycle`) — _how the engine runs_:
- `in-process` — runs inside the OmniRoute Node process (the native TS pipeline).
- `supervised` — a local child process OmniRoute installs/starts/stops/health-checks
via `ServiceSupervisor`, then consumes as a provider connection.
- `external` — an HTTP endpoint OmniRoute dispatches to but does **not** manage
(configured by an env base URL).
- `disabled` — registered but not selectable.
2. **Selection axis** (relay routing backend) — _whether the relay dispatches to it_:
`RelayRoutingBackend = "ts" | "bifrost" | "auto"` in
`src/app/api/v1/relay/chat/completions/routingBackend.ts`.
The mistake to avoid: treating "embedded service" and "routing backend" as one
list. They are not. A `supervised` engine (9router/cliproxy) is a **provider
connection consumed by the native pipeline**, not an alternate relay dispatch
backend. `bifrost` is the reverse — a relay dispatch backend that (historically)
was `external`-only.
## The registry — single source of truth
The `domain/routing/routerBackends.ts` contract (code lands with
[#5868](https://github.com/diegosouzapw/OmniRoute/pull/5868)) declares every engine once, with its
lifecycle, capabilities, service identity, default port, health config, and
telemetry support. Consumers look engines up via `getRouterBackend(id)`,
`listRouterBackends()`, and `listRouterBackendsByCapability(cap)` instead of
special-casing each sidecar.
| Backend | Lifecycle | Service (axis A) | Relay backend (axis B) | Health | Default port |
| ----------- | ------------ | ---------------- | ---------------------- | ------------- | ------------ |
| `ts` | `in-process` | — | `ts` (native) | — | — |
| `bifrost` | `external`¹ | —¹ | `bifrost` / `auto` | `/health` | — |
| `cliproxy` | `supervised` | `cliproxy` | — (provider) | `/v1/models` | 8317 |
| `9router` | `supervised` | `9router` | — (provider) | `/api/health` | 20130 |
| `vibeproxy` | `external` | — | — (provider adapter) | `/v1/models` | — |
¹ Bifrost's promotion to a `supervised` embedded service (installable/startable
from `/api/services/bifrost/`) is tracked in
[#5817](https://github.com/diegosouzapw/OmniRoute/pull/5817); until it merges,
Bifrost is `external`-only (reachable solely via `BIFROST_BASE_URL`).
`capabilities` (`chat`, `responses`, `streaming`, `tools`, `vision`,
`oauth-backed`, `dashboard-embed`, `model-sync`, `native-hot-path`) let callers
filter by what an engine can actually do rather than hard-coding per-id branches.
## Axis A — embedded services (supervised process side)
- **Registry of supervised processes:** `src/lib/services/bootstrap.ts` `SERVICES[]`
(today: `9router`, `cliproxy`).
- **Lifecycle owner:** `src/lib/services/ServiceSupervisor.ts` — `start()` spawns the
child, gates on `waitForHealthy()`, taps stdout/stderr into a ring buffer;
`stop()` SIGTERM→SIGKILL; all serialized under a lock.
- **State union** (`src/lib/services/types.ts`):
`not_installed | stopped | starting | running | stopping | error`, plus an
orthogonal `HealthState = healthy | unhealthy | unknown`.
- **Why a separate process (not an in-proc SDK)?** Process isolation is what makes
install/start/stop/health/logs independently controllable per sidecar and lets the
loopback spawn-guard apply. Modeling an in-proc adapter is future work — the
`native-hot-path` capability flag is where that would be expressed.
### Lifecycle route contract (`/api/services/<tool>/…`)
Status codes are **state/verb/path-specific by design** — this is the contract, not
inconsistency:
| Call | Condition | Status |
| ---------------------------- | ------------------------------- | ------------------------------------ |
| `POST .../start` | service `not_installed` | **409** (precondition) |
| `POST .../stop` | already stopped | **200** (idempotent no-op) |
| `GET .../status` | OK | **200** (`live ?? row ?? "unknown"`) |
| `POST .../start` | spawn failure | **503** (transient) |
| `GET .../status`, `.../stop` | uncaught error | **500** |
| `GET /api/services/<x>/logs` | unknown tool `<x>` | **404** `Service '<x>' not found` |
| `GET .../status?reveal=key` | missing `X-Reveal-Confirm: yes` | **403** (9router only) |
| **any** `/api/services/*` | caller not loopback/private-LAN | **403 LOCAL_ONLY** |
All error bodies are shaped by `createErrorResponse()` →
`{ error: { message, type }, requestId }`, where `type` is derived from the status
(`500→server_error`, `404→not_found`, `409→conflict`, else `invalid_request`) and is
the machine-actionable discriminator. Messages are pre-sanitized
(`sanitizeErrorMessage()`, Hard Rule #12).
**The loopback guard** is the most common source of a `403`: `/api/services/` is in
`LOCAL_ONLY_API_PREFIXES` (`src/server/authz/routeGuard.ts`) and
`src/server/authz/policies/management.ts` rejects any non-loopback / non-private-LAN
caller **before auth**, because these routes spawn child processes (Hard Rules 15
and 17). Reaching them through a public tunnel is `403` by design.
## Axis B — relay routing backend (dispatch side)
Only the relay proxy path `/api/v1/relay/chat/completions` selects a dispatch
backend; the main `/api/v1/chat/completions` surface never consults
`routingBackend.ts`.
- **Selection** (`resolveRelayRoutingBackend`): a single global env toggle —
`OMNIROUTE_RELAY_BACKEND` / `RELAY_ROUTING_BACKEND` ∈ {`ts`, `bifrost`, `auto`}.
If unset, `auto` when Bifrost is configured+enabled, else `ts`.
- **Behavior:**
- `bifrost` (forced): Bifrost failure → hard `502`, no fallback.
- `auto`: try Bifrost, on failure/cooldown silently fall through to native.
- `ts` / post-fallback: the native `open-sse` translator/executor pipeline.
- **Cooldown:** per-`baseUrl` failure cooldown in `bifrostCooldown.ts`.
Selection is **all-or-nothing at the relay level today** — there is no per-provider
or per-request engine swap on `release/v3.8.43`. The per-request gate is being added
by the sidecar-manifest work
([#5869](https://github.com/diegosouzapw/OmniRoute/pull/5869) manifest +
[#5870](https://github.com/diegosouzapw/OmniRoute/pull/5870) `shouldTryBifrostForRequest`),
which lets `auto` route only manifest-eligible providers through Bifrost.
## Dashboard integration
The services dashboard polls `GET /api/services/<tool>/status` every 5s via
`src/app/(dashboard)/dashboard/providers/services/hooks/useServiceStatus.ts`,
returning `{ tool, state, pid, port, health, installedVersion, latestVersion,
updateAvailable, autoStart, … }`. There is no shared availability-context provider —
each component calls the hook per tool. On `!res.ok` the hook currently surfaces a
bare `HTTP <status>`; mapping the `error.type` field to a human explanation is a
tracked UX improvement, not a contract change.
## Consequences
- New engines register once in `ROUTER_BACKENDS`; consumers gain them via capability
queries without new per-id branches.
- "Is this a service or a routing backend?" is answered by the `lifecycle` field, not
by which list an id happens to appear in.
- The Bifrost supervision (#5817) and native hot-path migration (#5670) build on this
shared contract instead of special-casing each sidecar.
docs/architecture/cluster-decisions.md
+101
View File
@@ -0,0 +1,101 @@
---
title: "Cluster Decisions"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Cluster Decisions — Optional Sidecar Profiles
**Status:** proposal (awaiting @diegosouzapw review)
**Date:** 2026-06-20
**Refs:** [#3932](https://github.com/diegosouzapw/OmniRoute/issues/3932), PR #4381
## TL;DR
Two opt-in compose profiles (`memory`, `bifrost`) for the existing 8-service deployment in [`docker-compose.yml`](../../docker-compose.yml). Default-up behaviour is **unchanged**: 3 × `omniroute` replicas + Caddy + Redis + CliproxyAPI. The two new profiles add Qdrant and Bifrost as optional sidecars, gated by `docker compose --profile <name> up`. **No existing service is removed or replaced.**
## Why this is conservative
OmniRoute's existing deployment shape is already lean and proven:
- **`redis:7-alpine`** handles the rate-limit/cache workload at production scale.
- **SQLite + sqlite-vec + FTS5** cover local memory + vector + text-search (see [`src/lib/memory/vectorStore.ts:108`](../../src/lib/memory/vectorStore.ts)).
- **Caddy** is already the LB + TLS terminator ([`docker-compose.yml`](../../docker-compose.yml)).
- **Bifrost** is already integrated as the Tier-1 router in [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (sidecar proxy with kill switch via `BIFROST_ENABLED` env var — set `=0` to bypass the sidecar and fall through to the TS path).
The two profiles here are **scale-out options for deployments that hit the SQLite ceiling** — not migrations. Both are default-off.
## The two profiles
### `memory` — Qdrant Vector Memory Sidecar
**When to flip on:**
- > 1M embeddings per deployment (sqlite-vec starts to slow at scale).
- Multi-replica deployment that needs shared vector state across `omniroute-1/2/3`.
- You already have an external Qdrant cluster (Qdrant Cloud, on-prem).
**What it adds:**
| Service | Image | Ports | Notes |
| -------- | ----------------------- | ----------- | ----------------------------------------------------- |
| `qdrant` | `qdrant/qdrant:v1.12.4` | `6333` HTTP | HNSW index; persistent volume `omniroute_qdrant_data` |
**Activation:** flip `qdrantEnabled = true` in the Settings UI **or** set `QDRANT_HOST=qdrant` env. See [`src/lib/memory/qdrant.ts:60`](../../src/lib/memory/qdrant.ts) for the precedence rules (settings table → env var → default).
**Env vars:** `QDRANT_HOST`, `QDRANT_PORT`, `QDRANT_API_KEY`, `QDRANT_COLLECTION`, `QDRANT_VECTOR_SIZE`, `QDRANT_HNSW_EF_CONSTRUCT` (see `.env.example` lines 1672-1683).
### `bifrost` — Bifrost Tier-1 Router Sidecar
**When to flip on:**
- You run ≥3 `omniroute` replicas and want provider rotation centralised in a single Go process.
- You want a single audit/logging surface for upstream-provider requests across all replicas.
- You want horizontal scaling of the Tier-1 routing layer independent of the OmniRoute replicas.
**What it adds:**
| Service | Image | Ports | Notes |
| --------- | -------------------------------- | ------ | ----------------------------------------------------------------------- |
| `bifrost` | `ghcr.io/maximhq/bifrost:1.5.21` | `8080` | Go-based Tier-1 router; persistent logs volume `omniroute_bifrost_logs` |
**Activation:** set `BIFROST_BASE_URL=http://bifrost:8080` in `.env.example`. The existing sidecar proxy route at [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (added in PR #4381) will pick this up automatically.
**Env vars:** `BIFROST_BASE_URL`, `BIFROST_API_KEY`, `BIFROST_STREAMING_ENABLED`, `BIFROST_TIMEOUT_MS` (see `.env.example` lines 1685-1695).
## What this PR explicitly does NOT do
The original issue thread floated a larger cluster rewrite. After auditing the actual workload shape, the following are **rejected** for the reasons given:
| Component | Verdict | Reason |
| ------------------------------------ | -------- | ------------------------------------------------------------------------------------------------------ |
| **Dragonfly** | **DROP** | `redis:7-alpine` is already fine for the rate-limit workload at production scale; no ceiling to break. |
| **NATS** | **DROP** | Each `omniroute` replica is a single Node.js process; no multi-process pub/sub workload exists. |
| **PostgreSQL** | **DROP** | SQLite + sqlite-vec + FTS5 cover all 3 use cases; 97 migrations + Electron packaging block migration. |
| **Neo4j** | **DROP** | Routing is a 5-table join; recursive CTE on SQLite is sufficient. |
| **MinIO** | **DROP** | No multi-MB blob workload; images/audio are passthrough proxies. |
| **pgvector / pg_ai / pg_textsearch** | **DROP** | Same SQLite-ceiling reason as PostgreSQL; pgvector ecosystem fragmented. |
| **HAProxy / Envoy** | **DROP** | Caddy already does LB + TLS; both were explicitly rejected as Tier-1 routers (see `AGENTS.md`). |
If a future use case proves out one of these, this doc is the place to amend.
## 4-week rollout (if approved)
1. **Wk 1** — Land this PR + verification of opt-in profiles with a 3-replica compose stack.
2. **Wk 2** — Bifrost full activation for OpenAI/Claude/Gemini/Ollama (4 of 14+ providers) using the sidecar proxy route at [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (gated by `BIFROST_ENABLED`, kill-switchable at runtime).
3. **Wk 3** — Qdrant memory profile enabled in a single test deployment; measure latency delta vs sqlite-vec.
4. **Wk 4** — Observability healthchecks (`docker compose ps` exit codes + `wget` smoke tests); 71-pillar refresh per ADR-041.
## Files changed in this PR
| File | Change |
| ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `docker-compose.yml` | +30 lines: `memory` profile (Qdrant), `bifrost` profile (Bifrost), persistent volumes, healthchecks. |
| `.env.example` | +24 lines: `QDRANT_*` (6 vars), `BIFROST_*` (4 vars). |
| `docs/reference/ENVIRONMENT.md` | +6 rows in section 25 for the `QDRANT_*` env vars. |
| `src/lib/memory/qdrant.ts` | +33 lines: env-var fallback chain (settings → env → default) for `QDRANT_HOST`/`QDRANT_PORT`/`QDRANT_API_KEY`/`QDRANT_COLLECTION`/`QDRANT_VECTOR_SIZE`/`QDRANT_HNSW_EF_CONSTRUCT`/`QDRANT_EMBEDDING_MODEL`. |
| `src/lib/memory/__tests__/qdrant-wiring.test.ts` | +88 lines: 9 new test cases pinning the env-var fallback precedence. |
| `docs/architecture/cluster-decisions.md` (this file) | NEW — decision record for the opt-in profiles. |
| `AGENTS.md` | +1 line: pointer to this doc in the reference documentation table. |
**Net touched code:** 4 production files (`docker-compose.yml`, `qdrant.ts`, `.env.example`, `ENVIRONMENT.md`), 1 test file (`qdrant-wiring.test.ts`), 2 doc files (`cluster-decisions.md`, `AGENTS.md`).
docs/architecture/meta.json
+11
View File
@@ -0,0 +1,11 @@
{
"title": "Architecture",
"pages": [
"ARCHITECTURE",
"AUTHZ_GUIDE",
"CODEBASE_DOCUMENTATION",
"REPOSITORY_MAP",
"RESILIENCE_GUIDE",
"QUALITY_GATES"
]
}
docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md
+70
View File
@@ -0,0 +1,70 @@
---
title: "OmniRoute vs Alternatives"
version: 3.8.43
lastUpdated: 2026-07-01
---
# OmniRoute vs Alternatives
Objective feature comparison vs popular open-source AI routers.
> **Methodology**: Public repos audited 2026-Q2. Versions as listed.
> Submit corrections via PR — we want this to be accurate.
| Feature | OmniRoute 3.8 | LiteLLM 1.x | OpenRouter (SaaS) | Portkey |
| -------------------------------------------------- | :----------------------------------------------: | :------------: | :---------------: | :---------: |
| **Providers** | **237+** | ~100 | ~50 | ~30 |
| **Free-tier providers** | **90+** | n/a | passthrough | n/a |
| **Self-hostable** | ✅ | ✅ | ❌ | ⚠ paid |
| **OAuth providers (Claude, Codex, Copilot, etc.)** | **15+** | partial | ❌ | ❌ |
| **Auto-fallback combos** | **17 strategies** | priority-based | tier-based | weighted |
| **Fusion (parallel panel + judge synthesis)** | ✅ | ❌ | ❌ | ❌ |
| **Tier 1/2/3 fallback (subscription→cheap→free)** | ✅ + UI | manual | n/a | manual |
| **Token compression** | 10-engine pipeline (RTK + Caveman + LLMLingua-2) | none | none | none |
| **Multimodal generation (speech/music/video)** | ✅ | ❌ | passthrough | ❌ |
| **Built-in MCP server** | ✅ 95 tools, 30 scopes | ❌ | ❌ | ❌ |
| **A2A protocol** | ✅ 6 skills | ❌ | ❌ | ❌ |
| **Memory (FTS5 + vector)** | ✅ | ❌ | ❌ | ❌ |
| **Guardrails (PII, injection, vision)** | ✅ | partial | ❌ | ✅ paid |
| **Cloud agent integrations** | Codex, Cursor, Devin, Jules | ❌ | ❌ | ❌ |
| **Circuit breaker per provider** | ✅ 3-state, lazy recovery | basic | ❌ | ✅ |
| **TLS fingerprint stealth (JA3/JA4)** | ✅ wreq-js | ❌ | ❌ | ❌ |
| **Eval framework** | ✅ built-in | ❌ | ❌ | ⚠ paid |
| **MITM proxy (intercepts Cursor/Antigravity)** | ✅ cross-platform | ❌ | ❌ | ❌ |
| **CLI with system tray (no Electron)** | ✅ | ❌ | n/a | n/a |
| **CLI machine-ID auto-auth** | ✅ | ❌ | n/a | n/a |
| **Dashboard** | Next.js 16 | basic | proprietary | proprietary |
| **i18n** | **42+ locales** | ❌ | ❌ | ⚠ |
| **Public agent skills (SKILL.md)** | ✅ 43 | ❌ | ❌ | ❌ |
| **Tunnel support (Cloudflared, Tailscale, Ngrok)** | ✅ | ❌ | n/a | n/a |
| **License** | MIT | MIT | proprietary | proprietary |
## When to choose OmniRoute
- You self-host and want **maximum provider coverage** (237+, 90+ with a free tier)
- You need a **built-in MCP server** (LLM tools, memory, skills exposed as tools)
- You need **A2A protocol** for agent-to-agent workflows
- You want **fingerprint stealth** (JA3/JA4) to avoid detection by upstream CAPTCHAs
- You need **enterprise features** (guardrails, evals, audit trail) without a SaaS bill
## When to choose LiteLLM
- You're **Python-first** and need tight integration with `litellm.completion()`
- You need **mature production deployment recipes** (k8s, Helm charts)
- Your team already runs Python microservices
## When to choose OpenRouter (SaaS)
- You don't want to self-host
- You're fine paying per-token at SaaS markup
- You need a **single payment method** across all providers
## When to choose Portkey
- You need a **commercial SLA** with uptime guarantees
- You prefer a **managed dashboard** without ops overhead
- You need **enterprise compliance** features out of the box
---
_Last updated: 2026-06-28. Submit corrections via PR to keep this table accurate._
docs/comparison/meta.json
+5
View File
@@ -0,0 +1,5 @@
{
"title": "Comparison",
"description": "How OmniRoute compares to alternatives",
"pages": ["OMNIROUTE_VS_ALTERNATIVES"]
}
docs/compression/COMPRESSION_ENGINES.md
+330
View File
@@ -0,0 +1,330 @@
---
title: "Compression Engines"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Compression Engines
OmniRoute compression is built around engine contracts. A mode can run one engine directly
(`caveman` or `rtk`) or a deterministic stacked pipeline that executes multiple engines in order.
## Modes
| Mode | Engine path | Intended input |
| ------------ | ---------------------------------- | -------------------------------------------- |
| `off` | none | Exact prompt preservation |
| `lite` | Caveman lite helpers | Low-risk always-on cleanup |
| `standard` | Caveman | Natural-language prompt condensation |
| `aggressive` | Caveman + history/tool summarizers | Long chat sessions |
| `ultra` | Caveman + pruning helpers | Context-limit recovery |
| `rtk` | RTK | Terminal, shell, build, test, and git output |
| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose, max savings |
## Engine Registry
The registry lives in `open-sse/services/compression/engines/registry.ts`. Engines expose a shared
contract:
- `id`: stable engine id such as `caveman` or `rtk`
- `apply(text, config)`: legacy execution path used by stacked pipelines
- `compress(input, config)`: primary execution path returning text + stats
- `getConfigSchema()`: returns the JSON-Schema-like shape of valid config
- `validateConfig(config)`: returns `{ valid, errors[] }`
Registration uses `registerCompressionEngine(engine)` (or `registerEngine` for advanced cases),
which calls `assertValidEngine()` and `validateConfig(defaultConfig)` before accepting.
Use `unregisterCompressionEngine(id)` to remove an engine at runtime.
`strategySelector.ts` registers the built-in engines before compression runs. This lets preview,
runtime compression, stacked mode, tests, and future engines use the same execution path.
### MCP description compression (related)
A separate registry compresses MCP tool description metadata at registry-level — see
`open-sse/mcp-server/descriptionCompressor.ts` and [MCP-SERVER.md](../frameworks/MCP-SERVER.md). It reuses
Caveman rules but operates on tool metadata, not request payloads.
### Additional built-in engines
Beyond Caveman, RTK, and LLMLingua-2, the registry ships several specialized lossless /
structural engines (used by stacked pipelines, the playground, and tests):
| Engine | Id | What it does |
| ------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CCR | `ccr` | Content-Compress-Retrieve (H4): replaces large contiguous text blocks with content-addressed references, so repeated/large blocks are sent once and referenced thereafter. |
| headroom | `headroom` | SmartCrusher (H3 + N5): lossless tabular compaction of homogeneous JSON-array payloads into a columnar `[N rows]` form. |
| ionizer | `ionizer` | Head/middle/tail row sampling for very large homogeneous blocks, storing the elided middle as a CCR content-addressed reference. |
| session-dedup | `session-dedup` | Content-addressed cross-turn deduplication (TokenMizer-inspired): elides text already seen in earlier turns of the same session. |
## Caveman
Caveman mode focuses on semantic condensation of normal prose:
- preserves code blocks, URLs, JSON, paths, and structured data
- removes filler, hedging, repeated context, and verbose connective phrasing
- supports language-aware file rule packs in `open-sse/services/compression/rules/`
- remains available through the legacy `standard`, `aggressive`, and `ultra` modes
The dashboard surface is `Dashboard -> Context & Cache -> Caveman`.
Caveman upstream reports `~75%` fewer output tokens, `65%` average output savings in benchmarks
with a `22-87%` range, and a `~46%` input-compression tool. OmniRoute uses the Caveman input-side
number when documenting stacked prompt/context savings; Caveman output mode remains a separate
response-behavior feature.
## RTK
RTK mode focuses on command and tool output:
- detects output classes such as `git status`, `git branch`, `git diff`, Vitest/Jest/Pytest,
Cargo/Go tests, TypeScript/Vite/Webpack builds, ESLint, npm audit/installs, Docker logs,
shell `find`/`grep`, stack traces, and generic logs
- applies 49 JSON filters from `open-sse/services/compression/engines/rtk/filters/`
- supports the RTK-style declarative pipeline: ANSI stripping, replace, match-output short-circuit,
strip/keep lines, per-line truncation, head/tail/max-line truncation, and on-empty fallback
- supports trust-gated project filters in `.rtk/filters.json` and global filters in
`DATA_DIR/rtk/filters.json`
- strips ANSI sequences, progress noise, repeated lines, and unhelpful boilerplate
- preserves actionable failures, warnings, summaries, changed files, and tail context
- can optionally retain redacted raw output for recovery/debugging through authenticated management
routes
The dashboard surface is `Dashboard -> Context & Cache -> RTK`.
Operational details for custom filters, trust, verify, and raw-output recovery live in
[`RTK_COMPRESSION.md`](./RTK_COMPRESSION.md).
RTK upstream reports `60-90%` savings for command-output compression. Its README example shows a
30-minute Claude Code session going from `~118,000` tokens to `~23,900`, or `79.7%` saved.
## LLMLingua-2 (Semantic Pruning)
LLMLingua-2 mode performs **semantic token pruning** on prose using a small ONNX token
classifier, complementing the rule-based Caveman and RTK engines:
- compresses prose in non-system messages only; fenced code blocks and other preserved
constructs are never altered
- runs the `@atjsh/llmlingua-2` backend (ONNX via `@huggingface/transformers`) in a
worker thread, so model inference never blocks the request event loop
- is **stackable** (`stackPriority` 35): in a stacked pipeline it runs after the
structural engines (CCR, session-dedup, headroom, Caveman) but before `ultra`, since
semantic pruning is most effective on already-structurally-compressed text — e.g.
`rtk -> caveman -> llmlingua`
- **fail-opens on any error** (missing optional deps, worker spawn, model load, inference,
or timeout) → the original text is returned unchanged, never an error
Engine location: `open-sse/services/compression/engines/llmlingua/`. The dashboard surface
is `Dashboard -> Context & Cache -> LLMLingua`.
### Models
The default model is **TinyBERT** (`atjsh/llmlingua-2-js-tinybert-meetingbank`, ~57 MB,
fast). A higher-accuracy **BERT-base** model (`Arcoldd/llmlingua4j-bert-base-onnx`,
~710 MB) is available via the engine config `model` field. `@huggingface/transformers`
downloads the selected model lazily from the HuggingFace Hub into
`${DATA_DIR}/models/llmlingua` on the first call (`modelStore.ts`); a `modelPath` config
override points it at a local copy instead (offline / air-gapped installs).
### Optional dependencies & on-demand install
The prunable LLMLingua runtime peer stack is **optional**. Three packages are declared as
`optionalDependencies` in `package.json` and kept **external** by the production build
(`scripts/build/prepublish.ts` does not bundle them):
| Package | Version (pin) | Notes |
| -------------------- | ------------- | ---------------------------------------------- |
| `@atjsh/llmlingua-2` | `2.0.3` | Entry package; declares the others as peers |
| `@tensorflow/tfjs` | `4.22.0` | Heaviest dep — dominates the ~800 MB footprint |
| `js-tiktoken` | `^1.0.20` | Tokenizer |
`@huggingface/transformers` is pinned at `3.5.2` as an **optional** dependency (shared with
the local embeddings path and also traced into the standalone bundle). Keeping it optional prevents
`onnxruntime-node` CUDA provider postinstall failures on CUDA 11 hosts from aborting the whole
OmniRoute install; when the optional stack is absent, LLMLingua still fail-opens. Only the three
packages above are prunable SLM peers. A standard `npm install` (dev) installs the optional stack
automatically unless optional dependencies are omitted.
**Why on-demand:** the npm-published package, the standalone bundle, and the Docker image
ship **without** these deps to stay slim. When they are absent, the worker's dependency
gate (a `@atjsh/llmlingua-2` resolve probe in `worker.ts`) fails and the engine
**fail-opens silently** — selecting LLMLingua becomes a no-op (text returned unchanged, no
error logged). To activate it in a pruned environment, install the optional stack:
```bash
# pin to the versions declared in package.json optionalDependencies
npm install @atjsh/llmlingua-2@2.0.3 @tensorflow/tfjs@4.22.0 js-tiktoken
```
Roughly **~800 MB** total: the TensorFlow.js + transformers runtimes dominate; the
TinyBERT model adds ~57 MB downloaded at first use (not via npm).
Per environment:
- **Dev / `npm install`** — installed automatically unless you passed `--omit=optional`
(or `--no-optional`). No action needed.
- **Global npm (`npm i -g omniroute`) / standalone** — run the install command above inside
the installed package directory, or reinstall without omitting optional deps.
- **Docker** — add the install command in a derived image layer; the published image
ships slim by design.
- **VPS (PM2)** — install into the app's `node_modules`, then restart the process so the
worker re-probes the gate.
**Verify it is active:** with LLMLingua selected, real prose actually shrinks (the engine
stops fail-opening), and the first request triggers the model download into
`${DATA_DIR}/models/llmlingua`. The gate intentionally probes only `@atjsh/llmlingua-2` —
the other peers are ESM-only and `require.resolve` throws on them even when present — so
the worker still fail-opens if any peer is genuinely missing at `import()` time.
## Stacked Pipelines
Stacked mode runs pipeline steps in order. The default is:
```txt
rtk -> caveman
```
Use this for coding-agent sessions where a prompt combines command output with human or assistant
prose. RTK reduces noisy tool logs first, then Caveman compresses remaining natural language.
Pipeline steps are configured with `stackedPipeline` in compression settings or through compression
combos.
When both engines reduce the same eligible payload, savings compound:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## MCP Accessibility Tree Filter
The MCP accessibility-tree smart filter is a post-execution compression layer that runs on MCP
**tool results**, not on prompts or context. It targets the verbose accessibility-tree and browser
snapshot payloads returned by tools like Playwright, computer-use, and browser-automation MCP
servers.
### What it does
1. **Noise stripping** — removes empty generic/text entries (`- generic:`, `- text: ""`)
2. **Sibling collapse** — when ≥ `collapseThreshold` (default 30) consecutive lines are structural
repeats, collapses them into the first `collapseKeepHead` (default 10) lines + a count summary +
the last `collapseKeepTail` (default 5) lines
3. **Ref preservation** — `[ref=eXX]` anchors required by Playwright/computer-use are never touched
4. **Hard truncation** — if the text after collapse still exceeds `maxTextChars` (default 50,000),
truncates with a navigation hint so the agent can continue working
### Engine location
```txt
open-sse/services/compression/engines/mcpAccessibility/
index.ts ← smartFilterText() entry point
collapseRepeated.ts ← sibling-collapse algorithm
constants.ts ← DEFAULT_MCP_ACCESSIBILITY_CONFIG
```
### Configuration
Controlled by `compression.mcpAccessibility` in global settings (migration 056). Default config:
```json
{
"enabled": true,
"maxTextChars": 50000,
"collapseThreshold": 30,
"collapseKeepHead": 10,
"collapseKeepTail": 5,
"minLengthToProcess": 2000
}
```
The filter is only applied to tool-result payloads whose `type` is `"text"` and whose length
exceeds `minLengthToProcess`. It does not affect prompt compression or request payloads.
### Expected savings
60–80% on browser snapshot tool results, depending on page complexity. The collapse algorithm
is O(n) in line count and adds negligible latency.
### This filter vs the compression engines above
| Aspect | Caveman / RTK / Stacked | MCP accessibility filter |
| ----------- | ------------------------- | -------------------------------------- |
| Target | Request prompts / context | MCP tool results |
| Trigger | Compression mode setting | `compression.mcpAccessibility.enabled` |
| Scope | All SSE messages | Tool results only |
| Ref anchors | N/A | Preserved unconditionally |
---
## Compression Combos
Compression combos are named compression profiles that can be assigned to routing combos:
- `compression_combos`: stores mode, pipeline, RTK config, language config, and default marker
- `compression_combo_assignments`: maps a compression combo to a routing combo
- runtime integration resolves an assigned compression combo before generic combo overrides
- analytics include `compression_combo_id` and `engine`
Dashboard surface: `Dashboard -> Context & Cache -> Compression Combos`.
## API Surface
| Route | Purpose |
| -------------------------------------- | ---------------------------------------------------------------- |
| `/api/settings/compression` | Global compression settings (includes `mcpAccessibility` config) |
| `/api/compression/preview` | Preview any compression mode |
| `/api/compression/language-packs` | List available Caveman language packs |
| `/api/context/caveman/config` | Caveman settings alias |
| `/api/context/rtk/config` | RTK defaults and settings |
| `/api/context/rtk/filters` | RTK filter catalog |
| `/api/context/rtk/test` | RTK preview/test endpoint |
| `/api/context/rtk/raw-output/[id]` | Authenticated redacted raw-output recovery |
| `/api/context/combos` | Compression combo CRUD |
| `/api/context/combos/[id]/assignments` | Routing-combo assignment CRUD |
| `/api/context/analytics` | Compression analytics alias |
Management routes require management authentication or API-key policy checks.
## MCP Tools
Compression exposes five MCP tools:
| Tool | Scope | Purpose |
| ----------------------------------- | ------------------- | -------------------------------- |
| `omniroute_compression_status` | `read:compression` | Settings, analytics, cache stats |
| `omniroute_compression_configure` | `write:compression` | Update global settings |
| `omniroute_set_compression_engine` | `write:compression` | Set mode and optional pipeline |
| `omniroute_list_compression_combos` | `read:compression` | List compression combos |
| `omniroute_compression_combo_stats` | `read:compression` | Read combo/engine analytics |
## Known limitations
- **LLMLingua-2 (SLM) requires co-located optional deps.** The worker only runs in a
production build when `@atjsh/llmlingua-2` + peers are co-located into
`dist/node_modules` (see `scripts/build/colocateOptionals.mjs`, #4286). Without them the
engine fail-opens (returns the original text). Worker resolution no longer depends on
`import.meta.url` (it dies in the standalone bundle) — it anchors on the runtime
cwd / `argv[1]`.
- **Caveman language packs `de` / `fr` / `ja` are partial.** They ship `context` +
`filler` + `structural` rules but no `dedup` / `ultra` packs, so `ultra` intensity is
no stronger than `full` for those languages (they use only their own rules — there is no
silent fall-back to the English `dedup`/`ultra` rules, which would mangle foreign text).
`en` / `es` / `id` / `pt-BR` are complete. Contributions of `dedup.json` + `ultra.json`
for the partial packs are welcome.
- **Stacked telemetry only lists engines that compressed.** A stacked-pipeline step whose
engine ran but produced 0 % savings returns `stats:null` and so does not appear in
`engineBreakdown` — indistinguishable from a step that was skipped. Distinguishing
"ran, 0 %" from "skipped" would require a breakdown-model change and is deferred.
## Validation
The focused gates for this area are:
```bash
node --import tsx/esm --test tests/unit/compression/rtk-*.test.ts tests/unit/compression/pipeline-integration.test.ts tests/unit/compression/context-compression-api.test.ts
node --import tsx/esm --test tests/unit/compression/*.test.ts tests/golden-set/*.test.ts tests/integration/compression-pipeline.test.ts tests/unit/api/compression/compression-api.test.ts
node --import tsx/esm --test tests/unit/compression/mcpAccessibility*.test.ts
npm run typecheck:core
```
docs/compression/COMPRESSION_GUIDE.md
+512
View File
@@ -0,0 +1,512 @@
---
title: "🗜️ Prompt Compression Guide — OmniRoute"
version: 3.8.40
lastUpdated: 2026-06-28
---
# 🗜️ Prompt Compression Guide — OmniRoute
> Save 15-95% on eligible context automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically).
## Overview
OmniRoute implements a modular prompt compression pipeline that runs **proactively** before requests hit upstream providers. This means your token savings happen transparently — no changes needed to your workflow.
```
Client Request
→ Compression Strategy Selector
→ Combo override? → Use combo setting
→ Auto-trigger threshold? → Use auto mode
→ Default mode? → Use global setting
→ Off? → Skip compression
→ Selected Compression Mode
→ Off: No compression
→ Lite: Safe whitespace/formatting cleanup (~15%)
→ Standard: Caveman-speak filler removal (~30%)
→ Aggressive: History aging + summarization (~50%)
→ Ultra: Heuristic pruning + code-block thinning (~75%)
→ RTK: Command-aware terminal/tool-output filtering (60-90% upstream range)
→ Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (78-95% eligible range)
→ Compressed Request → Provider
```
---
## Compression Modes
### Off
No compression applied. All messages pass through unchanged.
### Lite Mode (~15% savings, <1ms latency)
The safest mode — zero semantic change, only formatting cleanup:
| Technique | Description |
| ------------------------ | ------------------------------------------------- |
| `collapseWhitespace` | Merge consecutive blank lines and trailing spaces |
| `dedupSystemPrompt` | Remove duplicate system messages |
| `compressToolResults` | Compress verbose tool/function outputs |
| `removeRedundantContent` | Strip repeated instructions |
| `replaceImageUrls` | Shorten base64 image data URIs |
**Best for:** Always-on usage, safety-critical workflows.
### Standard Mode (~30% savings)
Inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — removes filler words and verbose phrasing while preserving meaning:
- Removes filler words ("please", "I think", "basically", "actually")
- Condenses verbose phrases ("in order to" → "to", "as a result of" → "because")
- Strips polite hedging ("Would you mind...", "If you could possibly...")
- 30+ regex rules tuned for coding prompts
**Best for:** Daily coding workflows, cost-conscious teams.
### Aggressive Mode (~50% savings)
Smart history management for long sessions:
- **Message Aging** — older messages get progressively compressed
- **Tool Result Summarization** — long tool outputs replaced with summaries
- **Structural Integrity Guards** — ensures `tool_use` + `tool_result` pairs stay consistent
- **Context Window Awareness** — respects per-model token limits
**Best for:** Extended debugging sessions, large codebases.
### Ultra Mode (~75% savings)
Maximum compression for token-critical scenarios:
- **Heuristic Pruning** — removes messages below relevance threshold
- **Code Block Thinning** — compresses repetitive code examples
- **Binary Search Truncation** — finds optimal cut point for context window
- All Aggressive mode features included
**Best for:** When you're hitting context limits repeatedly.
### RTK Mode (60-90% upstream range)
RTK mode is optimized for verbose tool outputs that appear in coding-agent sessions:
- Detects command/output classes such as `git status`, `git diff`, `git log`, test runners,
TypeScript/Vite/Webpack builds, ESLint/Biome/Prettier, npm audit/installs, Docker logs, infra
output, and generic shell output
- Applies JSON filter packs from `open-sse/services/compression/engines/rtk/filters/`
- Ships 49 built-in filters with inline verify samples
- Removes ANSI control sequences, progress bars, repeated lines, and non-actionable noise
- Preserves failures, errors, warnings, changed files, summaries, and the tail of long output
- Supports trust-gated project filters, global filters, and optional redacted raw-output recovery
**Best for:** Agent sessions with shell, build, test, git, grep, and file-output transcripts.
### Stacked Mode (78-95% eligible range)
Stacked mode runs multiple compression engines in a deterministic order. The default pipeline is:
```txt
RTK -> Caveman
```
That order keeps terminal/tool output compact first, then applies Caveman semantic condensation to
the remaining natural-language prompt. Stacked pipelines can be configured globally or through
compression combos assigned to routing combos.
**Best for:** Mixed context with large tool logs plus human instructions or assistant summaries.
---
## Upstream Savings Math
OmniRoute documents compression savings from two sources: upstream project benchmarks and
OmniRoute's own engine composition.
| Source | Upstream README number used here |
| ------- | --------------------------------------------------------------------------------------------------------------------- |
| Caveman | `~75%` fewer output tokens, `65%` benchmark average output savings, `22-87%` range, and `~46%` input compression tool |
| RTK | `60-90%` command-output savings; sample session `~118,000 -> ~23,900` tokens, or `79.7%` saved (`~80%`) |
For overlapping tool/context payloads, the default OmniRoute combo stacks the engines:
```txt
RTK -> Caveman
```
The combined savings are multiplicative, not additive:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
That `78-95%` number applies when both RTK and Caveman can reduce the same input/context payload.
Caveman response output mode is separate: when enabled, use Caveman's own output savings (`65%`
average, `~75%` headline, `22-87%` range). Total billing savings depend on your prompt/output mix.
---
## Token Savings Visualization
```
Without compression: 47K tokens sent to LLM
With Lite: 40K tokens sent (15% saved — safe, always-on)
With Standard: 33K tokens sent (30% saved — caveman-speak rules)
With Aggressive: 24K tokens sent (50% saved — aging + summarization)
With Ultra: 12K tokens sent (75% saved — heuristic pruning)
With RTK: 19K-5K tokens sent (60-90% saved on command/tool output)
With Stacked: 10K-2.5K tokens sent (78-95% eligible RTK+Caveman range)
```
---
## Configuration
### Dashboard
Navigate to `Dashboard → Context & Cache`:
- **Caveman** — mode selection, language packs, preview, and global defaults
- **RTK** — command-filter preview, RTK safety settings, and filter catalog
- **Compression Combos** — named engine pipelines assigned to routing combos
- **Auto-Trigger Threshold** — automatically engage compression when token count exceeds threshold
### Per-Combo Override
In `Dashboard → Context & Cache → Compression Combos`, assign a compression combo to a routing
combo:
```txt
Combo: "free-forever"
Compression Combo: "coding-agent-stack"
Pipeline: RTK -> Caveman
Targets:
1. if/kimi-k2-thinking
2. qw/qwen3-coder-plus
```
This lets you use stacked compression on free/coding providers while keeping lite mode on paid
subscriptions.
### Per-request override
Send the `x-omniroute-compression` request header to override the compression plan for a single
request. It has the highest precedence — it beats the routing-combo override, the active profile,
auto-trigger, and the panel Default. Unknown values are ignored (the request is never rejected) and
the global master switch still gates everything: when compression is off globally, the header cannot
turn it on. Values:
| Value | Effect |
| ------------- | -------------------------------------------------------------------- |
| `off` | No compression for this request. |
| `default` | The panel-derived Default profile (ignores the active profile). |
| `engine:<id>` | A single engine when enabled, e.g. `engine:rtk`. |
| `<combo>` | A named combo, matched by name (case-insensitive) first, then by id. |
The applied plan is echoed back in the `X-OmniRoute-Compression: <mode>; source=<source>` response
header, where `<source>` is one of `request-header`, `routing-override`, `active-profile`,
`auto-trigger`, `default`, or `off`.
### API
```bash
# Get compression settings
curl http://localhost:20128/api/settings/compression
# Update compression settings
curl -X PUT http://localhost:20128/api/settings/compression \
-H "Content-Type: application/json" \
-d '{"defaultMode":"stacked","autoTriggerMode":"stacked","autoTriggerTokens":32000}'
# Preview a specific RTK/stacked payload
curl -X POST http://localhost:20128/api/compression/preview \
-H "Content-Type: application/json" \
-d '{"mode":"rtk","messages":[{"role":"tool","content":"npm test output here"}]}'
# List RTK filter packs
curl http://localhost:20128/api/context/rtk/filters
# Test RTK directly with optional command metadata
curl -X POST http://localhost:20128/api/context/rtk/test \
-H "Content-Type: application/json" \
-d '{"command":"npm test","text":"FAIL tests/example.test.ts\nError: boom"}'
```
---
## What Gets Protected
The compression engine **always preserves:**
- ✅ Code blocks (fenced and inline)
- ✅ URLs and file paths
- ✅ JSON structures and structured data
- ✅ Identifiers and protected technical tokens
- ✅ Mathematical expressions
- ✅ Tool/function call definitions
- ✅ System prompts (in lite mode)
RTK raw-output recovery redacts common API keys, bearer tokens, Slack tokens, AWS access keys,
passwords, tokens, and secrets before anything is persisted.
---
## Compression Stats
Every compressed request includes stats in the server logs:
```json
{
"originalTokens": 47200,
"compressedTokens": 40120,
"savingsPercent": 15.0,
"techniquesUsed": ["collapseWhitespace", "dedupSystemPrompt"],
"mode": "lite",
"engine": "caveman",
"compressionComboId": "coding-agent-stack",
"durationMs": 0.8,
"rtkRawOutputPointers": []
}
```
---
## Phase Roadmap
| Phase | Modes | Status |
| ------- | -------------------------------------------------------------------- | ---------- |
| Phase 1 | Off, Lite | ✅ Shipped |
| Phase 2 | Standard, Aggressive, Ultra | ✅ Shipped |
| Phase 3 | RTK, Stacked, Compression Combos | ✅ Shipped |
| Phase 4 | Output Styles, SLM-tier Ultra, adaptive context-budget, eval harness | ✅ Shipped |
---
## Acknowledgments
Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project. Caveman reports `~75%` fewer output tokens, `65%` benchmark average output savings, a `22-87%` output range, and a `~46%` input-compression tool.
RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering. RTK reports `60-90%` savings, with its README sample session showing `~80%` saved.
---
## Advanced Compression Systems
Beyond the 7 standard modes, OmniRoute includes several advanced compression
systems that work automatically based on context.
### Cache-Aware Compression
Some providers (like Anthropic with prompt caching) support **prompt caching**,
which lets them cache parts of the prompt to reduce costs and latency. When
caching is enabled, aggressive compression can actually **hurt** performance
because it changes the cached tokens, invalidating the cache.
The `cachingAware.ts` module solves this by **detecting caching context** and
**adjusting the compression strategy** accordingly.
#### How it works
1. **Detect caching context** — Scans the request body for `cache_control` markers
2. **Identify caching providers** — Checks if the target provider supports caching
3. **Adjust strategy** — Downgrades `aggressive`/`ultra` to `standard` for caching providers
4. **Skip system prompt** — System prompts are usually cached, so don't compress them
5. **Use deterministic transformations** — Only use transformations that produce consistent output
#### Code example
```ts
import {
detectCachingContext,
getCacheAwareStrategy,
} from "@omniroute/open-sse/services/compression/cachingAware";
const body = {
model: "anthropic/claude-sonnet-4.5",
messages: [{ role: "user", content: "Hello" }],
cache_control: { type: "ephemeral" }, // ← Cache marker
};
const ctx = detectCachingContext(body, { provider: "anthropic" });
// → { hasCacheControl: true, provider: "anthropic", isCachingProvider: true }
const strategy = getCacheAwareStrategy("aggressive", ctx);
// → { strategy: "standard", skipSystemPrompt: true, deterministicOnly: true }
```
#### When to use
Cache-aware compression is **always on** — no configuration needed. It only kicks in
when:
- The request has `cache_control` markers
- The target provider supports prompt caching (Anthropic, OpenAI, etc.)
### Progressive Aging
Long conversations accumulate many message turns, but older turns become less
relevant. The `progressiveAging.ts` module **degrades messages by turn distance**:
- **Recent turns (0-3)**: Kept verbatim (full detail)
- **Medium turns (4-8)**: Lite compression (whitespace, formatting cleanup)
- **Old turns (9+)**: Caveman compression (filler removal, summarization)
- **Very old turns (20+)**: Heavily summarized or dropped
#### Code example
```ts
import { applyAging } from "@omniroute/open-sse/services/compression/progressiveAging";
const messages = [
{ role: "system", content: "You are a helpful assistant" },
{ role: "user", content: "What is 2+2?" },
{ role: "assistant", content: "4" },
// ... 50 more turns ...
];
const { messages: aged, saved } = applyAging(messages, {
verbatim: 3, // First 3 turns: verbatim
light: 8, // Turns 4-8: lite compression
moderate: 20, // Turns 9-20: caveman compression
// Turns 21+: heavy summarization
});
// saved = number of tokens saved
```
#### When to use
Progressive aging is **always on** for `aggressive` and `ultra` modes. It's
particularly effective for:
- Long-running coding sessions
- Multi-day conversations
- Agentic workflows with many tool calls
### Caveman Output Mode
The `outputMode.ts` module injects **system prompt instructions** to make the
model itself produce compressed, terse output (a "caveman" style).
#### How it works
Instead of compressing the input, this mode adds a system prompt like:
> "Reply in minimal words. Skip pleasantries. Use short sentences."
This works particularly well for:
- Code generation (terser output = fewer tokens)
- Quick Q&A (no need for elaborate explanations)
- Batch processing (maximize throughput)
#### When to use
Caveman output mode is **opt-in** — set it via the combo config:
```json
{
"strategy": "auto",
"config": {
"auto": {
"outputMode": "caveman"
}
}
}
```
### Tool Result Compression
The `toolResultCompressor.ts` module provides **5 specialized compression strategies**
for tool results (function calls, agent outputs, search results, etc.):
1. **Search result compression** — Removes redundant results, keeps top-N
2. **File read compression** — Truncates large files, preserves headers/imports
3. **Code execution compression** — Keeps only essential stdout/stderr
4. **Database query compression** — Limits rows, removes verbose metadata
5. **API response compression** — Strips null fields, condenses arrays
#### When to use
Tool result compression is **always on** when tool calls are present. No
configuration needed.
### Stacked Pipeline
The stacked mode runs **multiple engines in sequence** — usually RTK first
(60-90% savings on tool output), then Caveman (30% additional savings on the
remaining text). This achieves **78-95% total savings**.
#### How it works
```
Input (1000 tokens)
→ RTK (command-aware filter) → 200 tokens
→ Caveman (filler removal) → 140 tokens
→ Output (140 tokens, 86% savings)
```
#### When to use
Use stacked mode for:
- Tool-heavy workflows (agentic coding, research)
- Cost-sensitive batch processing
- When you need maximum token savings
Configure via combo:
```json
{
"strategy": "auto",
"config": {
"auto": {
"modePack": "stacked"
}
}
}
```
---
## Compression Combo Overrides
You can override the global compression mode **per combo** to fine-tune behavior
for different use cases:
```json
{
"id": "coding-combo",
"strategy": "priority",
"config": {
"auto": {
"weights": { "taskFit": 0.5 },
"modePack": "quality-first"
}
},
"compressionOverride": {
"mode": "aggressive",
"stackedPipelines": ["rtk", "caveman"],
"preserveToolDefinitions": true
}
}
```
This is useful for:
- **Coding combos**: Use `aggressive` mode for long sessions
- **Quick Q&A combos**: Use `lite` mode for fast responses
- **Tool-heavy combos**: Use `stacked` mode for max savings
- **Production combos**: Use `cache-aware` mode for caching providers
---
## See Also
- [Environment Config](../reference/ENVIRONMENT.md) — Compression environment variables
- [Architecture Guide](../architecture/ARCHITECTURE.md) — Compression pipeline internals
- [User Guide](../guides/USER_GUIDE.md) — Getting started with compression
- [RTK Compression](./RTK_COMPRESSION.md) — RTK filters, trust model, verify gate, raw-output recovery
- [Compression Engines](./COMPRESSION_ENGINES.md) — Caveman, RTK, stacked, APIs, MCP, dashboard
- [Compression Rules Format](./COMPRESSION_RULES_FORMAT.md) — JSON rule-pack format
- [Compression Language Packs](./COMPRESSION_LANGUAGE_PACKS.md) — Language-specific Caveman rules
docs/compression/COMPRESSION_LANGUAGE_PACKS.md
+156
View File
@@ -0,0 +1,156 @@
---
title: "Compression Language Packs"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Compression Language Packs
Caveman compression can load language-specific rule packs in addition to the built-in English rules.
This keeps the core engine stable while allowing Portuguese, Spanish, German, French, Japanese, and
future language packs to evolve independently.
## Location
Language packs live under:
```txt
open-sse/services/compression/rules/<language>/
```
Current shipped packs (verified against `rules/` directory contents):
| Language | Directory | Rule categories present |
| ------------------- | -------------- | --------------------------------------------------- |
| English | `rules/en/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| Spanish | `rules/es/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| Portuguese (Brazil) | `rules/pt-BR/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| Indonesian | `rules/id/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| German | `rules/de/` | `context`, `filler`, `structural` |
| French | `rules/fr/` | `context`, `filler`, `structural` |
| Japanese | `rules/ja/` | `context`, `filler`, `structural` |
> **Parity note:** `en`, `es`, `pt-BR`, and `id` packs have the full 5 categories; `de`, `fr`, `ja` ship 3 categories. The missing `dedup` and `ultra` categories silently fall back to the English built-ins. Contributions welcome to add `dedup.json` and `ultra.json` for the smaller packs.
>
> The `pt-BR` pack is based on **[Troglodita](https://github.com/leninejunior/troglodita)** by Lenine Júnior — a compression system designed from scratch for Brazilian Portuguese grammar (pleonasm reduction, PT-BR filler removal, technical abbreviations for the dev BR community).
>
> The canonical category list and per-category schema live in [`open-sse/services/compression/rules/_schema.json`](../../open-sse/services/compression/rules/_schema.json) (JSON Schema draft 2020-12).
## Language Detection
`languageDetector.ts` uses lightweight heuristics to infer the language from prompt text. The
configured default language is still respected, and detection can be disabled by config when exact
control is required.
Detection output is used only to choose rule packs. It does not change provider routing, locale
selection, or UI language.
## Config Shape
Compression settings can include:
```json
{
"languageConfig": {
"enabled": true,
"defaultLanguage": "en",
"autoDetect": true,
"enabledPacks": ["en", "pt-BR", "es", "id", "de", "fr", "ja"]
},
"cavemanConfig": {
"language": "en",
"autoDetectLanguage": true,
"enabledLanguagePacks": ["en", "pt-BR", "es", "id", "de", "fr", "ja"]
}
}
```
`languageConfig` controls dashboard/preview defaults. `cavemanConfig` is the runtime engine config
used when Caveman compresses message text.
## Adding a Language Pack
1. Create `open-sse/services/compression/rules/<language>/<pack>.json`.
2. Use the Caveman rule format from `docs/compression/COMPRESSION_RULES_FORMAT.md`.
3. Keep replacements conservative and avoid changing code, identifiers, URLs, or JSON.
4. Add or update tests for language selection and replacement behavior.
5. Expose new dashboard/i18n labels if the language appears in UI selectors.
## API
Available packs can be queried with:
```bash
curl http://localhost:20128/api/compression/language-packs
```
The preview endpoint accepts language config overrides:
```bash
curl -X POST http://localhost:20128/api/compression/preview \
-H "Content-Type: application/json" \
-d '{
"mode": "standard",
"text": "Por favor, eu gostaria que voce basicamente resumisse isso.",
"config": {
"languageConfig": {
"defaultLanguage": "pt-BR",
"autoDetect": true
}
}
}'
```
## SHARED_BOUNDARIES (v3.8.0)
All 6 language packs received a `SHARED_BOUNDARIES` clause in v3.8.0 that is applied at every
Caveman intensity (LITE, FULL, ULTRA). It instructs the engine to preserve these patterns verbatim,
regardless of surrounding filler removal:
| Pattern type | Example |
| -------------------------------- | -------------------------------------- |
| Fenced code blocks | ` ```python\n...\n``` ` |
| Inline code | `` `my_var` `` |
| URLs | `https://example.com/path` |
| File paths (absolute + relative) | `/etc/hosts`, `./src/index.ts` |
| Error headers | `Error:`, `TypeError:`, `SyntaxError:` |
| Stack trace lines | ` at functionName (file.ts:12:3)` |
These patterns are populated in `DEFAULT_CAVEMAN_CONFIG.preservePatterns` (previously `[]`). The
constant lives in `open-sse/services/compression/types.ts`.
### Why this matters
Without SHARED_BOUNDARIES, aggressive Caveman modes could strip content that looked like repetitive
prose but was actually a code snippet, file path, or error stack. SHARED_BOUNDARIES acts as a
language-agnostic safety net applied before filler rules run.
### Customizing preservePatterns
Additional patterns can be added at runtime via compression settings:
````json
{
"cavemanConfig": {
"preservePatterns": [
"```[\\s\\S]*?```",
"`[^`]+`",
"https?://\\S+",
"(?:/|\\./)[^\\s]+",
"\\b(?:Error|TypeError|SyntaxError|RangeError):",
"\\s+at\\s+\\S+\\s+\\(\\S+:\\d+:\\d+\\)"
]
}
}
````
Custom patterns extend (not replace) the 6 defaults.
---
## Operational Notes
- English built-in rules remain the fallback when a language pack is missing.
- Invalid built-in JSON packs fail validation so release assets do not silently degrade.
- Rule packs are data-only and should not import code or run arbitrary logic.
- The compression analytics layer records the selected mode and engine, not full prompt text.
docs/compression/COMPRESSION_RULES_FORMAT.md
+195
View File
@@ -0,0 +1,195 @@
---
title: "Compression Rules Format"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Compression Rules Format
Compression rules are JSON files loaded at runtime. They are intentionally data-only so new
language packs and RTK command filters can be reviewed without changing engine code.
> **Canonical schema (source of truth):** [`open-sse/services/compression/rules/_schema.json`](../../open-sse/services/compression/rules/_schema.json) (JSON Schema draft 2020-12).
> The examples below are illustrative — when in doubt, validate your pack against `_schema.json`.
## Caveman Rule Packs
Caveman rule packs live under:
```txt
open-sse/services/compression/rules/<language>/<pack>.json
```
Each pack contains replacements that apply to normal prose after protected regions are isolated.
```json
{
"language": "en",
"category": "filler",
"rules": [
{
"name": "question_to_directive",
"pattern": "\\b(?:Can you explain why|Could you show me how)\\b\\s*",
"replacement": "Explain why ",
"replacementMap": {
"can you explain why": "Explain why ",
"could you show me how": "Show how "
},
"flags": "gi",
"context": "all",
"category": "context",
"minIntensity": "lite",
"description": "Convert verbose questions into direct requests."
}
]
}
```
### Caveman Fields
| Field | Required | Description |
| ------------------------ | -------- | ---------------------------------------------------------------- |
| `language` | yes | BCP-47-like language key such as `en`, `pt-BR`, `es` |
| `category` | yes | Pack category filename/category, for example `filler` or `dedup` |
| `rules` | yes | Array of regex replacement rules |
| `rules[].name` | yes | Stable rule name |
| `rules[].pattern` | yes | JavaScript regex source |
| `rules[].flags` | no | JavaScript regex flags; default `gi` |
| `rules[].replacement` | no | Replacement string or fallback when `replacementMap` misses |
| `rules[].replacementMap` | no | Match-specific replacements keyed by normalized matched text |
| `rules[].context` | no | `all`, `user`, `assistant`, or `system`; default `all` |
| `rules[].category` | no | `filler`, `context`, `structural`, `dedup`, `terse`, or `ultra` |
| `rules[].minIntensity` | no | `lite`, `full`, or `ultra`; default `lite` |
| `rules[].description` | no | Human-readable rule summary |
Use `flags` when case-sensitive matching matters, for example article removal before lowercase prose
without stripping `the OpenAI API`. Use `replacementMap` when one regex has multiple alternatives
that need different outputs; this keeps JSON rule packs data-only while preserving the behavior of
the richer built-in TypeScript replacement functions.
## RTK Filter Packs
RTK filters live under:
```txt
open-sse/services/compression/engines/rtk/filters/<filter>.json
```
Each filter describes how to recognize and compress a command-output family.
```json
{
"id": "test-vitest",
"label": "Vitest output",
"category": "test",
"priority": 92,
"match": {
"outputTypes": ["test-vitest"],
"commands": ["vitest", "npm test", "npm run test"],
"patterns": ["\\bFAIL\\b", "\\bPASS\\b", "\\bTest Files\\b"]
},
"rules": {
"stripAnsi": true,
"replace": [{ "pattern": "\\s+\\[[0-9]+ms\\]", "replacement": "" }],
"matchOutput": [
{ "pattern": "All tests passed", "message": "vitest: ok", "unless": "FAIL|Error:" }
],
"includePatterns": ["FAIL", "Error:", "Test Files", "Tests"],
"dropPatterns": ["^\\s*$", "Duration\\s+\\d+"],
"collapsePatterns": ["^\\s+at "],
"deduplicate": true,
"truncateLineAt": 240,
"maxLines": 160,
"headLines": 24,
"tailLines": 40,
"onEmpty": "vitest: ok",
"filterStderr": false
},
"preserve": {
"errorPatterns": ["FAIL", "Error:", "AssertionError"],
"summaryPatterns": ["Test Files", "Tests", "Snapshots"]
},
"tests": [
{
"name": "keeps failing tests",
"command": "vitest",
"input": "FAIL test/a.test.ts\\nError: boom\\nTest Files 1 failed",
"expected": "FAIL test/a.test.ts\\nError: boom\\nTest Files 1 failed"
}
]
}
```
### RTK Fields
| Field | Required | Description |
| -------------------------- | -------- | ------------------------------------------------------------------------------ |
| `id` | yes | Stable filter id |
| `label` | yes | Dashboard-readable name |
| `category` | yes | Filter family: git, test, build, shell, docker, package, infra, cloud, generic |
| `priority` | no | Higher priority wins when multiple filters match |
| `match.outputTypes` | no | Detector output ids that select this filter |
| `match.commands` | no | Command tokens that select this filter |
| `match.patterns` | no | Regex patterns that select this filter from output text |
| `rules.stripAnsi` | no | Remove ANSI escape sequences before regex stages |
| `rules.replace` | no | Ordered regex substitutions applied line by line |
| `rules.matchOutput` | no | Short-circuit output rules with optional `unless` guard |
| `rules.includePatterns` | no | Lines to prefer preserving |
| `rules.dropPatterns` | no | Lines to remove as noise |
| `rules.collapsePatterns` | no | Repeated matching lines that can be collapsed |
| `rules.deduplicate` | no | Collapse duplicate normalized lines |
| `rules.truncateLineAt` | no | Unicode-safe per-line character limit |
| `rules.maxLines` | no | Maximum retained lines before tail preservation |
| `rules.headLines` | no | Head lines retained during truncation |
| `rules.tailLines` | no | Tail lines retained for recent context |
| `rules.onEmpty` | no | Fallback message when filtering removes all content |
| `rules.filterStderr` | no | Normalize common stderr prefixes before later filtering stages |
| `preserve.errorPatterns` | no | Error lines that should survive truncation |
| `preserve.summaryPatterns` | no | Summary lines that should survive truncation |
| `tests[]` | no | Inline verification samples used by the RTK verify gate |
RTK applies declarative stages in this order: `stripAnsi`, `filterStderr`, `replace`,
`matchOutput`, `dropPatterns`/`includePatterns`, `truncateLineAt`, `headLines`/`tailLines`,
`maxLines`, and `onEmpty`.
Custom filters can be loaded from:
1. Project `.rtk/filters.json` files only after a matching `.rtk/trust.json` hash is present or
`trustProjectFilters` is enabled.
2. Global `DATA_DIR/rtk/filters.json`.
3. Built-in filters.
Project/global custom files may contain one filter object or an array of filter objects. Invalid
custom filters are skipped with diagnostics; invalid built-in filters fail validation.
Project trust file:
```json
{
"filtersSha256": "0123456789abcdef..."
}
```
The environment override `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1` trusts project filters without a
hash and should be limited to controlled local development.
## Safety Rules
- Keep rules idempotent: running the same filter twice should not corrupt output.
- Preserve exact error text, file paths, line numbers, and command summaries where possible.
- Avoid rules that modify code blocks, JSON payloads, URLs, or secrets.
- Add unit coverage for new command families in detector/filter tests.
- Add `tests[]` samples to every built-in filter and to shared custom filters.
## Validation
Rule packs are validated before use. Built-in Caveman packs and built-in RTK filters fail fast
during validation so broken release assets are caught before shipment. Custom RTK filters are
skipped with diagnostics when parsing or trust validation fails.
Focused validation:
```bash
node --import tsx/esm --test tests/unit/compression/rule-loader.test.ts tests/unit/compression/language-packs.test.ts
node --import tsx/esm --test tests/unit/compression/rtk-verify.test.ts tests/unit/compression/rtk-dsl-pipeline.test.ts
```
docs/compression/CONTEXT_EDITING.md
+198
View File
@@ -0,0 +1,198 @@
---
title: "Delegated Context Editing (Anthropic)"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Delegated Context Editing (Anthropic)
Delegated **Context Editing** is a Claude-only context-management feature. Unlike OmniRoute's local
compression engines (Caveman, RTK, LLMLingua, stacked pipelines) — which rewrite the request body
_before_ it leaves the proxy — Context Editing asks the **provider** to clear stale
tool-use / tool-result blocks from its own running context window. OmniRoute only attaches a body
parameter (`context_management.edits[]`); Claude does the actual clearing against its own tokenizer.
This is a delegated capability by nature: other providers reject the parameter, so OmniRoute scopes
it strictly to Claude and Claude-Code-compatible relays.
Source of truth: `open-sse/config/contextEditing.ts` (strategy ids, body injection, telemetry
extraction), `open-sse/executors/base.ts` (injection gate + 400-fallback), and
`open-sse/services/compression/types.ts` (config shape + default).
## What `clear_tool_uses` does
OmniRoute injects a single edit into the outbound Anthropic Messages body:
```json
{
"context_management": {
"edits": [
{
"type": "clear_tool_uses_20250919",
"trigger": { "type": "input_tokens", "value": 100000 },
"keep": { "type": "tool_uses", "value": 3 }
}
]
}
}
```
- `type: "clear_tool_uses_20250919"` — the dated Anthropic strategy id (`CLEAR_TOOL_USES_STRATEGY`).
- `trigger.value: 100000` — once the request's input tokens exceed this threshold, Claude begins
clearing old tool-use/result pairs (`CONTEXT_EDITING_DEFAULT_TRIGGER_TOKENS`, Anthropic's default).
- `keep.value: 3` — the N most recent tool-use/result pairs are kept untouched
(`CONTEXT_EDITING_DEFAULT_KEEP_TOOL_USES`).
The beta is advertised via the `anthropic-beta: context-management-2025-06-27` header, which
OmniRoute already emits on Claude requests.
Injection is performed by `applyContextEditingToBody()` and is **idempotent**: if a `clear_tool_uses`
edit already exists on the body (added by a previous call or supplied by the client), the body is
left as-is. If a `clear_thinking_20251015` edit is also present, OmniRoute stable-sorts the
`clear_thinking` edit to the front, because Anthropic requires `clear_thinking` to precede
`clear_tool_uses` in the `edits[]` array.
## The per-combo enable toggle
Context Editing is **off by default** and opt-in. The toggle is a single boolean carried in the
compression config:
- Setting key: `contextEditing.enabled` (camelCase — **not** `context_editing` / `context-editing`).
- Type: `ContextEditingConfig { enabled: boolean }` in
`open-sse/services/compression/types.ts`.
- Default: `DEFAULT_CONTEXT_EDITING_CONFIG = { enabled: false }`.
- Zod schema: `contextEditingConfigSchema` in `src/shared/validation/compressionConfigSchemas.ts`.
- Storage: persisted with the rest of the compression settings (normalized in
`src/lib/db/compression.ts`).
In the dashboard the toggle lives in the compression hub
(`src/app/(dashboard)/dashboard/context/combos/CompressionHub.tsx`) and writes
`{ contextEditing: { enabled: … } }` back through `saveSettings()`. Because it rides on the
compression-settings object, it composes with the per-combo compression profile rather than being a
fully independent surface — the config carries only the on/off flag; all thresholds (`trigger`,
`keep`) are the constants documented above.
## Claude-only gating
Injection only happens for genuine Claude or Claude-Code-compatible relays. The gate in
`open-sse/executors/base.ts` is:
```ts
if (
(this.provider === "claude" || isClaudeCodeCompatible(this.provider)) &&
contextEditing?.enabled &&
!contextEditingDisabled
) {
applyContextEditingToBody(transformedBody, { enabled: true });
}
```
- `this.provider === "claude"` — real Anthropic key/OAuth.
- `isClaudeCodeCompatible(this.provider)` — relays whose provider id starts with the
`anthropic-compatible-cc-` prefix (they advertise Claude Code compatibility, so they are the relays
most likely to accept the beta). See `open-sse/services/provider.ts`.
Deliberately **excluded**:
- `claude-web` — a browser relay with a `create_conversation_params` request shape that never sees
`context_management`.
- Generic `anthropic-compatible-*` relays (without the `-cc-` prefix) — third-party endpoints with
uncertain beta support.
Non-Claude providers never receive the `context_management` parameter even when the toggle is on.
## The 400-fallback / relay coverage
A Claude-compatible relay may advertise the beta but still reject the `context_management` parameter
with an HTTP 400. To degrade gracefully instead of failing the request, the executor strips the
parameter and retries the same URL **once**:
```ts
if (
response.status === HTTP_STATUS.BAD_REQUEST &&
contextEditing?.enabled &&
!contextEditingDisabled &&
transformedBody?.context_management !== undefined
) {
const errText = await response
.clone()
.text()
.catch(() => "");
if (/context[_-]management|context editing/i.test(errText)) {
contextEditingDisabled = true;
delete transformedBody.context_management;
let retryBody = JSON.stringify(transformedBody);
if (isClaudeCodeCompatible(this.provider) || this.provider === "claude") {
retryBody = await signRequestBody(retryBody);
}
response = await fetch(url, { ...fetchOptions, body: retryBody });
}
}
```
Behavior:
1. Fires only on a `400` while context editing is enabled and the body actually carries
`context_management`.
2. The 400 body is read via a `clone()` so the original response stays intact for the non-matching
path.
3. The error text must match `/context[_-]management|context editing/i` — an unrelated 400 (e.g.
`max_tokens must be >= 1`) does **not** trigger the fallback; the original error propagates.
4. On a match it sets `contextEditingDisabled = true` (which suppresses re-injection if a fresh
`transformedBody` is later built for a retry/fallback URL), deletes `context_management`,
re-signs the body for Claude / Claude-Code-compatible relays (`signRequestBody`), and retries the
same URL once.
Genuine Claude carries the beta in `ANTHROPIC_BETA_BASE` and does not hit this fallback path.
## `applied_edits` telemetry
After a Claude response, OmniRoute records how much context the provider actually cleared. This is
**not** streamed — it is extracted from the non-streaming response body, best-effort, and never
affects the response (telemetry failures are swallowed).
- Extraction: `extractContextEditingTelemetry(responseBody)` in `open-sse/config/contextEditing.ts`.
It probes `applied_edits` in three locations (defensive over the response shape):
- `context_management.applied_edits`
- `usage.context_management.applied_edits`
- `usage.applied_edits`
- Per-edit fields read from each entry: `cleared_input_tokens` and `cleared_tool_uses`
(snake_case, Anthropic-native), with `clearedInputTokens` / `clearedToolUses` camelCase fallbacks.
- Returns `null` when no `applied_edits` array is found or nothing was actually cleared.
The receipt shape is `ContextEditingTelemetry { editCount, clearedInputTokens, clearedToolUses }`.
Recording happens in `open-sse/handlers/chatCore.ts` (gated to `provider === "claude"`) via
`recordContextEditingTelemetry()` (`src/lib/db/compressionAnalytics.ts`), which writes a compression
analytics row tagged:
- `mode: "context-editing"`
- `engine: "context-editing"`
- `tokens_saved` / `original_tokens` = the cleared input-token count
- `request_id` suffixed with `::context-editing`
So delegated clearing shows up in compression analytics alongside the local engines, under the
`context-editing` engine label, and is distinguishable from RTK/Caveman/LLMLingua savings.
## Relationship to the local compression engines
| Aspect | Local engines (Caveman / RTK / LLMLingua / stacked) | Delegated Context Editing |
| ----------------- | --------------------------------------------------- | ------------------------------------------- |
| Where it runs | In OmniRoute, before the request leaves the proxy | In the provider (Claude), server-side |
| What it edits | Prompt / context / tool-result text | Old tool-use / tool-result blocks |
| Provider scope | All providers | `claude` + `anthropic-compatible-cc-*` only |
| Toggle | Compression mode settings | `contextEditing.enabled` |
| Failure mode | Fail-open (original text) | 400-fallback: strip param, retry once |
| Savings telemetry | `engine: <engine id>` | `engine: "context-editing"` |
The two are complementary: local engines compress the bytes OmniRoute sends; Context Editing lets
Claude prune the running context across turns. They can be enabled together.
## See Also
- [COMPRESSION_ENGINES.md](./COMPRESSION_ENGINES.md) — engine registry and the local compression
engines
- [RTK_COMPRESSION.md](./RTK_COMPRESSION.md) — command/tool-output compression
- [../frameworks/MCP-SERVER.md](../frameworks/MCP-SERVER.md) — MCP description compression and
tool-cardinality reduction
- Source: `open-sse/config/contextEditing.ts`, `open-sse/executors/base.ts`,
`open-sse/services/compression/types.ts`, `src/lib/db/compressionAnalytics.ts`
docs/compression/EXTENDING_COMPRESSION.md
+615
View File
@@ -0,0 +1,615 @@
---
title: "Extending the Compression Pipeline"
version: 3.8.44
lastUpdated: 2026-07-02
---
# Extending the Compression Pipeline
> **TL;DR**: OmniRoute's compression engine is **pluggable** — you can register custom engines, ship language packs for new languages, and compose stacked pipelines. This guide shows how.
**Related guides:**
- [COMPRESSION_GUIDE.md](./COMPRESSION_GUIDE.md) — Full pipeline overview
- [COMPRESSION_ENGINES.md](./COMPRESSION_ENGINES.md) — Engine registry and built-in engines
- [RTK_COMPRESSION.md](./RTK_COMPRESSION.md) — RTK engine and custom filters
- [COMPRESSION_RULES_FORMAT.md](./COMPRESSION_RULES_FORMAT.md) — Rule pack format reference
---
## Overview
The compression system has **3 extension points**:
| Extension point | Use case | Difficulty |
| -------------------- | ------------------------------------------------------------------------ | ---------- |
| **Custom engine** | Add a brand-new compression algorithm (e.g., domain-specific summarizer) | Advanced |
| **Language pack** | Add support for a new natural language (e.g., Hindi, Arabic) | Medium |
| **Stacked pipeline** | Compose existing engines in a custom order | Beginner |
```
┌─────────────────────────────────────────────────────────────┐
│ Compression Strategy │
│ │
│ Input messages ──▶ getEffectiveMode() ──▶ mode │
│ │ │
│ ┌───────────────────────┼──────────┐ │
│ │ │ │ │ │ │
│ ▼ ▼ ▼ ▼ │ │
│ "rtk" "lite" "standard" "stacked" │
│ │ │ │ │ │ │
│ ▼ ▼ ▼ ▼ │ │
│ RTK Lite Caveman engines[] │
│ engine engine engine chained │
│ │ │ │ │ │ │
│ └─────────┴─────────┴─────────┘ │ │
│ │ │
│ ▼ │
│ Compressed output │
└─────────────────────────────────────────────────────────────┘
The strategy selector is MODE-BASED: each request selects ONE mode
(rtk / lite / standard / aggressive / ultra / stacked / off).
Only mode "stacked" chains multiple engines in sequence.
Default auto-trigger mode is "lite" (not a 3-tier priority chain).
```
---
## Writing a Custom Compression Engine
The engine interface (`open-sse/services/compression/engines/types.ts`) is the contract every engine must satisfy. It has 5 required methods.
### The `CompressionEngine` Interface
```ts
interface CompressionEngine {
id: string; // Unique engine ID
name: string; // Display name
description: string; // Short description
icon: string; // Icon (emoji or URL)
targets: CompressionEngineTarget[]; // ["messages", "tool_results", "code_blocks"]
stackable: boolean; // Can be used in a stacked pipeline
stackPriority: number; // Order in stacked pipelines (lower = earlier)
metadata: CompressionEngineMetadata;
apply(body, options?): CompressionResult;
compress(body, config?): CompressionResult;
getConfigSchema(): EngineConfigField[];
validateConfig(config): EngineValidationResult;
}
```
### Minimal Example: Whitespace Engine
The simplest possible engine — strip extra whitespace from messages.
````ts
import type { CompressionEngine } from "omniroute/compression/engines/types";
import { registerCompressionEngine } from "omniroute/compression/engines/registry";
function preserveCodeBlocks(text: string): string {
// Split by code block markers and preserve whitespace inside them
const parts = text.split(/(```[\s\S]*?```)/);
return parts
.map((part) => {
if (part.startsWith("```")) {
return part; // Don't modify code blocks
}
return part.replace(/\n{3,}/g, "\n\n"); // Only apply to prose
})
.join("");
}
const whitespaceEngine: CompressionEngine = {
id: "whitespace",
name: "Whitespace Stripper",
description: "Removes extra whitespace and blank lines",
icon: "📝",
targets: ["messages", "tool_results"],
stackable: true,
stackPriority: 100, // Run AFTER caveman/rtk
metadata: {
id: "whitespace",
name: "Whitespace Stripper",
description: "Removes extra whitespace and blank lines",
inputScope: "messages",
targetLatencyMs: 5,
supportsPreview: true,
stable: true,
},
apply(body, options) {
return this.compress(body, options?.config);
},
compress(body, config = {}) {
let originalLength = 0;
let compressedLength = 0;
// Traverse message array — handle both string and multipart content
const compressedBody = (body.messages || []).map((msg) => {
if (typeof msg.content === "string") {
originalLength += msg.content.length;
let compressed = msg.content
.replace(/[ \t]+/g, " ")
.replace(/\n{3,}/g, "\n\n")
.replace(/^\s+|\s+$/gm, "");
compressedLength += compressed.length;
return { ...msg, content: compressed };
}
// Multipart content: traverse parts, compress text parts only
if (Array.isArray(msg.content)) {
const newParts = msg.content.map((part) => {
if (part.type === "text" && typeof part.text === "string") {
originalLength += part.text.length;
let compressed = part.text
.replace(/[ \t]+/g, " ")
.replace(/\n{3,}/g, "\n\n")
.replace(/^\s+|\s+$/gm, "");
compressedLength += compressed.length;
return { ...part, text: compressed };
}
return part; // preserve image_url, tool_use, etc.
});
return { ...msg, content: newParts };
}
return msg;
});
return {
body: { ...body, messages: compressedBody },
stats: {
originalTokens: Math.ceil(originalLength / 4),
compressedTokens: Math.ceil(compressedLength / 4),
savingsPercent: originalLength > 0 ? 100 * (1 - compressedLength / originalLength) : 0,
techniques: ["whitespace-collapse"],
engineId: "whitespace",
},
};
},
getConfigSchema() {
return [
{
key: "preserveCodeBlocks",
type: "boolean",
label: "Preserve code blocks",
defaultValue: true,
description: "Don't touch whitespace inside ```code``` blocks",
},
];
},
validateConfig(config) {
if (config.preserveCodeBlocks !== undefined && typeof config.preserveCodeBlocks !== "boolean") {
return { valid: false, errors: ["preserveCodeBlocks must be a boolean"] };
}
return { valid: true, errors: [] };
},
};
// Register globally
registerCompressionEngine(whitespaceEngine);
````
### Where to Place Custom Engines
```
~/.omniroute/compression/engines/my-engine.ts # User-level
<project>/compression-engines/my-engine.ts # Project-level (loaded on startup)
```
Or load programmatically from a plugin:
```ts
// In your plugin
import {
registerCompressionEngine,
unregisterCompressionEngine,
} from "@omniroute/open-sse/services/compression/engines/registry";
import { myEngine } from "./engines/my-engine";
export default definePlugin({
name: "my-compression-plugin",
// The plugin SDK exposes onRequest / onResponse / onError hooks. Register the
// engine when the plugin module loads (or on first onRequest); unregister it
// from your own teardown path.
onRequest: async (ctx) => {
registerCompressionEngine(myEngine);
},
});
// On teardown:
// unregisterCompressionEngine("my-engine");
```
### Testing Your Engine
Register your engine in a plugin or startup function. Once registered, the engine will be available
in the strategy selector via its `id`. Test integration by composing it in a stacked pipeline:
---
## Creating Language Packs
Caveman-style compression uses **language-specific rule packs** to handle fillers, hedging, and verbose patterns in each natural language. OmniRoute ships with **6 language packs**: `en`, `es`, `fr`, `de`, `ja`, `pt-BR`.
### Pack Structure
A language pack is a directory of **JSON files** under `open-sse/services/compression/rules/<language>/`:
```
open-sse/services/compression/rules/
├── en/
│ ├── filler.json # Pleasantries, hedging, politeness
│ ├── context.json # Context-reducing rules
│ ├── dedup.json # Deduplication rules
│ ├── structural.json # Punctuation, formatting
│ └── ultra.json # Aggressive compression rules
├── es/ (same structure)
├── fr/ (same structure)
├── de/ (same structure)
├── ja/ (same structure)
└── pt-BR/ (same structure)
```
### Rule Anatomy
Each rule has this shape (from `open-sse/services/compression/ruleLoader.ts`):
```ts
interface FileRule {
name: string; // Human-readable name (kebab-case)
pattern: string; // JavaScript regex pattern
replacement?: string; // What to replace the match with
replacementMap?: Record<string, string>; // OR a key→replacement map
flags?: string; // Regex flags ("gi" typically)
context?: "all" | "user" | "system" | "assistant";
category?: "filler" | "context" | "structural" | "dedup" | "terse" | "ultra";
minIntensity?: "lite" | "full" | "ultra"; // Skip below this intensity
description?: string; // Documentation
}
```
### Example: Adding Hindi Filler Rules
```json
{
"language": "hi",
"category": "filler",
"rules": [
{
"name": "polite_opener",
"pattern": "\\b(?:नमस्ते|नमस्कार|आदरणीय)\\b[,!\\s]*",
"replacement": "",
"context": "all",
"category": "filler",
"minIntensity": "lite",
"description": "Strip polite openers like 'नमस्ते'"
},
{
"name": "filler_actually",
"pattern": "\\b(?:असल में|वास्तव में|दरअसल)\\b\\s*",
"replacement": "",
"context": "all",
"category": "filler",
"minIntensity": "lite",
"description": "Strip 'actually' fillers"
},
{
"name": "verbose_plea",
"pattern": "\\b(?:कृपया|कृपया आप|अनुरोध है कि आप)\\b\\s*",
"replacement": "",
"context": "all",
"category": "filler",
"minIntensity": "full",
"description": "Strip 'please' in Hindi"
}
]
}
```
### Validation
Rule packs are validated against `_schema.json` on load. A pack with bad structure will fail to load and log an error:
```
RULE_LOADER: pack "hi/filler.json" failed validation:
- rules.0.pattern: Invalid regex
- rules.1.context: must be one of [all, user, system, assistant]
```
Validation runs automatically when a pack is loaded (against `_schema.json`); an
invalid pack is rejected and the error above is logged. There is no separate
`npm run` script for pack validation — load the pack (e.g. start the server or
exercise the compression path) and watch the logs.
### Loading a Custom Language Pack
```ts
import { loadRulePack } from "omniroute/compression/ruleLoader";
await loadRulePack("./my-custom-rules/hi/filler.json");
```
Or place in a recognized location:
```
~/.omniroute/compression/rules/hi/filler.json # User-level
<project>/.compression/rules/hi/filler.json # Project-level
```
### Best Practices for Language Packs
1. **Start with `filler`** — these are the highest-impact rules
2. **Use `minIntensity`** to gate aggressive rules — protects against over-compression
3. **Include test cases** — add `tests[]` array in the JSON to verify behavior
4. **Order matters** — earlier rules apply first; place high-impact rules first
5. **Be conservative with `replacement`** — empty string is usually correct; never introduce new content
### Translation Strategy
When localizing rule packs to a new language:
1. **Translate the rule names** — they appear in debug output
2. **Adapt the regex patterns** — direct translation often fails (word boundaries differ)
3. **Test against real conversations** — the pack should be safe on actual input
4. **Match cultural conventions** — Japanese packs, for instance, have more honorific fillers than English
---
## Stacked Pipelines
A **stacked pipeline** runs multiple engines in sequence, with each engine's output feeding the next. This is how `mode: stacked` works internally.
### How Stacking Works
```
Input (10,000 tokens)
│
▼
┌──────────┐
│ Engine │ priority 10
│ A │ ──▶ output: 6,000 tokens (-40%)
└────┬─────┘
▼
┌──────────┐
│ Engine │ priority 50
│ B │ ──▶ output: 2,400 tokens (-60%)
└────┬─────┘
▼
┌──────────┐
│ Engine │ priority 100
│ C │ ──▶ output: 1,200 tokens (-80%)
└────┬─────┘
│
▼
Final output (1,200 tokens, ~88% savings combined)
```
When `mode: "stacked"` is selected, engines execute sequentially in the order specified in the `pipeline` array.
The output of engine N becomes the input of engine N+1.
### Compression Modes
OmniRoute selects **ONE mode per request** based on configuration, auto-trigger thresholds, and combo overrides.
The available modes are defined in `open-sse/services/compression/types.ts` (type `CompressionMode`):
| Mode | Engines | Use case |
| ------------ | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `off` | None | Disable all compression |
| `rtk` | RTK only | Command-output heavy sessions (80%+ savings) |
| `lite` | Lite only | Conservative compression (fast, safe) |
| `standard` | Caveman | Prose compression with language packs |
| `aggressive` | Caveman + Aggressive | Aggressive prose + aggressive final pass |
| `ultra` | Ultra | Maximum compression (lossy, last resort). Optionally routed through the **LLMLingua-2** SLM engine when `ultra.modelPath` is set (fail-opens to the rule-based path when the model is unavailable). |
| `stacked` | Custom pipeline | Compose engines in any order (see below) |
> Beyond the mode engines above, the registry also ships specialized stackable engines —
> **CCR**, **headroom**, **ionizer**, and **session-dedup** — documented in
> [COMPRESSION_ENGINES.md](./COMPRESSION_ENGINES.md#additional-built-in-engines).
Mode selection is determined by `getEffectiveMode()` in `open-sse/services/compression/strategySelector.ts`:
1. If compression is disabled: `"off"`
2. If a combo override exists: use the override
3. If auto-trigger threshold is exceeded: use `autoTriggerMode` (default: `"lite"`)
4. Otherwise: use `defaultMode`
### The Default Stacked Pipeline
When `mode: "stacked"` is explicitly configured, the default pipeline composes:
1. **RTK** — strip command output noise (~80% savings on terminal output)
2. **Caveman** — remove fillers, terse-ify prose (~46% on remaining text)
3. **Lite** — final whitespace + dedup pass
This composition achieves **78-95% savings** on tool-heavy sessions.
### Configuring Stacked Pipelines
In combo config:
```json
{
"compression": {
"mode": "stacked",
"pipeline": [
{ "engine": "rtk", "config": { "intensity": "aggressive" } },
{ "engine": "caveman", "config": { "intensity": "full" } },
{ "engine": "lite", "config": {} }
]
}
}
```
You can omit engines, add custom ones, or reorder them.
### State Passing
Engines can read metadata from the request context (in `options`):
```ts
compress(body, config) {
// Read metadata from previous engines
const original = options?.compressionComboId; // "my-coding-combo"
// ...
}
```
The metadata is **read-only** — engines cannot mutate the request context, only their own body output.
### Execution Order Gotchas
| Engine order | Effect |
| ----------------------------------- | ------------------------------------------------------------------------------ |
| RTK → Caveman → Lite | **Recommended** (strips noise first, then language, then whitespace) |
| Lite → RTK → Caveman | Bad — Lite strips whitespace from raw output, making RTK pattern matching fail |
| Caveman → RTK | Bad — Caveman may rewrite text in ways that RTK doesn't recognize |
| Any order with `tool_results` first | Better — tool output is the noisiest content |
### When NOT to Stack
Stacking isn't always better:
- **Simple messages** (no tool output) — single Caveman or Lite is enough
- **Cost-sensitive** — each engine adds ~5-50ms latency
- **Specific tools** — RTK alone is usually sufficient for shell output
### Building a Custom Pipeline
There is no named-pipeline registry. A stacked pipeline is just an **inline array
of steps** passed to `applyStackedCompression()` (exported from
`@omniroute/open-sse/services/compression/strategySelector`):
```ts
import { applyStackedCompression } from "@omniroute/open-sse/services/compression/strategySelector";
const result = applyStackedCompression(body, [
{ engine: "rtk", intensity: "aggressive" },
{ engine: "caveman", intensity: "full" },
]);
```
When you don't pass a pipeline, it defaults to `rtk(standard) → caveman(full)`.
To drive it from config, set `mode: "stacked"` and provide the step array under
`stackedPipeline` (read from `config.stackedPipeline`):
```json
{
"compression": {
"mode": "stacked",
"stackedPipeline": [
{ "engine": "rtk", "intensity": "aggressive" },
{ "engine": "caveman", "intensity": "full" }
]
}
}
```
---
## Upstream Sync Policy
OmniRoute's compression engines credit several upstream projects in the README
("inspired by RTK, Caveman, LLMLingua-2, Troglodita"). A common contributor
question is: **when upstream RTK adds a new tool filter or Caveman adds a rule
pack, how does that reach OmniRoute?** This section is the authoritative answer.
### Vendored copies vs. independent implementations
| Engine | Relationship to upstream | Location |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| **RTK** | **Independent reimplementation** (inspired-by, not a copy) | `open-sse/services/compression/engines/rtk/` |
| **Caveman** | **Independent reimplementation** (inspired-by) | `open-sse/services/compression/engines/cavemanAdapter.ts` |
| **Headroom** | Mostly internal; only the `gcf/` codec is **genuinely vendored** from `gcf-typescript` (MIT, SPDX-marked, generic profile only) | `open-sse/services/compression/engines/headroom/gcf/` |
| **LLMLingua-2 / Troglodita** | Inspired-by (drive the `llmlingua` + `session-dedup` engines) | `open-sse/services/compression/engines/llmlingua/`, `session-dedup` |
Key point: **RTK and Caveman are clean-room TypeScript implementations of the
_ideas_ (filter rules, rule packs), not vendored source trees.** There is no
upstream copy to `git pull` from — which is exactly why the README says
"inspired by" rather than "bundled".
### How upstream improvements are merged
There is **no automated upstream-release tracking and no `compression-sync`
label** — by design. Because the engines are reimplementations, an upstream RTK
filter or Caveman rule pack is not merged as code; it is **re-expressed as a new
rule/filter in OmniRoute's own format** (see
[COMPRESSION_RULES_FORMAT.md](./COMPRESSION_RULES_FORMAT.md)) and lands ad-hoc via
a normal PR. The extension points above (custom engine, language pack, RTK filter)
are the sanctioned way to contribute one.
Recent examples of exactly this flow:
- RTK filters for Gradle & `dotnet` build output (v3.8.42)
- RTK filters for kubectl / docker-build / composer / gh (#2824)
- Caveman Indonesian language pack (#3975), plus German / French / Japanese / Chinese packs
### Headroom (input-compression proxy)
Headroom is **fully internal** — a pinned vendored `gcf` codec snapshot plus
OmniRoute's own `smartcrusher` / `toon` / `tabular` layers. There is no live
upstream to track beyond the vendored copy; updates to `gcf` are refreshed
manually when the codec changes and re-validated against the compression budget
gate (`check:compression-budget`).
### Proposing an upstream-inspired improvement
1. **Don't vendor** — re-express the upstream rule/filter in OmniRoute's format.
2. Add it via the matching extension point below (language pack, RTK filter, or
custom engine).
3. Reference the upstream project in the PR description (attribution), not by
copying its license-bearing source.
4. Include tests and confirm the `check:compression-budget` gate still passes.
---
## Best Practices
### Engine Development
1. **Always implement `validateConfig`** — engines without validation cause silent failures
2. **Set realistic `targetLatencyMs`** — used by the strategy selector to choose engines
3. **Use `getConfigSchema` for the dashboard** — never hide config from users
4. **Support `stackable: true` if your engine is pure** — engines with side effects shouldn't stack
5. **Write inline tests** — engines should be verifiable in <1s
### Language Pack Development
1. **Start with `lite` intensity** — your rules should be safe at the lowest setting
2. **Use `context` to scope rules** — `user` only rules can't accidentally affect system prompts
3. **Avoid capturing JSON keys** — `\\bword\\b` can match inside JSON, breaking structured data
4. **Test with edge cases** — empty input, unicode, RTL text, emojis
5. **Use existing packs as templates** — `en/filler.json` is the most-developed example
### Pipeline Design
1. **Profile before optimizing** — measure with `compression_stats` first
2. **Prefer composition over reimplementation** — extend Caveman rules before writing a new engine
3. **Document the order rationale** — comment why engine A before engine B
4. **Test at all 3 intensity levels** — `lite` is fast but lossy, `ultra` is slow but precise
---
## Reference: Built-in Engines
| Engine ID | Stackable | Default stackPriority | Targets |
| -------------------- | --------- | --------------------- | ----------------------------------- |
| `lite` | Yes | 5 | messages, tool_results |
| `rtk` | Yes | 10 | tool_results |
| `standard` (caveman) | Yes | 20 | messages, tool_results, code_blocks |
| `aggressive` | Yes | 30 | messages |
| `ultra` | Yes | 40 | messages, code_blocks |
### See Also
- [COMPRESSION_GUIDE.md](./COMPRESSION_GUIDE.md) — Pipeline overview
- [COMPRESSION_ENGINES.md](./COMPRESSION_ENGINES.md) — Engine registry reference
- [COMPRESSION_RULES_FORMAT.md](./COMPRESSION_RULES_FORMAT.md) — Rule format spec
- [COMPRESSION_LANGUAGE_PACKS.md](./COMPRESSION_LANGUAGE_PACKS.md) — Language pack details
- [RTK_COMPRESSION.md](./RTK_COMPRESSION.md) — RTK engine and custom filters
- Source: `open-sse/services/compression/` (117 files, ~250KB)
docs/compression/RTK_COMPRESSION.md
+674
View File
@@ -0,0 +1,674 @@
---
title: "RTK Compression"
version: 3.8.40
lastUpdated: 2026-06-28
---
# RTK Compression
RTK compression is OmniRoute's command-aware compression engine for terminal and tool output. It is
designed for coding-agent sessions where most context growth comes from test logs, build output,
package manager noise, shell transcripts, Docker output, git output, and stack traces.
RTK can run directly with `defaultMode: "rtk"` or as the first step in a stacked pipeline, usually:
```txt
rtk -> caveman
```
That order compresses noisy machine output first, then lets Caveman condense remaining prose.
Upstream RTK reports `60-90%` command-output savings. Its README sample session goes from
`~118,000` standard tokens to `~23,900` RTK tokens, which is `79.7%` saved (`~80%`). OmniRoute uses
that upstream average for the stacked savings calculation with Caveman input compression:
```txt
RTK average: 80% saved
Caveman input: 46% saved
Stacked: 1 - (1 - 0.80) * (1 - 0.46) = 89.2% saved
Range: 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## What It Compresses
The built-in catalog currently ships 49 filters across these categories:
| Category | Examples |
| --------- | ------------------------------------------------------------- |
| `git` | `git status`, `git branch`, `git diff`, `git log` |
| `test` | Vitest, Jest, Pytest, Playwright, Go tests, Cargo tests |
| `build` | TypeScript, ESLint, Biome, Prettier, Vite, Webpack, Turbo, Nx |
| `package` | `npm install`, `npm audit`, `pip`, `uv sync`, Poetry, Bundler |
| `shell` | `ls`, `find`, `grep`, generic shell logs |
| `docker` | `docker ps`, Docker logs |
| `infra` | Terraform, OpenTofu, `systemctl status` |
| `generic` | JSON output, stack traces, generic output fallback |
The detector in `open-sse/services/compression/engines/rtk/commandDetector.ts` classifies output
before filter selection. Filters can also match by command pattern or output regex when a command
class is not enough.
## Filter Resolution
RTK loads filters in this order:
1. Project filters from `.rtk/filters.json`, only when trusted.
2. Global filters from `DATA_DIR/rtk/filters.json`.
3. Built-in filters from `open-sse/services/compression/engines/rtk/filters/`.
Project filters are intentionally trust-gated because regex filters can change how tool output is
shown to agents. A project filter file is accepted when one of these is true:
- `rtkConfig.trustProjectFilters` is `true`.
- `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1` is set.
- `.rtk/trust.json` contains the SHA-256 hash of `.rtk/filters.json`.
Trust file example:
```json
{
"filtersSha256": "0123456789abcdef..."
}
```
Custom filters can be one filter object or an array of filter objects. Invalid custom filters are
skipped and reported by `/api/context/rtk/filters` diagnostics. Invalid built-in filters fail fast.
## Filter DSL
Filters use the JSON schema described in [Compression Rules Format](./COMPRESSION_RULES_FORMAT.md).
The runtime applies these stages in order:
```txt
stripAnsi -> filterStderr -> replace -> matchOutput -> drop/include lines
-> truncateLineAt -> head/tail/maxLines -> onEmpty
```
Important fields:
| Field | Purpose |
| ---------------------------- | -------------------------------------------------------------- |
| `rules.stripAnsi` | Remove terminal color/control sequences before matching |
| `rules.filterStderr` | Normalize common stderr prefixes before matching/filtering |
| `rules.replace` | Apply ordered regex replacements |
| `rules.matchOutput` | Return a compact summary when output matches a known condition |
| `rules.matchOutput[].unless` | Skip the shortcut when an error/failure pattern is present |
| `rules.dropPatterns` | Remove noisy lines |
| `rules.includePatterns` | Prefer actionable lines |
| `rules.collapsePatterns` | Collapse repeated matching lines |
| `rules.deduplicate` | Per-filter opt-in: collapse consecutive duplicate lines |
| `rules.truncateLineAt` | Unicode-safe per-line truncation |
| `rules.onEmpty` | Fallback message if all lines are filtered out |
| `tests[]` | Inline samples used by the verify gate |
Built-in filters are expected to include inline `tests[]` samples. Custom filters should include
them too, especially when they are shared across projects.
## Line Deduplication (two layers)
RTK collapses duplicate lines at two independent layers:
1. **Per-filter `deduplicate` (opt-in, default `false`).** A filter can set `rules.deduplicate: true`
to collapse consecutive duplicate lines _within that filter's matched output_, before truncation.
This runs inside `lineFilter.ts`. For legacy filters, it is auto-enabled when the filter defines
`collapsePatterns`. Schema: `deduplicate: z.boolean().default(false)` in
`open-sse/services/compression/engines/rtk/filterSchema.ts`.
2. **Engine-wide `deduplicateThreshold` (default `3`).** After all filters run, the engine collapses
any run of `>= deduplicateThreshold` identical consecutive lines across the whole result
(`deduplicateRepeatedLines`, applied in `engines/rtk/index.ts`). The value is bounded to 2–100 on
normalization.
The per-filter pass runs first (inside the filter), the engine-wide pass runs last (over the joined
output), so the two compose without double-counting.
## Line Grouping (`enableGrouping`)
When `rtkConfig.enableGrouping` is `true` (default `false`), RTK runs an additional `groupSimilarLines`
pass over the post-dedup result that collapses runs of _near-equivalent_ (not byte-identical)
consecutive lines. `rtkConfig.groupingThreshold` (default `3`) is the minimum run length that triggers
grouping. This is the structural counterpart to `deduplicateThreshold`: dedup handles exact repeats,
grouping handles "the same shape with small differences". Both flags are part of the `rtkConfig` JSON
persisted in the `key_value` table (see Configuration above), so the setting survives restarts.
## Code Comment Stripping (`stripCodeComments` / `preserveDocstrings`)
When `rtkConfig.applyToCodeBlocks` is enabled, RTK can also strip comments from fenced code blocks:
- `stripCodeComments` (default `false`) — opt-in. When `true`, RTK removes comments from JavaScript
and TypeScript fenced blocks. The flag was historically read but never applied, so the default stays
at "preserve" to avoid a silent production change.
- `preserveDocstrings` (default `true`) — when stripping comments, JSDoc/`/** … */` block comments are
kept (they carry API documentation worth more than the bytes they cost). Set to `false` to strip
those too.
Comment removal is implemented in `open-sse/services/compression/engines/rtk/codeStripper.ts`. It uses
the **TypeScript parser** (not a regex) so that string, template, and regex literals are never mistaken
for comments, and it bails out entirely when JSX is detected (so JSX expression-container comments are
never corrupted). Comment stripping currently applies to **JavaScript and TypeScript only** — other
languages in the stripper's `CodeLanguage` set (Python, Rust, Go, Ruby, Java) have empty-line and
whitespace collapse but no comment removal. The stripped-block run is tagged `rtk:code-strip` in
`rulesApplied`.
> **Note — GCF / tabular encoding is a separate engine.** RTK does **not** contain the "GCF"
> (Graph Compact Format) tabular/columnar JSON encoder. That encoder — which replaced an older
> `omni-tabular` encoder — lives in the **headroom** engine
> (`open-sse/services/compression/engines/headroom/`, with the vendored codec under
> `headroom/gcf/`). It is unrelated to the RTK filter pipeline documented here.
## Configuration
Global settings are available through `/api/settings/compression`. RTK-specific settings are also
available through `/api/context/rtk/config`.
```json
{
"defaultMode": "stacked",
"autoTriggerMode": "stacked",
"autoTriggerTokens": 32000,
"stackedPipeline": [
{ "engine": "rtk", "intensity": "standard" },
{ "engine": "caveman", "intensity": "full" }
],
"rtkConfig": {
"enabled": true,
"intensity": "standard",
"applyToToolResults": true,
"applyToCodeBlocks": false,
"applyToAssistantMessages": false,
"enabledFilters": [],
"disabledFilters": [],
"maxLinesPerResult": 120,
"maxCharsPerResult": 12000,
"deduplicateThreshold": 3,
"customFiltersEnabled": true,
"trustProjectFilters": false,
"rawOutputRetention": "never",
"rawOutputMaxBytes": 1048576,
"enableGrouping": false,
"groupingThreshold": 3,
"stripCodeComments": false,
"preserveDocstrings": true
}
}
```
`enabledFilters` and `disabledFilters` use filter ids, for example `test-vitest` or `git-diff`.
The full `rtkConfig` shape is defined by `RtkConfig` / `DEFAULT_RTK_CONFIG` in
`open-sse/services/compression/types.ts`. The whole object is persisted as a single JSON value in
the SQLite `key_value` table under `namespace = "compression"`, `key = "rtkConfig"`
(`src/lib/db/compression.ts`), and normalized on read by `normalizeRtkConfig`. So every field below
— including `enableGrouping`, `groupingThreshold`, `stripCodeComments`, and `preserveDocstrings` —
round-trips through the same store and survives a restart.
| Key | Default | Purpose |
| ---------------------- | ------- | --------------------------------------------------------------------------- |
| `deduplicateThreshold` | `3` | Engine-wide: min consecutive identical lines to collapse (bounded 2–100) |
| `enableGrouping` | `false` | Opt-in: collapse runs of near-equivalent consecutive lines |
| `groupingThreshold` | `3` | Min consecutive similar-line run that triggers grouping |
| `stripCodeComments` | `false` | Opt-in: remove comments from fenced code blocks (needs `applyToCodeBlocks`) |
| `preserveDocstrings` | `true` | When stripping comments, keep JSDoc/`/** … */` blocks |
## API
| Route | Method | Purpose |
| ---------------------------------- | ------ | -------------------------------------------- |
| `/api/context/rtk/config` | GET | Read RTK config |
| `/api/context/rtk/config` | PUT | Update RTK config |
| `/api/context/rtk/filters` | GET | List filter catalog and load diagnostics |
| `/api/context/rtk/test` | POST | Preview RTK compression for one text payload |
| `/api/context/rtk/raw-output/[id]` | GET | Read retained redacted raw output |
| `/api/compression/preview` | POST | Preview any compression mode |
RTK test payload:
```json
{
"command": "npm test",
"text": "FAIL tests/example.test.ts\nAssertionError: expected true\nTest Files 1 failed",
"config": {
"intensity": "standard"
}
}
```
Compression preview payload:
```json
{
"mode": "stacked",
"messages": [
{
"role": "tool",
"content": "FAIL tests/example.test.ts\nAssertionError: expected true\nTest Files 1 failed"
}
],
"config": {
"rtkConfig": {
"rawOutputRetention": "failures"
}
}
}
```
Management routes require dashboard management auth or the matching API-key policy.
## Raw Output Recovery
RTK normally returns only compressed text. For debugging, `rawOutputRetention` can retain redacted
raw output:
| Value | Behavior |
| ---------- | ------------------------------------------------------- |
| `never` | Do not retain raw output |
| `failures` | Retain only likely failure output |
| `always` | Retain every compressed RTK raw output, after redaction |
Retained files are written under:
```txt
DATA_DIR/rtk/raw-output/
```
Secrets are redacted before persistence, including common bearer tokens, API keys, Slack tokens,
AWS access keys, and assignment-style `token=...`, `secret=...`, `password=...` values. Analytics
stores only the pointer id, size, and hash metadata.
## Verify Gate
The focused verify gate runs built-in inline filter tests without shelling out to external commands:
```bash
node --import tsx/esm --test tests/unit/compression/rtk-verify.test.ts
```
The broader RTK gate is:
```bash
node --import tsx/esm --test \
tests/unit/compression/rtk-*.test.ts \
tests/unit/compression/pipeline-integration.test.ts \
tests/unit/compression/context-compression-api.test.ts
```
Run the broad compression gate before release:
```bash
node --import tsx/esm --test \
tests/unit/compression/*.test.ts \
tests/golden-set/*.test.ts \
tests/integration/compression-pipeline.test.ts \
tests/unit/api/compression/compression-api.test.ts
```
## Extending RTK
1. Add or update a filter JSON file.
2. Include at least one `tests[]` sample that proves the important behavior.
3. Add a fixture under `tests/unit/compression/fixtures/rtk/` for new command families.
4. Add command detection coverage when introducing a new output class.
5. Run the verify and broad RTK gates.
6. If the filter is project-local, commit `.rtk/filters.json` and refresh `.rtk/trust.json` only after review.
---
## Intensity Levels (v3.8.16+)
RTK supports **3 intensity levels** that trade off between **compression aggressiveness** and **safety**. The level is set via `config.intensity` in the engine config.
### The 3 Levels
| Level | Truncation threshold | Token savings | Risk | Best for |
| -------------------- | -------------------- | ------------- | -------- | -------------------------------- |
| `minimal` | 24 lines per section | ~20-40% | Very low | Production with critical context |
| `standard` (default) | 24 lines per section | ~50-70% | Low | Daily coding sessions |
| `aggressive` | 16 lines per section | ~70-90% | Medium | Long sessions, max savings |
### Where the Truncation Happens
The truncation threshold affects `lineFilter.ts`:
```ts
// From open-sse/services/compression/engines/rtk/index.ts:329-330
config.intensity === "aggressive" ? 16 : 24,
config.intensity === "aggressive" ? 16 : 24,
```
Both the **head** and **tail** of each section are preserved; middle content is dropped when truncation kicks in.
### What Stays vs. What Gets Cut
| Content | minimal | standard | aggressive |
| -------------------------- | ------------ | ------------ | ------------ |
| Errors / stack traces | ✅ preserved | ✅ preserved | ✅ preserved |
| Test failures | ✅ preserved | ✅ preserved | ✅ preserved |
| Build errors | ✅ preserved | ✅ preserved | ✅ preserved |
| Test passes (verbose) | ✅ preserved | 🟡 collapsed | 🟡 collapsed |
| Routine output (info logs) | 🟡 collapsed | 🟡 collapsed | ❌ dropped |
| Progress bars | 🟡 collapsed | ❌ dropped | ❌ dropped |
| Banner / ASCII art | 🟡 collapsed | ❌ dropped | ❌ dropped |
### Choosing the Right Intensity
```
Is losing context catastrophic?
│
┌───────────┼───────────┐
│ │ │
YES NO NOT SURE
│ │ │
▼ │ │
minimal │ │
│ │ │
│ ▼ ▼
│ How critical Try `standard` first
│ is throughput? (works for 80% of
│ │ cases)
│ ┌────┴────┐
│ │ │
│ LOW HIGH
│ │ │
│ ▼ ▼
│ standard aggressive
│ │ │
└──────┴─────────┘
```
### Configuring Intensity
**Per-combo** (in combo config):
```json
{
"combo": "my-coding-combo",
"routing": {
/* ... */
},
"compression": {
"engine": "rtk",
"intensity": "aggressive"
}
}
```
**Programmatically**:
`rtkEngine` (`@omniroute/open-sse/services/compression/engines/rtk`) is a
`CompressionEngine` and has no `updateConfig` method. Update an engine's config
through the registry helper instead:
```ts
import { updateEngineConfig } from "@omniroute/open-sse/services/compression/engines/registry";
updateEngineConfig("rtk", { intensity: "aggressive" });
```
### Verifying the Effect
Use the **Verify Gate** (see below) to confirm your filter is safe at your chosen intensity:
```ts
import { runRtkFilterTests } from "omniroute/compression/engines/rtk/verify";
const result = runRtkFilterTests({ intensity: "aggressive" });
if (!result.passed) {
console.error("Filters failed at aggressive intensity");
}
```
---
## Custom Filter Development (v3.8.16+)
The `engines/rtk/filters/` directory contains **49+ built-in filter JSON files**. You can add your own to compress output from custom tools not covered by the defaults.
### Filter Schema (Zod)
```ts
{
"id": "string", // Required. Filter identifier (kebab-case, e.g., "python-traceback")
"label": "string", // Required. Human-readable filter name
"description": "string", // Optional (default: ""). Short description of what filter does
"category": "git|test|build|shell|docker|package|infra|cloud|generic",
"priority": number, // Optional (0-100, default: 50). Execution order (higher = first)
"match": {
"commands": ["string"], // Command names to match (e.g., "python", "pytest")
"patterns": ["string"], // Regex patterns to match output
"outputTypes": ["string"] // Detected output classes (e.g., "test-failure")
},
"rules": {
"stripAnsi": boolean, // Optional (default: false). Strip ANSI color codes
"replace": [ // Find-and-replace rules (default: [])
{ "pattern": "regex", "replacement": "..." }
],
"matchOutput": [ // Short-circuit on pattern match (default: [])
{
"pattern": "regex",
"message": "short summary",
"unless": "regex" // Skip if this pattern matches
}
],
"includePatterns": ["string"], // Lines to keep (regex patterns, default: [])
"dropPatterns": ["string"], // Lines to drop (regex patterns, default: [])
"collapsePatterns": ["string"], // Lines to collapse to single occurrence (default: [])
"deduplicate": boolean, // Optional (default: false). Remove duplicate lines
"truncateLineAt": number, // Optional (default: 0). Truncate lines to max chars
"maxLines": number, // Optional (default: 0). Hard cap on total lines
"headLines": number, // Optional (default: 20). Keep first N lines of matched output
"tailLines": number, // Optional (default: 20). Keep last N lines of matched output
"onEmpty": "string", // Optional (default: ""). Fallback message if all lines filtered
"filterStderr": boolean // Optional (default: false). Also filter stderr output
},
"preserve": {
"errorPatterns": ["string"], // Patterns that must always be preserved (default: [])
"summaryPatterns": ["string"] // Patterns for final summary line (default: [])
},
"tests": [ // Inline tests for verification (default: [])
{
"name": "string", // Required. Test name
"input": "sample output", // Required. Sample input text
"expected": "expected output", // Required. Expected compressed output
"command": "optional command" // Optional. Command context
}
]
}
```
### Example: Python Traceback Filter
```json
{
"id": "python-traceback",
"label": "Python Traceback Filter",
"description": "Compresses Python tracebacks to essential file/line locations and error type",
"category": "test",
"priority": 60,
"match": {
"commands": ["python", "python3", "pytest", "uv", "poetry"],
"patterns": ["Traceback \\(most recent call last\\)", "Error", "Exception"],
"outputTypes": ["error-traceback"]
},
"rules": {
"stripAnsi": true,
"includePatterns": [
"Traceback \\(most recent call last\\)",
"^\\s*File \".+\", line \\d+",
"^\\s*[A-Z][a-zA-Z]+Error:",
"^\\s*[A-Z][a-zA-Z]+Exception"
],
"dropPatterns": ["site-packages/", "^\\s+[a-z_]+\\([^)]*\\)$"],
"headLines": 5,
"tailLines": 3,
"maxLines": 25,
"filterStderr": true
},
"preserve": {
"errorPatterns": ["Error:", "Exception:", "Traceback"],
"summaryPatterns": ["^[A-Z][a-zA-Z]+(?:Error|Exception):"]
},
"tests": [
{
"name": "preserves-error-type-and-location",
"input": "Traceback (most recent call last):\n File \"app.py\", line 42, in main\n do_thing()\n File \"lib/utils.py\", line 17, in helper\n return 1 / 0\nZeroDivisionError: division by zero",
"expected": "Traceback (most recent call last):\n File \"app.py\", line 42, in main\n File \"lib/utils.py\", line 17, in helper\nZeroDivisionError: division by zero",
"command": "python app.py"
}
]
}
```
### Loading Custom Filters
Place the file in a recognized location:
```
~/.omniroute/rtk/filters/my-filter.json # User-level
<project>/.rtk/filters/my-filter.json # Project-level
```
Filters are loaded automatically on startup via `loadRtkFilters()` in `open-sse/services/compression/engines/rtk/filterLoader.ts`. The loader discovers filters from:
- Built-in catalog: `open-sse/services/compression/engines/rtk/filters/`
- User directory: `~/.omniroute/rtk/filters/`
- Project directory: `<project>/.rtk/filters/`
To load filters programmatically:
```ts
import { loadRtkFilters } from "@omniroute/open-sse/services/compression/engines/rtk/filterLoader";
// Options: customFiltersEnabled (load user/project filters, default on),
// trustProjectFilters, refresh.
const filters = loadRtkFilters({ customFiltersEnabled: true });
```
### Validation
Filters are validated against the Zod schema on load. A filter with bad structure will fail to load and log an error:
```
RTK_FILTER_LOADER: filter "my-filter" failed validation:
- rules.replace.0.pattern: Invalid regex
- match.commands: must not be empty
```
To validate all installed filters, call `runRtkFilterTests()` which is exported from `open-sse/services/compression/engines/rtk/verify.ts`.
### Best Practices
1. **Always include `tests[]`** — they prove your filter works and prevent regressions
2. **Use `matchOutput` for short-circuits** — if a single line tells the story, replace the whole block
3. **Prefer `keep` over `strip`** — explicit "always preserve" rules are safer than "always remove"
4. **Test at all 3 intensity levels** — `minimal` should be a no-op, `aggressive` should still preserve errors
5. **Use the `unless` field** — guard short-circuits with "don't trigger if X is present"
---
## Raw Output Recovery & Verify Gate
When RTK compresses output aggressively, you can **recover the original text** for debugging, audit, or replay.
### How Raw Output Recovery Works
```
Original output (10K tokens)
│
▼
RTK compress (with rawOutput.enabled=true)
│
├─▶ Compressed output (2K tokens) ──▶ to LLM
│
└─▶ Original output (10K tokens) ──▶ stored in DB
(linked by request_id)
```
### Enabling Raw Output Storage
**Per-request** (in combo config):
```json
{
"compression": {
"engine": "rtk",
"intensity": "aggressive",
"rawOutput": {
"enabled": true,
"maxBytes": 1048576 // 1MB cap
}
}
}
```
**Default**: `rawOutput.enabled: false` (saves storage).
### Storage Cost
| Per-request | 1MB cap | 10MB cap |
| ------------------------- | ------------ | ------------- |
| Average compressed output | ~5KB | ~5KB |
| Raw output stored | ~50-500KB | ~500KB-5MB |
| With 1000 requests/day | 50-500MB/day | 500MB-5GB/day |
> **Recommendation**: Only enable raw output for **debugging sessions** or **sampled auditing**, not always-on.
### Recovering the Original
```ts
import { readRtkRawOutput } from "omniroute/compression/engines/rtk/rawOutput";
const raw = readRtkRawOutput(pointerId); // pointerId from compression stats
if (raw) {
console.log("Original output:", raw);
}
```
The `pointerId` is returned in `CompressionStats.rtkRawOutputPointers[]` after compression.
See `open-sse/services/compression/engines/rtk/rawOutput.ts:102` for the function signature.
### The Verify Gate
The **RTK Filter Verification** (`open-sse/services/compression/engines/rtk/verify.ts`) validates all filters against their `tests[]` and ensures behavior is correct at all 3 intensity levels.
**Call `runRtkFilterTests()`** to run verification:
```ts
import { runRtkFilterTests } from "open-sse/services/compression/engines/rtk/verify";
const result = runRtkFilterTests();
console.log(`Passed: ${result.outcomes.filter((o) => o.passed).length}`);
console.log(`Failed: ${result.outcomes.filter((o) => !o.passed).length}`);
if (!result.passed) {
console.error("Filters failed verification");
result.outcomes
.filter((o) => !o.passed)
.forEach((o) => {
console.error(
` - ${o.filterId} / ${o.testName}: expected "${o.expected}", got "${o.actual}"`
);
});
}
```
**What it validates**:
1. Every filter loads and passes schema validation
2. Every `tests[]` entry produces expected output
3. `minimal` intensity is a no-op (preserves original, only applies structural filters)
4. `aggressive` intensity preserves errors, test failures, and stack traces
5. Compressed output is never larger than original input
- Source: `open-sse/services/compression/engines/rtk/` (63 files, ~70KB)
- **Before merging a filter change** — always ensure tests pass
- **After upgrading RTK engine** — schema may have changed
- **Periodically in monitoring** — protects against drift in test fixtures
- **When adding a new tool/command family** — proves the new filter works
---
## See Also
- [COMPRESSION_GUIDE.md](./COMPRESSION_GUIDE.md) — Full compression pipeline overview
- [COMPRESSION_ENGINES.md](./COMPRESSION_ENGINES.md) — Engine registry and built-in engines
- [EXTENDING_COMPRESSION.md](./EXTENDING_COMPRESSION.md) — Custom engines, language packs, stacked pipelines
- Source: `open-sse/services/compression/engines/rtk/` (63 files, ~70KB)
docs/compression/meta.json
+11
View File
@@ -0,0 +1,11 @@
{
"title": "Compression",
"pages": [
"COMPRESSION_GUIDE",
"COMPRESSION_ENGINES",
"RTK_COMPRESSION",
"COMPRESSION_LANGUAGE_PACKS",
"COMPRESSION_RULES_FORMAT",
"EXTENDING_COMPRESSION"
]
}
docs/diagrams/README.md
+62
View File
@@ -0,0 +1,62 @@
---
title: "Diagrams"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Diagrams
Mermaid sources (`.mmd`) and exported SVGs for OmniRoute v3.8.0 architecture flows.
## Canonical diagrams
| Source | Exported | Used in |
| ---------------------------------------------------- | ----------------------------------------- | ------------------------------------------------------------------------------ |
| [request-pipeline.mmd](./request-pipeline.mmd) | [SVG](./exported/request-pipeline.svg) | docs/architecture/ARCHITECTURE.md, docs/architecture/CODEBASE_DOCUMENTATION.md |
| [auto-combo-12factor.mmd](./auto-combo-12factor.mmd) | [SVG](./exported/auto-combo-12factor.svg) | docs/routing/AUTO-COMBO.md |
| [resilience-3layers.mmd](./resilience-3layers.mmd) | [SVG](./exported/resilience-3layers.svg) | docs/architecture/RESILIENCE_GUIDE.md, CLAUDE.md |
| [i18n-flow.mmd](./i18n-flow.mmd) | [SVG](./exported/i18n-flow.svg) | docs/guides/I18N.md |
| [mcp-tools-94.mmd](./mcp-tools-94.mmd) | [SVG](./exported/mcp-tools-94.svg) | docs/frameworks/MCP-SERVER.md |
| [cloud-agent-flow.mmd](./cloud-agent-flow.mmd) | [SVG](./exported/cloud-agent-flow.svg) | docs/frameworks/CLOUD_AGENT.md |
| [authz-pipeline.mmd](./authz-pipeline.mmd) | [SVG](./exported/authz-pipeline.svg) | docs/architecture/AUTHZ_GUIDE.md |
| [db-schema-overview.mmd](./db-schema-overview.mmd) | [SVG](./exported/db-schema-overview.svg) | docs/architecture/CODEBASE_DOCUMENTATION.md |
## How to update
1. Edit `*.mmd`.
2. Re-render: `npm run docs:render-diagrams` (uses `@mermaid-js/mermaid-cli`).
3. Commit both `.mmd` and `.svg`.
If `@mermaid-js/mermaid-cli` is not available locally, install it once:
```bash
npm install -g @mermaid-js/mermaid-cli
```
The script renders every `.mmd` in `docs/diagrams/` into `docs/diagrams/exported/*.svg`
with a white background, suitable for both dark and light themes.
## Linking from a doc
From a doc in `docs/<subfolder>/`, the relative path becomes `../diagrams/...`:
```markdown
![Request pipeline](../diagrams/exported/request-pipeline.svg)
> Source: [../diagrams/request-pipeline.mmd](../diagrams/request-pipeline.mmd)
```
From the repo root (e.g. `CLAUDE.md`):
```markdown
![Resilience layers](./exported/resilience-3layers.svg)
```
## Conventions
- One concept per diagram. Don't try to fit the whole platform in one chart.
- Keep node labels short (3-6 words). Use `<br/>` for line breaks inside nodes.
- Prefer `flowchart LR` for pipelines and `flowchart TB` for layered models.
- Use `sequenceDiagram` for interactive (request/response) flows.
- Use `erDiagram` for database schema overviews.
- Update both `.mmd` and `.svg` in the same commit. Keep them in lock-step.
docs/diagrams/authz-pipeline.mmd
+23
View File
@@ -0,0 +1,23 @@
%% AuthZ pipeline (3 route classes + policy evaluation)
%% Reflects: src/middleware.ts, src/server/authz/pipeline.ts,
%% src/server/authz/policies/{public,clientApi,management}.ts
%% v3.8.0
flowchart LR
Req["Incoming request"] --> Strip["Strip trusted internal headers<br/>(x-omniroute-auth-*)"]
Strip --> Classify["classifyRoute()"]
Classify -->|PUBLIC| PubP["publicPolicy<br/>(login, health, status)"]
Classify -->|CLIENT_API| CliP["clientApiPolicy<br/>(/api/v1/*, /v1/*)"]
Classify -->|MANAGEMENT| MgmtP["managementPolicy<br/>(dashboard, settings, admin)"]
PubP -->|allow| Stamp["Stamp x-omniroute-auth-*<br/>(kind / id / label / scopes)"]
CliP -->|extract Bearer| Validate["validateApiKey()"]
Validate -->|valid| Stamp
Validate -->|"invalid (REQUIRE_API_KEY=true)"| Reject401["401 AUTH_002"]
Validate -->|"absent (REQUIRE_API_KEY=false)"| Anon["anonymous fallthrough"] --> Stamp
MgmtP -->|session ok| Stamp
MgmtP -->|Bearer w/ manage scope| Stamp
MgmtP -->|"Bearer invalid"| Reject403["403 AUTH_001"]
MgmtP -->|no auth| Reject401Mgmt["401 / 302 /login"]
Stamp --> Handler["Handler<br/>(uses assertAuth)"]
docs/diagrams/auto-combo-12factor.mmd
+26
View File
@@ -0,0 +1,26 @@
%% Auto-Combo 12-factor scoring
%% Reflects: open-sse/services/autoCombo/scoring.ts (DEFAULT_WEIGHTS, sum = 1.0)
%% v3.8.40
flowchart TB
Request["Incoming request"] --> Candidates["Eligible candidates<br/>(provider × model × account)"]
Candidates --> Score["Compute composite score<br/>per candidate"]
subgraph Factors["12-factor scoring weights (sum = 1.0)"]
f1["health (0.20)"]
f2["quota (0.15)"]
f3["costInv (0.15)"]
f4["latencyInv (0.12)"]
f5["taskFit (0.08)"]
f6["stability (0.05)"]
f7["tierPriority (0.05)"]
f8["tierAffinity (0.05)"]
f9["specificityMatch (0.05)"]
f10["contextAffinity (0.05)"]
f11["connectionDensity (0.05)"]
f12["resetWindowAffinity (0.00)"]
end
Score --> Factors
Factors --> Final["Sort by score<br/>(desc)"]
Final --> Top["Pick top-N targets"]
Top --> Dispatch["Dispatch sequentially<br/>(short-circuit on success)"]
docs/diagrams/cloud-agent-flow.mmd
+32
View File
@@ -0,0 +1,32 @@
%% Cloud Agent task lifecycle
%% Reflects: src/lib/cloudAgent/* and src/app/api/v1/agents/tasks/*
%% Supported agents: codex-cloud, devin, jules
%% v3.8.0
sequenceDiagram
participant U as User
participant API as /v1/agents/tasks
participant Reg as Cloud Agent Registry
participant Ag as Cloud Agent<br/>(codex-cloud / devin / jules)
participant DB as SQLite domain DB
U->>API: POST createTask (description, sources)
API->>API: Zod validation + AuthZ (management)
API->>Reg: getAgent(providerId)
Reg->>Ag: instantiate with credentials
Ag->>Ag: createTask() returns planning
Ag-->>API: taskId + status: planning
API->>DB: insertCloudAgentTask
API-->>U: taskId
U->>API: GET /tasks/:id (poll)
API->>Ag: getStatus(externalId)
Ag-->>API: status, plan, messages
API->>DB: updateCloudAgentTask
API-->>U: snapshot
U->>API: POST approvePlan
API->>Ag: approvePlan(externalId)
Ag-->>API: status: running
Note over Ag: Async execution upstream
Ag-->>DB: stream events / final result
docs/diagrams/db-schema-overview.mmd
+16
View File
@@ -0,0 +1,16 @@
%% Database schema overview (selected core tables)
%% Reflects: src/lib/db/* (95+ modules, 110+ migrations)
%% v3.8.0
erDiagram
api_keys ||--o{ api_key_usage : tracks
api_keys ||--o{ rate_limits : enforces
providers ||--o{ provider_connections : holds
provider_connections ||--o{ domain_circuit_breakers : protected_by
combos ||--o{ combo_targets : contains
combos ||--o{ combo_executions : logs
spend_buffer ||--o{ usage_aggregations : flushed_to
memory_documents ||--o{ memory_chunks : split_into
skills_runs ||--o{ skills_logs : produced
agent_tasks ||--o{ agent_task_events : streams
audit_log }|--|| api_keys : by
mcp_audit }|--|| api_keys : by
docs/diagrams/exported/authz-pipeline.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

docs/diagrams/exported/auto-combo-12factor.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

docs/diagrams/exported/cloud-agent-flow.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

docs/diagrams/exported/db-schema-overview.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

docs/diagrams/exported/i18n-flow.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

docs/diagrams/exported/mcp-tools-94.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 22 KiB

docs/diagrams/exported/request-pipeline.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

docs/diagrams/exported/resilience-3layers.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

docs/diagrams/i18n-flow.mmd
+14
View File
@@ -0,0 +1,14 @@
%% Incremental hash-based i18n pipeline
%% Reflects: scripts/i18n/* and docs/i18n/<locale>/*.md mirror layout
%% v3.8.0
flowchart LR
Src["Source MDs<br/>(CLAUDE.md, docs/**/*.md)"] --> Hash["sha256 hash"]
Hash --> State[".i18n-state.json"]
State -->|diff| Dirty["Mark source dirty"]
Dirty --> Loop{"For each locale<br/>(39 langs)"}
Loop --> LLM["OmniRoute<br/>/chat/completions<br/>(cx/gpt-5.4-mini)"]
LLM --> Target["Write<br/>docs/i18n/&lt;locale&gt;/&lt;rel-path&gt;.md"]
Target --> State
Target --> CI{"CI drift check<br/>(npm run i18n:check)"}
CI -->|in sync| Pass["pass"]
CI -->|drift| Fail["fail<br/>(exit 1)"]
docs/diagrams/mcp-tools-94.mmd
+16
View File
@@ -0,0 +1,16 @@
%% MCP Server tool inventory by category
%% Reflects: open-sse/mcp-server/ — TOTAL_MCP_TOOL_COUNT in server.ts
%% (MCP_TOOLS 34 + memory 3 + skill 4 + agentSkill 3 + pool 6 + gamification 8 + plugin 8 + notion 6 + obsidian 22 = 94)
%% Regenerate the SVG with: npm run docs:render-diagrams
%% v3.8.40
flowchart LR
MCP["MCP Server<br/>94 tools total"]
MCP --> Core["Core (34)<br/>routing, cache, compression,<br/>quota, proxy, tunnels, search, web_fetch"]
MCP --> Mem["Memory (3)<br/>memory_search, memory_save,<br/>memory_delete"]
MCP --> Skl["Skills (4)<br/>skill_invoke, skill_list,<br/>skill_diagnose, skill_uninstall"]
MCP --> ASkl["Agent-Skills (3)<br/>cli-registry discovery"]
MCP --> Pool["Pool (6)<br/>pool_status, pool_sessions, pool_reset,<br/>pool_warm, pool_health, browser_pool_status"]
MCP --> Gam["Gamification (8)"]
MCP --> Plg["Plugins (8)"]
MCP --> Notion["Notion (6)<br/>search, get_page, query_database,<br/>get_database, list_block_children,<br/>append_blocks"]
MCP --> Obs["Obsidian (22)<br/>vault read/write, search,<br/>frontmatter, daily notes, …"]
docs/diagrams/request-pipeline.mmd
+24
View File
@@ -0,0 +1,24 @@
%% Request pipeline for /v1/chat/completions
%% Reflects: src/app/api/v1/chat/completions/route.ts → open-sse/handlers/chatCore.ts
%% v3.8.0
flowchart LR
Client["Client<br/>(IDE/CLI/SDK)"] --> Route["Next.js Route<br/>/v1/chat/completions"]
Route --> CORS["CORS preflight"]
CORS --> Validate["Zod validation<br/>(request body)"]
Validate --> Authz["AuthZ pipeline<br/>(extractApiKey +<br/>isValidApiKey)"]
Authz --> Policy["API key policy<br/>(allowlist + scopes)"]
Policy --> Guard["Prompt-injection<br/>guardrail"]
Guard --> ChatCore["handleChatCore"]
ChatCore --> Cache{"Cache hit?"}
Cache -->|yes| Return["Return cached"]
Cache -->|no| Rate["Rate limit<br/>(per-key, per-IP)"]
Rate --> Combo{"Combo<br/>target?"}
Combo -->|combo| Resolve["resolveComboTargets<br/>(17 strategies)"]
Resolve --> Single["handleSingleModel<br/>(per target)"]
Combo -->|single| Single
Single --> Translate["translateRequest<br/>(OpenAI↔Claude↔Gemini)"]
Translate --> Exec["getExecutor<br/>(31 executors)"]
Exec --> Upstream["Upstream Provider<br/>(177 providers)"]
Upstream --> Stream["SSE / JSON"]
Stream --> Transformer["responsesTransformer<br/>(Responses↔Chat)"]
Transformer --> Client
docs/diagrams/resilience-3layers.mmd
+21
View File
@@ -0,0 +1,21 @@
%% 3-layer resilience model
%% Reflects: src/shared/utils/circuitBreaker.ts, src/sse/services/auth.ts,
%% open-sse/services/accountFallback.ts
%% v3.8.0
flowchart TB
Req["Request"] --> L1{"Provider Circuit Breaker<br/>(scope: whole provider)"}
L1 -->|CLOSED| L2
L1 -->|OPEN| Block1["Skip provider<br/>(or 503 retry-after)"]
L1 -->|HALF_OPEN| Probe["Allow probe"]
Probe --> L2
L2{"Connection Cooldown<br/>(scope: one account/key)"} -->|available| L3
L2 -->|cooling down| Skip2["Skip this connection<br/>fall back to next"]
L3{"Model Lockout<br/>(scope: provider × conn × model)"} -->|allowed| Exec["Execute upstream"]
L3 -->|locked| Skip3["Fall back to next model"]
Exec -->|success| Clear["Clear cooldowns/lockouts"]
Exec -->|"408 / 500 / 502 / 503 / 504"| TripL1["Trip provider breaker"]
Exec -->|"401 / 403 (account)"| TripL2["Start connection cooldown"]
Exec -->|"429 quota (model)"| TripL3["Start model lockout"]
docs/frameworks/A2A-SERVER.md
+295
View File
@@ -0,0 +1,295 @@
---
title: "OmniRoute A2A Server Documentation"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute A2A Server Documentation
> Agent-to-Agent Protocol v0.3 — OmniRoute as an intelligent routing agent
The A2A surface has two faces:
- **JSON-RPC 2.0** at `POST /a2a` (canonical entry point, defined in `src/app/a2a/route.ts`).
- **REST** under `/api/a2a/*` for dashboards and tooling (status, task list, cancel).
Tasks are tracked by `A2ATaskManager` (`src/lib/a2a/taskManager.ts`, default 5-minute TTL). Skills are dispatched via `A2A_SKILL_HANDLERS` in `src/lib/a2a/taskExecution.ts`.
## Agent Discovery
```bash
curl http://localhost:20128/.well-known/agent.json
```
Returns the Agent Card describing OmniRoute's capabilities, skills, and authentication requirements.
The Agent Card's `version` field is sourced from `process.env.npm_package_version` (see `src/app/.well-known/agent.json/route.ts:13`), so it stays auto-synced with `package.json` on every release.
---
## Authentication
All `/a2a` requests require an API key via the `Authorization` header:
```
Authorization: Bearer YOUR_OMNIROUTE_API_KEY
```
If no API key is configured on the server, authentication is bypassed.
## Enablement
A2A is controlled by the **Endpoints → A2A** toggle and is disabled by default. When disabled,
`GET /api/a2a/status` reports `status: "disabled"` and `online: false`; JSON-RPC calls to
`POST /a2a` return HTTP 503 with JSON-RPC error code `-32000`.
---
## JSON-RPC 2.0 Methods
### `message/send` — Synchronous Execution
Sends a message to a skill and waits for the complete response.
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{
"jsonrpc": "2.0",
"id": "1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Write a hello world in Python"}],
"metadata": {"model": "auto", "combo": "fast-coding"}
}
}'
```
**Response:**
```json
{
"jsonrpc": "2.0",
"id": "1",
"result": {
"task": { "id": "uuid", "state": "completed" },
"artifacts": [{ "type": "text", "content": "..." }],
"metadata": {
"routing_explanation": "Selected claude-sonnet via provider \"anthropic\" (latency: 1200ms, cost: $0.003)",
"cost_envelope": { "estimated": 0.005, "actual": 0.003, "currency": "USD" },
"resilience_trace": [
{ "event": "primary_selected", "provider": "anthropic", "timestamp": "..." }
],
"policy_verdict": { "allowed": true, "reason": "within budget and quota limits" }
}
}
}
```
### `message/stream` — SSE Streaming
Same as `message/send` but returns Server-Sent Events for real-time streaming.
```bash
curl -N -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{
"jsonrpc": "2.0",
"id": "1",
"method": "message/stream",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Explain quantum computing"}]
}
}'
```
**SSE Events:**
```
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"working"},"chunk":{"type":"text","content":"..."}}}
: heartbeat 2026-03-03T17:00:00Z
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"completed"},"metadata":{...}}}
```
### `tasks/get` — Query Task Status
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"jsonrpc":"2.0","id":"2","method":"tasks/get","params":{"taskId":"TASK_UUID"}}'
```
### `tasks/cancel` — Cancel a Task
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"jsonrpc":"2.0","id":"3","method":"tasks/cancel","params":{"taskId":"TASK_UUID"}}'
```
---
## Available Skills
OmniRoute exposes 6 A2A skills wired in `src/lib/a2a/taskExecution.ts::A2A_SKILL_HANDLERS`. Each skill module lives in `src/lib/a2a/skills/`.
| Skill | ID | Description | Tags | Examples |
| :----------------- | :------------------- | :-------------------------------------------------------------------------------------------------------------- | :------------------------- | :------------------------------------- |
| Smart Routing | `smart-routing` | Routes a prompt through the optimal provider/combo using OmniRoute's combo engine + scoring | routing, providers | "Route this prompt via the best model" |
| Quota Management | `quota-management` | Reports per-provider quota state, helps callers decide when to throttle/switch | quota, providers | "Check quota for anthropic" |
| Provider Discovery | `provider-discovery` | Lists installed providers with capabilities, free-tier flags, OAuth status | providers, discovery | "What providers are available?" |
| Cost Analysis | `cost-analysis` | Estimates cost of a request/conversation given the catalog + recent usage | cost, usage | "Estimate cost for this conversation" |
| Health Report | `health-report` | Aggregates circuit breaker, cooldown, lockout state per provider | health, resilience | "Show health status of all providers" |
| List Capabilities | `list-capabilities` | Returns the full 42-entry Agent Skills catalog as a markdown table with raw SKILL.md URLs for context injection | catalog, discovery, skills | "List all OmniRoute capabilities" |
> Note: the Agent Card description currently advertises "36+ providers" (`src/app/.well-known/agent.json/route.ts:26` and `:55`). The actual catalog has grown to 180+ providers — the string should be updated in a follow-up change (tracked as a separate doc/code TODO; not modified here).
### `list-capabilities` Skill Detail
The `list-capabilities` skill is particularly useful for external agents that need to discover what OmniRoute exposes before sending API calls. It returns a structured markdown table artifact:
```
| ID | Name | Category | Area | Endpoints/Commands | Raw URL |
| --- | --- | --- | --- | --- | --- |
| omni-auth | Auth & Sessions | api | auth | POST /api/auth/login, ... | https://raw.githubusercontent.com/... |
...
```
Each row includes the `rawUrl` column so agents can immediately fetch the full SKILL.md. The `metadata.totalSkills` field is always `42`. Implementation: `src/lib/a2a/skills/listCapabilities.ts`. See also [AGENT-SKILLS.md](./AGENT-SKILLS.md).
---
## REST API (auxiliary)
The JSON-RPC endpoint `/a2a` is the canonical A2A entry point. The REST endpoints below provide auxiliary access for dashboards and external tooling:
| Endpoint | Method | Description | Auth |
| :--------------------------- | :----- | :------------------------------- | :--------------------- |
| `/api/a2a/status` | GET | Server status, registered skills | (public) |
| `/api/a2a/tasks` | GET | List tasks with filters | management |
| `/api/a2a/tasks/[id]` | GET | Get task by ID | management |
| `/api/a2a/tasks/[id]/cancel` | POST | Cancel running task | management |
| `/.well-known/agent.json` | GET | Agent Card (A2A discovery) | (public, cached 3600s) |
---
## Adding a New Skill
1. **Create skill file:** `src/lib/a2a/skills/<your-skill>.ts`
Export an async function `(task: A2ATask) => Promise<{ artifacts, metadata }>`. Follow the shape of existing skills such as `smartRouting.ts`.
2. **Register handler:** in `src/lib/a2a/taskExecution.ts`, add an entry to `A2A_SKILL_HANDLERS`:
```typescript
export const A2A_SKILL_HANDLERS = {
// ...existing skills
"your-skill": async (task) => {
const skillModule = await import("./skills/yourSkill");
return skillModule.executeYourSkill(task);
},
};
```
3. **Expose in Agent Card:** in `src/app/.well-known/agent.json/route.ts`, append to the `skills` array:
```json
{
"id": "your-skill",
"name": "Your Skill",
"description": "Brief, intent-focused description",
"tags": ["routing", "quota"],
"examples": ["Sample natural-language invocation"]
}
```
4. **Write tests:** `tests/unit/a2a-<your-skill>.test.ts`. Cover happy path + error path.
5. **Document** the new skill in this file's `Available Skills` table.
---
## Task TTL
Tasks expire after `ttlMinutes` (default 5 min) — configured in the `A2ATaskManager` constructor at `src/lib/a2a/taskManager.ts:82`. To customize, fork the `A2ATaskManager` instantiation and pass a different value (e.g., `new A2ATaskManager(15)` for 15-minute TTL). A background interval sweeps expired tasks every 60 seconds.
---
## Task Lifecycle
```
submitted → working → completed
→ failed
→ cancelled
```
- Tasks expire after 5 minutes by default (see [Task TTL](#task-ttl))
- Terminal states: `completed`, `failed`, `cancelled`
- Event log tracks every state transition
---
## Error Codes
| Code | Meaning |
| :----- | :----------------------------- |
| -32700 | Parse error (invalid JSON) |
| -32600 | Invalid request / Unauthorized |
| -32601 | Method or skill not found |
| -32602 | Invalid params |
| -32603 | Internal error |
| -32000 | A2A endpoint is disabled |
---
## Integration Examples
### Python (requests)
```python
import requests
resp = requests.post("http://localhost:20128/a2a", json={
"jsonrpc": "2.0", "id": "1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Hello"}]
}
}, headers={"Authorization": "Bearer YOUR_KEY"})
result = resp.json()["result"]
print(result["artifacts"][0]["content"])
print(result["metadata"]["routing_explanation"])
```
### TypeScript (fetch)
```typescript
const resp = await fetch("http://localhost:20128/a2a", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: "Bearer YOUR_KEY",
},
body: JSON.stringify({
jsonrpc: "2.0",
id: "1",
method: "message/send",
params: {
skill: "smart-routing",
messages: [{ role: "user", content: "Hello" }],
},
}),
});
const { result } = await resp.json();
console.log(result.metadata.routing_explanation);
```
docs/frameworks/ACP.md
+559
View File
@@ -0,0 +1,559 @@
---
title: ACP (Agent Client Protocol)
---
# ACP (Agent Client Protocol)
> **TL;DR**: ACP lets OmniRoute spawn CLI agents (like Claude Code, Codex) as child processes instead of using HTTP APIs. This gives you "CLI-as-backend" transport.
---
## What Is ACP?
ACP (Agent Client Protocol) is a **"CLI-as-backend" transport** for OmniRoute. Instead of intercepting HTTP API calls to AI providers, ACP **spawns CLI agents as child processes** and feeds prompts through their native interface.
### Why Use ACP?
| Benefit | Description |
| ---------------------- | ------------------------------------------ |
| **No API keys needed** | Uses your existing CLI authentication |
| **Native protocol** | Uses each CLI's native input/output format |
| **Auto-discovery** | Detects installed CLIs on your system |
| **14 built-in agents** | Pre-configured for popular CLI tools |
| **Custom agents** | Add your own CLI tools via settings |
| **Process management** | Handles lifecycle (spawn, send, kill) |
---
## Supported CLI Agents
ACP supports **14 built-in CLI agents** out of the box:
| Agent ID | Display Name | Binary | Protocol |
| ------------- | ------------------ | ------------- | -------- |
| `codex` | OpenAI Codex CLI | `codex` | stdio |
| `claude` | Claude Code CLI | `claude` | stdio |
| `goose` | Goose CLI | `goose` | stdio |
| `openclaw` | OpenClaw | `openclaw` | stdio |
| `aider` | Aider | `aider` | stdio |
| `opencode` | OpenCode | `opencode` | stdio |
| `cline` | Cline | `cline` | stdio |
| `qwen-code` | Qwen Code | `qwen` | stdio |
| `forge` | ForgeCode | `forge` | stdio |
| `amazon-q` | Amazon Q Developer | `q` | stdio |
| `interpreter` | Open Interpreter | `interpreter` | stdio |
| `cursor-cli` | Cursor CLI | `cursor` | stdio |
| `warp` | Warp AI | `warp` | stdio |
### Custom Agents
You can add your own CLI agents via settings. Custom agents support the same features as built-in agents.
---
## Quick Start
### Step 1: Install a CLI Agent
```bash
# Example: Install Claude Code CLI
npm install -g @anthropic-ai/claude-code
# Verify installation
claude --version
```
### Step 2: ACP Auto-Detection
ACP automatically detects installed CLI agents on your system. No configuration needed!
### Step 3: Use ACP Transport
Once detected, ACP can be used as a transport for any supported provider. OmniRoute will automatically use ACP when the CLI is available.
---
## How ACP Works
### Architecture
```
┌─────────────────┐
│ OmniRoute │
│ (HTTP Proxy) │
└────────┬────────┘
│
│ spawn()
▼
┌─────────────────┐
│ Child Process │
│ (CLI Agent) │
│ │
│ stdin ◄──────┤ Send prompt
│ stdout ──────►│ Receive response
│ stderr ──────►│ Receive errors
└─────────────────┘
```
### Process Lifecycle
1. **Spawn** — ACP creates a child process for the CLI agent
2. **Send** — ACP writes prompts to the process's stdin
3. **Receive** — ACP reads responses from stdout/stderr
4. **Idle Detection** — ACP waits 2 seconds of inactivity before considering the response complete
5. **Kill** — ACP terminates the process (SIGTERM, then SIGKILL after 5s)
### Communication Protocol
ACP uses **stdio** (standard input/output) for communication with CLI agents. The protocol is:
1. **Send prompt** — Write to stdin with a newline
2. **Wait for response** — Read from stdout until idle (2s of no output)
3. **Timeout** — Default 120 seconds (configurable)
---
## API Reference
### Registry Functions
#### `detectInstalledAgents()`
Detects all installed CLI agents on the system. Results are cached for 60 seconds.
```typescript
import { detectInstalledAgents } from "@/lib/acp";
const agents = detectInstalledAgents();
// Returns: CliAgentInfo[]
interface CliAgentInfo {
id: string; // e.g., "codex", "claude"
name: string; // Display name
binary: string; // Binary name to spawn
versionCommand: string; // Version detection command
version: string | null; // Detected version (null if not installed)
installed: boolean; // Whether the agent is installed
providerAlias: string; // Provider ID in OmniRoute
spawnArgs: string[]; // Arguments to pass when spawning
protocol: "stdio" | "http"; // Communication protocol
isCustom?: boolean; // Whether this is a user-defined custom agent
}
```
#### `getAvailableAgents()`
Gets only the agents that are installed and available for ACP.
```typescript
import { getAvailableAgents } from "@/lib/acp";
const available = getAvailableAgents();
// Returns: CliAgentInfo[] (only installed agents)
```
#### `getAgentById(id)`
Gets a specific agent by ID.
```typescript
import { getAgentById } from "@/lib/acp";
const agent = getAgentById("claude");
// Returns: CliAgentInfo | undefined
```
#### `setCustomAgents(agents)`
Sets custom agent definitions from settings.
```typescript
import { setCustomAgents } from "@/lib/acp";
setCustomAgents([
{
id: "my-custom-cli",
name: "My Custom CLI",
binary: "mycli",
versionCommand: "mycli --version",
providerAlias: "my-provider",
spawnArgs: [],
protocol: "stdio",
},
]);
```
### Manager Functions
#### `acpManager.spawn(agentId, binary, args, env)`
Spawns a new CLI agent process.
```typescript
import { acpManager } from "@/lib/acp";
const session = acpManager.spawn("claude", "claude", ["--print", "--output-format", "json"], {
/* custom env vars */
});
// Returns: AcpSession
```
**Allowed agent IDs**: `["claude", "codex", "gemini", "qwen"]`
#### `acpManager.sendPrompt(sessionId, prompt, timeoutMs)`
Sends a prompt to a CLI agent and collects the response.
```typescript
import { acpManager } from "@/lib/acp";
const response = await acpManager.sendPrompt(
"acp-claude-1234567890-abc123",
"What is 2+2?",
120000 // 2 minutes timeout
);
// Returns: Promise<string>
```
#### `acpManager.kill(sessionId)`
Kills a session and cleans up.
```typescript
import { acpManager } from "@/lib/acp";
const killed = acpManager.kill("acp-claude-1234567890-abc123");
// Returns: boolean
```
#### `acpManager.getActiveSessions()`
Gets all active sessions.
```typescript
import { acpManager } from "@/lib/acp";
const sessions = acpManager.getActiveSessions();
// Returns: AcpSession[]
```
#### `acpManager.killAll()`
Kills all sessions.
```typescript
import { acpManager } from "@/lib/acp";
acpManager.killAll();
```
### Session Interface
```typescript
interface AcpSession {
id: string; // Unique session ID
agentId: string; // Agent ID (e.g., "claude")
process: ChildProcess; // Child process handle
alive: boolean; // Whether the process is alive
stdoutBuffer: string; // Accumulated stdout buffer
stderrBuffer: string; // Accumulated stderr buffer
createdAt: Date; // Created timestamp
}
```
### Events
The `AcpManager` extends `EventEmitter` and emits the following events:
#### `stdout`
Emitted when the CLI agent writes to stdout.
```typescript
acpManager.on("stdout", ({ sessionId, data }) => {
console.log(`[${sessionId}] stdout: ${data}`);
});
```
#### `stderr`
Emitted when the CLI agent writes to stderr.
```typescript
acpManager.on("stderr", ({ sessionId, data }) => {
console.error(`[${sessionId}] stderr: ${data}`);
});
```
#### `exit`
Emitted when the CLI agent process exits.
```typescript
acpManager.on("exit", ({ sessionId, code, signal }) => {
console.log(`[${sessionId}] exited with code ${code}, signal ${signal}`);
});
```
#### `error`
Emitted when the CLI agent process errors.
```typescript
acpManager.on("error", ({ sessionId, error }) => {
console.error(`[${sessionId}] error: ${error}`);
});
```
---
## Configuration
### Environment Variables
ACP inherits all environment variables from the parent process and can be extended with custom env vars:
```typescript
acpManager.spawn("claude", "claude", [], {
ANTHROPIC_API_KEY: "sk-...",
DEBUG: "true",
});
```
### Spawn Arguments
Each agent has default spawn arguments defined in the registry. You can override them:
```typescript
acpManager.spawn("claude", "claude", ["--print", "--verbose"], {});
```
### Timeouts
Default prompt timeout is **120 seconds** (2 minutes). You can override:
```typescript
await acpManager.sendPrompt(sessionId, prompt, 300000); // 5 minutes
```
### Detection Cache
Agent detection is cached for **60 seconds** to avoid expensive filesystem scans. Force refresh:
```typescript
import { refreshAgentCache } from "@/lib/acp";
refreshAgentCache();
```
---
## Security
### Command Injection Prevention
ACP validates version commands to prevent command injection attacks:
```typescript
const DISALLOWED_VERSION_COMMAND_CHARS = /[;&|<>`$\r\n]/;
```
Version commands containing these characters are rejected:
- `;` — Command separator
- `&` — Background process
- `|` — Pipe
- `<`, `>` — Redirection
- `` ` `` — Command substitution
- `$` — Variable expansion
- `\r`, `\n` — Line breaks
### Binary Name Validation
ACP validates that the version command binary matches the expected binary name (unless it's a custom agent).
### Process Isolation
Each ACP session runs in its own child process. The process is killed when the session ends or times out.
---
## Performance
### Detection Performance
- **First call**: ~50-200ms (runs `version` command for each agent)
- **Cached calls**: <1ms (returns from cache)
- **Cache TTL**: 60 seconds
### Prompt Performance
- **Spawn**: ~50-100ms
- **Send prompt**: ~10-50ms
- **Wait for response**: Depends on CLI agent (typically 1-30 seconds)
- **Kill**: ~5 seconds (SIGTERM) + immediate (SIGKILL)
### Resource Usage
- **Memory per session**: ~10-50MB (depends on CLI agent)
- **CPU**: Minimal (I/O bound)
- **Disk**: None
---
## Troubleshooting
### "Unknown agent" Error
**Problem**: `acpManager.spawn()` throws `Unknown agent: <id>`
**Solution**: Only 4 agents are allowed in `spawn()`:
- `claude`
- `codex`
- `gemini`
- `qwen`
Other agents must be spawned manually or via custom agent definitions.
### "Session not alive" Error
**Problem**: `acpManager.sendPrompt()` throws `Session ${sessionId} is not alive`
**Solution**: The session may have exited or been killed. Check session status:
```typescript
const session = acpManager.getSession(sessionId);
if (!session?.alive) {
// Re-spawn the session
acpManager.spawn("claude", "claude", [], {});
}
```
### "ACP timeout" Error
**Problem**: `acpManager.sendPrompt()` throws `ACP timeout after 120000ms`
**Solution**: Increase the timeout:
```typescript
await acpManager.sendPrompt(sessionId, prompt, 300000); // 5 minutes
```
### CLI Not Detected
**Problem**: `detectInstalledAgents()` doesn't find your CLI
**Solutions**:
1. **Check PATH**: Ensure the CLI is in your system PATH
2. **Check version command**: Run `claude --version` manually
3. **Check permissions**: Ensure the CLI is executable
4. **Custom agent**: Add a custom agent definition for non-standard CLIs
### Permission Denied
**Problem**: ACP can't execute the CLI
**Solutions**:
1. **Check file permissions**: `chmod +x /usr/local/bin/claude`
2. **Check ownership**: Ensure OmniRoute has read/execute permissions
3. **Check SELinux/AppArmor**: May block process spawning
---
## Examples
### Example 1: Spawn and Use Claude Code
```typescript
import { acpManager, detectInstalledAgents } from "@/lib/acp";
// Detect installed agents
const agents = detectInstalledAgents();
const claude = agents.find((a) => a.id === "claude");
if (claude?.installed) {
// Spawn a new session
const session = acpManager.spawn("claude", claude.binary, ["--print", "--output-format", "json"]);
// Send a prompt
const response = await acpManager.sendPrompt(
session.id,
"Explain quantum computing in 100 words"
);
console.log("Claude's response:", response);
// Clean up
acpManager.kill(session.id);
}
```
### Example 2: Auto-Discovery with Fallback
```typescript
import { acpManager, getAvailableAgents } from "@/lib/acp";
const available = getAvailableAgents();
// Try Claude first, fallback to Codex
let agentId = "claude";
if (!available.find((a) => a.id === "claude")) {
if (available.find((a) => a.id === "codex")) {
agentId = "codex";
} else {
throw new Error("No ACP-compatible CLI agent found");
}
}
const agent = available.find((a) => a.id === agentId)!;
const session = acpManager.spawn(agentId, agent.binary, agent.spawnArgs);
const response = await acpManager.sendPrompt(session.id, "Hello!");
acpManager.kill(session.id);
```
### Example 3: Custom Agent
```typescript
import { setCustomAgents, detectInstalledAgents } from "@/lib/acp";
// Register a custom CLI agent
setCustomAgents([
{
id: "my-llm-cli",
name: "My LLM CLI",
binary: "myllm",
versionCommand: "myllm --version",
providerAlias: "my-llm-provider",
spawnArgs: ["--format", "json"],
protocol: "stdio",
},
]);
// Now detectInstalledAgents() will include "my-llm-cli"
const agents = detectInstalledAgents();
```
---
## What's Next?
- **[API Reference](../reference/API_REFERENCE.md)** — REST API endpoints
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 226 providers
- **[MCP Server](./MCP-SERVER.md)** — Model Context Protocol integration
- **[A2A Server](./A2A-SERVER.md)** — Agent-to-Agent protocol
- **[Cloud Agent](./CLOUD_AGENT.md)** — Cloud-based agents
---
## Reference
- [AionUi Project](https://github.com/iOfficeAI/AionUi) — Inspiration for ACP auto-detection
- [ACP Source Code](../../src/lib/acp/) — Implementation details
- `manager.ts` — Process lifecycle management
- `registry.ts` — Agent discovery and registration
- `index.ts` — Public API exports
docs/frameworks/AGENT-SKILLS.md
+309
View File
@@ -0,0 +1,309 @@
---
title: "OmniRoute Agent Skills Catalog"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute Agent Skills Catalog
> **Source of truth:** `src/lib/agentSkills/` (catalog, generator, parsers) + `skills/` directory (SKILL.md files)
> **Last updated:** 2026-06-28 — v3.8.40
Agent Skills are structured SKILL.md files that teach external agents, MCP clients, and A2A orchestrators how to use OmniRoute's REST API and CLI. Unlike [Omni Skills](./SKILLS.md) (which are LLM tool definitions executed inside OmniRoute), Agent Skills are a _documentation catalog_ — static markdown that can be fed directly into agent context.
---
## Overview
The catalog contains **42 canonical Agent Skills** (22 REST API + 20 CLI). Each skill has:
- A **canonical ID** (`omni-auth`, `cli-serve`, etc.)
- A **SKILL.md** file in `skills/{id}/SKILL.md` with YAML frontmatter (`name`, `description`) + rich markdown body
- **REST endpoints** (API skills) or **CLI subcommands** (CLI skills) derived from the OpenAPI spec and CLI registry
- A **GitHub raw URL** for live fetch: `https://raw.githubusercontent.com/diegosouzapw/OmniRoute/refs/heads/main/skills/{id}/SKILL.md`
---
## Architecture
```
src/shared/constants/agentSkills.ts — 42-entry curated list (name/desc/category/area/icon)
src/lib/agentSkills/
catalog.ts — getCatalog(), getSkillById(), filterCatalog(), computeCoverage()
generator.ts — generateAgentSkills() writes SKILL.md to skills/{id}/
openapiParser.ts — extracts REST endpoints from docs/openapi.yaml
cliRegistryParser.ts — extracts CLI subcommands from bin/cli-registry.ts
schemas.ts — Zod schemas: AgentSkillSchema, SkillCoverageSchema, etc.
types.ts — TypeScript interfaces: AgentSkill, SkillCoverage, etc.
skills/{id}/SKILL.md — Generated + curated markdown files (42 total)
src/app/api/agent-skills/
route.ts — GET /api/agent-skills
[id]/route.ts — GET /api/agent-skills/{id}
[id]/raw/route.ts — GET /api/agent-skills/{id}/raw (text/markdown)
coverage/route.ts — GET /api/agent-skills/coverage
generate/route.ts — POST /api/agent-skills/generate (auth required)
open-sse/mcp-server/tools/agentSkillTools.ts — 3 MCP tools (list, get, coverage)
src/lib/a2a/skills/listCapabilities.ts — A2A skill: list-capabilities
```
---
## SKILL.md Format
```markdown
---
name: omni-providers
description: "Manage provider connections: add, test, rotate, and remove credentials."
---
<!-- generated by src/lib/agentSkills/generator.ts; manual edits will be overwritten -->
## Overview
...
## Authentication
...
## Endpoints
...
<!-- skill:custom-start -->
## Custom Section (preserved across regeneration)
...
<!-- skill:custom-end -->
```
The generator preserves content between `<!-- skill:custom-start -->` and `<!-- skill:custom-end -->` on regeneration. Ten skills have curated custom blocks:
`omni-mcp`, `omni-compression`, `cli-providers`, `cli-eval`, `omni-agents-a2a`, `omni-combos-routing`, `omni-auth`, `omni-resilience`, `omni-inference`, `cli-serve`.
---
## REST API Discovery
| Endpoint | Method | Description | Auth |
| :--------------------------- | :----- | :------------------------------------------------------- | :--------- |
| `/api/agent-skills` | GET | List catalog (optional `?category=api\|cli&area=<area>`) | none |
| `/api/agent-skills/{id}` | GET | Get single skill metadata | none |
| `/api/agent-skills/{id}/raw` | GET | Fetch SKILL.md as `text/markdown` | none |
| `/api/agent-skills/coverage` | GET | Coverage stats (how many SKILL.md files exist) | none |
| `/api/agent-skills/generate` | POST | Trigger generator (dryRun/prune/onlyIds) | management |
Example — list all API skills:
```bash
curl "http://localhost:20128/api/agent-skills?category=api"
```
Example — fetch a single SKILL.md:
```bash
curl -H "Accept: text/markdown" "http://localhost:20128/api/agent-skills/omni-providers/raw"
```
---
## MCP Discovery
Three MCP tools are registered under scope `read:catalog`:
| Tool | Description |
| :-------------------------------- | :------------------------------------------------- |
| `omniroute_agent_skills_list` | List skills (optional `category` / `area` filters) |
| `omniroute_agent_skills_get` | Get metadata + SKILL.md for one skill by `id` |
| `omniroute_agent_skills_coverage` | Coverage stats (API/CLI have/total) |
See [MCP-SERVER.md](./MCP-SERVER.md) for scope wiring and authentication.
---
## A2A Discovery
The A2A skill `list-capabilities` returns the full 42-skill catalog as a markdown table artifact. External orchestrators can invoke it via:
```json
{
"jsonrpc": "2.0",
"id": "1",
"method": "message/send",
"params": {
"skill": "list-capabilities",
"messages": [{ "role": "user", "content": "List all capabilities" }]
}
}
```
See [A2A-SERVER.md](./A2A-SERVER.md) for protocol details.
---
## Catalog — 42 Skill IDs
### API Skills (22)
| ID | Area | Entry Point |
| :--------------------- | :-------------- | :---------------------------------- |
| `omni-auth` | auth | Auth + session management |
| `omni-providers` | providers | Provider connection management |
| `omni-models` | models | Model catalog and capabilities |
| `omni-combos-routing` | combos-routing | Combo routing strategies |
| `omni-api-keys` | api-keys | API key management |
| `omni-usage-logs` | usage-logs | Usage and cost logs |
| `omni-budget` | budget | Budget guards |
| `omni-settings` | settings | Global settings |
| `omni-proxies` | proxies | Proxy pool management |
| `omni-cache` | cache | Semantic + prompt cache |
| `omni-compression` | compression | Context compression engines |
| `omni-context-rtk` | context-rtk | RTK compression |
| `omni-resilience` | resilience | Circuit breakers + cooldowns |
| `omni-cli-tools` | cli-tools | CLI tools REST proxy |
| `omni-tunnels` | tunnels | Tunnel management |
| `omni-sync-cloud` | sync-cloud | Cloud sync |
| `omni-db-backups` | db-backups | Database backups |
| `omni-webhooks` | webhooks | Webhook event dispatcher |
| `omni-mcp` | mcp | MCP server (87 tools, 3 transports) |
| `omni-agents-a2a` | agents-a2a | A2A agent protocol |
| `omni-version-manager` | version-manager | Version and update management |
| `omni-inference` | inference | Direct inference / completions |
### CLI Skills (20)
| ID | Area | CLI Command Root |
| :------------------- | :----------------- | :---------------------- |
| `cli-serve` | cli-serve | `omniroute serve` |
| `cli-health` | cli-health | `omniroute health` |
| `cli-providers` | cli-providers | `omniroute providers` |
| `cli-keys` | cli-keys | `omniroute keys` |
| `cli-models` | cli-models | `omniroute models` |
| `cli-chat` | cli-chat | `omniroute chat` |
| `cli-routing` | cli-routing | `omniroute routing` |
| `cli-resilience` | cli-resilience | `omniroute resilience` |
| `cli-compression` | cli-compression | `omniroute compression` |
| `cli-contexts` | cli-contexts | `omniroute contexts` |
| `cli-cost-usage` | cli-cost-usage | `omniroute cost` |
| `cli-mcp` | cli-mcp | `omniroute mcp` |
| `cli-a2a` | cli-a2a | `omniroute a2a` |
| `cli-tunnel` | cli-tunnel | `omniroute tunnel` |
| `cli-backup-sync` | cli-backup-sync | `omniroute backup` |
| `cli-policy-audit` | cli-policy-audit | `omniroute policy` |
| `cli-batches` | cli-batches | `omniroute batch` |
| `cli-eval` | cli-eval | `omniroute eval` |
| `cli-plugins-skills` | cli-plugins-skills | `omniroute plugins` |
| `cli-setup` | cli-setup | `omniroute setup` |
---
## How External Agents Consume Skills
### 1. Discovery via REST
```bash
# Get the full catalog
curl "http://your-omniroute/api/agent-skills" | jq '.skills[] | {id, name, category}'
# Get SKILL.md for context injection
curl "http://your-omniroute/api/agent-skills/omni-providers/raw" > omni-providers.md
```
### 2. Discovery via MCP
```typescript
// In a Claude Desktop / Cursor MCP client:
const result = await client.callTool("omniroute_agent_skills_list", { category: "api" });
// result.skills → array of AgentSkill with rawUrl for each
```
### 3. Discovery via A2A
```python
import requests
resp = requests.post("http://your-omniroute/a2a", json={
"jsonrpc": "2.0", "id": "1",
"method": "message/send",
"params": {"skill": "list-capabilities", "messages": [{"role": "user", "content": "list"}]}
})
table = resp.json()["result"]["artifacts"][0]["content"]
# table is a markdown table with all 42 skill IDs + rawUrl columns
```
### 4. Direct GitHub raw fetch (no server required)
```bash
BASE="https://raw.githubusercontent.com/diegosouzapw/OmniRoute/refs/heads/main/skills"
curl "${BASE}/omni-providers/SKILL.md"
```
---
## Generator
The generator reads the curated catalog + OpenAPI spec + CLI registry and writes `skills/{id}/SKILL.md` for each entry:
```bash
# Preview (dry run, no writes)
curl -X POST http://localhost:20128/api/agent-skills/generate \
-H "Authorization: Bearer <admin-key>" \
-H "Content-Type: application/json" \
-d '{"dryRun":true}'
# Full regeneration
curl -X POST http://localhost:20128/api/agent-skills/generate \
-H "Authorization: Bearer <admin-key>" \
-H "Content-Type: application/json" \
-d '{"dryRun":false,"prune":false}'
# Regenerate specific IDs
curl -X POST http://localhost:20128/api/agent-skills/generate \
-H "Authorization: Bearer <admin-key>" \
-H "Content-Type: application/json" \
-d '{"dryRun":false,"onlyIds":["omni-providers","cli-serve"]}'
```
The generator response is a `GeneratorReport`:
```json
{
"generated": ["omni-providers", "cli-serve"],
"unchanged": [],
"pruned": [],
"orphansDetected": [],
"errors": []
}
```
---
## Coverage API
```bash
curl "http://localhost:20128/api/agent-skills/coverage"
```
```json
{
"api": { "have": 22, "total": 22 },
"cli": { "have": 20, "total": 20 },
"totalSkills": 42,
"generatedAt": "2026-05-28T00:00:00.000Z"
}
```
---
## Related
- [SKILLS.md](./SKILLS.md) — Omni Skills framework (LLM tool injection + marketplace)
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP tool catalog (`omniroute_agent_skills_*` tools)
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A protocol (`list-capabilities` skill)
- `src/lib/agentSkills/` — catalog, generator, parsers
- `skills/` — generated SKILL.md files (42 entries)
docs/frameworks/AGENTBRIDGE.md
+537
View File
@@ -0,0 +1,537 @@
---
title: "AgentBridge"
version: 3.8.40
lastUpdated: 2026-06-28
---
# AgentBridge
AgentBridge is OmniRoute's MITM (Man-in-the-Middle) proxy that intercepts HTTPS traffic from IDE AI agents and reroutes it through OmniRoute's unified routing engine. It supports **9 IDE agents** — Antigravity, Kiro, GitHub Copilot, OpenAI Codex, Cursor, Zed, Claude Code, Open Code, and Trae (investigating) — making OmniRoute the broadest-coverage MITM proxy for AI coding assistants on the market.
**Dashboard location:** `/dashboard/tools/agent-bridge`
**Sidebar group:** Tools (after Cloud Agents)
**See also:** [`TRAFFIC_INSPECTOR.md`](./TRAFFIC_INSPECTOR.md) — monitor all intercepted traffic in real-time; [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md) — the Linux TPROXY transparent-decrypt capture mode driven by the `/api/tools/agent-bridge/tproxy` route.
---
## §1 Overview
### What is AgentBridge?
When an IDE agent (e.g., GitHub Copilot, Cursor, Claude Code) makes an API call, it connects directly to the upstream AI provider (OpenAI, Anthropic, etc.). AgentBridge intercepts that connection transparently at the TLS level — without requiring any agent configuration change — and rewrites the request through OmniRoute.
This means you can:
- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 226+ providers.
- **Apply model mappings**: `gemini-3-flash` → `claude-sonnet-4.7` transparently at the handler level.
- **Observe all agent traffic**: every intercepted request is published to the [Traffic Inspector](./TRAFFIC_INSPECTOR.md).
- **Apply OmniRoute resilience**: combo routing, circuit breakers, fallbacks, and cost tracking work for IDE agent traffic too.
### Positioning vs. the market
| Feature | 9router | anti-api | llm-interceptor | **OmniRoute AgentBridge** |
| ----------------- | :-----: | :------: | :-------------: | :-----------------------: |
| Antigravity | ✓ | ✓ | — | ✓ |
| GitHub Copilot | ✓ | ✓ | — | ✓ |
| Kiro (AWS) | ✓ | ✓ | — | ✓ |
| OpenAI Codex | — | ✓ | — | ✓ |
| Cursor IDE | ✓ | ✓ | — | ✓ |
| Zed Industries | — | ✓ | — | ✓ |
| Claude Code | — | — | ✓ | ✓ |
| Open Code | — | — | ✓ | ✓ |
| Trae | — | — | — | 🔍 Investigating |
| Dashboard UI | ✓ | ✗ | ✗ | ✓ |
| Traffic Inspector | ✗ | ✗ | ✓ | ✓ |
| OmniRoute routing | ✗ | ✗ | ✗ | ✓ |
| Model mapping UI | ✗ | ✗ | ✗ | ✓ |
| Bypass list | ✗ | ✗ | ✓ | ✓ |
| Upstream CA cert | ✗ | ✗ | ✓ | ✓ |
---
## §2 Architecture
### 2.1 Components overview
```
IDE Agent (VS Code / Cursor / etc.)
│ HTTPS (port 443)
▼
/etc/hosts — 127.0.0.1 api.githubcopilot.com ← DNS redirect
│
▼
src/mitm/server.cjs (port 443, CJS child process)
│ resolves target by Host header SNI
│ generates per-SNI TLS cert signed by AgentBridge CA
├── Bypass list match? → TCP passthrough (no decrypt)
├── Target match? → fetch → OmniRoute router (port 20128)
│ └── handler.intercept() — TypeScript
│ ├── maskSecrets() on request body/headers
│ ├── TrafficBuffer.push() — publishes to Traffic Inspector
│ └── fetchRouter() → /v1/chat/completions
└── No match? → TCP passthrough (no decrypt)
```
### 2.2 MITM server (`src/mitm/server.cjs`)
The core MITM server runs as a Node.js CJS child process (to avoid rewriting the existing CJS codebase). It:
- Listens on port 443 (requires privilege or `authbind`/`setcap`)
- Receives CONNECT tunnels from the OS (via `/etc/hosts` DNS redirect)
- Generates per-SNI TLS certificates signed by the AgentBridge CA (`DATA_DIR/mitm/ca.crt`)
- Resolves the target agent by Host header via `targets/index.ts` registry
- Dispatches to the TypeScript handler layer via HTTP to `http://127.0.0.1:20128`
`TARGET_HOSTS` is loaded from `DATA_DIR/mitm/targets.json` (written by `targets/index.ts` at boot), allowing dynamic updates without restarting the CJS server.
### 2.3 Handler base (`src/mitm/handlers/base.ts`)
All agent handlers extend `MitmHandlerBase`:
```ts
export abstract class MitmHandlerBase {
abstract readonly agentId: AgentId;
abstract intercept(
req: IncomingMessage,
res: ServerResponse,
body: Buffer,
mappedModel: string
): Promise<void>;
// Protected helpers: fetchRouter, pipeSSE, hookBufferStart, hookBufferUpdate
}
```
Each handler calls `hookBufferStart()` before proxying and `hookBufferUpdate()` when complete. These push `InterceptedRequest` entries into `globalTrafficBuffer` (see [Traffic Inspector](./TRAFFIC_INSPECTOR.md) §4).
### 2.4 Targets registry (`src/mitm/targets/`)
Each agent has a declarative target file:
```ts
// src/mitm/targets/copilot.ts
export const COPILOT_TARGET: MitmTarget = {
id: "copilot",
name: "GitHub Copilot",
hosts: ["api.githubcopilot.com", "copilot-proxy.githubusercontent.com"],
port: 443,
endpointPatterns: ["/chat/completions", "/v1/chat/completions"],
defaultModels: [{ id: "gpt-4o", name: "GPT-4o", alias: "gpt-4o" }],
handler: () => import("../handlers/copilot"),
riskNoticeKey: "providers.riskNotice.oauth",
};
```
The registry (`targets/index.ts`) exports `ALL_TARGETS` and emits `DATA_DIR/mitm/targets.json` on boot.
### 2.5 Passthrough and bypass list (`src/mitm/passthrough.ts`)
**Bypass list** (checked first, with precedence over target match):
- Default patterns: banking hosts, `.gov.`, OAuth/SSO providers (Okta, Auth0), etc.
- User patterns: stored in DB table `agent_bridge_bypass`
- Bypassed hosts receive a transparent TCP tunnel — TLS is **never decrypted**
**Passthrough default** (no target match and not in bypass):
- Also receives a TCP tunnel — connections are never broken
- Prevents the AgentBridge from disrupting general system HTTPS traffic
Routing precedence:
```
bypass list → target match → passthrough
```
### 2.6 Upstream CA cert (`src/mitm/upstreamTrust.ts`)
For corporate network environments with a custom CA:
```bash
AGENTBRIDGE_UPSTREAM_CA_CERT=/path/to/corporate-ca.pem
```
When set, configures `undici`'s global dispatcher with the extra CA cert, allowing AgentBridge to reach upstream providers through corporate TLS termination proxies.
### 2.7 Secret masking (`src/mitm/maskSecrets.ts`)
Applied to all request bodies and headers **before** they enter the Traffic Inspector buffer or any log:
- `sk-` / `ak-` / `pk-` prefixed tokens (OpenAI/Anthropic-style)
- `Authorization: Bearer <token>` headers
- Generic long tokens (≥40 chars)
---
## §3 Setup
### 3.1 Start/stop the MITM server
Use the AgentBridge Server Card at `/dashboard/tools/agent-bridge`:
| Action | Description |
| --------------- | ----------------------------------------------------------------------- |
| Start Server | Spawns `src/mitm/server.cjs` on port 443 |
| Stop Server | Gracefully shuts down the child process |
| Restart Server | Stop + start (picks up target changes) |
| Trust Cert | Installs `DATA_DIR/mitm/ca.crt` into OS trust store |
| Download Cert | Downloads `ca.crt` for manual installation |
| Regenerate Cert | Creates a new CA keypair (all existing per-agent certs are invalidated) |
### 3.2 Trust the certificate
The AgentBridge CA certificate must be trusted by the OS before IDEs will accept the MITM connection.
**Linux (NSS — Chrome/Firefox):**
```bash
certutil -A -d sql:$HOME/.pki/nssdb -n "OmniRoute AgentBridge" -t CT,, -i ~/.omniroute/mitm/ca.crt
```
**macOS (Keychain):**
```bash
sudo security add-trusted-cert -d -r trustRoot \
-k /Library/Keychains/System.keychain ~/.omniroute/mitm/ca.crt
```
**Windows (certmgr):**
```powershell
certutil -addstore -f Root $env:USERPROFILE\.omniroute\mitm\ca.crt
```
Or use the "Trust Cert" button in the dashboard (runs the appropriate command for your OS, with sudo prompt if needed).
#### Electron-based IDEs ignore the OS trust store (`NODE_EXTRA_CA_CERTS`)
Some IDEs — notably **Antigravity IDE**, and other Electron / VS Code-derived apps — bundle
their own Node.js runtime that **does not consult the OS trust store** for outbound
`fetch`/HTTPS. Trusting the CA at the OS/NSS level is enough for the IDE's native **backend**
(e.g. a Go language server, which uses the OS CA bundle), but the **Electron frontend** will
still fail TLS — it surfaces as the app being _logged out_ or showing a _"connection error"_
even though the MITM log shows the backend's bootstrap calls returning `200`. Two steps are
required, and both matter:
1. Point the runtime at the CA explicitly:
```bash
export NODE_EXTRA_CA_CERTS=/path/to/omniroute-agentbridge-ca.crt
```
2. **Launch the IDE from that shell.** Starting it from the desktop icon / Dock / Start menu
does **not** inherit shell exports, and `~/.config/environment.d/*.conf` only applies after
a fresh graphical login. Fully quit the IDE first — Electron's singleton lock means a second
launch just focuses the existing process and the new environment is ignored.
The OS-trust + NSS step above remains necessary (the Chromium network stack used by some auth
flows reads the per-user NSS store, and has its own static pins for `*.googleapis.com` that a
locally-trusted CA overrides). `NODE_EXTRA_CA_CERTS` covers the Node `fetch` path on top of it.
### 3.3 DNS routing
For each agent you want to intercept, its API host(s) must resolve to `127.0.0.1`. AgentBridge manages `/etc/hosts` entries automatically when you toggle DNS for an agent in the Setup Wizard.
Example `/etc/hosts` entries for GitHub Copilot:
```
127.0.0.1 api.githubcopilot.com
127.0.0.1 copilot-proxy.githubusercontent.com
```
### 3.4 Model mapping
Use the Model Mapping Table in each agent card to define source → target mappings:
| Source model (agent native) | Target model (OmniRoute) |
| --------------------------- | ------------------------ |
| `gpt-4o` | `claude-sonnet-4.7` |
| `*` (wildcard) | `claude-haiku-4.7` |
Wildcard `*` maps any unrecognized model to the specified target. Persisted in `agent_bridge_mappings` table.
> **Tip — discover the agent's real model IDs.** An IDE may send model names that differ from
> its UI labels and that change between major versions. For example **Antigravity 2** sends
> `gemini-3.1-pro-low`, `gemini-pro-agent`, and `gemini-3.1-flash-lite` over the wire — not the
> `gemini-2.5-pro` shown in older docs. Send one chat with no matching mapping in place: the MITM
> logs the exact incoming `model:` and passes the request through. Map that literal value, then
> the next request is intercepted and routed to your target.
### 3.5 Risk notice
AgentBridge intercepts credentials (OAuth tokens, API keys) that the IDE uses to authenticate with upstream providers. These are **masked before logging** (see §2.7) but are visible to OmniRoute's MITM layer. First activation of each agent shows a dismissible risk notice modal.
### 3.6 Maintenance & Diagnostics
The dashboard exposes a **Maintenance & Diagnostics** card (`AgentBridgeMaintenanceCard`, in `src/app/(dashboard)/dashboard/tools/agent-bridge/components/`) that surfaces operational MITM routes which previously had no UI. Its subtitle: _"Self-test the capture pipeline, undo leftover system state, and move your setup between machines."_ The card client helpers live in `src/lib/inspector/agentBridgeMaintenanceApi.ts`.
| Button | Route | What it does |
| ----------------- | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Diagnose** | `GET /api/tools/agent-bridge/diagnose` | Runs the capture-pipeline self-test and shows a per-check report (✓/✗ + remediation hint). |
| **Repair** | `POST /api/tools/agent-bridge/repair` | Undoes orphaned MITM system state (DNS spoof entries, root CA, system proxy) left behind by a crash or SIGKILL. Idempotent — reports "Nothing to repair" when state is clean. |
| **Remove CA** | `DELETE /api/tools/agent-bridge/cert` | Untrusts and removes the MITM root CA from the OS trust store (explicit, idempotent). Shown only when the CA is currently trusted; requires an inline "Remove CA?" confirmation. |
| **Export config** | `GET /api/tools/agent-bridge/config` | Downloads the portable config JSON (see §3.7). |
| **Import config** | `POST /api/tools/agent-bridge/config` | Uploads a previously-exported config JSON (see §3.7). |
**Diagnostics checks** (`summarizeDiagnostics()` in `src/mitm/inspector/diagnostics.ts`). The route runs the effectful probe for each and feeds the booleans into the pure summarizer; a single `healthy` verdict plus a per-failure hint is returned:
| Check name | What it verifies | Hint on failure |
| ------------------ | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `server-running` | The MITM server process is active | "The MITM server is not running. Start it from the AgentBridge tab." |
| `server-reachable` | The MITM server accepts connections on its port (TCP probe) | "The MITM server is not accepting connections on its port. Check that the port is free and that you have privileges to bind it." |
| `cert-exists` | The MITM certificate has been generated on disk | "No MITM certificate has been generated yet. Generate one from the AgentBridge tab." |
| `cert-trusted` | The MITM root CA is in the OS trust store | "The MITM root CA is not trusted by the OS store, so TLS interception will fail. Trust the certificate from the AgentBridge tab." |
| `dns-configured` | Target hostnames are spoofed in `/etc/hosts` | "Target hostnames are not spoofed in /etc/hosts, so traffic never reaches the proxy. Enable DNS for the agent(s) you want to capture." |
**Orphaned-state banner:** when the page detects state left behind by a crash (DNS spoof / CA / system proxy), the card shows an amber banner — _"A previous session left system state behind (DNS spoof, CA, or system proxy). Run Repair to clean it up."_ — and highlights the **Repair** button. `Repair` is the application-layer analogue of ProxyBridge's `--cleanup` flag (it delegates to `repairMitm()` in `src/mitm/manager.ts`).
> The MITM root CA is kept installed across stop/start to avoid repeated sudo
> prompts (the same behavior as mitmproxy/Charles), so removing it is an explicit
> **Remove CA** action rather than something that happens automatically on stop.
### 3.7 Portable config import/export
AgentBridge can serialize the **operator-tunable** state into a versioned JSON blob so a setup can be replicated across machines. The serializer is `src/lib/inspector/configPortability.ts` (`exportConfig()` / `importConfig()`), validated by `AgentBridgeConfigSchema`.
The export includes exactly three pieces (built-in defaults are intentionally **NOT** exported, so importing never duplicates or fights them):
| Field | Source | Notes |
| ---------------- | --------------------------------------------------------- | ----------------------------------------------------------------------- |
| `bypassPatterns` | user-defined bypass patterns (`agent_bridge_bypass`) | default bank/gov/okta patterns are excluded |
| `customHosts` | Traffic Inspector custom hosts (`inspector_custom_hosts`) | each: `{ host, kind: "llm"\|"app"\|"custom", label? }` |
| `agentMappings` | per-agent model mappings (`agent_bridge_mappings`) | `{ [agentId]: [{ source, target }] }` for every agent that has mappings |
```jsonc
// GET /api/tools/agent-bridge/config
{
"version": 1,
"bypassPatterns": ["*.internal.example.com"],
"customHosts": [{ "host": "api.example.com", "kind": "llm", "label": null }],
"agentMappings": { "copilot": [{ "source": "gpt-4o", "target": "claude-sonnet-4.7" }] },
}
```
**Import behavior** (`POST /api/tools/agent-bridge/config`): bypass patterns and per-agent mappings **replace wholesale**; custom hosts are added **idempotently** (`INSERT OR IGNORE`). The response reports how many of each were applied:
```jsonc
{ "ok": true, "bypassPatterns": 1, "customHosts": 1, "agents": 1 }
```
What is **NOT** in the config: server running state, cert paths, per-agent DNS state, upstream CA path, and TPROXY settings — those are host/runtime state, not portable preferences.
---
## §4 Per-agent reference
| # | Agent | Status | Hosts intercepted | Auth type |
| --- | ------------------ | ---------------- | ------------------------------------------------------------------ | -------------- |
| 1 | **Antigravity** | ✅ Supported | `daily-cloudcode-pa.googleapis.com`, `cloudcode-pa.googleapis.com` | Firebase OAuth |
| 2 | **Kiro (AWS)** | ✅ Supported | `prod.kiro.aws`, `dev.kiro.aws` | AWS SigV4 |
| 3 | **GitHub Copilot** | ✅ Supported | `api.githubcopilot.com`, `copilot-proxy.githubusercontent.com` | GitHub OAuth |
| 4 | **OpenAI Codex** | ✅ Supported | `api.openai.com` (Codex paths), `chatgpt.com` | OpenAI key |
| 5 | **Cursor IDE** | ✅ Supported | `api2.cursor.sh`, `api.cursor.sh` | Cursor OAuth |
| 6 | **Zed Industries** | ✅ Supported | `api.zed.dev`, `llm.zed.dev` | Zed OAuth |
| 7 | **Claude Code** | ✅ Supported | `api.anthropic.com` (opt-in) | Anthropic key |
| 8 | **Open Code** | ✅ Supported | `openrouter.ai`, `api.openai.com` (zen paths) | API key |
| 9 | **Trae** | 🔍 Investigating | TBD — see §8 | TBD |
### Setup wizard steps (per agent)
Each agent card has a 3-step setup wizard:
1. **Verify prerequisites** — Server running? Cert trusted? IDE installed (auto-detected)?
2. **Enable DNS** — Adds `/etc/hosts` entries (requires sudo). Shows exactly which lines will be added.
3. **Map models** — Optional model mapping table. Wildcards accepted.
### Agent detection
For agents 1–8, AgentBridge attempts to auto-detect IDE installation:
```ts
export async function detectAgent(agentId: AgentId): Promise<DetectionResult>;
// Returns: { installed: boolean, version?: string, path?: string }
```
Detection uses OS-specific paths and binary checks (e.g., `code --list-extensions | grep github.copilot` for Copilot, `~/.config/antigravity/` for Antigravity).
---
## §5 Security
### Hard Rules applied
| Rule | Application |
| --------------------------------- | ---------------------------------------------------------------------------------------- |
| **#12** `sanitizeErrorMessage` | All handler errors are sanitized before response or buffer entry |
| **#13** Shell env-passing | `/etc/hosts` edits use `env` option — no string interpolation of paths |
| **#15 + #17** `isLocalOnlyPath()` | `/api/tools/agent-bridge/` is LOCAL_ONLY + SPAWN_CAPABLE — loopback enforced before auth |
### Bypass list for sensitive hosts
The bypass list ensures that financial institutions, OAuth/SSO providers, and other sensitive hosts are **never decrypted**. Their TLS traffic passes through as a transparent TCP tunnel — OmniRoute never sees the plaintext.
Default bypass patterns include:
- `*.bank.*`, `*.gov.*` (financial/government)
- `*.okta.com`, `*.auth0.com`, `*.microsoft.com` (SSO/identity)
- `*.apple.com`, `*.icloud.com` (Apple system services)
User-added bypass patterns are stored in `agent_bridge_bypass` table and take precedence over everything.
### Secret masking
`maskSecrets()` from `src/mitm/maskSecrets.ts` is applied:
- On every request body before `TrafficBuffer.push()`
- On every header before logging or broadcasting
Patterns: `sk-`/`ak-`/`pk-` prefix tokens, `Bearer` tokens, and generic tokens ≥40 characters.
### Upstream CA cert
When `AGENTBRIDGE_UPSTREAM_CA_CERT` is set, the file is read at startup. If the path exists but the file is unreadable, AgentBridge logs a clear error and refuses to start (prevents silent TLS failures in corporate environments).
### Known limitations
- **Port 443 requires privilege**: On Linux, AgentBridge needs `setcap 'cap_net_bind_service=+ep'` on the Node binary, or run via `authbind`. The Setup Wizard displays OS-specific instructions.
- **IDE restart required**: After DNS redirect, the IDE must be restarted for the new host resolution to take effect.
- **Hardcoded OAuth tokens**: Some agents (Kiro, Antigravity) store OAuth refresh tokens locally. These are transparent to AgentBridge — it sees the Bearer token in each request, which is masked before logging.
- **Electron frontends need `NODE_EXTRA_CA_CERTS`**: IDEs whose frontend runs on a bundled Node/Electron runtime ignore the OS/NSS trust store and must be launched from a shell with `NODE_EXTRA_CA_CERTS` set (see §3.2). Symptom when missing: the IDE backend authenticates (MITM shows `200`s) but the UI stays logged out.
- **Multiple installs of the same IDE are independent**: a system install (e.g. `/usr/share/antigravity/antigravity`) and a user-local "Full" install (e.g. `~/AntigravityIDE_Full/antigravity-ide`) are separate processes with their own runtimes — each must be relaunched with the CA injected. Identify which one is running by its binary path before relaunching.
- **Identity is set by the agent's system prompt, not the routed model**: when you remap an agent's model to a different provider, the reply still claims the agent's native identity (e.g. Antigravity answers "I am powered by Gemini") because the IDE injects that into the system prompt. Confirm the real backend in `call_logs` / `proxy_logs` (`provider`, `model`, `target_format`), not by asking the model who it is.
---
## §6 Troubleshooting
### Port 443 conflict
If another process is already listening on port 443 (web server, VPN, etc.):
```bash
lsof -i :443 # find the process
sudo fuser -k 443/tcp # force-kill (use with care)
```
Alternatively, configure a non-privileged port in AgentBridge settings and set up `iptables` / `pf` redirect rules.
### Certificate not trusted
If the IDE shows TLS errors after starting AgentBridge:
1. Verify the cert was installed: `security find-certificate -c "OmniRoute AgentBridge"` (macOS) or `certutil -L -d sql:$HOME/.pki/nssdb` (Linux/NSS)
2. Some apps maintain their own trust store (Firefox, Chrome on Linux). Run "Trust Cert" again and check the NSS/Firefox-specific cert store.
3. Restart the IDE after trusting — in-flight TLS sessions use the old trust state.
### IDE logged out / "connection error" despite a trusted CA
Symptom: after redirecting DNS and trusting the CA, an Electron-based IDE (e.g. Antigravity)
opens **logged out** or shows an authentication/connection error, yet the MITM log shows the
bootstrap calls (`loadCodeAssist`, `fetchAvailableModels`, …) returning `200`.
Cause: the IDE's **bundled Node/Electron runtime ignores the OS trust store**. The native
backend (a Go language server) trusts the OS CA and authenticates, but the Electron frontend
does not — so the UI believes it is offline.
Fix (both steps): export `NODE_EXTRA_CA_CERTS=<ca.crt>` **and relaunch the IDE from that
shell**, not from the desktop icon. Fully quit the IDE first — Electron's singleton lock means
a second launch just focuses the existing process and the new environment is ignored. See §3.2.
This mirrors an open upstream report where a standalone agent works through a MITM but the IDE
variant fails under the same setup.
### DNS not propagated
Check that `/etc/hosts` was updated:
```bash
grep "omniroute\|127.0.0.1.*github\|127.0.0.1.*cursor" /etc/hosts
```
Flush DNS cache:
```bash
# macOS
sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder
# Linux (systemd-resolved)
sudo systemctl restart systemd-resolved
# Windows
ipconfig /flushdns
```
### IDE not detected
Auto-detection uses common installation paths. If detection fails but the IDE is installed:
- Check if the IDE binary is in a non-standard location
- The Setup Wizard still works — detection failure just means the badge won't show the install path
### Handler errors (upstream fetch fails)
If AgentBridge intercepts but all requests fail:
1. Verify at least one provider is connected at `/dashboard/providers`
2. Check OmniRoute server logs: `APP_LOG_LEVEL=debug` in `.env`
3. Verify `OMNIROUTE_BASE_URL` points to the correct router endpoint (default: `http://127.0.0.1:20128`)
---
## §7 API reference
All routes are `LOCAL_ONLY` (loopback-only, enforced before auth) and `SPAWN_CAPABLE`. See `src/server/authz/routeGuard.ts`.
Base path: `/api/tools/agent-bridge/`
| Method | Path | Description |
| ------------------- | ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| GET | `/api/tools/agent-bridge/state` | Global server state + per-agent detection/status |
| GET | `/api/tools/agent-bridge/agents` | List registered agents (id, name, hosts, viability, state) |
| GET | `/api/tools/agent-bridge/agents/{id}` | State of one agent (target config + detection + stored state) |
| PATCH | `/api/tools/agent-bridge/agents/{id}` | Update `setup_completed` for agent |
| GET | `/api/tools/agent-bridge/agents/{id}/detect` | Run detection probe for agent (`installed`, `version?`, `path?`) |
| POST | `/api/tools/agent-bridge/agents/{id}/dns` | Enable/disable DNS for agent (`{enabled: boolean}`) |
| GET | `/api/tools/agent-bridge/agents/{id}/mappings` | Model mappings for agent |
| PUT | `/api/tools/agent-bridge/agents/{id}/mappings` | Replace model mappings |
| POST | `/api/tools/agent-bridge/server` | Start/stop/restart server (`action: "start"\|"stop"\|"restart"\|"trust-cert"\|"regenerate-cert"`) |
| GET | `/api/tools/agent-bridge/cert` | Cert status (`exists`, `trusted`, `path`) |
| POST | `/api/tools/agent-bridge/cert` | Trust (install) the MITM root CA |
| DELETE | `/api/tools/agent-bridge/cert` | Untrust (remove) the MITM root CA — idempotent (see §3.6) |
| POST | `/api/tools/agent-bridge/cert/regenerate` | Regenerate the self-signed MITM cert |
| GET | `/api/tools/agent-bridge/cert/download` | Stream the PEM cert for download |
| GET | `/api/tools/agent-bridge/bypass` | List bypass patterns (`default` + `user`) |
| POST | `/api/tools/agent-bridge/bypass` | Replace user-defined bypass patterns wholesale |
| DELETE | `/api/tools/agent-bridge/bypass?pattern=...` | Remove a single user-defined bypass pattern |
| GET | `/api/tools/agent-bridge/diagnose` | Capture-pipeline self-test (see §3.6) |
| POST | `/api/tools/agent-bridge/repair` | Undo orphaned MITM system state (see §3.6) |
| GET | `/api/tools/agent-bridge/config` | Export portable config JSON (see §3.7) |
| POST | `/api/tools/agent-bridge/config` | Import portable config JSON (see §3.7) |
| GET | `/api/tools/agent-bridge/upstream-ca` | Get configured upstream CA path |
| POST | `/api/tools/agent-bridge/upstream-ca` | Validate + persist upstream CA path |
| POST | `/api/tools/agent-bridge/upstream-ca/test` | Validate-only (dry-run) an upstream CA path — does not persist |
| GET / POST / DELETE | `/api/tools/agent-bridge/tproxy` | TPROXY transparent-decrypt capture mode — see [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md) |
Full OpenAPI schemas: `docs/openapi.yaml` → tag `AgentBridge`.
---
## §8 Roadmap
### Trae investigation
Trae is a relatively new AI coding assistant. Before implementing a handler:
1. Identify the binary/extension in VS Code / JetBrains marketplaces or as a standalone app
2. Capture traffic with mitmproxy to discover API hosts and endpoint shapes
3. Determine authentication mechanism
4. Assess go/no-go based on TOS and API discoverability
Until investigation completes, the Trae card in the dashboard shows a "Investigating" badge with a "Report viability" link. The handler stub at `src/mitm/handlers/trae.ts` throws a structured `Not yet implemented` error.
### Backlog agents (MITM required — no custom base URL support)
The following tools do not support custom base URLs in their current versions, making MITM the only interception path. Viability assessment is pending:
- **Windsurf** (Codeium/Cognition)
- **Amp** (Sourcegraph)
- **Amazon Q / Kiro CLI** (AWS Bedrock — separate from Kiro IDE)
- **Cowork** (Anthropic desktop)
Note: GitHub Copilot CLI ≥v1.0.19 supports `COPILOT_PROVIDER_BASE_URL` — use direct config instead of MITM for that tool.
docs/frameworks/AGENT_PROTOCOLS_GUIDE.md
+283
View File
@@ -0,0 +1,283 @@
---
title: "Agent Protocols Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Agent Protocols Guide
> **Source:** `src/lib/{a2a,acp,cloudAgent}/`, `src/app/api/{a2a,acp,cloud}/`, `src/app/api/v1/agents/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute exposes three different agent-related surfaces. They look similar at first glance but solve different problems. Use this page to pick the right one.
## TL;DR
| Surface | Best for | Transport | Standard |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- | -------------------- |
| **A2A — Agent-to-Agent** | Cross-agent collaboration with peer agents that speak the A2A protocol | JSON-RPC 2.0 over HTTP | A2A v0.3 (open spec) |
| **ACP — CLI Agents Registry** | Detecting / registering / launching CLI coding agents installed on the user's machine (Cursor, Cline, Codex CLI, Claude Code, Aider, etc.) | HTTP REST | OmniRoute-specific |
| **Cloud Agents** | Submitting long-running coding tasks to external cloud services (Codex Cloud, Devin, Jules, Cursor Cloud) | HTTP REST + DB-backed tasks | OmniRoute-specific |
The three are independent — pick any subset.
## Decision Tree
```
Do you need a cloud service to do work outside this machine (Codex Cloud / Devin / Jules)?
├─ YES → Cloud Agents (POST /api/v1/agents/tasks)
└─ NO → Continue
│
Do you have a peer agent that speaks A2A and wants to collaborate?
├─ YES → A2A (POST /a2a)
└─ NO → Continue
│
Do you need to list / configure CLI coding agents installed locally?
├─ YES → ACP (GET /api/acp/agents)
└─ NO → Use plain /v1/chat/completions
```
## 1. A2A — Agent-to-Agent
**Spec:** [A2A v0.3](https://a2a-protocol.org)
**OmniRoute endpoint:** `POST /a2a` (JSON-RPC 2.0)
**Agent Card:** `GET /.well-known/agent.json`
### When to use
- Building a multi-agent system where OmniRoute is one of the peers
- Exposing OmniRoute's routing intelligence (smart-routing, quota-management, etc.) to agents in frameworks like Google ADK or generic agent meshes
- Wrapping OmniRoute behind a standard discovery + invocation surface
### Methods
- `message/send` — submit a message, receive sync response
- `message/stream` — submit + receive SSE-streamed progress events
- `tasks/get` — read task by ID
- `tasks/cancel` — cancel a running task
### Built-in skills (6)
- `smart-routing` — route a prompt through the optimal combo
- `quota-management` — report per-provider quota state
- `provider-discovery` — list installed providers with capabilities
- `cost-analysis` — estimate cost of a request/conversation
- `health-report` — aggregate breaker/cooldown/lockout state per provider
- `list-capabilities` — enumerate the agent's available skills and metadata
### Deep dive
See [A2A-SERVER.md](./A2A-SERVER.md) for transport details, agent card structure, task TTL config, and the template for adding new skills.
## 2. ACP — CLI Agents Registry
**OmniRoute endpoint:** `GET /api/acp/agents`
**Source:** `src/lib/acp/{index,manager,registry}.ts`
### What it is
ACP is OmniRoute's **local CLI agent inventory**. It detects which coding CLIs are installed on the host (Cursor, Cline, Claude Code, Codex CLI, Continue, etc.), resolves their versions, and surfaces them to the dashboard so the user can wire each CLI to point at OmniRoute.
This is NOT an external protocol — it's an internal registry that powers the "CLI Tools" UI and the CLI fingerprint tracking (see [CLI-TOOLS.md](../reference/CLI-TOOLS.md)).
### What it does
- Probes the host for installed CLI binaries (uses `which` / `where` per OS)
- Reads each CLI's version (calls `<bin> --version`)
- Optionally accepts user-defined custom agents (binary path + version probe + spawn args)
- Persists custom agents in settings
- Returns the unified list to the dashboard
### REST API
| Endpoint | Method | Description | Auth |
| ----------------- | ------ | ------------------------------------------------------------- | ------- |
| `/api/acp/agents` | GET | List detected + custom agents (installed/total counts) | API key |
| `/api/acp/agents` | POST | Add/update/remove custom agent (action discriminator in body) | API key |
Body shape for POST (`customAgentBodySchema` in `src/app/api/acp/agents/route.ts`):
```json
{
"action": "add|update|remove",
"id": "cursor",
"name": "Cursor",
"binary": "/usr/local/bin/cursor",
"versionCommand": "--version",
"providerAlias": "cursor",
"spawnArgs": ["--api-base", "http://localhost:20128"],
"protocol": "stdio"
}
```
### Use cases
- Dashboard "CLI Tools" page lists what's installed and helps you point each at OmniRoute
- Custom agents let power users register internal/proprietary CLIs that OmniRoute doesn't know about by default
- Detection result fuels the `cli-tools` fingerprint matrix
### When NOT to use ACP
- ACP doesn't _run_ tasks. It only detects + configures CLIs. To actually invoke a CLI, you launch it yourself with the env vars OmniRoute provides (`OPENAI_BASE_URL`, `OPENAI_API_KEY`, etc.).
## 3. Cloud Agents
**OmniRoute endpoints:** `/api/v1/agents/tasks/*` (lifecycle) + `/api/cloud/*` (plumbing)
**Source:** `src/lib/cloudAgent/`
### What it is
A uniform interface over third-party cloud coding agents. You submit a prompt + repo URL, OmniRoute dispatches to the right cloud agent, polls status, returns results.
### Supported agents (3, all confirmed in `src/lib/cloudAgent/agents/`)
- `codex-cloud` — OpenAI Codex Cloud
- `devin` — Cognition Devin
- `jules` — Google Jules
### Lifecycle
```
POST /api/v1/agents/tasks
→ BaseAgent.createTask() per agent class
→ external service starts work
→ task row created in DB (cloud_agent_tasks)
↓
GET /api/v1/agents/tasks/[id]
→ lazy status sync from provider
→ returns current status + plan + activity log
↓
POST /api/v1/agents/tasks/[id] (action: "approve" | "message" | "cancel")
→ forwards to provider (or marks cancelled locally)
↓
DELETE /api/v1/agents/tasks/[id]
→ local cancel
```
### Auth
⚠️ **All `/api/v1/agents/tasks/*` endpoints require management auth** (commit `588a0333`). Bearer-only callers receive 401 since v3.8.0.
### Deep dive
See [CLOUD_AGENT.md](./CLOUD_AGENT.md) for the `CloudAgentBase` contract, per-agent specifics, schema details, and credential plumbing endpoints.
## Comparison: A2A vs Cloud Agents
Both have "long-running tasks" but at different layers:
| Aspect | A2A | Cloud Agents |
| ------------------ | --------------------------------------------------------------------------------- | ---------------------------------------- |
| Standard | Open A2A v0.3 | OmniRoute-specific |
| Where compute runs | Inside OmniRoute (uses configured combos) | External (Codex / Devin / Jules servers) |
| Task duration | Default TTL 5 min (configurable in `TaskManager`) | Minutes to hours |
| Repo-aware | No (passes prompts only) | Yes (repo URL + branch) |
| Use case | Cross-agent collab, smart routing as a service | Delegate "implement feature X in repo Y" |
| Auth | Optional `OMNIROUTE_API_KEY` for `/a2a`; management for `/api/a2a/*` REST helpers | Always management |
## Integration Examples
### Discover OmniRoute's A2A capabilities
```bash
curl http://localhost:20128/.well-known/agent.json
```
Returns the Agent Card with all 5 skills, transports, and version.
### Call OmniRoute as an A2A agent
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"method": "message/send",
"params": {
"messages": [{"role": "user", "content": "Route this prompt"}],
"skillId": "smart-routing"
},
"id": 1
}'
```
### List installed CLI agents via ACP
```bash
curl http://localhost:20128/api/acp/agents \
-H "Authorization: Bearer <api-key>"
```
### Add a custom CLI agent
```bash
curl -X POST http://localhost:20128/api/acp/agents \
-H "Authorization: Bearer <api-key>" \
-H "Content-Type: application/json" \
-d '{
"action": "add",
"id": "my-custom-cli",
"name": "My Custom CLI",
"binary": "/opt/mycli/bin/mycli",
"versionCommand": "--version",
"providerAlias": "openai"
}'
```
### Submit a Cloud Agent task
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"agentId": "devin",
"prompt": "Implement feature X in repo Y",
"repo": "https://github.com/user/repo",
"branch": "main"
}'
```
### Poll cloud task status
```bash
curl http://localhost:20128/api/v1/agents/tasks/<task-id> \
-H "Cookie: auth_token=..."
```
## When to Use What
- **Chatbot / copilot frontend** → `/v1/chat/completions` (OpenAI-compat — not an agent protocol)
- **Multi-agent collaboration** → A2A
- **Listing local CLIs in the dashboard** → ACP
- **Delegating long-running coding tasks to cloud services** → Cloud Agents
## Internal Architecture
```
┌─────────────────────┐
│ OmniRoute Core │
└─────────────────────┘
↑ ↑ ↑
┌─────────┘ │ └─────────┐
│ │ │
┌───────┐ ┌─────────┐ ┌────────────┐
│ A2A │ │ ACP │ │ Cloud │
│ (/a2a)│ │ (/acp) │ │ Agents │
└───────┘ └─────────┘ │ (/v1/agents│
│ │ │ /tasks) │
↓ ↓ └────────────┘
External peer Local CLI │
agents that binaries on ↓
speak A2A v0.3 the host Codex Cloud,
Devin, Jules
```
## See Also
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A deep dive
- [CLOUD_AGENT.md](./CLOUD_AGENT.md) — Cloud Agents deep dive
- [CLI-TOOLS.md](../reference/CLI-TOOLS.md) — External CLI integrations (uses ACP)
- [SKILLS.md](./SKILLS.md) — Skills framework (different from A2A skills — local execution sandbox)
- [API_REFERENCE.md](../reference/API_REFERENCE.md#agents-protocol) — endpoint reference
- Source: `src/lib/{a2a,acp,cloudAgent}/`
docs/frameworks/CLOUD_AGENT.md
+393
View File
@@ -0,0 +1,393 @@
---
title: "Cloud Agents"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Cloud Agents
> **Source of truth:** `src/lib/cloudAgent/` and `src/app/api/v1/agents/tasks/`
> **Last updated:** 2026-06-28 — v3.8.40 (frontmatter refresh; 4 agents incl. cursor-cloud)
OmniRoute orchestrates third-party cloud-hosted coding agents (Codex Cloud, Cursor,
Devin, Jules) as long-running tasks. Each agent is wrapped behind a uniform interface so
clients can submit a prompt + repo URL and receive results without dealing with
provider-specific APIs.
A Cloud Agent task is **not** a regular chat completion. It is a durable, multi-step
unit of work that may take minutes to hours, can produce a Pull Request as its
artifact, and supports follow-up messages and (in some providers) plan approval gates.
![Cloud Agent task lifecycle](../diagrams/exported/cloud-agent-flow.svg)
> Source: [diagrams/cloud-agent-flow.mmd](../diagrams/cloud-agent-flow.mmd)
## Supported Agents
| Provider ID | Class | Source | Upstream Base URL | Plan Approval |
| -------------- | ------------------ | ------------------------------------- | --------------------------------------- | ------------- |
| `jules` | `JulesAgent` | `src/lib/cloudAgent/agents/jules.ts` | `https://jules.googleapis.com/v1alpha` | Yes |
| `devin` | `DevinAgent` | `src/lib/cloudAgent/agents/devin.ts` | `https://api.devin.ai/v1` | Yes |
| `codex-cloud` | `CodexCloudAgent` | `src/lib/cloudAgent/agents/codex.ts` | `https://api.openai.com/v1/codex/cloud` | No (auto) |
| `cursor-cloud` | `CursorCloudAgent` | `src/lib/cloudAgent/agents/cursor.ts` | `https://api.cursor.com/v0` | No (auto) |
Registry: `src/lib/cloudAgent/registry.ts` — exports `getAgent(providerId)`,
`getAvailableAgents()`, and `isCloudAgentProvider(providerId)`. The registry is a
plain in-memory `Record<string, CloudAgentBase>` populated at module load.
## Architecture
```
Client (Dashboard / CLI / API)
→ POST /api/v1/agents/tasks (management auth required)
→ CreateCloudAgentTaskSchema validation (Zod)
→ registry.getAgent(providerId)
→ getCloudAgentCredentials(providerId)
└─ pulls from getProviderConnections({ provider, isActive: true })
(apiKey first, fallback to accessToken)
→ agent.createTask({ prompt, source, options }, credentials)
└─ HTTP POST to upstream provider API
└─ returns CloudAgentTask with internal id + externalId
→ insertCloudAgentTask(...) into cloud_agent_tasks (SQLite)
Polling (lazy sync on read):
GET /api/v1/agents/tasks/[id]
→ getCloudAgentTaskById(id)
→ agent.getStatus(externalId, credentials) // refreshes status + activities
→ updateCloudAgentTask(...) with new status, result, completed_at
→ return serialized task
Interactions:
POST /api/v1/agents/tasks/[id] body: { action: "approve" | "message" | "cancel" }
→ agent.approvePlan(externalId, credentials) for "approve"
→ agent.sendMessage(externalId, message, credentials) for "message"
→ status flips to "cancelled" for "cancel" (local-only)
```
Sync is **lazy**: status is refreshed from the upstream on every `GET /tasks/[id]`.
There is no background poller. Dashboards that need fresh state should poll the GET
endpoint at a sensible interval.
## `CloudAgentBase` Interface
Source: `src/lib/cloudAgent/baseAgent.ts`
```typescript
export interface AgentCredentials {
apiKey: string;
baseUrl?: string;
}
export interface CreateTaskParams {
prompt: string;
source: CloudAgentSource;
options: {
autoCreatePr?: boolean;
planApprovalRequired?: boolean;
environment?: Record<string, string>;
};
}
export interface GetStatusResult {
status: CloudAgentStatus;
externalId?: string;
result?: CloudAgentResult;
activities: CloudAgentActivity[];
error?: string;
}
export abstract class CloudAgentBase {
abstract readonly providerId: string;
abstract readonly baseUrl: string;
abstract createTask(p: CreateTaskParams, c: AgentCredentials): Promise<CloudAgentTask>;
abstract getStatus(externalId: string, c: AgentCredentials): Promise<GetStatusResult>;
abstract approvePlan(externalId: string, c: AgentCredentials): Promise<void>;
abstract sendMessage(
externalId: string,
message: string,
c: AgentCredentials
): Promise<CloudAgentActivity>;
abstract listSources(
c: AgentCredentials
): Promise<{ name: string; url: string; branch?: string }[]>;
protected mapStatus(raw: string): CloudAgentStatus; // heuristic upstream-string → enum
protected generateTaskId(): string; // `task_<ts>_<rand>`
protected generateActivityId(): string; // `act_<ts>_<rand>`
}
```
`CodexCloudAgent.approvePlan` intentionally throws — Codex Cloud auto-plans and has
no approval gate. `CodexCloudAgent.listSources` returns `[]`.
`CursorCloudAgent` drives Cursor's Background / Cloud Agents through its official REST
API (`api.cursor.com/v0`) with a **user or service-account API key** — the safer,
first-party alternative to re-using the Cursor IDE's OAuth session (provider `cursor`,
which carries a ban-risk warning). It is a plain REST adapter (no `@cursor/sdk` native
dependency). `approvePlan` throws (Cursor agents run autonomously); `listSources` lists
the repositories reachable by the key. Cursor returns UPPERCASE status enums
(`CREATING`/`RUNNING`/`FINISHED`/`ERROR`), mapped explicitly to the shared
`CloudAgentStatus`. `baseUrl` is overridable per-credential so the API version/path can
be corrected without a code change.
## Domain Types
Source: `src/lib/cloudAgent/types.ts`
```typescript
export const CLOUD_AGENT_STATUS = {
QUEUED: "queued",
RUNNING: "running",
AWAITING_APPROVAL: "awaiting_approval",
COMPLETED: "completed",
FAILED: "failed",
CANCELLED: "cancelled",
} as const;
export interface CloudAgentSource {
repoName: string;
repoUrl: string; // must be a valid URL
branch?: string;
}
export interface CloudAgentResult {
prUrl?: string;
prNumber?: number;
commitMessage?: string;
diffUrl?: string;
summary?: string;
duration?: number; // seconds, positive int
cost?: number; // positive float
}
export interface CloudAgentActivity {
id: string;
type: "plan" | "command" | "code_change" | "message" | "error" | "completion";
content: string;
timestamp: string; // ISO 8601
metadata?: Record<string, unknown>;
}
export interface CloudAgentTask {
id: string; // internal `task_...` id
providerId: "jules" | "devin" | "codex-cloud" | "cursor-cloud";
externalId?: string; // upstream provider's id
status: CloudAgentStatus;
prompt: string; // 1..10000 chars
source: CloudAgentSource;
options: {
autoCreatePr?: boolean;
planApprovalRequired?: boolean;
environment?: Record<string, string>;
};
result?: CloudAgentResult;
activities: CloudAgentActivity[];
error?: string;
createdAt: string;
updatedAt: string;
completedAt?: string;
}
```
Validation schemas (`CreateCloudAgentTaskSchema`, `UpdateCloudAgentTaskSchema`) are
exported alongside the types and are used by the route handlers.
## Database
Source: `src/lib/cloudAgent/db.ts` — table is created lazily via
`createCloudAgentTaskTable()` (also called from `src/lib/cloudAgent/index.ts` at
module import).
```sql
CREATE TABLE IF NOT EXISTS cloud_agent_tasks (
id TEXT PRIMARY KEY,
provider_id TEXT NOT NULL,
external_id TEXT,
status TEXT NOT NULL DEFAULT 'queued',
prompt TEXT NOT NULL,
source TEXT NOT NULL, -- JSON
options TEXT DEFAULT '{}', -- JSON
result TEXT, -- JSON
activities TEXT DEFAULT '[]', -- JSON
error TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
completed_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_provider ON cloud_agent_tasks(provider_id);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_status ON cloud_agent_tasks(status);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_created ON cloud_agent_tasks(created_at DESC);
```
`updateCloudAgentTask` enforces a **column whitelist** to prevent SQL injection:
`status`, `prompt`, `source`, `options`, `result`, `activities`, `error`,
`completed_at`. Any other key in the partial update is silently dropped.
## REST API — Task Lifecycle
**Auth:** All `/api/v1/agents/tasks*` endpoints require **management auth**
(`requireCloudAgentManagementAuth` wraps `requireManagementAuth` from
`src/lib/api/requireManagementAuth`). This is enforced after commit `588a0333`
(_"fix(auth): require management auth for agent and cooldown APIs"_).
| Method | Path | Purpose |
| ------- | ----------------------------- | ------------------------------------------------------ |
| OPTIONS | `/api/v1/agents/tasks` | CORS preflight |
| GET | `/api/v1/agents/tasks` | List tasks (filter: `provider`, `status`, `limit≤500`) |
| POST | `/api/v1/agents/tasks` | Create task (dispatches to upstream + persists) |
| DELETE | `/api/v1/agents/tasks?id=...` | Delete task by query id (does **not** cancel upstream) |
| OPTIONS | `/api/v1/agents/tasks/[id]` | CORS preflight |
| GET | `/api/v1/agents/tasks/[id]` | Read task + lazy-sync status from upstream |
| POST | `/api/v1/agents/tasks/[id]` | Action: `approve` / `message` / `cancel` |
| DELETE | `/api/v1/agents/tasks/[id]` | Delete task by path id |
### Create task
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"providerId": "devin",
"prompt": "Fix the bug in src/foo.ts where the parser returns null",
"source": {
"repoName": "user/repo",
"repoUrl": "https://github.com/user/repo",
"branch": "main"
},
"options": {
"autoCreatePr": true,
"planApprovalRequired": false
}
}'
```
Response `201`:
```json
{
"data": {
"id": "task_1731512345678_abc123def",
"providerId": "devin",
"externalId": "session_xyz",
"status": "queued",
"prompt": "...",
"source": { "repoName": "user/repo", "repoUrl": "...", "branch": "main" },
"options": { "autoCreatePr": true },
"createdAt": "2026-05-13T12:34:56.789Z"
}
}
```
### Approve a plan
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{"action":"approve"}'
```
### Send a follow-up message
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-d '{"action":"message","message":"Also add a unit test for the parser"}'
```
### Cancel (local status only)
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-d '{"action":"cancel"}'
```
`cancel` flips `status` to `"cancelled"` in the local DB but does **not** call the
upstream provider — there is no abort RPC in `CloudAgentBase`. To stop billing
upstream, terminate the task in the provider's own console.
## REST API — Cloud Provider Plumbing
These auxiliary endpoints under `src/app/api/cloud/` are used by remote clients
(the CLI, the Electron app, or sync workers) to read provider connection metadata
and resolve model aliases. They are authenticated with a **regular API key**
(via `validateApiKey`), not the management auth used by the task endpoints.
| Method | Path | Purpose |
| ------ | ------------------------------- | ------------------------------------------------------------------- |
| POST | `/api/cloud/auth` | Validate API key, return masked connection metadata + model aliases |
| PUT | `/api/cloud/credentials/update` | Refresh `accessToken` / `refreshToken` / `expiresAt` |
| POST | `/api/cloud/model/resolve` | Resolve a model alias to `{ provider, model }` |
| GET | `/api/cloud/models/alias` | List all model aliases |
| PUT | `/api/cloud/models/alias` | Set a model alias (and auto-sync to Cloud if enabled) |
`/api/cloud/auth` never returns raw `apiKey` / `accessToken` / `refreshToken`. It
returns `hasApiKey`, `hasAccessToken`, `hasRefreshToken`, and a masked preview
(`maskedApiKey`: first 4 + `****` + last 4).
## Credentials Resolution
`getCloudAgentCredentials(providerId)` in `src/lib/cloudAgent/api.ts`:
1. Loads active provider connections via `getProviderConnections({ provider: providerId, isActive: true })`.
2. For each connection, prefers `apiKey` (trimmed). Falls back to `accessToken`.
3. Returns the first non-empty token wrapped as `{ apiKey: token }`.
4. Returns `null` if no usable token is found — the API responds `400` with
`"No active credentials configured for cloud agent provider: <id>"`.
This means Cloud Agents reuse the same Provider Connection table as regular LLM
providers. To enable Jules, create an active connection with `provider: "jules"`
and a populated `apiKey`.
## Dashboard
Source: `src/app/(dashboard)/dashboard/cloud-agents/page.tsx`
A `"use client"` React page that:
- Lists tasks (polled via `GET /api/v1/agents/tasks`).
- Submits new tasks via a form that maps to `CreateCloudAgentTaskSchema`.
- Shows status badges (`queued`, `running`, `awaiting_approval`, `completed`,
`failed`, `cancelled`) and renders the `activities[]` timeline.
- Surfaces the `result.prUrl` / `commitMessage` / `summary` when `status === "completed"`.
## Integration with A2A
Cloud Agents can be exposed as A2A skills by registering an A2A skill that delegates
its `tasks/send` handler to `getAgent(...).createTask(...)` and translates A2A task
status events to the JSON-RPC 2.0 protocol. See [A2A-SERVER.md](./A2A-SERVER.md).
## Adding a New Cloud Agent
1. Create `src/lib/cloudAgent/agents/<name>.ts` extending `CloudAgentBase`.
2. Implement `createTask`, `getStatus`, `approvePlan` (or throw if N/A),
`sendMessage`, `listSources`. Use `this.mapStatus(...)` for status normalization.
3. Register in `src/lib/cloudAgent/registry.ts` under a stable `providerId`.
4. Extend the `providerId` literal union in `src/lib/cloudAgent/types.ts`
(`CloudAgentTask.providerId` and `CreateCloudAgentTaskSchema`).
5. Add the provider to `src/shared/constants/providers.ts` if it needs a connection
record. OAuth-based providers also need `src/lib/oauth/providers/`.
6. Add tests under `tests/unit/cloud-agent-*.test.ts`.
7. Update this doc and the dashboard's `CLOUD_AGENTS` constant.
## Configuration
| Env Var | Purpose |
| ---------------- | ----------------------------------------------------------- |
| `DATA_DIR` | Location of the SQLite database holding `cloud_agent_tasks` |
| `JWT_SECRET` | Required for management auth on task endpoints |
| `API_KEY_SECRET` | Required to encrypt provider connection credentials at rest |
No Cloud-Agent-specific env vars exist today — every secret lives in the
`provider_connections` table.
## See Also
- [A2A-SERVER.md](./A2A-SERVER.md)
- [API_REFERENCE.md](../reference/API_REFERENCE.md)
- [SKILLS.md](./SKILLS.md)
- [MEMORY.md](./MEMORY.md)
- Source: `src/lib/cloudAgent/`
- Routes: `src/app/api/v1/agents/tasks/`, `src/app/api/cloud/`
- Dashboard: `src/app/(dashboard)/dashboard/cloud-agents/page.tsx`
docs/frameworks/EMBEDDED-SERVICES.md
+856
View File
@@ -0,0 +1,856 @@
---
title: "Embedded Services"
description: "Reference for 9Router, CLIProxyAPI, Mux, and Bifrost"
---
# Embedded Services
> **Version:** v3.8.44
> **Last updated:** 2026-07-03
> **Audience:** Engineers adding, maintaining, or debugging embedded services (9Router, CLIProxyAPI, Mux, Bifrost).
Embedded services are locally-installed process sidecar tools that OmniRoute installs, supervises, and
exposes as first-class routing targets. Unlike external providers (which are reached over the internet
via API keys), embedded services run on the same machine as OmniRoute and communicate over loopback.
---
## Table of Contents
1. [Overview](#1-overview)
2. [Architecture — 4 layers](#2-architecture--4-layers)
3. [Lifecycle state machine](#3-lifecycle-state-machine)
4. [API reference](#4-api-reference)
5. [Security](#5-security)
6. [Adding a new embedded service](#6-adding-a-new-embedded-service)
7. [Troubleshooting](#7-troubleshooting)
8. [FAQ](#8-faq)
---
## 1. Overview
### Why embedded services?
Four services are embedded as of v3.8.44:
| Service | npm package | Default port | Purpose |
| --------------- | ----------------------------------------------- | :----------: | ------------------------------------------------------------------------------------------------------------------ |
| **9Router** | `9router` | 20130 | AI router that OmniRoute can use as a sub-provider. Models exposed as `9router/{sub}/{model}` |
| **CLIProxyAPI** | `@anthropic/cli-proxy` (via `cliproxy` binary) | auto | Local proxy adapter for Anthropic CLI auth flows. Provides fallback routing when OAuth tokens expire |
| **Mux** | `mux` (headless `mux server`) | 8322 | Local agent-orchestration daemon (coder/mux). Lifecycle-managed only — not a routing target (no LLM proxying). |
| **Bifrost** | `@maximhq/bifrost` | 8080 | Go AI-gateway relay backend. When running, auto-selected by the relay route (`/v1/relay/`) |
All four follow the same supervisory model:
- OmniRoute installs them under `DATA_DIR/services/{name}/` (isolated from OmniRoute's own `package.json`)
- OmniRoute spawns and monitors them as child processes
- OmniRoute injects an ephemeral API key into the child's environment and rotates it without downtime (where applicable)
- All management routes (`/api/services/*`) are **LOCAL_ONLY** — accessible only from loopback (hard rule #17)
### Key decisions (from design plan)
| Decision | Value |
| ------------------------------------- | ------------------------------------------------------------------------ |
| Dashboard access to 9Router native UI | Reverse proxy at `/dashboard/providers/services/9router/embed/*` |
| Installation mechanism | `npm install {package}` via `execFile` (no shell interpolation) |
| Consumption mode | Provider registered as `9router/{sub}/{model}` in routing engine |
| API key management | OmniRoute generates, encrypts at-rest (AES-256-GCM), and injects via env |
| Dashboard location | `/dashboard/providers/services` (three tabs) |
| Auto-start | Toggle per service, default OFF |
---
## 2. Architecture — 4 layers
```
┌────────────────────────────────────────────────────────────────────┐
│ Layer 1 — UI │
│ /dashboard/providers/services (tabs: CLIProxyAPI | 9Router | Mux)│
│ Logs live (SSE), Start/Stop/Restart/Update, Settings, Install │
│ │
│ src/app/(dashboard)/dashboard/providers/services/ │
│ ├── page.tsx Shell + tab routing by ?tab= │
│ ├── tabs/ CliproxyServiceTab, NinerouterServiceTab,│
│ │ MuxServiceTab │
│ └── components/ ServiceStatusCard, ServiceLifecycleButtons,│
│ ServiceLogsPanel, ApiKeyCard, ... │
└──────────────────────┬─────────────────────────────────────────────┘
│ HTTP (Next.js fetch)
┌──────────────────────▼─────────────────────────────────────────────┐
│ Layer 2 — API (LOCAL_ONLY — loopback only) │
│ │
│ /api/services/9router/{install|start|stop|restart|update| │
│ rotate-key|status|auto-start|logs} │
│ /api/services/cliproxy/{install|start|stop|restart|update| │
│ status|auto-start|logs} │
│ /api/services/mux/{install|start|stop|restart|update| │
│ status|auto-start|logs} │
│ /dashboard/providers/services/9router/embed/[...path] │
│ (reverse HTTP + WebSocket proxy → 9Router upstream) │
│ │
│ Gate: LOCAL_ONLY_API_PREFIXES includes "/api/services/" and │
│ "/dashboard/providers/services/*/embed/" │
└──────────────────────┬─────────────────────────────────────────────┘
│ in-process calls
┌──────────────────────▼─────────────────────────────────────────────┐
│ Layer 3 — ServiceSupervisor (src/lib/services/) │
│ │
│ ServiceSupervisor.ts Generic supervisor (child_process.spawn) │
│ ├── install: execFile('npm', ['install', pkg, '--prefix']) │
│ ├── start: spawn(node, [entrypoint], {env, cwd}) │
│ ├── api_key: crypto.randomBytes(32) → env NINEROUTER_API_KEY │
│ ├── port: 20130 for 9Router (configurable) │
│ ├── logs: stdio ring buffer 5 MB → SSE events │
│ ├── health: HTTP GET /health every 2–5 s, lazy recovery │
│ └── lifecycle: SIGTERM 15 s → SIGKILL │
│ │
│ registry.ts getSupervisor(name) / registerSupervisor() │
│ bootstrap.ts Bootstraps all SERVICES[] at process start │
│ apiKey.ts getOrCreateApiKey(), generateServiceApiKey() │
│ modelSync.ts Periodic GET /v1/models → service_models table │
│ ringBuffer.ts Circular log buffer (5 MB per service) │
│ healthCheck.ts Polling HTTP health probe │
│ installers/ ninerouter.ts, cliproxy.ts, mux.ts │
│ (installer adapters) │
└──────────────────────┬─────────────────────────────────────────────┘
│ OpenAI-compatible HTTP (loopback)
┌──────────────────────▼─────────────────────────────────────────────┐
│ Layer 4 — Provider / Routing │
│ │
│ open-sse/executors/ninerouter.ts │
│ Re-looks up port and API key per-request (no caching). │
│ Strips "9router/" prefix from model id before proxying. │
│ Returns 503 service_not_running if supervisor not in "running". │
│ │
│ src/shared/constants/providers.ts │
│ Entry for "9router": isEmbeddedService: true │
│ │
│ open-sse/config/providerRegistry.ts │
│ Models stored as "9router/{sub}/{model}" (prefixed). │
│ Synced every 5 min by modelSync.ts. │
│ │
│ Mux is lifecycle-managed ONLY (Layers 1-3) — it is an agent- │
│ orchestration daemon, not an LLM proxy, so it has no Layer 4 │
│ executor/provider entry and is never a routing target. │
└────────────────────────────────────────────────────────────────────┘
```
### Key source files
| File | Role |
| ------------------------------------------- | ------------------------------------------------ |
| `src/lib/services/ServiceSupervisor.ts` | Core class: lifecycle, lock, health, ring buffer |
| `src/lib/services/bootstrap.ts` | Process-level registration and auto-start |
| `src/lib/services/registry.ts` | Singleton map `tool → supervisor` |
| `src/lib/services/apiKey.ts` | Key generation, AES-256-GCM encryption at-rest |
| `src/lib/services/modelSync.ts` | Periodic model sync (5 min) + on-demand |
| `src/lib/services/ringBuffer.ts` | 5 MB circular log buffer with SSE subscribe |
| `src/lib/services/healthCheck.ts` | HTTP health probe (configurable interval) |
| `src/lib/services/installers/ninerouter.ts` | npm install/update/uninstall for 9Router |
| `src/lib/services/installers/cliproxy.ts` | npm install/update/uninstall for CLIProxyAPI |
| `src/lib/services/installers/mux.ts` | npm install/update/uninstall for Mux |
| `src/app/api/services/9router/_lib.ts` | `getOrInitSupervisor()` helper |
| `src/app/api/services/[name]/logs/route.ts` | Shared SSE logs endpoint |
| `open-sse/executors/ninerouter.ts` | Provider executor (Layer 4) |
---
## 3. Lifecycle state machine
```
install()
┌─────────────┐ ──────────► ┌─────────────┐
│ not_installed│ │ stopped │◄──────────────────┐
└─────────────┘ └──────┬──────┘ │
│ start() │
▼ │ stop()
┌──────────┐ │
│ starting │ │
└────┬─────┘ │
health probe ok │ crash / SIGTERM │
┌────▼─────┐ (exit within 5s) │
│ running │──── crash ──────────►┤
└────┬─────┘ ┌─▼────┐
stop() │ │error │
▼ └──────┘
┌──────────┐
│ stopping │
└──────────┘
```
States stored in the `version_manager` DB table (`status` column) and mirrored
in `ServiceSupervisor` in-memory state. The in-memory state is authoritative for
a running process; the DB state is the durable fallback at boot.
### State transitions
| From | Event | To |
| --------------- | ---------------------------------- | ---------------------- |
| `not_installed` | `install()` succeeds | `stopped` |
| `stopped` | `start()` called | `starting` |
| `starting` | health probe returns 200 | `running` |
| `starting` | process exits before healthy | `error` |
| `running` | `stop()` called | `stopping` → `stopped` |
| `running` | process exits unexpectedly (< 5 s) | `error` (fast crash) |
| `running` | process exits unexpectedly (> 5 s) | `error` |
| `error` | `start()` called | `starting` |
| any | `stop()` while `stopping` | no-op |
### Operation lock
`ServiceSupervisor` serializes lifecycle operations through an async operation lock
(`withLock()`). Concurrent `start()` calls on the same supervisor result in exactly
one spawn; the second caller waits and returns the existing status. This prevents
race conditions when, for example, auto-start and a UI button fire simultaneously.
---
## 4. API reference
All routes under `/api/services/` are **LOCAL_ONLY** (loopback only, hard rule #17).
Non-loopback requests receive `403 LOCAL_ONLY` regardless of auth token.
### 4.1 9Router endpoints (8 routes)
#### `POST /api/services/9router/install`
Install 9Router from npm. Creates `DATA_DIR/services/9router/` with its own
`package.json` and `node_modules/`. Does not conflict with OmniRoute's own deps.
**Request body** (all optional):
```json
{ "version": "latest" }
```
| Field | Type | Default | Description |
| --------- | -------- | ---------- | ------------------------------------ |
| `version` | `string` | `"latest"` | npm version tag or semver to install |
**Responses:**
| Status | Description |
| ------ | ------------------------------------------------------ |
| `200` | `{ ok: true, installedVersion: "x.y.z", path: "..." }` |
| `400` | Invalid request body (Zod validation failure) |
| `409` | Already installing (lock held) |
| `500` | npm install failed — see `message` for friendly error |
**Notes:** Uses `execFile('npm', [...])` — no shell, no interpolation (hard rule #13).
EACCES errors are surfaced as friendly messages.
---
#### `POST /api/services/9router/start`
Start 9Router. Registers a supervisor if not already registered, then calls
`supervisor.start()`. Idempotent when already running.
**Request body:** none
**Responses:**
| Status | Description |
| ------ | ---------------------------------------------------- |
| `200` | `ServiceStatus` object (see schema below) |
| `409` | 9Router is not installed (`status: "not_installed"`) |
| `503` | Start failed (process error — see `lastError`) |
**ServiceStatus schema:**
```json
{
"tool": "9router",
"state": "running",
"pid": 12345,
"port": 20130,
"health": "healthy",
"startedAt": "2026-05-25T10:00:00.000Z",
"lastError": null
}
```
---
#### `POST /api/services/9router/stop`
Gracefully stop 9Router. Sends SIGTERM, waits 15 s, then SIGKILL if still alive.
Idempotent when already stopped.
**Request body:** none
**Responses:**
| Status | Description |
| ------ | ---------------------------------- |
| `200` | `ServiceStatus` (state: "stopped") |
| `503` | Stop failed unexpectedly |
---
#### `POST /api/services/9router/restart`
Equivalent to `stop()` then `start()` under the operation lock.
**Request body:** none
**Responses:** same as `start` (returns final `ServiceStatus`).
---
#### `POST /api/services/9router/update`
Updates 9Router to a newer npm version. If the service is running, it is stopped
first, npm install is run (installing the newer version in-place), and then the
service is restarted.
**Request body** (all optional):
```json
{ "version": "latest" }
```
**Responses:**
| Status | Description |
| ------ | --------------------------------------------------------------- |
| `200` | `{ ok: true, previousVersion: "...", installedVersion: "..." }` |
| `400` | Invalid body |
| `500` | npm update failed |
---
#### `POST /api/services/9router/rotate-key`
Generates a new API key for 9Router, encrypts it at-rest, and restarts the service
(if running) so it picks up the new key from its environment. The old key is
invalidated immediately.
**Request body:** none
**Responses:**
| Status | Description |
| ------ | ------------------------------------------ |
| `200` | `{ keyRotated: true, restarted: boolean }` |
| `500` | Rotation failed |
**Security:** The new key is never returned in the response (no credential leak).
It is stored encrypted (AES-256-GCM) in the `version_manager` table.
---
#### `GET /api/services/9router/status`
Returns combined live + DB status including version metadata and API key preview.
**Responses:**
| Status | Description |
| ------ | ------------------ |
| `200` | See schema below |
| `500` | Status read failed |
**Response schema:**
```json
{
"tool": "9router",
"state": "running",
"pid": 12345,
"port": 20130,
"health": "healthy",
"startedAt": "2026-05-25T10:00:00.000Z",
"lastError": null,
"installedVersion": "1.2.3",
"latestVersion": "1.2.4",
"updateAvailable": true,
"apiKeyMasked": "nr_****abcd",
"autoStart": false,
"providerExpose": false
}
```
---
#### `POST /api/services/9router/auto-start`
Toggle the auto-start flag. When `enabled: true`, the service starts automatically
the next time OmniRoute boots (if the service is installed).
**Request body:**
```json
{ "enabled": true }
```
**Responses:**
| Status | Description |
| ------ | --------------------- |
| `200` | `{ autoStart: true }` |
| `400` | Invalid body |
---
#### `GET /api/services/9router/logs`
SSE stream of live logs from 9Router's stdout/stderr ring buffer.
**Query parameters:**
| Param | Type | Default | Description |
| -------- | --------- | ------- | --------------------------------------------------------- |
| `tail` | `integer` | 200 | How many historical lines to send first (max 1000) |
| `filter` | `string` | none | Case-insensitive substring filter (no regex — ReDoS-safe) |
**SSE events:**
| Event | Data | Description |
| ----------- | ----------- | ----------------------- |
| `snapshot` | `LogLine[]` | Initial historical tail |
| `log` | `LogLine` | Live log line |
| `heartbeat` | `{}` | Keep-alive every 15 s |
**LogLine schema:**
```json
{ "ts": 1716633600000, "stream": "stdout", "line": "[9router] Listening on :20130" }
```
**Responses:**
| Status | Description |
| ------ | --------------------------------------------- |
| `200` | `text/event-stream` |
| `400` | `filter` parameter too long (> 200 chars) |
| `404` | Service not found (supervisor not registered) |
---
### 4.2 CLIProxyAPI endpoints (7 routes)
CLIProxyAPI has the same endpoint shape as 9Router minus `rotate-key` (CLIProxyAPI
does not require an injected API key; it authenticates via the host's existing CLI
config) and `status` includes fewer fields.
| Method | Path | Description |
| ------ | ----------------------------------- | ------------------------------------ |
| `POST` | `/api/services/cliproxy/install` | Install CLIProxyAPI from npm |
| `POST` | `/api/services/cliproxy/start` | Start CLIProxyAPI |
| `POST` | `/api/services/cliproxy/stop` | Stop CLIProxyAPI |
| `POST` | `/api/services/cliproxy/restart` | Restart CLIProxyAPI |
| `POST` | `/api/services/cliproxy/update` | Update to newer version |
| `GET` | `/api/services/cliproxy/status` | Live + DB status (no `apiKeyMasked`) |
| `POST` | `/api/services/cliproxy/auto-start` | Toggle auto-start |
The shared `GET /api/services/{name}/logs` endpoint (see §4.1) works for all
four services using the `[name]` dynamic segment.
---
### 4.3 Mux endpoints (7 routes)
Mux has the same endpoint shape as CLIProxyAPI — no `rotate-key` route in the API
surface (the bearer token is generated the same way as 9Router's via
`getOrCreateApiKey("mux")` and injected via the `MUX_SERVER_AUTH_TOKEN` env var, but
there is no dedicated rotation endpoint yet). Mux is lifecycle-managed only: unlike
9Router, it has no Layer 4 executor and is never registered as a routing provider.
| Method | Path | Description |
| ------ | -------------------------------- | ------------------------------------- |
| `POST` | `/api/services/mux/install` | Install Mux from npm (`npm i mux`) |
| `POST` | `/api/services/mux/start` | Start Mux (`mux server`) |
| `POST` | `/api/services/mux/stop` | Stop Mux |
| `POST` | `/api/services/mux/restart` | Restart Mux |
| `POST` | `/api/services/mux/update` | Update to newer npm version |
| `GET` | `/api/services/mux/status` | Live + DB status |
| `POST` | `/api/services/mux/auto-start` | Toggle auto-start |
---
### 4.4 Bifrost endpoints (7 routes)
Bifrost is a Go AI-gateway relay backend (`@maximhq/bifrost`). It uses the same
endpoint shape as CLIProxyAPI (no `rotate-key` — Bifrost manages its own provider
keys in `config.json` under its `-app-dir`).
| Method | Path | Description |
| ------ | ---------------------------------- | ------------------------------------------------------ |
| `POST` | `/api/services/bifrost/install` | Install Bifrost from npm (`@maximhq/bifrost`) |
| `POST` | `/api/services/bifrost/start` | Start Bifrost on port 8080 (default) |
| `POST` | `/api/services/bifrost/stop` | Stop Bifrost |
| `POST` | `/api/services/bifrost/restart` | Restart Bifrost |
| `POST` | `/api/services/bifrost/update` | Update to newer version |
| `GET` | `/api/services/bifrost/status` | Live + DB status |
| `POST` | `/api/services/bifrost/auto-start` | Toggle auto-start |
| `GET` | `/api/services/bifrost/logs` | SSE log tail (via shared `[name]/logs` dynamic route) |
**Routing wiring:** When `BIFROST_BASE_URL` is unset and the supervised Bifrost
instance is running, `getBifrostRoutingConfig()` (in `routingBackend.ts`) automatically
uses `http://127.0.0.1:{port}` as the relay base URL. Explicit `BIFROST_BASE_URL` env
always takes precedence.
---
### 4.4 Reverse proxy (9Router dashboard embed)
The dashboard embeds the 9Router web UI inside an iframe via an internal reverse
proxy at:
```
GET|POST|... /dashboard/providers/services/9router/embed/[...path]
```
This proxy:
- Forwards the request to `http://127.0.0.1:{port}/{path}` (loopback only)
- Strips incoming `cookie` and `authorization` headers (no leakage of OmniRoute session)
- Injects `Authorization: Bearer {apiKey}` for 9Router authentication
- Strips `set-cookie`, `content-security-policy`, `x-frame-options`, `cross-origin-*` from the response
- Rewrites HTML responses to inject `<base href>` and normalize absolute paths (`/foo` → `/dashboard/.../embed/foo`)
WebSocket upgrades for the embedded dashboard are handled by a companion server on a
dedicated port (see `src/lib/services/embedWsProxy.ts`).
**Security:** The embed proxy routes are classified under `LOCAL_ONLY_API_PREFIXES`
and can only be reached from loopback. An attacker who obtains a JWT via a
Cloudflare/Ngrok tunnel cannot proxy into embedded services.
---
## 5. Security
### LOCAL_ONLY enforcement (hard rule #17)
All routes under `/api/services/` and `/dashboard/providers/services/*/embed/` are
classified as LOCAL_ONLY in `src/server/authz/routeGuard.ts`. The loopback check
runs unconditionally before any auth branch:
```
request arrives
→ isLocalOnlyPath(path)?
→ non-loopback → 403 LOCAL_ONLY (always, before auth check)
→ loopback → fall through to normal auth
```
This prevents a leaked JWT (e.g., via a tunnel) from triggering `npm install` or
process spawning. See `docs/security/ROUTE_GUARD_TIERS.md` for the full tier
matrix.
### API key injection
9Router and Mux require an API key/bearer token for their own HTTP endpoints.
OmniRoute:
1. Generates a key via `crypto.randomBytes(32).toString("base64url")` with a
service-specific prefix (`nr_` for 9Router, `mx_` for Mux).
2. Encrypts it at-rest using AES-256-GCM (same cipher used for provider credentials).
3. Decrypts and injects it as an environment variable at spawn time —
`NINEROUTER_API_KEY` for 9Router, `MUX_SERVER_AUTH_TOKEN` for Mux (never a CLI
flag, so the token never appears in `ps`/process listings).
4. Never returns the plaintext key in any HTTP response.
CLIProxyAPI does not require an injected key (it authenticates via the host's
existing CLI config).
### SSRF defense
The reverse HTTP proxy (`/dashboard/.../embed/[...path]`) is hardcoded to forward
only to `http://127.0.0.1:{port}`. It never follows redirects to non-loopback
destinations. The `ssrf-req-filter` library is used to reject any upstream URL that
resolves outside the loopback range.
### Shell safety (hard rule #13)
`npm install` is invoked via `execFile('npm', ['install', pkg, '--prefix', dir])` —
no template literals, no shell, no interpolation of external paths into the command
string. Runtime values (ports, API keys) are passed via the child's `env` object.
### Error sanitization (hard rule #12)
All error responses from `/api/services/*` go through `buildErrorBody()` or
`sanitizeErrorMessage()`. Raw `err.stack` and `err.message` are never returned
verbatim to the caller.
---
## 6. Adding a new embedded service
Follow these 8 steps. Read the existing implementations in `src/lib/services/installers/`
and `src/app/api/services/` as the canonical reference.
### Step 1 — Create the installer
Create `src/lib/services/installers/{name}.ts` modeled on `ninerouter.ts`:
```typescript
export const NAME_PACKAGE = "your-npm-package";
export const NAME_DEFAULT_PORT = 20132; // pick a free port
export async function install(version = "latest"): Promise<InstallResult> { ... }
export async function update(version = "latest"): Promise<InstallResult> { ... }
export async function uninstall(): Promise<void> { ... }
export function resolveSpawnArgs(apiKey: string, port: number): SpawnArgs { ... }
export async function getInstalledVersion(): Promise<string | null> { ... }
export async function getLatestVersion(): Promise<string | null> { ... }
```
Use `runNpm(['install', NAME_PACKAGE, '--prefix', dir])` from `installers/utils.ts`
— never `execSync` or shell interpolation.
### Step 2 — Register in bootstrap
Add a `ServiceEntry` to the `SERVICES` array in `src/lib/services/bootstrap.ts`:
```typescript
{
tool: "myservice",
port: NAME_DEFAULT_PORT,
healthPath: "/health",
healthIntervalMs: 5_000,
stopTimeoutMs: 15_000,
logsBufferBytes: 5_242_880,
needsApiKey: true, // false if no API key needed
}
```
Extend `buildSpawnArgsFactory()` to handle `cfg.tool === "myservice"`.
### Step 3 — Add migration and DB seed
Ensure the service has a row in `version_manager` via a migration in
`src/lib/db/migrations/`. The row should have:
```sql
INSERT OR IGNORE INTO version_manager (tool, status, auto_start, provider_expose)
VALUES ('myservice', 'not_installed', 0, 0);
```
### Step 4 — Create the 7 API endpoints
Under `src/app/api/services/{name}/`:
```
_lib.ts getOrInitSupervisor() helper
install/route.ts POST — calls installer.install()
start/route.ts POST — calls supervisor.start()
stop/route.ts POST — calls supervisor.stop()
restart/route.ts POST — calls supervisor.restart()
update/route.ts POST — calls installer.update()
status/route.ts GET — merges live + DB status
auto-start/route.ts POST — toggles auto_start flag
```
The shared `GET /api/services/[name]/logs` route is already wired — no changes
needed there.
Delegate all error responses through `createErrorResponse()` / `buildErrorBody()`.
### Step 5 — Add to LOCAL_ONLY_API_PREFIXES
In `src/server/authz/routeGuard.ts`, verify that `/api/services/` is already listed.
If you introduce a new prefix (e.g., `/api/tools/`), add it to both
`LOCAL_ONLY_API_PREFIXES` and, if it spawns processes, to `SPAWN_CAPABLE_PREFIXES`.
Add a test in `tests/unit/authz/routeGuard.test.ts`.
### Step 6 — Add the UI tab
Create `src/app/(dashboard)/dashboard/providers/services/tabs/{Name}ServiceTab.tsx`.
Reuse shared components:
- `ServiceStatusCard` — live state + health badge
- `ServiceLifecycleButtons` — Start / Stop / Restart / Update
- `ServiceLogsPanel` — SSE log tail (connects to `/api/services/{name}/logs`)
- `ApiKeyCard` — key reveal + rotate (if `needsApiKey: true`)
Register the tab in `ServicesPageShell.tsx`.
### Step 7 — Add the provider entry (if the service is a routing target)
If the embedded service exposes an OpenAI-compatible `/v1/chat/completions` endpoint:
1. Add a provider entry in `src/shared/constants/providers.ts` with `isEmbeddedService: true`.
2. Create `open-sse/executors/{name}.ts` extending `BaseExecutor`. Re-lookup port and
API key per-request (never cache in the constructor). Return a `503 service_not_running`
response when the supervisor state is not `"running"`.
3. Register models in `open-sse/config/providerRegistry.ts` with the service prefix
(e.g., `myservice/sub/model`). `modelSync.ts` will keep them updated.
### Step 8 — Document and test
1. Update `docs/frameworks/EMBEDDED-SERVICES.md` (this file) — add the service to the
table in §1 and any new endpoints to §4.
2. Add unit tests in `tests/unit/services/` (lifecycle, installer, API shape).
3. Add integration test in `tests/integration/services/` (behind `RUN_SERVICES_INT=1`).
4. Update `docs/openapi.yaml` with the new endpoints.
---
## 7. Troubleshooting
### Service does not start
**Symptoms:** Start button returns 503, state stays `"error"` or `"starting"`.
**Checklist:**
1. Check `GET /api/services/{name}/logs` (or the Logs panel in the dashboard). Look
for lines like `Error: ENOENT`, `address already in use`, or `Cannot find module`.
2. Verify `npm` is in PATH: `which npm` from the same user account that runs OmniRoute.
3. Verify the service is installed: check `GET /api/services/{name}/status` for
`installedVersion`. If `null`, run install first.
4. Check `DATA_DIR/services/{name}/node_modules/` exists and is not empty.
5. Check the `lastError` field in the status response for the sanitized exit reason.
---
### Cold start is slow (> 10 s to reach `running`)
**Symptoms:** State stays `"starting"` for a long time before going to `"running"` or `"error"`.
**Explanation:** 9Router's cold start includes importing large dependency trees (DNS,
tunnel, MITM modules). Default health interval is 2 s with 3 attempts before the
supervisor declares a timeout (but continues polling).
**Fix:** The `healthIntervalMs` and the `waitForHealthy` timeout
(`healthIntervalMs * 3`) are configurable in `bootstrap.ts`. For services with longer
startup times, increase `healthIntervalMs` to 5000 and `stopTimeoutMs` to 30 000.
---
### Port collision (`EADDRINUSE`)
**Symptoms:** Logs show `address already in use :::20130`.
**Causes:**
- Another process is already using port 20130.
- A previous 9Router process was not fully stopped (zombie PID).
**Fix:**
1. Change the default port via `NINEROUTER_PORT` environment variable in `.env`.
2. Find and kill the conflicting process: `lsof -ti :20130 | xargs kill -9`.
3. The port is configurable per service in `bootstrap.ts` via the `port` field.
**Note:** 9Router defaults to port 20130 specifically to avoid colliding with
OmniRoute's default port 20128.
---
### Permission denied (EACCES) on install
**Symptoms:** Install returns 500, logs show `EACCES` or `permission denied`.
**Causes:**
- `DATA_DIR` or its parent is not writable by the OmniRoute process.
- Running inside Docker rootless without write access to the mapped volume.
**Fix:**
1. Check `DATA_DIR` (default: `~/.omniroute/`): `ls -la ~/.omniroute/`
2. Ensure the OmniRoute process user owns the directory: `chown -R $USER ~/.omniroute/`
3. In Docker, ensure the volume mount has the correct permissions for the container user.
---
### Update fails (`npm install` timeout or network error)
**Symptoms:** Update returns 500 with `InstallError`, logs show network timeout.
**Checklist:**
1. Confirm npm registry is reachable: `npm ping`.
2. Check for corporate proxy: `npm config get proxy`, `npm config get https-proxy`.
3. Try the install manually: `npm install {package}@latest --prefix ~/.omniroute/services/{name}/`.
4. If behind an air-gap, pre-download the tarball and use `npm install /path/to/tarball.tgz`.
---
### Service shows `"error"` state immediately after start (fast crash)
**Symptoms:** State transitions from `"starting"` to `"error"` in under 5 seconds.
`lastError` shows `"Fast crash (exited with code 1)"`.
**Checklist:**
1. Read the full log tail: `GET /api/services/{name}/logs?tail=500`.
2. Common cause: missing environment variables expected by the service.
3. For 9Router: verify `NINEROUTER_DISABLE_MITM=true` and
`NINEROUTER_DISABLE_TUNNEL=true` are in the env passed at spawn (see
`installers/ninerouter.ts` `resolveSpawnArgs`).
---
## 8. FAQ
**Q: Can I expose the embedded services endpoints to non-loopback clients?**
No. The LOCAL_ONLY tier is intentional (hard rule #17). Routes that can invoke
`npm install` or spawn `node` processes must not be reachable from non-loopback
traffic, because a leaked JWT via a tunnel (Cloudflare, Ngrok, Tailscale) would
otherwise allow arbitrary process spawning. There is no opt-out carve-out for
`/api/services/` — unlike `/api/mcp/`, it is excluded from the manage-scope bypass
list. See `docs/security/ROUTE_GUARD_TIERS.md`.
---
**Q: Will 9Router and CLIProxyAPI be available in production/cloud deployments?**
Yes. Both services follow the same local-first model as OmniRoute itself. They run
on the same machine and communicate over loopback. "Production" here means the VPS
or local server where OmniRoute is deployed, not a remote cloud provider.
---
**Q: How do I debug the supervisor?**
1. Tail the SSE log stream: `curl -N http://localhost:20128/api/services/9router/logs`.
2. Check structured logs in OmniRoute's pino output filtered by
`service:supervisor` namespace.
3. Inspect the DB row: `sqlite3 ~/.omniroute/omniroute.db "SELECT * FROM version_manager WHERE tool='9router'"`.
4. Use `GET /api/services/9router/status` to see the current live state, PID, health,
and `lastError` in one call.
---
**Q: The supervisor shows `health: "degraded"` or `health: "unknown"` but state is `"running"`. Is that a problem?**
`"degraded"` means the health probe returned a non-200 response. `"unknown"` means no
probe has completed yet (race with first poll). Both are transient during startup.
If health stays `"degraded"` for more than `healthIntervalMs * 3` ms after
`"running"`, the embedded service is running but its HTTP API is not responding. Check
whether the port is correct in the status response and whether the service is actually
listening on that port.
---
**Q: Can I change the 9Router API key without a full restart?**
No. The API key is passed to 9Router via an environment variable at spawn time.
Environment variables cannot be changed in a running process. `POST .../rotate-key`
automatically stops and restarts the service to apply the new key. The key rotation
takes effect within the service's `stopTimeoutMs` (default 15 s) plus its startup
time.
---
**Q: What is the ring buffer limit and what happens when it fills?**
Each service has a dedicated 5 MB ring buffer. When the buffer is full, the oldest
log lines are evicted to make room for new ones. The SSE `snapshot` event returns
the most recent lines within the `tail` limit. Logs are not persisted to disk unless
`logsBufferPath` is set in the DB row.
---
## See also
- `docs/security/ROUTE_GUARD_TIERS.md` — LOCAL_ONLY tier details
- `docs/architecture/CODEBASE_DOCUMENTATION.md` — §3.2 Embedded Services module mapping
- `docs/architecture/ARCHITECTURE.md` — system-level context
- `docs/openapi.yaml` — machine-readable endpoint definitions
- `CLAUDE.md` §"Adding a New Embedded Service" — quick-reference checklist
docs/frameworks/EVALS.md
+250
View File
@@ -0,0 +1,250 @@
---
title: "Evaluations (Evals)"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Evaluations (Evals)
> **Source of truth:** `src/lib/evals/`, `src/lib/db/evals.ts`, `src/app/api/evals/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute ships a generic evaluation framework you can use to benchmark routing
configurations, single providers/models, or the bundled "golden set" suites.
Use it to verify routing changes, validate new providers, and gate releases
before promoting them to production traffic.
The framework is implemented as:
- A pure runner (`src/lib/evals/evalRunner.ts`) that registers in-memory
built-in suites, evaluates outputs against expected criteria, and aggregates
scorecards.
- A persistence layer (`src/lib/db/evals.ts`) for custom (user-defined) suites
and historical runs in SQLite.
- An orchestration layer (`src/lib/evals/runtime.ts`) that executes each case
by dispatching real calls to `POST /v1/chat/completions`, captures latency
and outputs, and persists the run.
- REST endpoints under `/api/evals/*` (management-auth only).
- A dashboard surface at `Dashboard → Usage → Evals` (`EvalsTab.tsx`).
## Concepts
### Suite
A suite is a named collection of test cases with a `description` and one or
more cases. Suites come from two sources:
| Source | Where defined | Mutable at runtime? |
| ---------- | --------------------------------------------- | ------------------- |
| `built-in` | Registered via `registerSuite()` at boot | No (code-defined) |
| `custom` | Stored in SQLite `eval_suites` + `eval_cases` | Yes (via API/UI) |
The current built-in suites (see `src/lib/evals/evalRunner.ts`):
- `golden-set` — 10 baseline cases across greeting/math/translation/safety
- `coding-proficiency` — Python/JS/SQL/TS/bug detection
- `reasoning-logic` — syllogisms, word problems, pattern recognition
- `multilingual` — translation and language detection
- `safety-guardrails` — PII, jailbreak, refusal, bias awareness
- `instruction-following` — JSON-only, numbered lists, language constraints
- `codex-comparison` — head-to-head coding tasks intended for compare mode
### Case
Each case carries:
| Field | Description |
| ---------- | ------------------------------------------------------------ |
| `id` | Stable identifier (used to key outputs and metrics) |
| `name` | Human-readable label |
| `model` | Default model when the run uses `suite-default` targeting |
| `input` | `{ messages, max_tokens? }` — sent to `/v1/chat/completions` |
| `expected` | `{ strategy, value }` — scoring rubric (see below) |
| `tags` | Optional labels (e.g. `safety`, `pii`, `jailbreak`) |
### Target
The same suite can be run against different targets. The target schema is
`evalTargetSchema` in `src/shared/validation/schemas.ts`:
| Target type | `id` | Behavior |
| --------------- | ---------- | --------------------------------------------------------------- |
| `suite-default` | `null` | Each case uses its built-in `model` field |
| `model` | model name | Force every case through one direct model (e.g. `gpt-4o`) |
| `combo` | combo name | Run every case through one combo (exercises the routing engine) |
For `model` and `combo`, the `id` field is required (enforced by Zod
`superRefine`). When `compareTarget` is provided, both targets must differ —
the runner persists both runs under the same `runGroupId` for A/B comparison.
## Scoring Rubrics
Implemented in `evaluateCase()` (evalRunner.ts):
| Strategy | Pass when… |
| ---------- | -------------------------------------------------------------------- |
| `exact` | `actualOutput === expected.value` |
| `contains` | `actualOutput.toLowerCase().includes(expected.value.toLowerCase())` |
| `regex` | `new RegExp(expected.value).test(actualOutput)` is truthy |
| `custom` | `expected.fn(actualOutput, evalCase)` returns truthy (built-in only) |
**Note:** Custom-function scoring is reserved for code-defined (built-in)
suites because functions cannot be serialized through the API. The
`evalCaseBuilderSchema` only accepts `contains | exact | regex` for
user-created suites.
There is no LLM-as-judge or embedding-based similarity scorer today — it would
be a clean extension point in `evaluateCase()`.
## Database Schema
Three tables (migrations `030_create_eval_runs.sql` and
`031_create_eval_suites.sql`):
| Table | Purpose |
| ------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `eval_suites` | Custom suite metadata (`id`, `name`, `description`) |
| `eval_cases` | Cases per suite — `input_json`, `expected_*`, `tags_json` |
| `eval_runs` | Historical runs — `pass_rate`, `total`, `passed`, `failed`, `avg_latency_ms`, `summary_json`, `results_json`, `outputs_json` |
Built-in suites are **not** stored in the DB. They live in memory and are
re-registered every time `evalRunner.ts` is imported.
## REST API
All endpoints require management auth (`requireManagementAuth`) — they are not
part of the public proxy surface.
| Endpoint | Method | Description |
| ----------------------------- | -------- | ------------------------------------------------------------- |
| `/api/evals` | `GET` | List suites + recent runs + scorecard + targets + keys |
| `/api/evals` | `POST` | Run a suite (single or compare) — schema `evalRunSuiteSchema` |
| `/api/evals/{suiteId}` | `GET` | Fetch one suite (built-in or custom) |
| `/api/evals/suites` | `POST` | Create a custom suite — schema `evalSuiteSaveSchema` |
| `/api/evals/suites/{suiteId}` | `GET` | Fetch a custom suite |
| `/api/evals/suites/{suiteId}` | `PUT` | Replace a custom suite (cases get re-inserted) |
| `/api/evals/suites/{suiteId}` | `DELETE` | Delete a custom suite and its cases |
### Running a suite
```bash
curl -X POST http://localhost:20128/api/evals \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"suiteId": "golden-set",
"target": { "type": "combo", "id": "my-combo" },
"apiKeyId": "optional-api-key-uuid"
}'
```
Optional fields:
- `outputs` — `Record<caseId, string>` of pre-computed outputs. When provided,
the runner **skips dispatch** and only scores the cached outputs (useful for
offline evaluation).
- `compareTarget` — second target to run in parallel; both runs share a
generated `runGroupId` for head-to-head viewing.
- `apiKeyId` — internal API key used to authenticate the dispatched
`/v1/chat/completions` calls. Required when `REQUIRE_API_KEY` is enabled.
### Creating a custom suite
```bash
curl -X POST http://localhost:20128/api/evals/suites \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"name": "Production smoke",
"description": "Quick sanity check before deploy",
"cases": [
{
"name": "JSON shape",
"model": "gpt-4o",
"input": { "messages": [{ "role": "user", "content": "Reply with {\"ok\": true}" }] },
"expected": { "strategy": "regex", "value": "\"ok\"\\s*:\\s*true" }
}
]
}'
```
## Dispatch Pipeline
`runEvalSuiteAgainstTarget()` (`src/lib/evals/runtime.ts`):
1. Resolves the suite (built-in or custom).
2. For each case, builds a `Request` to `/v1/chat/completions` with the case's
`messages`, the resolved `model`, `stream: false`, and `max_tokens: 512`
(or the case override).
3. Calls the chat handler directly (in-process — no extra HTTP hop).
4. Captures latency and extracts text from either `choices[0].message.content`
or the Responses-API `output[]` payload.
5. Scores all outputs via `runSuite()`, then persists via `saveEvalRun()`.
Cases run **sequentially**. There is no concurrency flag today.
## Dashboard
The UI lives at `Dashboard → Usage → Evals`
(`src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx`). From there you
can:
- Browse built-in and custom suites with case-by-case preview.
- Create/edit/delete custom suites with the case builder.
- Pick a target (suite defaults / model / combo), optionally a second
`compareTarget`, optionally an API key, then run on demand.
- Inspect run history, per-case pass/fail, latency, and captured outputs.
- See the rolling scorecard aggregated across the latest run per
`(suite, target)` scope.
## Relationship with the Auto-Assessment RFC
A separate, narrower assessment subsystem lives at `src/domain/assessment/`
(see also [AUTO-COMBO.md](../routing/AUTO-COMBO.md) for the live scoring engine).
That subsystem targets the Auto Combo engine — automatically scoring providers and
models so combos can self-heal when upstreams fail. It uses its own runner,
its own categorizer, and its own scoring logic.
The Evals framework documented here is the **broader, general-purpose
testing surface**. Prefer it for arbitrary regression suites, A/B comparisons,
and per-release smoke tests. Use the Auto-Assessment subsystem when you need
real-time provider health to influence routing decisions.
## CI Integration
There is no dedicated `eval:ci` npm script today. Two paths if you want to
gate releases on eval results:
- **HTTP path**: stand up the server, hit `POST /api/evals` with a known
`suiteId` + `target`, and assert `runs[].summary.passRate >= N` in the
response.
- **In-process path**: import `runEvalSuiteAgainstTarget()` from
`@/lib/evals/runtime` from a script, run against a test DB, and check the
returned `PersistedEvalRun.summary`.
Tests covering the route and history live at
`tests/unit/evals-route.test.ts` and `tests/unit/evals-history.test.ts`.
## Extension Points
Common changes and where to make them:
- **New scoring strategy** — extend the `switch (evalCase.expected.strategy)`
block in `evaluateCase()` (`evalRunner.ts`) and widen `EvalCaseStrategy` in
`src/lib/db/evals.ts` plus `evalCaseBuilderSchema` in `schemas.ts`.
- **New built-in suite** — define a suite object and call `registerSuite()` at
the bottom of `evalRunner.ts`. It will be auto-discovered by `listSuites()`.
- **Run with concurrency** — change the sequential `for` loop in
`runEvalSuiteAgainstTarget()` to a bounded `Promise.all` (no concurrency
control exists today).
- **Stream/tool-call cases** — currently the runner forces `stream: false`.
Streaming or tool-aware evaluation would require changes in `runtime.ts`
(capture and aggregate SSE chunks before scoring).
## See Also
- [USER_GUIDE.md](../guides/USER_GUIDE.md) — overall product walkthrough
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — request pipeline reference
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — Auto Combo scoring engine (live runtime)
- Source: `src/lib/evals/`, `src/lib/db/evals.ts`, `src/app/api/evals/`
- UI: `src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx`
docs/frameworks/GAMIFICATION.md
+1082
View File
File diff suppressed because it is too large Load Diff
docs/frameworks/MCP-SERVER.md
+421
View File
@@ -0,0 +1,421 @@
---
title: "OmniRoute MCP Server Documentation"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute MCP Server Documentation
> Model Context Protocol server with 94 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations.
>
> Source of truth: `open-sse/mcp-server/schemas/tools.ts` (34 base) + `memoryTools.ts` (3) + `skillTools.ts` (4) + `agentSkillTools.ts` (3) + `poolTools.ts` (6) + `gamificationTools.ts` (8) + `pluginTools.ts` (8) + `notionTools.ts` (6) + `obsidianTools.ts` (22) = **94** (`TOTAL_MCP_TOOL_COUNT`). Tool registration and scope wiring lives in `open-sse/mcp-server/server.ts`.
![MCP tool inventory (94 tools by category)](../diagrams/exported/mcp-tools-94.svg)
> Source: [diagrams/mcp-tools-94.mmd](../diagrams/mcp-tools-94.mmd) (regenerate via `npm run docs:render-diagrams`).
## Installation
OmniRoute MCP is built-in. Start it with:
```bash
omniroute --mcp
```
Or via the open-sse transport:
```bash
# HTTP streamable transport (port 20130)
omniroute --dev # MCP auto-starts on /mcp endpoint
```
## Transports
The MCP server exposes three transports, all backed by the same `createMcpServer()` factory:
| Transport | Where | When to use |
| :---------------- | :------------------------------------------ | :--------------------------------------------------- |
| `stdio` | `open-sse/mcp-server/server.ts` | IDE integrations (Claude Desktop, Cursor, etc.) |
| `sse` | `POST/GET /api/mcp/sse` via `httpTransport` | Browser/agent clients that need an event stream |
| `streamable-http` | `POST/GET/DELETE /api/mcp/stream` | Multi-session HTTP clients (`mcp-session-id` header) |
The active HTTP transport (`sse` or `streamable-http`) is selected by the `mcpTransport` setting. Switching transports closes existing sessions on the other transport.
### Remote access (manage-scope bypass)
`/api/mcp/*` is in the LOCAL_ONLY tier (`src/server/authz/routeGuard.ts`) — by default only loopback hosts (`localhost`, `127.0.0.1`, `::1`) can reach it. Since v3.8.2, non-loopback clients may connect if they present an `Authorization: Bearer <api-key>` whose key carries the `manage` scope. This is the only way to reach the remote MCP server through a tunnel, reverse proxy, or public hostname.
```bash
# Grant manage scope: open the dashboard API Keys page and toggle
# "Management Access" on the key, or POST scopes:["manage"] when creating.
# Then connect from a remote MCP client:
curl -i \
-H "Host: your-public-host.example" \
-H "Authorization: Bearer sk-…" \
-H "Content-Type: application/json" \
-H "Accept: application/json, text/event-stream" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"my-client","version":"0"}}}' \
https://your-public-host.example/api/mcp/stream
```
A non-manage key (or no Bearer) returns `403 LOCAL_ONLY`. The sibling prefix `/api/cli-tools/runtime/*` is intentionally NOT bypassable — see [Route Guard Tiers — Manage-scope carve-out](../security/ROUTE_GUARD_TIERS.md#manage-scope-carve-out).
## IDE Configuration
See [MCP Client Configuration](../guides/SETUP_GUIDE.md#mcp-client-configuration) for Claude Desktop,
Cursor, Cline, and compatible MCP client setup.
---
## Essential Tools (8) — Phase 1
| Tool | Scopes | Description |
| :------------------------------ | :-------------------- | :------------------------------------------------------------ |
| `omniroute_get_health` | `read:health` | Uptime, memory, circuit breakers, rate limits, cache stats |
| `omniroute_list_combos` | `read:combos` | All configured combos with strategies (optional metrics) |
| `omniroute_get_combo_metrics` | `read:combos` | Performance metrics for a specific combo |
| `omniroute_switch_combo` | `write:combos` | Activate or deactivate a combo |
| `omniroute_check_quota` | `read:quota` | Quota used/total, percent remaining, reset time, token health |
| `omniroute_route_request` | `execute:completions` | Send a chat completion through OmniRoute routing |
| `omniroute_cost_report` | `read:usage` | Cost report by period (session/day/week/month) |
| `omniroute_list_models_catalog` | `read:models` | Full model catalog with capabilities, status, pricing |
## Phase 1 — Search
| Tool | Scopes | Description |
| :--------------------- | :--------------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| `omniroute_web_search` | `execute:search` | Web search through OmniRoute search gateway (Serper/Brave/Perplexity/Exa/Tavily/Google PSE/Linkup/SearchAPI/SearXNG) with failover |
## Advanced Tools (11) — Phase 2
| Tool | Scopes | Description |
| :--------------------------------- | :----------------------------------- | :---------------------------------------------------------------------------------------- |
| `omniroute_simulate_route` | `read:health`, `read:combos` | Dry-run routing simulation with fallback tree |
| `omniroute_set_budget_guard` | `write:budget` | Session budget with degrade/block/alert action |
| `omniroute_set_routing_strategy` | `write:combos` | Update combo strategy at runtime (priority/weighted/auto/etc.) |
| `omniroute_set_resilience_profile` | `write:resilience` | Apply `aggressive` / `balanced` / `conservative` resilience preset |
| `omniroute_test_combo` | `execute:completions`, `read:combos` | Live test of every provider in a combo using a real upstream call |
| `omniroute_get_provider_metrics` | `read:health` | Per-provider metrics with p50/p95/p99 latency and circuit breaker state |
| `omniroute_best_combo_for_task` | `read:combos`, `read:health` | Recommend combo by task type with budget/latency constraints |
| `omniroute_explain_route` | `read:health`, `read:usage` | Explain why a request was routed to a provider (scoring factors + fallbacks) |
| `omniroute_get_session_snapshot` | `read:usage` | Full session snapshot: cost, tokens, top models/providers, errors, budget guard |
| `omniroute_db_health_check` | `read:health`, `write:resilience` | Diagnose (and optionally auto-repair) database drift like broken combo refs / orphan rows |
| `omniroute_sync_pricing` | `pricing:write` | Sync pricing data from external sources (LiteLLM); supports `dryRun` |
## Cache Tools (2)
| Tool | Scopes | Description |
| :---------------------- | :------------ | :-------------------------------------------------- |
| `omniroute_cache_stats` | `read:cache` | Semantic cache, prompt-cache, and idempotency stats |
| `omniroute_cache_flush` | `write:cache` | Flush cache globally or by signature/model |
## Compression Tools (5)
| Tool | Scopes | Description |
| :---------------------------------- | :------------------ | :----------------------------------------------------------------------------------------------------------------------- |
| `omniroute_compression_status` | `read:compression` | Compression settings, analytics summary, and cache-aware stats (includes `analytics.mcpDescriptionCompression` metadata) |
| `omniroute_compression_configure` | `write:compression` | Configure compression mode, threshold, target ratio, system-prompt preservation, MCP description compression toggle |
| `omniroute_set_compression_engine` | `write:compression` | Pick the active engine (off/caveman/rtk/stacked) and Caveman/RTK intensity |
| `omniroute_list_compression_combos` | `read:compression` | List named compression combos and their engine pipelines |
| `omniroute_compression_combo_stats` | `read:compression` | Analytics grouped by compression combo and engine |
`omniroute_compression_status` reports MCP description compression separately under
`analytics.mcpDescriptionCompression`. Those values are metadata-size estimates for MCP listable
descriptions (`tools`, `prompts`, `resources`, and `resourceTemplates`); they are not provider usage
receipts and are marked with `source: "mcp_metadata_estimate"`.
### MCP Accessibility Tree Filter (v3.8.0)
Separate from the 5 compression tools above, OmniRoute includes a post-execution filter that
compresses the **tool results** of MCP browser/accessibility tools before they are returned to the
agent. This filter is not itself a tool — it runs transparently on any tool result that contains
verbose accessibility-tree or browser-snapshot text (≥2000 chars).
Key behaviors:
- Collapses ≥30 consecutive repeated sibling lines into head + tail summary
- Preserves `[ref=eXX]` anchors required by Playwright/computer-use
- Hard-truncates oversized text (>50,000 chars) with a navigation hint
- Expected savings: **60–80%** on browser snapshot payloads
Configuration: `compression.mcpAccessibility` in global settings (migration 056).
Implementation: `open-sse/services/compression/engines/mcpAccessibility/`.
Full docs: [Compression Engines — MCP Accessibility Tree Filter](../compression/COMPRESSION_ENGINES.md#mcp-accessibility-tree-filter).
See [Compression Engines](../compression/COMPRESSION_ENGINES.md) and [RTK Compression](../compression/RTK_COMPRESSION.md) for
the runtime compression model behind these tools.
## 1Proxy Tools (3)
| Tool | Scopes | Description |
| :-------------------------- | :------------- | :-------------------------------------------------------------------------------------- |
| `omniroute_oneproxy_fetch` | `read:proxies` | Fetch free proxies from the 1proxy marketplace (protocol/country/quality/limit filters) |
| `omniroute_oneproxy_rotate` | `read:proxies` | Get the next available proxy by strategy (`random` / `quality` / `sequential`) |
| `omniroute_oneproxy_stats` | `read:proxies` | Pool stats, sync status, distribution by protocol and country |
## Memory Tools (3)
Defined in `open-sse/mcp-server/tools/memoryTools.ts`. Auth/scope is enforced through the standard MCP scope pipeline.
| Tool | Scopes | Description |
| :------------------------ | :------------- | :---------------------------------------------------------------------------------- |
| `omniroute_memory_search` | `read:memory` | Search memories by query / type / API key with token-budget enforcement |
| `omniroute_memory_add` | `write:memory` | Add a new memory entry (`factual` / `episodic` / `procedural` / `semantic`) |
| `omniroute_memory_clear` | `write:memory` | Clear memories for an API key, optionally filtered by type or `olderThan` timestamp |
## Skill Tools (4)
Defined in `open-sse/mcp-server/tools/skillTools.ts`. Backed by `src/lib/skills/registry` + `src/lib/skills/executor`.
| Tool | Scopes | Description |
| :---------------------------- | :--------------- | :-------------------------------------------------------------------------------- |
| `omniroute_skills_list` | `read:skills` | List registered skills with optional filtering by API key, name, or enabled state |
| `omniroute_skills_enable` | `write:skills` | Enable or disable a specific skill by ID |
| `omniroute_skills_execute` | `execute:skills` | Execute a skill with provided input and return the execution record |
| `omniroute_skills_executions` | `read:skills` | List recent skill execution history |
## Notion Context Source (6)
Defined in `open-sse/mcp-server/tools/notionTools.ts`. Token stored in `key_value` table via `src/lib/db/notion.ts`. REST client in `src/lib/notion/api.ts`. Settings API in `src/app/api/settings/notion/route.ts`. Dashboard UI in `src/app/(dashboard)/dashboard/endpoint/components/NotionSourceCard.tsx`.
Configure your Notion integration token from the **Context Sources** tab in the Endpoint dashboard, or via the REST API:
```bash
# Set token
curl -X POST http://localhost:20128/api/settings/notion \
-H "Content-Type: application/json" \
-d '{"token": "ntn_..."}'
# Check status
curl http://localhost:20128/api/settings/notion
# Disconnect
curl -X DELETE http://localhost:20128/api/settings/notion
```
| Tool | Scopes | Description |
| :--------------------------- | :------------- | :------------------------------------------------------------- |
| `notion_search` | `read:notion` | Full-text search across all pages and databases |
| `notion_get_page` | `read:notion` | Get a page by ID with its properties |
| `notion_list_block_children` | `read:notion` | List the child blocks of a page or block |
| `notion_query_database` | `read:notion` | Query a database with filters, sorts, and pagination |
| `notion_get_database` | `read:notion` | Get database schema by ID |
| `notion_append_blocks` | `write:notion` | Append children blocks to a parent block (max 100 per request) |
## Agent Skill Catalog Tools (3)
Defined in `open-sse/mcp-server/tools/agentSkillTools.ts`. Backed by `src/lib/agentSkills/catalog`. These tools expose the 42-entry Agent Skills documentation catalog to MCP clients and external agents. Scope: `read:catalog`.
| Tool | Scopes | Description |
| :-------------------------------- | :------------- | :--------------------------------------------------------------------------------------------------------------- |
| `omniroute_agent_skills_list` | `read:catalog` | List all 42 agent skills with optional `category` (api\|cli) and `area` filters; returns metadata + coverage |
| `omniroute_agent_skills_get` | `read:catalog` | Get full metadata + SKILL.md content for a single skill by canonical `id` |
| `omniroute_agent_skills_coverage` | `read:catalog` | Coverage stats: how many of the 22 API and 20 CLI skills have SKILL.md files on the filesystem vs catalog totals |
See [AGENT-SKILLS.md](./AGENT-SKILLS.md) for the full catalog and how external agents consume it.
## Related Frameworks (v3.8.0)
The MCP tool inventory above (94 tools = 34 core + 3 memory + 4 skills + 3 agent-skills + 6 pool + 8 gamification + 8 plugins + 6 notion + 22 obsidian) is intentionally
scoped to runtime routing/cache/compression/memory/skills/proxy/context-source operations. Two adjacent
frameworks ship alongside the MCP server in v3.8.0 and are documented separately:
### Cloud Agents
Cloud Agents are out-of-process AI coding agents (codex-cloud, devin, jules) wired into
OmniRoute through the same connection model used for LLM providers. They are exposed via
their own REST surface (`/api/v1/agents/*`) and are **not** part of the MCP tool catalog
— calling a Cloud Agent does not consume an MCP scope.
- Implementation: `src/lib/cloudAgent/` (`registry.ts`, `agents/codex-cloud.ts`, `agents/devin.ts`, `agents/jules.ts`).
- Lifecycle: `createTask`, `getStatus`, `approvePlan`, `sendMessage`, `listSources`.
- Documentation: [docs/frameworks/CLOUD_AGENT.md](./CLOUD_AGENT.md).
### Guardrails
Guardrails are pre/post-execution filters (vision-bridge, pii-masker, prompt-injection)
applied inside the chat pipeline. They run before the MCP tool/route layer is reached
and emit structured violations to the audit pipeline; they are not invoked as MCP tools.
- Implementation: `src/lib/guardrails/`.
- Documentation: [docs/security/GUARDRAILS.md](../security/GUARDRAILS.md).
When debugging an MCP call that appears blocked, check both the MCP audit log
(`scope_denied:*` entries) and the guardrails audit trail — a request may be rejected by
a guardrail **before** it ever reaches the MCP scope enforcement layer.
---
## REST API Endpoints
| Endpoint | Method | Description | Auth |
| :--------------------- | :-------------------- | :-------------------------------------------------------------------------------------------------- | :------------------------- |
| `/api/mcp/status` | `GET` | Server status: heartbeat, HTTP transport state, audit activity summary | Management (session/admin) |
| `/api/mcp/tools` | `GET` | Tool catalog (name, description, scopes, phase, source endpoints) | Management |
| `/api/mcp/sse` | `GET` / `POST` | SSE transport endpoint (gated by `mcpEnabled` + `mcpTransport === "sse"`) | API key + scopes |
| `/api/mcp/stream` | `POST`/`GET`/`DELETE` | Streamable HTTP transport (uses `mcp-session-id` header; `DELETE` ends the session) | API key + scopes |
| `/api/mcp/audit` | `GET` | Audit log entries from `mcp_tool_audit` (filters: `limit`, `offset`, `tool`, `success`, `apiKeyId`) | Management |
| `/api/mcp/audit/stats` | `GET` | Aggregated audit stats (`totalCalls`, `successRate`, `avgDurationMs`, top tools) | Management |
Source files: `src/app/api/mcp/{status,tools,sse,stream,audit,audit/stats}/route.ts`.
Both SSE and Streamable HTTP transports are blocked until the MCP server is enabled in Settings (`mcpEnabled`) and the appropriate `mcpTransport` is selected. If the wrong transport is configured the route returns HTTP 400 with a hint to switch settings.
---
## Authentication & Scopes
MCP tools are authenticated through API key scopes. Scope enforcement is centralized in
`open-sse/mcp-server/scopeEnforcement.ts`. Each tool requires specific scopes:
| Scope | Tools |
| :-------------------- | :---------------------------------------------------------------------------------------------------------------- |
| `read:health` | `get_health`, `get_provider_metrics`, `simulate_route`, `explain_route`, `best_combo_for_task`, `db_health_check` |
| `read:combos` | `list_combos`, `get_combo_metrics`, `simulate_route`, `best_combo_for_task`, `test_combo` |
| `write:combos` | `switch_combo`, `set_routing_strategy` |
| `read:quota` | `check_quota` |
| `read:usage` | `cost_report`, `get_session_snapshot`, `explain_route` |
| `read:models` | `list_models_catalog` |
| `execute:completions` | `route_request`, `test_combo` |
| `execute:search` | `web_search` |
| `write:budget` | `set_budget_guard` |
| `write:resilience` | `set_resilience_profile`, `db_health_check` |
| `pricing:write` | `sync_pricing` |
| `read:cache` | `cache_stats` |
| `write:cache` | `cache_flush` |
| `read:compression` | `compression_status`, `list_compression_combos`, `compression_combo_stats` |
| `write:compression` | `compression_configure`, `set_compression_engine` |
| `read:proxies` | `oneproxy_fetch`, `oneproxy_rotate`, `oneproxy_stats` |
| `read:notion` | `notion_search`, `notion_list_databases`, `notion_get_database`, `notion_query_database`, `notion_read` |
| `write:notion` | `notion_append_blocks` |
| `read:memory` | `memory_search` |
| `write:memory` | `memory_add`, `memory_clear` |
| `read:skills` | `skills_list`, `skills_executions` |
| `write:skills` | `skills_enable` |
| `execute:skills` | `skills_execute` |
| `read:catalog` | `agent_skills_list`, `agent_skills_get`, `agent_skills_coverage` |
Wildcard scopes are supported: `read:*` grants all read-scopes, `*` grants full access.
---
## Environment Variables
| Variable | Default | Purpose |
| :-------------------------------------- | :--------------------------------- | :----------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_BASE_URL` | `http://localhost:20128` | Base URL the MCP server uses when calling OmniRoute internal APIs |
| `OMNIROUTE_API_KEY` | (empty) | API key forwarded as `Authorization: Bearer` to internal API calls |
| `OMNIROUTE_MCP_ENFORCE_SCOPES` | `false` (only `"true"` enables it) | When enabled, missing scopes deny tool calls and log `scope_denied:<reason>` in audit log |
| `OMNIROUTE_MCP_SCOPES` | (empty) | Comma-separated allowlist of scopes considered "available" by default (used when caller does not provide its own scopes) |
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | (unset = on) | When set to `0/false/off/no`, disables MCP description compression at registration time |
| `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION` | (unset = on) | Alternate alias for the same toggle as above |
| `MCP_TOOL_DENY` | (unset = no filter) | Comma-separated tool names to drop from `tools/list` (tool-cardinality reduction — see below) |
| `MCP_TOOL_ALLOW` | (unset = no filter) | Comma-separated tool names to keep exclusively (allow-list mode — see below) |
| `DATA_DIR` | `~/.omniroute` | Heartbeat file is written to `${DATA_DIR}/runtime/mcp-heartbeat.json` |
---
## Description Compression
MCP tool, prompt, and resource registries can compress descriptions at registration/list time to reduce the metadata footprint exposed to clients (and therefore the prompt context cost). The implementation lives in `open-sse/mcp-server/descriptionCompressor.ts` and is wired into the MCP server via `compressMcpRegistryMetadata` inside `createMcpServer()`.
- Compression runs over the description text using the Caveman ruleset (`getRulesForContext("all", "full")`) with preserved-block extraction (code spans, fenced blocks, etc.) so structural content is not altered.
- Toggle per-deployment via the `compression.mcpDescriptionCompressionEnabled` value in the `key_value` settings table (default: enabled) — exposed in the UI as **Analytics → MCP description compression**.
- Toggle process-wide via either `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS=false` or `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION=false`.
- Realtime stats are surfaced via `omniroute_compression_status` under `analytics.mcpDescriptionCompression` and tagged `source: "mcp_metadata_estimate"` to disambiguate from real provider usage receipts.
---
## Tool Cardinality Reduction (F4.3)
Description compression shrinks each tool's metadata; **tool-cardinality reduction** goes one step further by reducing _how many_ tools are announced at all. Advertising fewer tools in the `tools/list` manifest cuts the per-request token cost the client's model pays for the tool catalog ("layer 5" compression). The implementation is a pure, stateless filter in `open-sse/mcp-server/toolCardinality.ts` (`reduceToolManifest`), wired into the registration loop in `createMcpServer()` (`open-sse/mcp-server/server.ts`).
**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 94 tools are announced unchanged.
| Variable | Mode |
| :--------------- | :-------------------------------------------------------------------------------------- |
| `MCP_TOOL_DENY` | Blacklist — comma-separated tool names that are always dropped from `tools/list` |
| `MCP_TOOL_ALLOW` | Allow-list — comma-separated tool names; only these survive, everything else is dropped |
`deny` takes priority over `allow`. Names are comma-separated, trimmed, and empty entries are ignored. Examples:
```bash
# Drop two tools from the catalog
MCP_TOOL_DENY="omniroute_get_health,omniroute_list_combos" omniroute --mcp
# Announce only the routing + quota tools (allow-list mode)
MCP_TOOL_ALLOW="omniroute_route_request,omniroute_check_quota" omniroute --mcp
```
**How filtered tools are removed:** registration always succeeds; a tool the profile rejects is then `.disable()`d on the MCP SDK handle, so it never appears in `tools/list` but the wiring stays intact (clean enable/disable, no re-registration). The profile parser is `readMcpToolProfileFromEnv(process.env)`, which returns `null` (no filtering) when both vars are empty.
The richer `ToolProfile` shape behind `reduceToolManifest` also supports scope-intersection filtering (`allowScopes`, with `read:*`-style wildcard matching) and a deterministic `maxTools` cap, but those two knobs need the full manifest at registration time and are **not** exposed through the environment variables today (a `tools/list`-level hook is a tracked follow-up). `estimateManifestTokens()` is available to compare the manifest token cost before and after reduction.
---
## Runtime Heartbeat
The stdio transport persists liveness to `${DATA_DIR}/runtime/mcp-heartbeat.json` every 5 seconds. The dashboard (`/api/mcp/status`) reads this file plus PID liveness to derive `online`. HTTP transports report state from in-process `getMcpHttpStatus()` instead (no file write).
The heartbeat snapshot contains:
```json
{
"pid": 12345,
"startedAt": "2026-05-13T12:34:56.000Z",
"lastHeartbeatAt": "2026-05-13T12:35:01.000Z",
"version": "1.8.1",
"transport": "stdio",
"scopesEnforced": false,
"allowedScopes": [],
"toolCount": 43
}
```
---
## Audit Logging
Every tool call is logged to the SQLite `mcp_tool_audit` table by `open-sse/mcp-server/audit.ts`:
- Tool name, arguments (hashed/truncated as per per-tool `auditLevel`), result
- Duration in ms, success/failure flag, error message (when applicable)
- API key hash, timestamp
- Scope denials are logged as `scope_denied:<reason>` with the missing scope list
Use the dashboard or the `/api/mcp/audit` and `/api/mcp/audit/stats` REST endpoints to inspect recent calls.
---
## Files
| File | Purpose |
| :----------------------------------------------------------------------- | :--------------------------------------------------------------- |
| `open-sse/mcp-server/server.ts` | MCP server factory, stdio entry point, scoped tool registrations |
| `open-sse/mcp-server/httpTransport.ts` | SSE + Streamable HTTP transport (session management) |
| `open-sse/mcp-server/scopeEnforcement.ts` | Tool scope evaluation and caller resolution |
| `open-sse/mcp-server/audit.ts` | Tool call audit logging (`mcp_tool_audit`) |
| `open-sse/mcp-server/runtimeHeartbeat.ts` | stdio heartbeat writer (`mcp-heartbeat.json`) |
| `open-sse/mcp-server/descriptionCompressor.ts` | Description compression for tool / prompt / resource registries |
| `open-sse/mcp-server/schemas/tools.ts` | Zod schemas + tool registry (`MCP_TOOLS`, 34 entries) |
| `open-sse/mcp-server/tools/advancedTools.ts` | Phase 2 + cache + 1proxy tool handlers |
| `open-sse/mcp-server/tools/compressionTools.ts` | Compression tool handlers |
| `open-sse/mcp-server/tools/memoryTools.ts` | Memory tool definitions (3 tools) |
| `open-sse/mcp-server/tools/skillTools.ts` | Skill tool definitions (4 tools) |
| `open-sse/mcp-server/tools/notionTools.ts` | Notion context source tool definitions (6 tools) |
| `open-sse/mcp-server/tools/gamificationTools.ts` | Gamification tool definitions (8 tools) |
| `open-sse/mcp-server/tools/pluginTools.ts` | Plugin registration and management tools (8 tools) |
| `src/app/api/mcp/status/route.ts` | `/api/mcp/status` endpoint |
| `src/app/api/mcp/tools/route.ts` | `/api/mcp/tools` endpoint |
| `src/app/api/mcp/sse/route.ts` | `/api/mcp/sse` SSE transport route |
| `src/app/api/mcp/stream/route.ts` | `/api/mcp/stream` Streamable HTTP transport route |
| `src/app/api/mcp/audit/route.ts` | `/api/mcp/audit` audit log query |
| `src/app/api/mcp/audit/stats/route.ts` | `/api/mcp/audit/stats` aggregated audit metrics |
| `src/lib/notion/api.ts` | Notion REST API client (retry, timeout, error classification) |
| `src/lib/db/notion.ts` | Notion token persistence (`key_value` table) |
| `src/app/api/settings/notion/route.ts` | Notion settings API (GET/POST/DELETE) |
| `src/app/(dashboard)/dashboard/endpoint/components/NotionSourceCard.tsx` | Notion token management UI |
| `tests/unit/notion-api.test.ts` | Notion API client tests (7) |
| `tests/unit/notion-tools.test.ts` | Notion tools scope enforcement tests (10) |
| `tests/unit/db/notion.test.mjs` | Notion DB module tests (3) |
docs/frameworks/MEMORY.md
+880
View File
@@ -0,0 +1,880 @@
---
title: "Memory System"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Memory System
> **Source of truth:** `src/lib/memory/` and `src/app/api/memory/`
> **Last updated:** 2026-06-28 — v3.8.40 (off-by-default + int8 quantization catch-up)
OmniRoute provides persistent conversational memory keyed by API key (and
optionally session id). Memories are extracted automatically from LLM responses
via lightweight regex pattern matching and injected back into subsequent
requests as a leading system message (or first user message for providers that
reject the system role).
> **Memory is OFF by default (v3.8.30+).** `DEFAULT_MEMORY_SETTINGS.enabled` is
> now `false` (`src/lib/memory/settings.ts`). Enabling memory injects up to
> `maxTokens` (~2k) of retrieved context into **every** chat request, which is
> billed — a surprising cost for new installs and for clients that manage their
> own context. Opt in explicitly under **Settings → Memory** (the
> `MemorySkillsTab` shows a token-cost warning callout when memory is enabled).
> A client can opt a single request out with the `x-omniroute-no-memory`
> request header (`true`/`1`/`yes`) — see the request-header table in
> [API_REFERENCE.md](../reference/API_REFERENCE.md). A no-memory request sets
> `memoryOwnerId = null`, which disables **both** memory and skill injection for
> that request (`open-sse/handlers/chatCore/headers.ts::isNoMemoryRequested`).
Memory is **scoped per API key**, not per user — every request authenticated
with the same API key shares the same memory pool, with optional further
scoping by `sessionId`.
## Architecture
```
Client → /v1/chat/completions (apiKeyInfo resolved upstream)
→ handleChatCore() [open-sse/handlers/chatCore.ts]
→ resolveMemoryOwnerId(apiKeyInfo) # extracts id
→ getMemorySettings() # cached settings
→ shouldInjectMemory(body, {enabled}) # gate
→ retrieveMemories(apiKeyId, config) # SQL + FTS5 + optional vector
→ injectMemory(body, memories, provider) # system or user message
→ upstream provider call
→ on response: extractFacts(text, apiKeyId, sessionId) # non-blocking
→ setImmediate → createMemory(fact) per match
→ embed(content) + upsertVector(id, vec)
```
The injection and extraction call-sites are wired in
`open-sse/handlers/chatCore.ts` (look for `retrieveMemories`, `injectMemory`,
and `extractFacts`).
## Engine architecture (3-tier resolution)
The Memory Engine resolves the retrieval path at runtime based on available
infrastructure and settings. Three tiers exist, applied in priority order:
```
┌─────────────────────────────────────────────────────────────┐
│ TIER 0 — Keyword (FTS5) │
│ Always available. SQLite FTS5 full-text search over │
│ content + key. Used when strategy = "exact" or as fallback. │
└──────────────────────────────────┬──────────────────────────┘
│ strategy = semantic|hybrid?
▼
┌─────────────────────────────────────────────────────────────┐
│ TIER 1 — Embedded Vector (sqlite-vec) │
│ sqlite-vec v0.1.9 loaded via db.loadExtension(). │
│ KNN brute-force over Float32 vectors. Active when: │
│ • sqlite-vec loadExtension succeeds │
│ • An embedding source is available (remote | static | │
│ transformers) that can produce a Float32Array │
│ • vec_memories table exists (created on first ready()) │
└──────────────────────────────────┬──────────────────────────┘
│ qdrant.enabled?
▼
┌─────────────────────────────────────────────────────────────┐
│ TIER 2 — Qdrant (opt-in external vector database) │
│ When enabled, replaces sqlite-vec for semantic/hybrid. │
│ Requires running Qdrant instance + configured host/port. │
└─────────────────────────────────────────────────────────────┘
```
Degradation is automatic and transparent:
- If sqlite-vec fails to load, tier 1 is unavailable → falls back to tier 0.
- If embedding source returns an error, tier 1 falls back to tier 0.
- If Qdrant is unhealthy, tier 2 falls back to tier 1 (or tier 0 if tier 1
is also unavailable).
## Embedding sources
The embedding layer (`src/lib/memory/embedding/`) resolves which source to use
based on `MemorySettingsExtended.embeddingSource`:
| Source | Description | Key required | Cold start |
| -------------- | ---------------------------------------------------------------------------- | ------------ | ---------------- |
| `remote` | Uses a configured provider's embedding API (OpenAI, Cohere, etc.) | Yes | None |
| `static` | Local lookup-table embedding via `potion-base-8M` (WordPiece + mean pooling) | No | ~200ms |
| `transformers` | Local ONNX inference via `@huggingface/transformers` v4, `all-MiniLM-L6-v2` | No | ~3s + ~400MB RAM |
| `auto` | Runtime resolution: remote (if key exists) → static → transformers → null | Depends | Depends |
**Resolution order for `auto`:**
1. Find first provider in `listEmbeddingProviders()` with `hasKey === true` → `remote`.
2. If `settings.staticEnabled === true` → `static`.
3. If `settings.transformersEnabled === true` → `transformers`.
4. Otherwise → `null` (degrades to FTS5 keyword search).
The embedding cache (`src/lib/memory/embedding/cache.ts`) uses an in-memory
LRU map keyed by `${source}:${model}:${dim}:${sha256(text)}`, capped at
`MEMORY_EMBEDDING_CACHE_MAX` entries (default 1000) with a TTL of
`MEMORY_EMBEDDING_CACHE_TTL_MS` (default 5 min). Shared across all callers
per process lifecycle.
## Hybrid RRF (k=60)
When `strategy = "hybrid"` and the vector store is available, retrieval uses
Reciprocal Rank Fusion to merge FTS5 and vector results:
```
RRF(d) = Σ 1 / (k + rank_i(d)) where k = 60 (configurable via MEMORY_RRF_K)
i
```
Concretely:
1. Run FTS5 search → ranked list `R_fts` (position 1..N).
2. Run KNN vector search → ranked list `R_vec` (position 1..M).
3. For each unique `memoryId`:
`rrf_score = 1/(60 + fts_rank)` + `1/(60 + vec_rank)` (0 if not in list).
4. Sort by `rrf_score` DESC, apply token budget walk.
RRF is well-known to be effective without needing score normalization across
heterogeneous retrieval systems. The default `k=60` is from the original
Cormack et al. paper and works well for small corpora (<10k memories).
## Backfill (lazy + reindex)
When the embedding model changes (detected via `embedding_signature`), the
vector store is rebuilt and all existing memories are marked
`needs_reindex = 1` in the `memories` table.
**Lazy backfill**: On the next retrieval, any memory missing a vector entry is
embedded and inserted into `vec_memories` before the search runs. This
amortizes the backfill cost across real requests without blocking startup.
**Explicit reindex**: The Engine tab in `/dashboard/memory` provides a
"Reindex Now" button that calls `POST /api/memory/reindex`. The handler calls
`runReindexBatch()` from `src/lib/memory/reindex.ts`, which processes up to
`limit` pending entries per request. Progress can be polled via
`GET /api/memory/engine-status` (`vectorStore.needsReindex`).
The `memory_vec_meta` table (migration `073_memory_vec.sql`) stores:
- `active_dim` — current vector dimension (null = not yet calibrated).
- `embedding_signature` — `${source}:${model}:${dim}` used to detect changes.
- `last_reset_at` — timestamp of last full reset.
- `vec_loaded` — 0/1 flag whether sqlite-vec loaded successfully.
## Settings extension
Seven new fields were added to `MemorySettingsExtended` (plan 21, D9) in
`src/shared/schemas/memory.ts`, persisted via `src/lib/db/settings.ts`:
| Field | Type | Default | Description |
| ------------------------ | -------------------------------------------------- | -------- | ------------------------------------------------ |
| `embeddingSource` | `"remote" \| "static" \| "transformers" \| "auto"` | `"auto"` | Which embedding source to use |
| `embeddingProviderModel` | `string \| null` | `null` | Provider/model in `provider/model` format |
| `transformersEnabled` | `boolean` | `false` | Opt-in for Transformers.js (MiniLM, ~400MB) |
| `staticEnabled` | `boolean` | `false` | Opt-in for static potion-base-8M local model |
| `rerankEnabled` | `boolean` | `false` | Enable reranking step (adds +200-500ms/req) |
| `rerankProviderModel` | `string \| null` | `null` | Rerank provider/model in `provider/model` format |
| `vectorStore` | `"sqlite-vec" \| "qdrant" \| "auto"` | `"auto"` | Which vector backend to use |
These are exposed via `GET /PUT /api/settings/memory` (schema `MemorySettingsExtendedSchema`).
> **TODO (D20):** Scope `global` (sharing memories across all API keys) is not
> implemented in this release. It requires schema changes and a global retrieval
> path. Track separately.
## Storage Layers
### Primary: SQLite (`memories` table)
Created by migration `015_create_memories.sql`:
| Column | Type | Notes |
| --------------------------- | ------------------ | -------------------------------------------------------------------- |
| `id` | `TEXT PRIMARY KEY` | UUID generated via `crypto.randomUUID()` |
| `api_key_id` | `TEXT NOT NULL` | Owning API key |
| `session_id` | `TEXT` | Optional per-conversation scope |
| `type` | `TEXT NOT NULL` | One of `factual`, `episodic`, `procedural`, `semantic` |
| `key` | `TEXT` | Stable upsert key, e.g. `preference:i_prefer_python` |
| `content` | `TEXT NOT NULL` | The actual fact text |
| `metadata` | `TEXT` | JSON blob (category, extractedAt, source, ...) |
| `created_at` / `updated_at` | `TEXT` | ISO 8601 strings |
| `expires_at` | `TEXT` | Optional expiry; `NULL` means permanent |
| `memory_id` | `INTEGER UNIQUE` | Added by `023_fix_memory_fts_uuid.sql` to bridge UUIDs ↔ FTS5 rowids |
Indexes: `api_key_id`, `session_id`, `type`, `expires_at`, plus the unique
`memory_id` index.
**Upsert semantics**: `createMemory()` looks for an existing row with the same
`(api_key_id, key)` and updates it in place when found (merging `metadata` via
shallow spread). This keeps the table from growing unbounded for repeated
preference statements.
### Full-text Search (`memory_fts` virtual table)
`022_add_memory_fts5.sql` creates an FTS5 virtual table over `content` and
`key`. `023_fix_memory_fts_uuid.sql` fixes a real-world bug where the UUID
primary key did not join to FTS5's integer rowid — the migration adds the
`memory_id` column, recreates the FTS table, and wires triggers
(`memory_fts_ai`, `memory_fts_ad`, `memory_fts_au`) that keep FTS in sync on
INSERT, DELETE, and UPDATE.
Used by `retrieval.ts` for the `semantic` and `hybrid` strategies (see below).
The retrieval code guards with `hasTable("memory_fts")` and falls back to
chronological order if the FTS table is missing or the FTS query throws.
### Optional: Qdrant (vector store tier 2)
`src/lib/memory/qdrant.ts` implements an optional Qdrant integration as tier 2
vector store. Retrieval only routes to Qdrant when the engine selector
`memoryVectorStore === "qdrant"` — the default `"auto"` (and `"sqlite-vec"`)
**never** select Qdrant. The Engine-tab toggle sets **both** `qdrantEnabled` and
`memoryVectorStore` together: enabling makes Qdrant the primary store, disabling
resets to `"auto"` (#5597 — before that fix, enabling was inert because nothing
wrote the engine selector). If Qdrant is unreachable or returns nothing, retrieval
falls back to sqlite-vec → FTS5.
- `upsertSemanticMemoryPoint()` — embed `key + content` with the configured
embedding model, ensure the collection exists (creates cosine-distance
vectors on first use), and upsert a point with payload `{memoryId,
apiKeyId, sessionId, key, content, metadata, createdAtUnix, expiresAtUnix}`.
- `searchSemanticMemory(query, topK, scope)` — embed the query, search the
collection filtered by `kind = "omniroute_memory"` and optionally by
`apiKeyId` / `sessionId`. Caps `topK` to `[1, 20]`.
- `deleteSemanticMemoryPoint(id)` — single point delete. Called by
`deleteMemory()` after the SQLite row is removed (D15).
- `cleanupSemanticMemoryPoints({retentionDays})` — bulk delete points whose
`expiresAtUnix` is in the past or whose `createdAtUnix` is older than the
retention cutoff. Counts first so the dashboard can show actual numbers.
- `checkQdrantHealth()` — `GET /readyz` health probe with latency.
The settings UI exposes Qdrant config, health check, semantic search test,
and cleanup in the **Engine tab** of `/dashboard/memory`. The corresponding
routes under `src/app/api/settings/qdrant/` are all wired as of v3.8.6:
| Route | Method | Description |
| --------------------------------------- | ------------- | ------------------------------- |
| `/api/settings/qdrant` | `GET` / `PUT` | Read / update Qdrant settings |
| `/api/settings/qdrant/health` | `GET` | Liveness probe + latency |
| `/api/settings/qdrant/search` | `POST` | Semantic search test |
| `/api/settings/qdrant/cleanup` | `POST` | Remove expired / old points |
| `/api/settings/qdrant/embedding-models` | `GET` | List available embedding models |
**Behavior notes (what to expect):**
- **Engine selection** — enabling Qdrant in the Engine tab makes it the primary
store (sets `memoryVectorStore="qdrant"`); disabling resets to `"auto"` (#5597).
- **No back-fill** — only memories created/updated **after** Qdrant is enabled are
written to it (fire-and-forget dual-write). Pre-existing SQLite memories are **not**
migrated; "Reindex Now" rebuilds the sqlite-vec index only, not Qdrant.
- **Vector dimension is auto-detected** from the actual embedding on first use — there
is no dimension field to fill in. Changing the embedding model after a collection
exists is **not** auto-handled: the existing collection is left untouched, dimension-
mismatched writes/searches fail and fall back to sqlite-vec. Recreate the collection
(new name, or delete it in Qdrant) to switch embedders.
- **Distance metric** — always **Cosine** (hardcoded on collection creation; not
configurable).
- **Auth** — API key only (sent as the `api-key` header; optional for unauthenticated
local Docker). JWT/RBAC are not used.
- **Config fields** — the UI exposes `host`, `port`, `collection`, `embeddingModel`,
`apiKey`. `vectorSize` / `hnswEfConstruct` are env/DB only and `vectorSize` is not
used for collection creation (dimension comes from the embedding).
### Vector quantization (int8 — opt-in, both backends)
Both vector backends support **opt-in int8 quantization** to cut the memory
footprint of stored vectors (~4× smaller than Float32) at a small recall cost.
Default is **off** on both — vectors stay full-precision unless explicitly
enabled.
| Backend | Setting | Type | Default | Where read |
| ---------- | ------------------------------- | ------------------------------ | -------- | ----------------------------------------------------------- |
| Qdrant | `qdrantQuantization` (DB key) | `"none" \| "int8" \| "binary"` | `"none"` | `src/lib/memory/qdrant.ts::normalizeQdrantConfig()` |
| sqlite-vec | `MEMORY_VEC_QUANTIZATION` (env) | `"none" \| "int8"` | `"none"` | `src/lib/memory/vectorStore.ts::requestedVecQuantization()` |
- **Qdrant** is configured per-instance via the `qdrantQuantization` setting
key (exposed as the `quantization` field on `PUT /api/settings/qdrant`). When
`"int8"`, `buildQuantizationConfig()` requests scalar quantization
(`always_ram`, quantile `0.99`) and searches enable `rescore: true` so the
full-precision vectors refine the int8 candidate set.
- **sqlite-vec** quantization is **environment-only** (not a DB setting): set
`MEMORY_VEC_QUANTIZATION=int8` to store the local vectors as an `int8[dim]`
column via `vec_quantize_int8(?, 'unit')`. The chosen mode is folded into the
`embedding_signature` (an `:int8` suffix), so switching modes triggers a full
reindex of the `vec_memories` table — the same lazy-backfill path used when
the embedding model changes.
## Memory Types
`MemoryType` (`src/lib/memory/types.ts`):
| Type | Used for |
| ------------ | ------------------------------------------------------------ |
| `factual` | Preferences, stable user facts, behavioral patterns |
| `episodic` | Decisions tied to a specific moment ("I chose Postgres") |
| `procedural` | Workflow / how-to memory (reserved; no auto-extractor today) |
| `semantic` | Reserved for vector-store entries |
`MemoryConfig` retrieval strategy is one of `exact`, `semantic`, or `hybrid`,
and scope is one of `session`, `apiKey`, or `global`. The default scope from
`getMemorySettings()` is `apiKey`.
## Fact Extraction (`extraction.ts`)
Extraction is **regex-based**, not LLM-based — it runs in-process with
`setImmediate()` so it never blocks the response stream:
- **Preference patterns** → `MemoryType.FACTUAL`
(e.g. `I prefer …`, `I really like …`, `my favorite is …`, `I hate …`)
- **Decision patterns** → `MemoryType.EPISODIC`
(e.g. `I'll use …`, `I chose …`, `I went with …`, `I'm going to adopt …`)
- **Pattern patterns** → `MemoryType.FACTUAL`
(e.g. `I usually …`, `I always …`, `I tend to …`)
Each match is sanitised (`trim`, whitespace-collapse, capped at 500 chars),
deduplicated within the batch via a stable `factKey(category, content)`, and
stored via `createMemory()` with metadata
`{category, extractedAt, source: "llm_response"}`. Input text is capped at
64 KiB (`MAX_EXTRACTION_TEXT_LENGTH`) — when longer, the **tail** of the text
is used so the most recent assistant content always participates.
`extractFactsFromText(text)` is exported for tests and returns the structured
facts without storing them.
## Retrieval (`retrieval.ts`)
`retrieveMemories(apiKeyId, config)` is the main entry point. It:
1. Normalises and validates the config through `MemoryConfigSchema`.
2. Returns `[]` immediately when `enabled` is false or `maxTokens <= 0`.
3. Clamps `maxTokens` to `[1, 8000]`.
4. Detects whether the modern `memories` table exists (vs the legacy `memory`
table) so older databases keep working.
5. Builds the base query with expiry guard
(`expires_at IS NULL OR datetime(expires_at) > datetime('now')`), optional
session scope, and optional `retentionDays` cutoff.
6. Branches on strategy:
- **`exact`** (default): chronological `ORDER BY created_at DESC LIMIT 100`.
- **`semantic`**: if `config.query` and `memory_fts` exists, JOIN
`memory_fts MATCH ?` and order by FTS rank; fall back to chronological
when FTS returns 0 rows.
- **`hybrid`**: union of FTS results (higher relevance) and the
chronological set, deduplicated by id.
7. Computes a keyword relevance score (`getRelevanceScore`) over
`content`, `key`, and `metadata` JSON when a query is provided. Rows with
zero score are filtered out.
8. Sorts by score desc, then `createdAt` desc.
9. Walks the ranked list and accepts entries while a running
`estimateTokens(content)` (≈ `length / 4`) stays under the budget. Always
returns at least one entry when any matched.
`estimateTokens` is exported and used by retrieval, summarisation, and the MCP
`omniroute_memory_search` tool.
## Injection (`injection.ts`)
`injectMemory(request, memories, provider)`:
1. Joins all memory contents into a single `Memory context: …` string.
2. Picks a strategy by provider name:
- **System message** (default for OpenAI, Anthropic, Gemini, …) — prepends
a `{role: "system", content: memoryText}` ahead of any existing system
messages so user system prompts still take precedence.
- **User message** (fallback) — for providers in
`PROVIDERS_WITHOUT_SYSTEM_MESSAGE`: `o1`, `o1-mini`, `o1-preview`,
`glm`, `glmt`, `glm-cn`, `zai`, `qianfan`. These reject the system role
and would 400 otherwise (cf. issue #1701 for GLM/Zhipu).
3. Logs the count, strategy, and model under `memory.injection.injected`.
`providerSupportsSystemMessage(provider)` is exported for callers that need to
make routing decisions of their own. Unknown providers default to `true`
(system role allowed) for safety.
## Settings (`settings.ts`)
Memory configuration is **stored in the DB settings table**, not in env vars.
`getMemorySettings()` reads from `getSettings()` and caches the result
in-process; `invalidateMemorySettingsCache()` is called by the settings PUT
route after writes.
### Legacy fields (all versions)
| DB key | Type | Default | UI control |
| --------------------- | ------- | -------------------------------------------------- | ----------------------------------------------- |
| `memoryEnabled` | boolean | `false` (off by default since v3.8.30) | Memory on/off |
| `memoryMaxTokens` | integer | `2000` (range `0–16000`) | Token budget for injection |
| `memoryRetentionDays` | integer | `30` (range `1–365`) | Retention window |
| `memoryStrategy` | enum | `"hybrid"` (one of `recent`, `semantic`, `hybrid`) | Retrieval strategy |
| `skillsEnabled` | boolean | `false` | Toggles per-key skill injection (see SKILLS.md) |
Note: the UI strategy `"recent"` maps to the internal `"exact"` retrieval
strategy via `toMemoryRetrievalConfig()` (chronological order).
### New fields (v3.8.6, plan 21 D9)
See also the "Settings extension" section above for field descriptions.
| DB key | API field | Default |
| --------------------------- | ------------------------ | -------- |
| `memoryEmbeddingSource` | `embeddingSource` | `"auto"` |
| `memoryEmbeddingModel` | `embeddingProviderModel` | `null` |
| `memoryTransformersEnabled` | `transformersEnabled` | `false` |
| `memoryStaticEnabled` | `staticEnabled` | `false` |
| `memoryRerankEnabled` | `rerankEnabled` | `false` |
| `memoryRerankModel` | `rerankProviderModel` | `null` |
| `memoryVectorStore` | `vectorStore` | `"auto"` |
Qdrant-related DB keys (`qdrantEnabled`, `qdrantHost`, `qdrantPort`,
`qdrantApiKey`, `qdrantCollection` default `"omniroute_memory"`,
`qdrantEmbeddingModel` default `"openai/text-embedding-3-small"`) are read by
`normalizeQdrantConfig()` in `qdrant.ts`.
### Environment variables (v3.8.6)
Six optional env vars tune the engine's runtime behaviour (documented in `.env.example`):
| Variable | Default | Description |
| ------------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `MEMORY_EMBEDDING_CACHE_TTL_MS` | `300000` | Embedding cache TTL (5 min) |
| `MEMORY_EMBEDDING_CACHE_MAX` | `1000` | Max entries in embedding LRU cache |
| `MEMORY_TRANSFORMERS_MODEL` | `Xenova/all-MiniLM-L6-v2` | HF repo for Transformers.js model |
| `MEMORY_STATIC_MODEL` | `minishlab/potion-base-8M` | HF repo for static potion model |
| `MEMORY_STATIC_CACHE_DIR` | `<DATA_DIR>/embeddings` | Where to store downloaded models |
| `MEMORY_VEC_TOP_K` | `20` | Default top-K for vector search |
| `MEMORY_RRF_K` | `60` | RRF k constant for hybrid search |
| `MEMORY_VEC_QUANTIZATION` | `none` | Set to `int8` to store local sqlite-vec vectors quantized (~4× smaller; opt-in). Mode change forces a reindex. |
## Summarisation (`summarization.ts`)
`summarizeMemories(apiKeyId, sessionId?, maxTokens = 4000)` compacts older
content when the running token total over a key's memories exceeds the
budget. It iterates rows DESC by `created_at`, keeps rows that fit, and for
the rest replaces `content` in place with the first three sentences of the
original. `tokensSaved` is the difference in `estimateTokens` between old and
new content.
This routine is **available but not called automatically** in the current
chat pipeline — call it from a cron, an admin action, or
`MemoryConfig.autoSummarize` glue if you need ongoing compaction. The data
loss is one-way: original text is overwritten.
## REST API
All endpoints require management auth (`requireManagementAuth`).
### Core memory endpoints (existing + updated)
| Method | Path | Description |
| -------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET` | `/api/memory` | Paginated list with filters: `apiKeyId`, `type`, `sessionId`, `q`, `limit`, `page`, `offset`. Response includes `stats.total`, `stats.tokensUsed`, `stats.hitRate`, `cacheStats` |
| `POST` | `/api/memory` | Create entry (Zod-validated: `content`, `key`, optional `type`, `sessionId`, `apiKeyId`, `metadata`, `expiresAt`). Calls `createMemory()` which upserts on `(apiKeyId, key)` |
| `GET` | `/api/memory/[id]` | Fetch a single entry by UUID |
| `PUT` | `/api/memory/[id]` | Update entry fields (`type`, `key`, `content`, `metadata`). Body: `MemoryUpdatePutSchema`. Also syncs vector if embedding source available. |
| `DELETE` | `/api/memory/[id]` | Delete an entry; also deletes from `vec_memories` (D15) and Qdrant best-effort. Returns 404 when missing. |
| `GET` | `/api/memory/health` | Runs `verifyExtractionPipeline("health-check")` — round-trip create→list→delete. Returns `{working, latencyMs, error?}` |
### New memory engine endpoints (plan 21)
| Method | Path | Description |
| ------ | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `POST` | `/api/memory/retrieve-preview` | Dry-run of `retrieveMemories` — returns ranked results with score, tier, tokens. Body: `RetrievePreviewSchema`. Does NOT inject or modify memories. |
| `GET` | `/api/memory/embedding-providers` | Lists providers with embedding models, indicating which have a configured API key. |
| `GET` | `/api/memory/engine-status` | Returns full engine status: keyword tier, embedding resolution, vector store stats, Qdrant health, rerank config. Shape: `MemoryEngineStatusSchema`. |
| `POST` | `/api/memory/summarize` | Manually trigger memory compaction. Body: `MemorySummarizeSchema` (`olderThanDays`, `apiKeyId?`, `dryRun`). Returns `{candidates, tokensSaved}`. |
| `POST` | `/api/memory/reindex` | Trigger vector reindex for memories with `needs_reindex=1`. Body: `MemoryReindexSchema` (`force`). Returns `{started, pending}`. |
### Settings endpoints
| Method | Path | Description |
| ------ | --------------------------------------- | ------------------------------------------------------------------------------------------------ |
| `GET` | `/api/settings/memory` | Current normalised `MemorySettingsExtended` (7 new fields + legacy) |
| `PUT` | `/api/settings/memory` | Update any field from `MemorySettingsExtendedSchema` (12 total fields) |
| `GET` | `/api/settings/qdrant` | Current Qdrant settings (`QdrantSettingsSchema`) |
| `PUT` | `/api/settings/qdrant` | Update Qdrant settings. Body: `QdrantSettingsUpdateSchema`. `apiKey` = empty string removes key. |
| `GET` | `/api/settings/qdrant/health` | Liveness probe against configured Qdrant instance. Returns `QdrantHealthResultSchema`. |
| `POST` | `/api/settings/qdrant/search` | Semantic search test against Qdrant. Body: `QdrantSearchSchema` (`query`, `topK`). |
| `POST` | `/api/settings/qdrant/cleanup` | Remove Qdrant points for expired / old memories. |
| `GET` | `/api/settings/qdrant/embedding-models` | List embedding models available for Qdrant. |
The `/api/memory` list query supports either `page`-based pagination
(`parsePaginationParams`) **or** raw `offset` — when `offset` is present it
takes precedence and a derived `page` is computed for the response shape.
## MCP Tools (`open-sse/mcp-server/tools/memoryTools.ts`)
When the MCP server is enabled, three memory tools are registered:
- `omniroute_memory_search` — `{apiKeyId, query?, type?, maxTokens?, limit?}`
→ wraps `retrieveMemories()`. As of v3.8.6 (D16), the `strategy` is read
from `getMemorySettings()` instead of being hardcoded to `"exact"`. If
`query` is provided and `strategy` is `semantic` or `hybrid`, the vector
store is used when available.
- `omniroute_memory_add` — `{apiKeyId, sessionId?, type, key, content,
metadata?}` → wraps `createMemory()`. Accepts only the 4 canonical types:
`factual`, `episodic`, `procedural`, `semantic` (D17).
- `omniroute_memory_clear` — `{apiKeyId, type?, olderThan?}` → lists matching
entries, optionally filters by created-before timestamp, then deletes each
via `deleteMemory()` (which also removes vectors from sqlite-vec + Qdrant).
See [MCP-SERVER.md](./MCP-SERVER.md) for transport and scope details.
## Dashboard (Memory Studio)
`src/app/(dashboard)/dashboard/memory/page.tsx` is now a **3-tab Studio**:
### Tab: Memories
- Concept card (collapsible "How it works" explainer).
- Real-time list, search, and pagination (debounced 300 ms).
- Type filter (`factual` / `episodic` / `procedural` / `semantic` / all).
- Add-memory modal (key, content, type).
- Inline edit (pencil button → `PUT /api/memory/[id]`).
- Delete per row (with confirmation dialog).
- JSON export of the current page; JSON import via file picker.
- Stat cards: `totalEntries`, `tokensUsed`, `hitRate`.
- "Compact old" button → `POST /api/memory/summarize` (dry-run first shows
candidate count, then confirms).
- A green/red health dot driven by `GET /api/memory/health`.
### Tab: Playground
- Query input + strategy selector (Exact / Semantic / Hybrid) + token budget.
- "Simulate" → `POST /api/memory/retrieve-preview` — shows ranked results with
`score`, `tier`, `tokens`, `vecScore`, `ftsScore`.
- Resolution panel showing which embedding source / vector store was used and
whether a fallback occurred.
### Tab: Engine
- Engine status panel (keyword FTS5 chip, embedding chip, vector store chip,
Qdrant health chip, rerank chip).
- "Reindex Now" button → `POST /api/memory/reindex`.
- Embedding source selector (auto / remote / static / transformers + toggles).
- Qdrant config card (enable toggle, host/port/collection/key, test connection,
semantic search test, cleanup).
- Rerank config card (enable toggle, provider/model selector).
Memory and Qdrant settings also live under
`/dashboard/settings → Memory & Skills` (`MemorySkillsTab.tsx`) for
the legacy/global settings surface.
## Caching
`src/lib/memory/store.ts` keeps an in-process LRU-ish cache
(`MEMORY_CACHE_TTL = 5 min`, `MEMORY_MAX_CACHE_SIZE = 10 000`, with 20 %
oldest eviction) for `getMemory(id)` reads, plus a generic key/value
`memoryCache` layer (`src/lib/memory/cache.ts`) with `get`/`set`/`invalidate`
methods used by callers that want their own scoped cache (1 000-entry LRU,
default TTL 5 min).
## Privacy & Lifecycle
- Memory ownership is the API key id (`resolveMemoryOwnerId` in
`chatCore.ts`). Without an `apiKeyInfo.id` neither retrieval nor injection
nor extraction runs.
- Entries with a future `expires_at` are filtered out of retrieval; old
entries beyond `retentionDays` are excluded by the
`created_at >= cutoff` clause in `retrieveMemories`.
- For hard deletion, use `DELETE /api/memory/[id]` or `omniroute_memory_clear`.
- Extraction is fire-and-forget via `setImmediate`; failures are logged under
`memory.extraction.background.failed` and never surface to the caller.
- Verification round-trips (`verifyExtractionPipeline`) clean up their own
test entries in a `finally` block.
## See Also
- [SKILLS.md](./SKILLS.md) — the `skillsEnabled` setting injects tool
definitions alongside memory.
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP transport / scopes.
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — broader API surface.
- Source modules:
- `src/lib/memory/types.ts`, `schemas.ts`
- `src/lib/memory/store.ts`, `retrieval.ts`, `injection.ts`, `reindex.ts`
- `src/lib/memory/extraction.ts`, `summarization.ts`, `verify.ts`
- `src/lib/memory/settings.ts`, `qdrant.ts`, `cache.ts`
- `src/lib/memory/vectorStore.ts` — sqlite-vec + hybrid RRF
- `src/lib/memory/embedding/index.ts` — multi-source embedding layer
- `src/lib/memory/embedding/types.ts`, `remote.ts`, `staticPotion.ts`,
`transformersLocal.ts`, `cache.ts`
- `src/shared/schemas/memory.ts` — Zod schemas for all memory API bodies
- `src/shared/schemas/qdrant.ts` — Zod schemas for Qdrant settings/ops
- `src/lib/db/memoryVec.ts` — CRUD for `memory_vec_meta`
- `src/lib/db/migrations/015_create_memories.sql`,
`022_add_memory_fts5.sql`, `023_fix_memory_fts_uuid.sql`,
`073_memory_vec.sql`
- `src/app/api/memory/route.ts`, `[id]/route.ts`, `health/route.ts`
- `src/app/api/memory/retrieve-preview/route.ts`
- `src/app/api/memory/engine-status/route.ts`
- `src/app/api/memory/embedding-providers/route.ts`
- `src/app/api/memory/summarize/route.ts`
- `src/app/api/memory/reindex/route.ts`
- `src/app/api/settings/memory/route.ts`
- `src/app/api/settings/qdrant/route.ts` + sub-routes
- `src/app/(dashboard)/dashboard/memory/` — Studio UI (page + components +
tabs + hooks)
- `open-sse/handlers/chatCore.ts` (injection / extraction wiring)
- `open-sse/mcp-server/tools/memoryTools.ts`
---
## Choosing an Embedding Provider (v3.8.16+)
OmniRoute's memory engine supports **four embedding sources** (`src/lib/memory/embedding/`). Each has different trade-offs in **latency, cost, model quality, and setup complexity**.
### The Four Providers
| Provider | Source | Latency | Cost | Quality | Setup |
| -------------- | ------------------------------------------ | ------------------------------- | -------------------- | -------------------------- | ------------------ |
| `transformers` | Local ONNX model (Xenova/all-MiniLM-L6-v2) | ~50-150ms (CPU) | Free | Good | `npm install` only |
| `static` | Pre-computed vectors (cached) | <1ms | Free | N/A (depends on cache hit) | None |
| `remote` | OpenAI / Cohere / Voyage API | ~100-300ms | $0.02-0.10/1M tokens | Excellent | API key |
| `cache` | In-memory LRU layer over any source | <1ms (hit), full latency (miss) | Free | Same as underlying | None |
### Decision Tree
```
What's your deployment context?
│
┌───────────┼───────────┬──────────────┐
│ │ │ │
DEV/TEST SMALL PROD LARGE PROD EDGE / OFFLINE
│ │ │ │
▼ ▼ ▼ ▼
transformers transformers remote (Qdrant) transformers
(free, no API) (best quality) (no internet)
│ │ │ │
└────────┬──┴───────────┴──────────────┘
│
▼
ALWAYS add `cache` layer on top
(LruCache wraps any provider)
```
### Database & API Configuration
Memory embedding options are configured via the Settings API/UI, not environment variables. The relevant settings database keys under Settings (`normalizeMemorySettings` in `src/lib/memory/settings.ts`) are:
- `memoryEmbeddingSource`: `"transformers"` (local), `"remote"` (API-based, e.g. OpenAI), `"static"` (external store), or `"auto"`
- `memoryEmbeddingProviderModel`: Model identifier for remote/static sources (e.g., `"text-embedding-3-small"`)
- `memoryTransformersEnabled`: `true` | `false`
- `memoryStaticEnabled`: `true` | `false`
- `memoryVectorStore`: `"sqlite-vec"`, `"qdrant"`, or `"auto"`
#### Local Model (`transformers`)
Uses transformers.js internally to run local models:
```bash
# Env vars read in code (src/lib/memory/embedding/index.ts):
MEMORY_TRANSFORMERS_MODEL=Xenova/all-MiniLM-L6-v2 # HF model repo
MEMORY_STATIC_MODEL=minishlab/potion-base-8M # HF static potion model
MEMORY_STATIC_CACHE_DIR=<DATA_DIR>/embeddings # Cache directory
```
#### LRU Embedding Cache
The cache is always on by default and configured via env vars:
```bash
MEMORY_EMBEDDING_CACHE_MAX=1000 # Max cached items
MEMORY_EMBEDDING_CACHE_TTL_MS=300000 # TTL (5 min)
```
### Performance Numbers
Benchmark on a typical 4-core x86 server (texts ~100 tokens each):
| Provider | p50 | p95 | p99 | Cost / 1M embeddings |
| -------------------- | ----- | ----- | ----- | ---------------------------------- |
| `transformers` (CPU) | 80ms | 180ms | 350ms | Free |
| `remote` (OpenAI) | 120ms | 220ms | 400ms | ~$0.02 (ada-002) / $0.13 (3-large) |
| `static` (Qdrant) | 15ms | 30ms | 60ms | Depends on Qdrant hosting |
| `cache` (hit) | <1ms | <1ms | 2ms | Free |
---
## Fact Extraction Patterns (v3.8.16+)
The `extraction.ts` module (`src/lib/memory/extraction.ts`) uses **regex pattern matching** to extract structured facts from conversation messages. Understanding these patterns helps you tune extraction quality for your use case.
### Default Pattern Categories
| Category | Example pattern | Captures |
| ------------------- | ----------------------------------------------------------- | ------------------------------ |
| PREFERENCE_PATTERNS | `"I prefer <X>"`, `"I like <X>"`, `"I hate <X>"` | User preferences |
| DECISION_PATTERNS | `"I'll use <X>"`, `"I decided to <X>"`, `"I went with <X>"` | User decisions (episodic) |
| PATTERN_PATTERNS | `"I usually <X>"`, `"I always <X>"`, `"I never <X>"` | Persistent behavioral patterns |
### Example Patterns (Simplified)
```ts
// From src/lib/memory/extraction.ts
const PREFERENCE_PATTERNS = [
/\bI\s+(?:really\s+)?prefer\s+([^.,\n]+)/gi,
/\bI\s+(?:really\s+)?like\s+([^.,\n]+)/gi,
/\bI\s+(?:hate|dislike|avoid)\s+([^.,\n]+)/gi,
];
const DECISION_PATTERNS = [
/\bI'?(?:ll|will)\s+use\s+([^.,\n]+)/gi,
/\bI\s+(?:have\s+)?decided\s+(?:to\s+)?([^.,\n]+)/gi,
];
const PATTERN_PATTERNS = [/\bI\s+usually\s+([^.,\n]+)/gi, /\bI\s+always\s+([^.,\n]+)/gi];
```
### What Gets Extracted
When a user says:
> "I prefer TypeScript. I'll use Postgres for this project. I always commit before pushing. I don't like Python."
> Extraction produces 4 memories:
>
> | Key | Category | Type | Content |
> | ------------------------------------ | ---------- | -------- | --------------------------- |
> | `preference:typescript` | preference | factual | "TypeScript" |
> | `decision:postgres_for_this_project` | decision | episodic | "Postgres for this project" |
> | `pattern:commit_before_pushing` | pattern | factual | "commit before pushing" |
> | `preference:python` | preference | factual | "Python" |
### Extraction Limits
To prevent runaway extraction, the following limits apply:
| Min content length | 3 chars |
| Max content length | 500 chars |
### When to Disable Extraction
Extraction runs automatically whenever memory is enabled; there is no separate
extraction-only toggle. To turn it off, disable memory entirely (`enabled: false`
via `PUT /api/settings/memory`). Consider doing so when:
- You have high message volume and the extraction cost is non-trivial
- Your conversations are mostly transient (chat, debugging) with no long-term value
- You're already capturing context via custom plugins
---
## Hybrid RRF Tuning (v3.8.16+)
The **Reciprocal Rank Fusion (RRF)** algorithm combines FTS5 (keyword) and vector (semantic) results. The `k` parameter controls how much weight is given to lower-ranked results.
### The Formula
For each candidate memory, the RRF score is:
```
RRF(d) = Σ 1 / (k + rank_i(d))
```
Where:
- `k` is the constant (default 60)
- `rank_i(d)` is the rank of document `d` in the i-th retrieval system (FTS, vector)
- The sum runs over all retrieval systems
### How `k` Affects Results
| `k` value | Effect | Best for |
| -------------------- | ------------------------------------------------------------------------------- | -------------------------------------- |
| `k=0` | Pure rank fusion (no smoothing) | Theoretical baseline |
| `k=10-30` | Heavily weights top results, low-rank barely contributes | When top-3 results are usually correct |
| **`k=60`** (default) | Balanced — top-10 results all contribute meaningfully | General-purpose retrieval |
| `k=100+` | Flatter — even low-rank results can dominate if they appear in multiple systems | When recall > precision is critical |
### Tuning `k` in Practice
```bash
# Default
MEMORY_RRF_K=60
# Aggressive precision (small memory, few docs)
MEMORY_RRF_K=20
# Maximum recall (large memory, varied queries)
MEMORY_RRF_K=120
```
**Example with `k=20`:**
- FTS rank 1 → contribution `1/21 = 0.048`
- FTS rank 10 → contribution `1/30 = 0.033`
- Vector rank 1 → contribution `0.048`
- Combined max: `0.096`
**Example with `k=60`:**
- FTS rank 1 → contribution `1/61 = 0.016`
- FTS rank 10 → contribution `1/70 = 0.014`
- Vector rank 1 → contribution `0.016`
- Combined max: `0.033`
With higher `k`, the **relative difference** between top-1 and rank-10 is smaller, so the algorithm relies more on **consensus across retrieval systems** than on top-rank confidence.
### When to Change `k`
| Symptom | Try |
| -------------------------------------- | ------------------------------------------------------------ |
| Top result always wins, but it's wrong | **Lower** k (e.g., 20) — top-rank confidence matters more |
| Right answer is in top-5 but not top-1 | **Higher** k (e.g., 100) — flatter scoring rewards consensus |
| Recall is high but precision is low | **Lower** k — sharpen the ranking |
| Recall is low (missing relevant docs) | **Higher** k — give lower-ranked docs a chance |
### RRF Weighting
The reciprocal rank fusion uses equal weights for semantic vector rank and full-text search rank:
```
RRF(d) = 1/(k + rank_vector) + 1/(k + rank_fts)
```
There are no environment variables to adjust individual weights (`MEMORY_RRF_VECTOR_WEIGHT`/`MEMORY_RRF_FTS_WEIGHT` do not exist).
---
## Summarization Strategy (v3.8.16+)
The `summarization.ts` module (`src/lib/memory/summarization.ts`) compresses older memories to keep the active set small while preserving recall.
### When Summarization Triggers
| Trigger | Threshold (default) |
| ---------------------- | ------------------- |
| Manual trigger via API | n/a |
### What Gets Summarized
Two entry points are exported from `summarization.ts`:
- **`summarizeMemories(apiKeyId, sessionId?, maxTokens = 4000)`** — condenses the
memories for a session into a single summary text bounded by a token budget.
- **`summarizeMemoriesOlderThan(apiKeyId, days, dryRun)`** — the age-based
compaction used by the API: it selects every memory older than `days`, builds
one condensed summary memory from them, and (when `dryRun` is `false`) deletes
the originals. Pass `dryRun: true` to preview the candidate set and token total
without modifying anything.
There is no tag/key clustering pass or per-memory "core vs summarizable" scoring —
selection is purely the age cutoff, and the summary text is a condensed,
type-prefixed line per candidate.
### Triggering Summarization
Summarization is **manual / opt-in** — the `autoSummarize` setting is `false` by
default, so nothing is compacted automatically. Trigger it via the API:
```bash
curl -X POST http://localhost:20128/api/memory/summarize \
-H "Authorization: Bearer $OMNIROUTE_KEY"
```
To leave it off, simply keep `autoSummarize` at its default (`false`).
### Summarization Quality Tips
- **Preview first with `dryRun`** — `summarizeMemoriesOlderThan(..., true)` returns
the candidate list and total token count so you can confirm what would be merged
before deleting the originals.
- **Run summarization during low-traffic hours** if you have a large memory corpus — the LLM call is the slow part
```bash
# Cron-style: summarize at 3am daily
0 3 * * * curl -X POST http://localhost:20128/api/memory/summarize \
-H "Authorization: Bearer $OMNIROUTE_KEY"
```
docs/frameworks/NOTION_CONTEXT.md
+131
View File
@@ -0,0 +1,131 @@
---
title: "Notion Context Source"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Notion Context Source
> **Source of truth:** `src/lib/notion/api.ts` (REST client), `src/lib/db/notion.ts`
> (token persistence), `open-sse/mcp-server/tools/notionTools.ts` (6 MCP tools),
> `src/app/api/settings/notion/route.ts` (settings API). Tool registration and scope
> wiring lives in `open-sse/mcp-server/server.ts`.
## What it is
OmniRoute can connect to a **Notion** workspace as a **context source** — a read/write
knowledge base that agents reach through the built-in MCP server. Once a Notion
integration token is configured, the MCP tools let an LLM search pages and databases,
read page content and block trees, query databases with filters/sorts, and append new
blocks — all proxied through OmniRoute (with retry, timeout, and error classification)
so the model never touches the Notion API directly.
The integration is a thin, hardened wrapper over the official Notion REST API
(`https://api.notion.com/v1`, `Notion-Version: 2026-03-11`). The client
(`src/lib/notion/api.ts`) adds:
- **Retry with exponential backoff** (up to 3 attempts) for `429` and `5xx`.
- **55-second request timeout** via `AbortController`.
- **Typed error classification** — `NotionAuthError` (401/403),
`NotionNotFoundError` (404), `NotionRateLimitError` (429, honors `retry after`
hints), `NotionValidationError` (400/409), `NotionServerError` (5xx),
`NotionTimeoutError`.
- **Message sanitization** that strips stack-trace-like fragments before surfacing.
## Setup
There is **no environment variable** for the Notion token — it is stored in the
SQLite `key_value` table (namespace `notion`, key `integration_token`) via
`src/lib/db/notion.ts`. Configure it from the **Context Sources** tab of the Endpoint
dashboard (`ObsidianSourceCard`'s sibling `NotionSourceCard`), or via the settings REST API.
> [!NOTE]
> The token is a **Notion internal integration token**. Create an integration at
> <https://www.notion.com/my-integrations>, then share the pages/databases you want
> OmniRoute to access with that integration (Notion's permission model is share-based,
> not workspace-wide).
### Configure via REST
```bash
# Save + validate the integration token (POST validates by issuing a test search)
curl -X POST http://localhost:20128/api/settings/notion \
-H "Content-Type: application/json" \
-d '{"token":"ntn_xxx"}'
# Check connection status
curl http://localhost:20128/api/settings/notion
# Disconnect (clears the stored token)
curl -X DELETE http://localhost:20128/api/settings/notion
```
All three methods require dashboard authentication (`isAuthenticated`). On `POST`,
OmniRoute saves the token and immediately runs a 1-result test search; if Notion
returns an error object the token is cleared and the call fails with `400`.
## MCP tools (6)
Defined in `open-sse/mcp-server/tools/notionTools.ts`. The token is resolved at call
time via `getNotionToken()`; if none is configured the tool throws
`"Notion integration token not configured. Set it in Settings > Context Sources."`
| Tool | Scope | Description |
| ---------------------------- | -------------- | --------------------------------------------------------------------------------- |
| `notion_search` | `read:notion` | Search pages and databases by text query (returns titles, IDs, URLs). Paginated. |
| `notion_get_page` | `read:notion` | Get content and metadata of a page by its ID. |
| `notion_list_block_children` | `read:notion` | List all block children of a block or page (the block tree). Paginated. |
| `notion_query_database` | `read:notion` | Query a database with optional `filter` + `sorts` (Notion API format). Paginated. |
| `notion_get_database` | `read:notion` | Get the schema/metadata of a database by ID. |
| `notion_append_blocks` | `write:notion` | Append block children to an existing block or page (max 100 blocks per request). |
### Input parameters
- `notion_search` — `query` (1–500 chars), `pageSize` (1–100, default 20),
`startCursor` (optional).
- `notion_get_page` — `pageId` (32-char hex or UUID).
- `notion_list_block_children` — `blockId`, `pageSize` (1–100, default 50),
`startCursor` (optional).
- `notion_query_database` — `databaseId`, `filter` (optional, Notion filter format),
`sorts` (optional array), `pageSize` (1–100, default 50), `startCursor` (optional).
- `notion_get_database` — `databaseId`.
- `notion_append_blocks` — `blockId`, `children` (array of block objects),
`after` (optional position).
### Scopes
The read tools require `read:notion` and the write tool requires `write:notion`.
Scopes are enforced by `withScopeEnforcement()` in
`open-sse/mcp-server/server.ts` only when `OMNIROUTE_MCP_ENFORCE_SCOPES=true`; the
caller's allowed scopes come from `OMNIROUTE_MCP_SCOPES` (comma-separated) or the
authenticated API key's scope context. See [MCP-SERVER.md](./MCP-SERVER.md) for the
full scope model.
## Endpoints
| Method | Path | Purpose |
| -------- | ---------------------- | -------------------------------------- |
| `GET` | `/api/settings/notion` | Return `{ connected, hasToken }`. |
| `POST` | `/api/settings/notion` | Save + validate the integration token. |
| `DELETE` | `/api/settings/notion` | Disconnect (clear the stored token). |
> These are dashboard settings routes. There is **no public `/v1` Notion proxy
> endpoint** — Notion is reached exclusively through the MCP tools above.
## Use cases
- **Knowledge-grounded answers** — let an agent `notion_search` the workspace and
`notion_get_page` the top hit before answering, so responses cite real internal docs.
- **Database-backed workflows** — `notion_query_database` a tasks/CRM database with
filters + sorts, then summarize or triage the rows.
- **Write-back / logging** — `notion_append_blocks` to append meeting notes, run
summaries, or agent output into an existing page (append-only; no destructive edits).
- **Structure exploration** — `notion_list_block_children` to walk a page's block tree,
or `notion_get_database` to discover a database's property schema before querying it.
## Related
- [MCP Server](./MCP-SERVER.md) — transports, scope enforcement, full tool inventory.
- [Obsidian Context Source](./OBSIDIAN_CONTEXT.md) — the other built-in context source.
- [Memory System](./MEMORY.md) — persistent conversational memory (complementary
context layer, injected automatically rather than tool-fetched).
docs/frameworks/OBSIDIAN_CONTEXT.md
+191
View File
@@ -0,0 +1,191 @@
---
title: "Obsidian Context Source"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Obsidian Context Source
> **Source of truth:** `src/lib/obsidian/api.ts` (REST + sync client),
> `src/lib/db/obsidian.ts` (token / base-URL / WebDAV persistence),
> `src/lib/obsidianSync.ts` (WebDAV vault sync), `open-sse/mcp-server/tools/obsidianTools.ts`
> (22 MCP tools), `src/app/api/settings/obsidian/route.ts` +
> `src/app/api/settings/obsidian/webdav/route.ts` (settings APIs). Tool registration
> and scope wiring lives in `open-sse/mcp-server/server.ts`.
## What it is
OmniRoute connects to an **Obsidian** vault as a **context source** — a local Markdown
knowledge base that agents read and write through the built-in MCP server. The
integration talks to the **Obsidian Local REST API** community plugin running inside the
desktop app, so agents can search notes, read/write/patch files, list the vault, work
with daily/weekly periodic notes, manage tags, run Obsidian commands, and (optionally)
coordinate a bidirectional desktop↔mobile vault sync.
The client (`src/lib/obsidian/api.ts`) wraps the Local REST API with:
- **Retry with backoff** for transient `5xx`, **30-second timeout** via `AbortController`.
- **Typed error classification** — `ObsidianAuthError` (401/403),
`ObsidianNotFoundError` (404), `ObsidianServerError` (5xx), `ObsidianTimeoutError`.
- A **friendly "cannot reach Obsidian" hint** that calls out the common port mistake
(HTTP on `27123`, **not** the MCP endpoint on `27124`) and the Tailscale form.
- Vault-relative **path encoding** so note paths with spaces/slashes are safe.
## Setup
There is **no environment variable** for the Obsidian token or base URL — both are
stored in the SQLite `key_value` table (namespace `obsidian`) via
`src/lib/db/obsidian.ts`. The token is **encrypted at rest** (AES-256-GCM, with
plaintext backward-compat fallback). Configure from the **Context Sources** tab of the
Endpoint dashboard (`ObsidianSourceCard`), or via the settings REST API.
> [!IMPORTANT]
> The **Obsidian Local REST API** plugin must be installed and running. Its REST
> interface listens on **HTTP `127.0.0.1:27123`** (the default base URL). Port `27124`
> is a _separate_ MCP/HTTPS endpoint and is explicitly rejected by the settings route.
> If connecting from another device, use `http://<tailscale-ip>:27123`.
### Configuration keys (SQLite `key_value`, namespace `obsidian`)
| Key | Purpose | Encrypted |
| ----------------- | ------------------------------------------------ | --------- |
| `api_key` | Local REST API bearer token | yes |
| `base_url` | REST base URL (default `http://127.0.0.1:27123`) | no |
| `vault_path` | Absolute path to the vault directory (for sync) | no |
| `webdav_username` | Generated WebDAV username (vault sync) | no |
| `webdav_password` | Generated WebDAV password (vault sync) | yes |
| `webdav_enabled` | Whether WebDAV vault sync is enabled | no |
### Configure via REST
```bash
# Save + validate the Local REST API token (POST validates via a status check)
curl -X POST http://localhost:20128/api/settings/obsidian \
-H "Content-Type: application/json" \
-d '{"token":"<obsidian-rest-api-key>","baseUrl":"http://127.0.0.1:27123"}'
# Check connection status (returns connected, hasToken, baseUrl, vaultPath)
curl http://localhost:20128/api/settings/obsidian
# Disconnect (clears the stored token)
curl -X DELETE http://localhost:20128/api/settings/obsidian
```
All methods require dashboard authentication. `POST` rejects any URL on port `27124`
and validates the token by calling the Local REST API status endpoint before persisting.
### WebDAV vault sync
`src/app/api/settings/obsidian/webdav/route.ts` manages an optional WebDAV-backed
vault sync (driven by `src/lib/obsidianSync.ts`). Enabling it points OmniRoute at a
local vault directory and mints a random WebDAV username/password pair:
```bash
# Enable WebDAV sync for a vault directory (mints username/password)
curl -X POST http://localhost:20128/api/settings/obsidian/webdav \
-H "Content-Type: application/json" \
-d '{"vaultPath":"/home/me/MyVault"}'
# Get WebDAV sync status (credentials returned only while enabled)
curl http://localhost:20128/api/settings/obsidian/webdav
# Disable WebDAV sync (clears credentials + managed .stignore)
curl -X DELETE http://localhost:20128/api/settings/obsidian/webdav
```
### Per-API-key context source (optional)
Obsidian config can be scoped **per API key** via the `api_key_context_sources` table
(`src/lib/db/apiKeyContextSources.ts`). When an MCP call carries an authenticated API
key id, `getObsidianConfigForApiKey()` prefers that key's own token/base-URL/vault-path
(`source: "api_key"`) and otherwise falls back to the global config (`source: "global"`).
## MCP tools (22)
Defined in `open-sse/mcp-server/tools/obsidianTools.ts`. The token/base-URL are resolved
per call (per-API-key first, then global). Tools that hit the OmniRoute **sync server**
(the four `obsidian_sync_*` tools) additionally require the sync auth token configured
in OmniRoute settings.
### Read tools (`read:obsidian`)
| Tool | Description |
| ---------------------------- | ------------------------------------------------------------------------------------ |
| `obsidian_check_status` | Check whether the Local REST API is reachable and authenticated. |
| `obsidian_search_simple` | Full-text search of note content; returns snippets with file paths. |
| `obsidian_search_structured` | Search using a JSON Logic expression (and/or/regex/path filters). |
| `obsidian_read_note` | Read a note by vault-relative path; optionally a specific heading/block/frontmatter. |
| `obsidian_list_vault` | List files and directories in the vault (tree of entries). |
| `obsidian_get_document_map` | Get the note's heading structure as a map of headings → line numbers. |
| `obsidian_get_note_metadata` | Get frontmatter, tags, links, char/word count without the full content. |
| `obsidian_get_active_file` | Get the path + content of the currently active file in Obsidian. |
| `obsidian_get_periodic_note` | Get the daily/weekly/monthly periodic note for a date (today if omitted). |
| `obsidian_get_tags` | List all vault tags with their frequencies. |
| `obsidian_list_commands` | List available Obsidian command IDs (use with `obsidian_execute_command`). |
| `obsidian_sync_status` | OmniRoute sync server status: running, vault name, port, uptime, last sync. |
| `obsidian_sync_conflicts` | List unresolved sync conflicts (path, conflict path, detected-at). |
### Write tools (`write:obsidian`)
| Tool | Description |
| -------------------------------- | ----------------------------------------------------------------------------------- |
| `obsidian_write_note` | Create or overwrite a note with given Markdown content. |
| `obsidian_append_note` | Append content to a note; optionally to a specific heading/block. |
| `obsidian_patch_note` | Surgically append/prepend/replace at a heading, block, or frontmatter field. |
| `obsidian_delete_note` | Permanently delete a note from the vault. |
| `obsidian_move_note` | Move or rename a note within the vault. |
| `obsidian_execute_command` | Execute an Obsidian command by its command ID. |
| `obsidian_open_file` | Open a file in Obsidian (creates it if it does not exist). |
| `obsidian_sync_trigger` | Trigger an immediate bidirectional desktop↔mobile vault sync. |
| `obsidian_sync_resolve_conflict` | Resolve a sync conflict: keep `local` (mobile), `remote` (desktop), or `keep-both`. |
> [!NOTE]
> `obsidian_patch_note` targets accept `targetType` of `heading | block | frontmatter`
> and `operation` of `append | prepend | replace`, with an optional
> `createTargetIfMissing`. The four `obsidian_sync_*` tools talk to the local sync
> server (`http://127.0.0.1:27781` by default) and require the sync token.
### Scopes
Read tools require `read:obsidian`; write tools require `write:obsidian`. Enforcement
is identical to Notion — handled by `withScopeEnforcement()` in
`open-sse/mcp-server/server.ts`, gated on `OMNIROUTE_MCP_ENFORCE_SCOPES=true`, with
allowed scopes sourced from `OMNIROUTE_MCP_SCOPES` or the API key's scope context. See
[MCP-SERVER.md](./MCP-SERVER.md).
## Endpoints
| Method | Path | Purpose |
| -------- | ------------------------------- | ----------------------------------------------------- |
| `GET` | `/api/settings/obsidian` | Return `{ connected, hasToken, baseUrl, vaultPath }`. |
| `POST` | `/api/settings/obsidian` | Save + validate token (rejects port `27124`). |
| `DELETE` | `/api/settings/obsidian` | Disconnect (clear stored token). |
| `GET` | `/api/settings/obsidian/webdav` | WebDAV sync status + credentials (while enabled). |
| `POST` | `/api/settings/obsidian/webdav` | Enable WebDAV sync for a vault directory. |
| `DELETE` | `/api/settings/obsidian/webdav` | Disable WebDAV sync. |
> These are dashboard settings routes. The vault itself is reached through the Obsidian
> Local REST API (the configured `base_url`) and through the MCP tools above — there is
> no public `/v1` Obsidian proxy endpoint.
## Use cases
- **Vault-grounded answers** — `obsidian_search_simple` / `obsidian_search_structured`
then `obsidian_read_note` so an agent answers from your real notes.
- **Note authoring / journaling** — `obsidian_write_note`, `obsidian_append_note`, or
the surgical `obsidian_patch_note` to log agent output, summaries, or daily notes
(`obsidian_get_periodic_note`) into the vault.
- **Vault navigation** — `obsidian_list_vault`, `obsidian_get_document_map`, and
`obsidian_get_tags` to explore structure before reading/writing.
- **Obsidian automation** — `obsidian_list_commands` + `obsidian_execute_command` to
drive plugins/commands from an agent; `obsidian_open_file` to surface a note in the UI.
- **Mobile sync** — enable WebDAV sync, then `obsidian_sync_trigger` /
`obsidian_sync_status` / `obsidian_sync_conflicts` / `obsidian_sync_resolve_conflict`
to coordinate desktop↔mobile and resolve conflicts.
## Related
- [MCP Server](./MCP-SERVER.md) — transports, scope enforcement, full tool inventory.
- [Notion Context Source](./NOTION_CONTEXT.md) — the other built-in context source.
- [Memory System](./MEMORY.md) — persistent conversational memory (complementary
context layer, injected automatically rather than tool-fetched).
docs/frameworks/OPENCODE.md
+165
View File
@@ -0,0 +1,165 @@
---
title: "OpenCode Integration"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OpenCode Integration
> **Status:** Generally available.
> **Audience:** Operators wiring OpenCode to an OmniRoute deployment.
> **Source of truth (config schema):** `src/shared/services/opencodeConfig.ts`
> **Source of truth (npm package):** `@omniroute/opencode-provider/` (publishable workspace)
[OpenCode](https://opencode.ai) is an agentic CLI/desktop AI client. It reads its provider catalog from `~/.config/opencode/opencode.json` (or `opencode.jsonc`) and follows the schema at `https://opencode.ai/config.json`. OmniRoute exposes itself to OpenCode as one of those providers — every request flows through OmniRoute's standard OpenAI-compatible `/v1` surface, so OpenCode automatically benefits from Auto-Combo routing, circuit breakers, key policies, observability, etc.
There are **two supported integration paths**. Pick one — they generate the same config.
---
## Path 1 — CLI generator (no npm install)
Recommended for end users. Ships with OmniRoute. Writes `opencode.json` in place.
```bash
# After installing OmniRoute (npm i -g @omniroute/cli or local clone)
omniroute config opencode \
--baseUrl http://localhost:20128 \
--apiKey "$OMNIROUTE_API_KEY"
```
Behind the scenes the CLI calls `mergeOpenCodeConfigText()` (`src/shared/services/opencodeConfig.ts:104`), so an existing `opencode.json` keeps its other providers and comments. The OmniRoute entry is added/replaced atomically.
Resulting file (default model catalog):
```jsonc
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"omniroute": {
"npm": "@ai-sdk/openai-compatible",
"name": "OmniRoute",
"options": {
"baseURL": "http://localhost:20128/v1",
"apiKey": "<your-key>",
},
"models": {
"claude-opus-4-5-thinking": { "name": "claude-opus-4-5-thinking" },
"claude-sonnet-4-5-thinking": { "name": "claude-sonnet-4-5-thinking" },
"gemini-3.1-pro-high": { "name": "gemini-3.1-pro-high" },
"gemini-3-flash": { "name": "gemini-3-flash" },
},
},
},
}
```
---
## Path 2 — npm package `@omniroute/opencode-provider`
Recommended when you're scripting the config from Node/TS (CI pipelines, monorepos, custom installer flows).
```bash
npm install --save-dev @omniroute/opencode-provider
```
```ts
import { writeFileSync } from "node:fs";
import { buildOmniRouteOpenCodeConfig } from "@omniroute/opencode-provider";
const config = buildOmniRouteOpenCodeConfig({
baseURL: "http://localhost:20128",
apiKey: process.env.OMNIROUTE_API_KEY ?? "sk_omniroute",
// Optional: override the model catalog exposed to OpenCode
models: ["auto", "claude-opus-4-7", "gpt-5.5"],
modelLabels: { auto: "Auto-Combo" },
});
writeFileSync("opencode.json", JSON.stringify(config, null, 2));
```
For a non-destructive merge against an existing file, replicate `mergeOpenCodeConfigText()` from `opencodeConfig.ts` or call the CLI generator.
See the [package README](../../@omniroute/opencode-provider/README.md) for the full API.
---
## What the runtime actually does
Both paths produce the same `provider.omniroute.npm: "@ai-sdk/openai-compatible"`. At runtime, OpenCode loads `@ai-sdk/openai-compatible` (already a transitive dependency of OpenCode) and configures it with `baseURL` + `apiKey`. From there:
```
OpenCode UI/agent
→ @ai-sdk/openai-compatible
→ HTTP POST {baseURL}/chat/completions (OmniRoute OpenAI surface)
→ OmniRoute /v1/chat/completions handler (open-sse/handlers/chatCore.ts)
→ combo routing / Auto-Combo / executor
→ upstream provider
```
The plugin never touches HTTP. It only emits configuration.
---
## Model catalog defaults
```ts
export const OMNIROUTE_DEFAULT_OPENCODE_MODELS = [
"claude-opus-4-5-thinking",
"claude-sonnet-4-5-thinking",
"gemini-3.1-pro-high",
"gemini-3-flash",
] as const;
```
You can override via `models: [...]`. Recommended additions:
- `"auto"` — surfaces OmniRoute's [Auto-Combo](../routing/AUTO-COMBO.md) zero-config router. Lets OpenCode pick "the best available model" without you hard-coding the catalog.
- `"<combo-name>"` — any combo you've defined in the dashboard; OmniRoute resolves it transparently.
---
## URL normalisation
The helper accepts both forms and emits exactly one `/v1`:
| Input | Output (`options.baseURL`) |
| ------------------------------ | --------------------------- |
| `http://localhost:20128` | `http://localhost:20128/v1` |
| `http://localhost:20128/` | `http://localhost:20128/v1` |
| `http://localhost:20128/v1` | `http://localhost:20128/v1` |
| `http://localhost:20128/v1///` | `http://localhost:20128/v1` |
This deduplication is **the most common breakage** seen in older configs. If you have an `opencode.json` from before v3.8.0 that points at `/v1/v1/...`, re-run the generator or call `createOmniRouteProvider` again.
---
## Authentication modes
| OmniRoute setting | Recommended `apiKey` value |
| ------------------------------------------- | -------------------------------------------------- |
| `REQUIRE_API_KEY=false` (default for local) | `sk_omniroute` (literal placeholder) |
| `REQUIRE_API_KEY=true` | A real per-user API key from Dashboard → API Keys. |
For Anthropic-style clients that send `x-api-key` + `anthropic-version`, OmniRoute's `extractApiKey` also honours the key from `x-api-key`. OpenCode uses the OpenAI surface, so it'll always send `Authorization: Bearer ${apiKey}` — no Anthropic special-case applies here.
---
## Troubleshooting
| Symptom | Cause | Fix |
| ---------------------------------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `404` on every request with URL containing `/v1/v1/` | Stale config from pre-v3.8 plugin that double-suffixed `/v1`. | Regenerate via Path 1 or 2. |
| `401 Invalid API key` | OmniRoute has `REQUIRE_API_KEY=true` and the key is unknown. | Create the key in the dashboard, or set `REQUIRE_API_KEY=false` (local only) and use `sk_omniroute`. |
| Model list empty in OpenCode UI | All 4 default models are hidden in OmniRoute's provider visibility. | Pass `models: ["auto", ...]` to surface ones you've enabled. |
| OpenCode 500 with `cannot read property 'models'` | Older OpenCode (< 0.1.x) didn't accept inline `models`. | Upgrade OpenCode to a version that follows the v1 schema (`opencode.ai/config.json`). |
---
## See also
- [API reference](../reference/API_REFERENCE.md) — full OmniRoute REST surface
- [Auto-Combo](../routing/AUTO-COMBO.md) — what `model: "auto"` means
- [`@omniroute/opencode-provider` README](../../@omniroute/opencode-provider/README.md)
- Source: `src/shared/services/opencodeConfig.ts`, `src/lib/cli-helper/config-generator/opencode.ts`, `@omniroute/opencode-provider/src/index.ts`
docs/frameworks/OPEN_SSE_ARCHITECTURE.md
+576
View File
@@ -0,0 +1,576 @@
---
title: "open-sse Architecture"
version: 3.8.40
lastUpdated: 2026-06-28
---
# open-sse Architecture
> **TL;DR**: `open-sse/` is the core streaming engine that powers every LLM request in OmniRoute. It contains ~900 files implementing the request pipeline, executors, services, MCP server, and translation layer. This guide explains how the pieces fit together.
**Source:** `open-sse/` (workspace package, ~900 files; 811 `.ts`)
---
## Why a Separate Workspace Package?
`open-sse/` is a **standalone workspace** in the OmniRoute monorepo for several reasons:
1. **Reusability** — `open-sse` is published as `@omniroute/open-sse` on npm, so other projects can use it independently
2. **Clean boundaries** — the streaming engine is decoupled from the OmniRoute-specific UI/DB layer
3. **Performance** — the engine has no Next.js dependencies, enabling faster cold starts in CLI/serverless contexts
4. **Versioning** — `open-sse` can release on its own cadence
```json
// package.json
"workspaces": ["open-sse"]
```
---
## Top-Level Structure
```
open-sse/
├── index.ts # Public entry point
├── types.d.ts # Public type exports
├── package.json # @omniroute/open-sse
├── config/ # Provider configs, constants, registries
├── executors/ # Per-provider HTTP executors (67 + base.ts/index.ts)
├── handlers/ # Request handlers (chatCore, responses, etc.)
├── lib/ # Internal utilities
├── mcp-server/ # Model Context Protocol server
├── services/ # ~298 service modules
├── transformer/ # Responses API format transformer
├── translator/ # Format translation (OpenAI ↔ Claude ↔ Gemini)
└── utils/ # Shared utilities (logging, error, stream, etc.)
```
### Module Counts
| Directory | Files | Purpose |
| `executors/` | 68 | Per-provider HTTP executors (unified via DefaultExecutor factory) |
| `handlers/` | 16 | Request entry points (chatCore, responses, embeddings) |
| `services/` | ~298 | Routing, caching, rate limiting, refresh, etc. |
| `translator/` | ~27 | Format conversion (OpenAI ↔ Claude ↔ Gemini) |
| `mcp-server/` | 32 | MCP tools and transports |
| `utils/` | ~65 | Cross-cutting utilities (logging, error, stream) |
| `config/` | ~10 | Provider configs, constants, registries |
---
## The Request Pipeline
Every LLM request flows through a **5-stage pipeline**:
```
┌──────────────┐
HTTP request │ 1. ROUTE │ combo resolution, model selection
(Next.js route) └──────┬───────┘
│
▼
┌──────────────┐
│ 2. TRANSLATE│ format conversion (OpenAI ↔ Claude ↔ Gemini)
└──────┬───────┘
│
▼
┌──────────────┐
│ 3. EXECUTE │ provider executor, HTTP, retry, breaker
└──────┬───────┘
│
▼
┌──────────────┐
│ 4. STREAM │ SSE transformation, backpressure
└──────┬───────┘
│
▼
┌──────────────┐
│ 5. RECORD │ usage tracking, call log, error classification
└──────┬───────┘
│
▼
HTTP response (SSE or JSON)
```
### Stage 1: Route (services/combo.ts)
**Entry point**: `handleComboChat()` in `services/combo.ts`
Resolves the request to a concrete `(provider, model, account, credentials)` tuple:
- Look up the combo by ID (or build a virtual combo for `auto/*` models)
- Apply routing strategy (priority, weighted, round-robin, etc.)
- Filter out unhealthy providers (circuit breaker)
- Pick the next viable target
For `auto/*` models, this stage also:
- Runs the **9-factor scoring** algorithm (`services/autoCombo/`)
- Selects a `provider+model` pair based on health, cost, latency, etc.
### Stage 2: Translate (translator/)
If the source format (e.g., OpenAI) differs from the target format (e.g., Claude), the request is **translated**:
- System prompt → system message
- Tool definitions → provider-specific tool format
- Reasoning/thinking parameters → provider-specific equivalents
- Message role normalization (`developer` → `system` for non-OpenAI)
The `translator/index.ts` exposes:
```ts
translateRequest(body, sourceFormat, targetFormat): TranslatedRequest
needsTranslation(source, target): boolean
```
### Stage 3: Execute (executors/)
**Entry point**: `getExecutor(providerId).execute(request, options)`
All providers use `DefaultExecutor` (`executors/default.ts`) via the `getExecutor()` factory fallback. The executor:
- Builds the upstream URL (`buildUrl()`)
- Adds provider-specific headers (`buildHeaders()`)
- Transforms the request body (`transformRequest()`)
- Sends the HTTP request with retry + exponential backoff
- Handles auth refresh if needed (OAuth providers)
All executors extend `BaseExecutor` (`executors/base.ts`, 1170 LOC) which provides:
- Common retry logic
- Proxy integration
- Circuit breaker integration
- Usage recording hooks
### Stage 4: Stream (utils/stream.ts)
For streaming responses, the executor returns a **ReadableStream**. The handler:
- Pipes through an SSE transform (`createSSETransformStreamWithLogger`)
- Applies heartbeat pings to detect dead connections
- Handles client disconnect gracefully (`pipeWithDisconnect`)
- Transforms SSE → JSON for non-streaming clients
For non-streaming responses, the executor returns a parsed JSON object that is passed through unchanged.
### Stage 5: Record (services/usage.ts)
After the response (success or failure), usage is recorded:
- `prompt_tokens`, `completion_tokens`, `cached_tokens` from the response
- `cost_usd` computed from pricing data
- `latency_ms`, `status`, `error_class` if failed
- Persisted to `usage_history` table
Call log artifacts (if enabled) are written to `${DATA_DIR}/call_logs/`.
---
## Key Files Deep-Dive
### chatCore.ts (5977 lines)
The **main request handler**. Despite its size, it has a clear structure:
```ts
// Pseudo-structure of chatCore.ts
export async function handleChat(request: NextRequest) {
// 1. Auth + CORS
await authenticateRequest(request);
applyCorsHeaders(response);
// 2. Body validation
const body = await parseRequestBody(request);
// 3. Format detection + translation
const sourceFormat = detectFormat(request);
const targetFormat = getTargetFormat(providerId);
if (needsTranslation(sourceFormat, targetFormat)) {
body = translateRequest(body, sourceFormat, targetFormat);
}
// 4. Combo routing
const targets = await resolveComboTargets(comboId, body);
for (const target of targets) {
try {
const result = await executeOnTarget(target, body);
await recordUsage(result);
return result;
} catch (err) {
// Continue to next target
}
}
// 5. Emergency fallback
return await emergencyFallback(body);
}
```
Despite being one giant function, it's organized into **commented sections** that map to the 5-stage pipeline.
### combo.ts (4456 LOC)
The **routing engine** that resolves a combo to ordered targets.
```ts
// services/combo.ts
export async function handleComboChat(body, comboId): Promise<ChatResult> {
const targets = await resolveComboTargets(comboId, body);
for (const target of targets) {
try {
return await handleSingleModel(target, body);
} catch (err) {
log.warn("target failed, trying next", { target, err });
}
}
throw new ComboExhaustedError("All targets failed");
}
```
Supports **17 routing strategies** (see `src/shared/constants/routingStrategies.ts`):
| Strategy | Behavior |
| ------------------- | ------------------------------------------------------------------------- |
| `priority` | First-target ordered list |
| `weighted` | Probabilistic by per-target weight |
| `round-robin` | Cycle through targets in order |
| `context-relay` | Hand off context across targets |
| `fill-first` | Fill quota before moving to next |
| `p2c` | Power of two choices |
| `random` | Uniform random |
| `least-used` | Pick the one with fewest recent uses |
| `cost-optimized` | Cheapest healthy target first |
| `reset-aware` | Aware of provider reset windows |
| `reset-window` | Reset window-based routing |
| `headroom` | Most remaining quota headroom first |
| `strict-random` | Truly uniform (no quality weighting) |
| `auto` | Use 9-factor scoring (`autoCombo/`) |
| `lkgp` | Last known good provider first |
| `context-optimized` | Best for long-context requests |
| `fusion` | Fan out to a panel in parallel, then synthesize via a judge (`fusion.ts`) |
### base.ts (1170 LOC)
The **abstract executor** that all 67 executors extend. It contains:
- `buildUrl()` — default URL construction (subclasses override for custom)
- `buildHeaders()` — default headers (auth, content-type)
- `transformRequest()` — pass-through by default
- `execute()` — the main HTTP loop with retry/backoff/breaker
```ts
// open-sse/executors/default.ts
export class DefaultExecutor extends BaseExecutor {
// Handles all OpenAI/Anthropic-compatible providers
// Providers register configurations (URL, auth, headers) but share executor logic
}
```
Provider-specific behavior (auth headers, base URL, version headers) is configured via the provider registry, not separate executor classes.
````
---
## Services (117 modules)
Services are **focused, single-purpose modules** that handlers compose. The big categories:
### Routing & Combo
- `combo.ts` — entry point for combo-routed requests
- `services/autoCombo/` — 9-factor scoring, 8 auto routing strategies
- `wildcardRouter.ts` — matches wildcard routes (`gpt-*`)
- `modelFamilyFallback.ts` — T5 intra-family fallback
### Rate Limiting & Quota
- `rateLimitManager.ts` — token bucket per key+provider
- `usage.ts` — usage recording
- `quotaCache.ts` — in-memory quota snapshots
### Account & Token
- `tokenRefresh.ts` — OAuth refresh on 401
- `accountFallback.ts` — switch to alternate account
- `sessionManager.ts` — multi-turn session state
### Intelligence
- `intentClassifier.ts` — classify request intent
- `taskAwareRouter.ts` — route by task type
- `thinkingBudget.ts` — allocate thinking tokens
- `contextManager.ts` — inject routing context
### Resilience
- `resilience.ts` — retry, backoff, breaker orchestration
- `emergencyFallback.ts` — last-resort fallback
- `modelDeprecation.ts` — auto-route to successor models
### State
- `signatureCache.ts` — dedup by request signature
- `volumeDetector.ts` — load shedding
- `contextHandoff.ts` — session serialization
### Compression
- `compression/` (subdirectory) — full compression pipeline
- 39 files covering engines, rule packs, adapters
### Skills
- (covered in [SKILLS.md](./SKILLS.md))
### Memory
- (covered in [MEMORY.md](./MEMORY.md))
---
## Executors (75+ files)
One file per provider. They all extend `BaseExecutor` and override what differs.
### Common Patterns
Providers are resolved via `getExecutor(providerId)`, which returns the configured executor. OpenAI/Anthropic-compatible providers use `DefaultExecutor` (`executors/default.ts`). Provider-specific behavior (base URL, auth headers, API version) is configured in `open-sse/config/providers/`, while request body transformations are handled in `open-sse/translator/`.
**Custom URL** is set via provider configuration:
```ts
// Provider config in open-sse/config/providers/
export default {
id: "together",
baseURL: "https://api.together.xyz/v1/chat/completions",
}
````
**Custom auth** is handled through the provider registry's auth configuration (API key, OAuth, header profiles).
**Custom request body** transformations (e.g., Anthropic separating `system` from `messages`) are registered per-provider in `open-sse/translator/`.
````
### The Executor Factory
`executors/index.ts` exports `getExecutor(providerId)`:
```ts
import { getExecutor } from "@omniroute/open-sse/executors";
const executor = getExecutor("anthropic");
const result = await executor.execute({
model: "claude-sonnet-4-5",
messages: [...],
});
````
The factory is generated from `config/providerRegistry.ts` which lists all 212+ providers and their executor class.
---
## Translators
Translate between **3 formats**: OpenAI, Anthropic, Gemini, plus the new Responses API.
### When Translation Happens
```ts
import { needsTranslation, translateRequest } from "@omniroute/open-sse/translator";
if (needsTranslation(sourceFormat, targetFormat)) {
body = translateRequest(body, sourceFormat, targetFormat);
}
```
Common translations:
- `OpenAI → Anthropic`: separate `system` field, `x-api-key` header
- `OpenAI → Gemini`: `contents` instead of `messages`, `systemInstruction`
- `OpenAI → Responses API`: `input` array, `previous_response_id` state
### Edge Cases Handled
- `developer` role → `system` for non-OpenAI
- `system` role → merged into first user message for GLM/ERNIE
- `json_schema` → Gemini's `responseMimeType` + `responseSchema`
- `tools` → provider-specific tool format
- Thinking parameters (o1, Claude) → provider-specific equivalents
---
## MCP Server
`open-sse/mcp-server/` implements the **Model Context Protocol** server:
- **30+ tools** (provider management, combos, memory, cache, compression, 1proxy, skills)
- **3 transports**: stdio, SSE, Streamable HTTP
- **13 scopes** for fine-grained authorization
### Tool Registration
Tools are registered as standalone files in `open-sse/mcp-server/tools/`, each exporting a name, schema, handler, and scope:
```ts
// open-sse/mcp-server/tools/getHealth.ts
import { z } from "zod";
export default {
name: "omniroute_get_health",
description: "Get system health snapshot",
scope: "read:health",
inputSchema: z.object({}),
handler: async (_args, ctx) => {
return await getSystemHealth();
},
};
```
### Transports
```ts
// stdio (CLI usage)
startMcpStdio(server);
// SSE (HTTP-based streaming)
startMcpSse(server, port);
// Streamable HTTP (modern MCP)
startMcpStreamable(server, port);
```
### Authorization
Every tool call goes through scope checks (`open-sse/mcp-server/auth/`):
```ts
if (!hasScope(apiKey, "providers:read")) {
throw new Error("Insufficient scope");
}
```
---
## Transformers
`open-sse/transformer/` converts between **Chat Completions** and **Responses API** formats.
### Why a Separate Transformer?
The Responses API is OpenAI's new format with **stateful conversations** (`previous_response_id`). When a client sends a Responses request, OmniRoute:
1. Converts Responses → Chat Completions internally
2. Sends to provider (any provider that supports Chat Completions)
3. Converts the response back to Responses format
4. Streams the converted response to the client
The transformer (`transformer/responsesTransformer.ts`) provides:
```ts
createResponsesApiTransformStream(): TransformStream
```
This handles:
- `response.output_item.added` events
- `response.output_text.delta` events
- `response.completed` event
- Tool call mapping (`function_call` ↔ `tool_calls`)
---
## Configuration
`open-sse/config/` holds the configuration layer:
| File | Purpose |
| ----------------------------- | --------------------------------- |
| `providerRegistry.ts` | 212+ provider definitions |
| `providerModels.ts` | Model aliases, format mapping |
| `constants.ts` | Timeouts, limits, status codes |
| `defaultThinkingSignature.ts` | Default Claude thinking signature |
| `modelStrip.ts` (in services) | Per-provider field stripping |
### Provider Registry Schema
```ts
interface ProviderConfig {
id: string;
name: string;
baseUrl: string;
authType: "bearer" | "api-key" | "oauth" | "cookie";
executorClass: string;
defaultModel: string;
capabilities: ProviderCapabilities;
models: ModelDefinition[];
}
```
Zod validation at module load ensures all provider configs are valid.
---
## Performance Constraints
The routing engine has strict performance budgets:
| Operation | Target | Measurement |
| --------------------------------------- | ------ | ------------------------- |
| Combo resolution | <10ms | For 50 targets |
| Rate limit check | <1ms | In-memory token bucket |
| Model family fallback | <5ms | Cached family definitions |
| Request routing dispatch | <2ms | Hot path |
| **No blocking I/O in routing hot path** | — | All async |
---
## Anti-Patterns
❌ **Synchronous DB calls in `combo.ts`** — pre-compute and cache
❌ **Retry logic in handlers** — use `retry()` from resilience service
❌ **Direct provider config access** — use `providerRegistry` getters
❌ **Hardcoded fallback chains** — define in `modelFamilyFallback.ts`
❌ **State mutations across concurrent requests** — use request-scoped context only
---
## Adding a New Component
### Adding a New Service
1. Create `open-sse/services/[serviceName].ts` with focused responsibility
2. Export main handler function and any constants
3. Add unit tests in `tests/unit/services/[serviceName].test.mjs`
4. Integrate into request pipeline in `handlers/chatCore.ts` (if routing-related)
5. Update routing logic in `combo.ts` if service affects target selection
6. Document in this file
### Adding a New Executor
1. Create `open-sse/executors/[provider].ts` extending `BaseExecutor`
2. Register in `config/providerRegistry.ts`
3. Add to `executors/index.ts` factory
4. Add unit tests for the executor
5. Document in `docs/architecture/ARCHITECTURE.md`
### Adding a New MCP Tool
1. Create or update `open-sse/mcp-server/tools/[category]Tools.ts`
2. Define Zod schema for inputs
3. Register tool in `mcp-server/index.ts`
4. Add to scope matrix in `mcp-server/auth/`
5. Add unit tests
---
## See Also
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — high-level architecture
- [CODEBASE_DOCUMENTATION.md](../architecture/CODEBASE_DOCUMENTATION.md) — engineering reference
- [REPOSITORY_MAP.md](../architecture/REPOSITORY_MAP.md) — directory-by-directory
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — 9-factor scoring
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP server
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A server
- Source: `open-sse/` (400+ files, ~143K LOC)
docs/frameworks/PLAYGROUND_STUDIO.md
+212
View File
@@ -0,0 +1,212 @@
---
title: "Playground Studio"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Playground Studio
> **Feature:** Playground Studio — unified AI testing workspace for `/dashboard/playground`.
> **Plans:** `17-playground-studio-redesign.plan.md` + `_orchestration/master-plan-group-C.md`
> **Status:** Released in v3.8.6
---
## Overview
Playground Studio transforms `/dashboard/playground` from a simple Monaco-based editor into
a full-featured testing workspace. It replaces the legacy `page.tsx` with a `PlaygroundStudio`
shell that renders four tabs and a shared config pane.
```
┌ Playground ──────────────────────────────────────────────────────────┐
│ [💬 Chat] [⚖ Compare] [{} API] [🔧 Build] 142↑ 38↓ · $0.002 </>│
├──────────────────────────────────────────┬───────────────────────────┤
│ {active tab content} │ ─ Config │
│ │ Endpoint [chat ∨] │
│ │ Model [gpt-5.4 ∨] │
│ │ System [textarea] │
│ │ Temp ▕▕▔▔ 0.7 │
│ │ Presets [▾ load][save] │
│ │ [✨ Improve prompt] │
└──────────────────────────────────────────┴───────────────────────────┘
```
---
## Tabs
### Chat Tab
Evolves `ChatPlayground.tsx` into a multi-turn streaming workbench:
- Full markdown rendering via `MarkdownMessage.tsx` (code blocks, tables, lists, links).
- System prompt sourced from the shared Config pane.
- Token/cost per message (prompt + completion tokens).
- Regenerate last response.
- Sends to `POST /v1/chat/completions` with SSE streaming.
### Compare Tab
The key differentiator for a proxy: run 1 prompt across up to **4 models in parallel**.
- Up to 4 columns, each independently streaming from `/v1/chat/completions`.
- `+ Add model` button (Cmd+K shortcut) to add columns.
- `Run all ▶` triggers all streams simultaneously via `Promise.all` + per-column `AbortController`.
- Global **Cancel all** aborts every in-flight stream.
- Per-column `ProviderMetrics` shows TTFT, TPS, tokens, and estimated cost in real time.
- Metrics labelled **"client-side estimate"** (D12) — measured from first SSE chunk.
### API Tab
Preserves 100% of the original Monaco editor for power users (D14):
- 10 endpoints: chat completions, completions, embeddings, images, audio, speech, transcriptions, moderations, rerank, search.
- Multimodal file upload.
- SSE streaming with real-time output.
- Wrapped as `ApiTab.tsx` (lazy-loaded, `ssr: false`).
### Build Tab
Tools/function calling and structured output UI:
- `ToolsBuilder.tsx` — add/edit/remove `tools[]` with JSON schema editor per tool.
Validates parameters via `ToolDefinitionSchema` (Zod).
- `StructuredOutputEditor.tsx` — toggle JSON mode + JSON schema editor.
Validates response against schema via `StructuredOutputSchema` (Zod).
- Sends request to `/v1/chat/completions` with `tools[]` and/or `response_format`.
---
## Config Pane (Shared)
`StudioConfigPane.tsx` — always visible, collapsible.
| Field | Component | Notes |
| -------------- | --------------------- | ---------------------------------------------------------------------- |
| Endpoint | `<select>` | 10 options matching `PlaygroundEndpoint` |
| Model | `<input>` | free text, e.g. `openai/gpt-4o` |
| System prompt | `<textarea>` | fed into all tabs |
| Parameters | `ParamSliders` | temperature, max_tokens, top_p, presence/frequency penalty, seed, stop |
| Presets | `PresetPicker` | load/save named config snapshots (persisted in DB) |
| Improve prompt | `ImprovePromptButton` | opens quota-warning modal, calls `/api/playground/improve-prompt` |
State is lifted to `PlaygroundStudio.tsx` and passed down to all tabs. Switching tabs
preserves config state.
---
## Top Bar
`StudioTopBar.tsx`:
- Tab switcher (role="tablist").
- `TokenCostCounter` — live token (↑/↓) and estimated cost display.
- Export code button (`</>`) — opens `ExportCodeModal`.
---
## Export Code Modal
`ExportCodeModal.tsx` uses `codeExport.ts` to generate curl / Python / TypeScript snippets
from the current `PlaygroundState`. API key placeholder is always `$OMNIROUTE_API_KEY` (D11).
---
## Prompt Improver
`ImprovePromptButton.tsx` → `useImprovePrompt.ts` → `POST /api/playground/improve-prompt`:
1. Modal warns "will consume quota".
2. On confirm, sends `{ system, prompt, model, tone }` to the route.
3. Route calls `/v1/chat/completions` internally with `promptImprover.META_SYSTEM_PROMPT`.
4. Returns `{ improvedSystem?, improvedPrompt?, tokensIn, tokensOut }`.
5. UI patches the Config pane system prompt and the Chat tab user prompt.
---
## Presets
`PresetPicker.tsx` → `usePresets.ts` → `/api/playground/presets/*`:
- Stored in `playground_presets` SQLite table (migration `084_playground_presets.sql`).
- Each preset stores: `name`, `endpoint`, `model`, `system`, `params_json`, `created_at`.
- CRUD: `GET` list, `POST` create, `GET /:id`, `PUT /:id`, `DELETE /:id`.
---
## Stream Metrics
`useStreamMetrics.ts` + `streamMetrics.ts` (pure function):
- `start()` — records request start time.
- `onFirstChunk()` — records TTFT.
- `onChunk(n)` — accumulates completion token count.
- `finish(usage?)` — computes final metrics: `ttftMs`, `totalMs`, `tps`, `tokensIn`, `tokensOut`, `costUsd`.
- Pricing from static table in `src/lib/playground/types.ts` (labelled "estimated" — D13).
---
## Backend Routes
| Method | Path | Handler |
| -------- | -------------------------------- | ----------------------------------------------------------------------------------------- |
| `POST` | `/api/playground/improve-prompt` | Zod-validates `ImprovePromptRequestSchema`; calls `/v1/chat/completions` with meta-prompt |
| `GET` | `/api/playground/presets` | Returns `{ presets: PlaygroundPresetListItem[] }` |
| `POST` | `/api/playground/presets` | Creates preset; validates `PlaygroundPresetCreateSchema` |
| `GET` | `/api/playground/presets/:id` | Returns one preset or 404 |
| `PUT` | `/api/playground/presets/:id` | Partial update |
| `DELETE` | `/api/playground/presets/:id` | 204 |
Auth: optional (`REQUIRE_API_KEY`). Errors via `buildErrorBody()` (Hard Rule #12).
---
## Key Files
| Path | Purpose |
| -------------------------------------------------------------------------- | --------------------------------------------------- |
| `src/app/(dashboard)/dashboard/playground/PlaygroundStudio.tsx` | Shell component, tab orchestrator |
| `src/app/(dashboard)/dashboard/playground/components/StudioTopBar.tsx` | Tabs + counter + export button |
| `src/app/(dashboard)/dashboard/playground/components/StudioConfigPane.tsx` | Shared config panel |
| `src/app/(dashboard)/dashboard/playground/components/tabs/ChatTab.tsx` | Chat workbench |
| `src/app/(dashboard)/dashboard/playground/components/tabs/CompareTab.tsx` | Multi-model compare |
| `src/app/(dashboard)/dashboard/playground/components/tabs/ApiTab.tsx` | Monaco editor (preserved) |
| `src/app/(dashboard)/dashboard/playground/components/tabs/BuildTab.tsx` | Tools + structured output |
| `src/app/(dashboard)/dashboard/playground/components/ExportCodeModal.tsx` | Code export modal |
| `src/app/(dashboard)/dashboard/playground/components/CompareColumn.tsx` | Single compare column |
| `src/app/(dashboard)/dashboard/playground/components/ProviderMetrics.tsx` | TTFT/TPS display |
| `src/app/(dashboard)/dashboard/playground/hooks/useStreamMetrics.ts` | Client-side metric hook |
| `src/app/(dashboard)/dashboard/playground/hooks/usePresets.ts` | Presets CRUD hook |
| `src/app/(dashboard)/dashboard/playground/hooks/useImprovePrompt.ts` | Improve-prompt hook |
| `src/lib/playground/codeExport.ts` | curl/Python/TS generator (shared with Search Tools) |
| `src/lib/playground/promptImprover.ts` | Meta-prompt builder |
| `src/lib/playground/streamMetrics.ts` | Pure metrics computation |
| `src/lib/db/playgroundPresets.ts` | DB module (CRUD) |
| `src/app/api/playground/improve-prompt/route.ts` | Improve-prompt REST route |
| `src/app/api/playground/presets/route.ts` | Presets list + create |
| `src/app/api/playground/presets/[id]/route.ts` | Presets get/update/delete |
| `src/lib/db/migrations/084_playground_presets.sql` | DB migration |
---
## Troubleshooting
| Symptom | Cause | Fix |
| -------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------- |
| Monaco editor not rendering in API tab | SSR loaded Monaco | Verify `ApiTab` uses `dynamic(..., { ssr: false })` |
| Compare streams fire sequentially | Wrong `Promise.all` usage | All stream starts must be dispatched in one `Promise.all` call |
| Metrics show `null` TTFT | First chunk handler not wired | Check `useStreamMetrics.onFirstChunk()` is called in the SSE reader loop |
| Preset not persisting | DB migration not run | Run `npm run db:migrate` or restart the server (migration auto-runs on startup) |
| Improve prompt returns 502 | Model not set in Config | User must enter a model name in the Config pane before improving |
| Export code shows `MISSING_API_KEY` | Placeholder not inserted | `codeExport.ts` always uses `API_KEY_PLACEHOLDER = "$OMNIROUTE_API_KEY"` |
---
## References
- Master plan: `_tasks/features-v3.8.6/refactorpages/_orchestration/master-plan-group-C.md`
- Feature plan: `_tasks/features-v3.8.6/refactorpages/17-playground-studio-redesign.plan.md`
- Code export: `src/lib/playground/codeExport.ts`
- Prompt improver: `src/lib/playground/promptImprover.ts`
- Search Tools Studio: `docs/frameworks/SEARCH_TOOLS_STUDIO.md`
docs/frameworks/PLUGINS.md
+114
View File
@@ -0,0 +1,114 @@
---
title: "OmniRoute CLI Plugin System"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute CLI Plugin System
Extend the `omniroute` CLI without modifying its core. Plugins follow the `omniroute-cmd-*` naming convention, similar to `gh extension` or `kubectl plugin`.
## Quick start
```bash
# Install a plugin from npm
omniroute plugin install stripe
# Install a local plugin in development
omniroute plugin install ./my-plugin
# List installed plugins
omniroute plugin list
# Scaffold a new plugin
omniroute plugin scaffold myplugin
cd omniroute-cmd-myplugin
omniroute plugin install .
```
## Plugin anatomy
A plugin is an npm package named `omniroute-cmd-<name>` (or `@scope/omniroute-cmd-<name>`).
```
omniroute-cmd-myplugin/
├── package.json # must have "type": "module" and "main": "index.mjs"
├── index.mjs # exports register(program, ctx) + optional meta
└── README.md
```
### `package.json`
```json
{
"name": "omniroute-cmd-myplugin",
"version": "0.1.0",
"type": "module",
"main": "index.mjs",
"engines": { "omniroute": ">=4.0.0" },
"keywords": ["omniroute-plugin", "omniroute-cmd"]
}
```
### `index.mjs`
```js
export const meta = {
name: "myplugin",
version: "0.1.0",
description: "My plugin for OmniRoute",
omnirouteApi: ">=4.0.0",
};
export function register(program, ctx) {
program
.command("myplugin")
.description(meta.description)
.option("-n, --name <name>")
.action(async (opts, cmd) => {
const gOpts = cmd.optsWithGlobals();
const res = await ctx.apiFetch("/api/combos", {
baseUrl: gOpts.baseUrl,
apiKey: gOpts.apiKey,
});
const data = await res.json();
ctx.emit(data, gOpts);
});
}
```
## Plugin context API
The `ctx` object passed to `register(program, ctx)`:
| Property | Type | Description |
| ---------------------------- | ---------------- | -------------------------------------------------- |
| `ctx.apiFetch(path, opts)` | `async function` | Authenticated fetch to the OmniRoute server |
| `ctx.emit(data, opts)` | `function` | Output in table/json/jsonl/csv per `--output` flag |
| `ctx.t(key)` | `async function` | i18n translation lookup |
| `ctx.withSpinner(label, fn)` | `async function` | Wraps async fn with ora spinner |
| `ctx.baseUrl` | `string` | Resolved base URL |
| `ctx.apiKey` | `string \| null` | API key if provided |
## Discovery
Plugins are discovered from:
1. `~/.omniroute/plugins/<name>/` — user-local installs
2. `OMNIROUTE_PLUGIN_PATH` env var — custom directory
Loading errors are caught and printed as warnings — a broken plugin never crashes the CLI.
## Security
Plugins run with the same Node.js process privileges as `omniroute`. Only install plugins from sources you trust. `omniroute plugin install` shows an explicit warning and requires `--yes` or interactive confirmation.
## Publishing
1. Ensure `package.json` has `"keywords": ["omniroute-plugin"]`
2. `npm publish` as normal
3. Users discover via `omniroute plugin search <query>` (searches npm registry)
## Example plugin
See [`examples/omniroute-cmd-hello/`](../../examples/omniroute-cmd-hello/index.mjs) for a minimal working example with `meta` + `register()`.
docs/frameworks/PLUGIN_MARKETPLACE.md
+348
View File
@@ -0,0 +1,348 @@
---
title: "Plugin Marketplace"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Plugin Marketplace
> **Source of truth:** `src/lib/plugins/` (`marketplace.ts`, `manager.ts`, `manifest.ts`,
> `scanner.ts`, `loader.ts`), `src/app/api/plugins/`, and
> `src/app/(dashboard)/dashboard/plugins/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute ships a WordPress-style plugin system. Plugins are self-contained
directories — each with a `plugin.json` manifest and an entry file — that hook
into the request pipeline (`onRequest` / `onResponse` / `onError`) and into
lifecycle events (`onInstall` / `onActivate` / `onDeactivate` / `onUninstall`).
The **Plugin Marketplace** is the discovery layer on top of that system. It
exposes a browsable catalog of installable plugins. By default the catalog is a
small built-in seed registry; an operator can point it at a custom remote
registry URL, in which case the fetch is hardened by a DNS-resolving SSRF guard
(see [Security](#security)).
Every plugin route is **loopback-only** (Tier 1 — `LOCAL_ONLY`): plugins load
and execute code in child processes, so the routes are unreachable from a
non-loopback origin regardless of auth. See
[`docs/security/ROUTE_GUARD_TIERS.md`](../security/ROUTE_GUARD_TIERS.md).
## How It Fits Together
```
Dashboard (/dashboard/plugins)
├─ "Installed" tab → GET /api/plugins (listPlugins)
│ POST /api/plugins/scan (pluginManager.scan)
│ POST /api/plugins/{name}/activate|deactivate
│ DELETE /api/plugins/{name} (uninstall)
└─ "Marketplace" tab → GET /api/plugins/marketplace
→ listMarketplacePlugins()
├─ no custom URL → built-in SEED_REGISTRY
└─ custom URL → isSafeMarketplaceUrl() SSRF guard
→ safeOutboundFetch(guard:"public-only")
```
- **Registry layer** — `src/lib/plugins/marketplace.ts`: lists / searches the
catalog, falling back to the seed registry on any failure.
- **Lifecycle layer** — `src/lib/plugins/manager.ts` (`pluginManager` singleton):
install, upgrade, activate, deactivate, uninstall, scan, startup load.
- **Manifest layer** — `src/lib/plugins/manifest.ts`: Zod schema + defaults for
`plugin.json`.
- **Scanner** — `src/lib/plugins/scanner.ts`: discovers plugins on disk under
the plugin directory.
- **Loader** — `src/lib/plugins/loader.ts`: spawns each plugin in an isolated
child process and brokers hook calls over IPC.
## Marketplace Catalog
`listMarketplacePlugins()` (`src/lib/plugins/marketplace.ts`) returns a list of
`MarketplaceEntry` objects:
| Field | Type | Notes |
| ------------- | -------- | ------------------------------------ |
| `name` | string | kebab-case plugin name |
| `version` | string | semver |
| `description` | string | Short summary |
| `author` | string | Author / org |
| `license` | string | SPDX-style license id |
| `downloadUrl` | string | Source download URL (may be empty) |
| `repository` | string? | Optional repository URL |
| `tags` | string[] | Search/filter tags |
| `downloads` | number | Download count |
| `rating` | number | 0–5 |
| `verified` | boolean | Whether the entry is marked verified |
| `lastUpdated` | string | ISO-ish date string |
When no custom registry URL is configured, the catalog is the built-in
`SEED_REGISTRY` (currently `request-logger`, `rate-limiter`, `cost-tracker`, and
`theme-manager`). The seed registry is always available — if a configured remote
registry is unreachable, returns a non-`200` status, or returns an unrecognized
body, `listMarketplacePlugins()` logs a warning and falls back to the seed list.
> Note: the marketplace **catalog** (browse/search) is wired end to end, but
> one-click marketplace **install** from the catalog is not yet implemented — the
> dashboard's "Install" button on a marketplace entry currently shows a
> "coming soon" notice. Installation today goes through the local-path install
> flow (`POST /api/plugins`) and on-disk discovery (`POST /api/plugins/scan`).
## REST API
All endpoints require management auth (`requireManagementAuth`) **and** are
loopback-only — `/api/plugins` and `/api/plugins/` are listed in
`LOCAL_ONLY_API_PREFIXES` (`src/server/authz/routeGuard.ts`).
| Endpoint | Method | Description |
| -------------------------------- | ------ | --------------------------------------------------- |
| `/api/plugins` | GET | List installed plugins (optional `?status=` filter) |
| `/api/plugins` | POST | Install a plugin from an absolute local path |
| `/api/plugins/scan` | POST | Scan the plugin directory and register new plugins |
| `/api/plugins/marketplace` | GET | List marketplace catalog entries |
| `/api/plugins/[name]` | GET | Get installed plugin details |
| `/api/plugins/[name]` | DELETE | Uninstall a plugin |
| `/api/plugins/[name]/activate` | POST | Activate (load + register hooks) |
| `/api/plugins/[name]/deactivate` | POST | Deactivate (fire `onDeactivate`, unregister hooks) |
| `/api/plugins/[name]/config` | GET | Get plugin config + config schema |
| `/api/plugins/[name]/config` | PUT | Update plugin config (validated against schema) |
The `GET /api/plugins` `status` filter accepts one of
`installed` / `active` / `inactive` / `error`. An invalid value returns `400`.
### List installed plugins
```bash
curl http://localhost:20128/api/plugins \
-H "Cookie: auth_token=..."
```
### Install from a local path
```bash
curl -X POST http://localhost:20128/api/plugins \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{ "path": "/absolute/path/to/my-plugin" }'
```
The `path` must be **absolute** and may not contain `..` traversal segments or
null bytes (enforced by Zod). The source directory must contain a valid
`plugin.json` (or be a parent of one). On success the response is `201` with the
installed plugin row.
### Browse the marketplace
```bash
curl http://localhost:20128/api/plugins/marketplace \
-H "Cookie: auth_token=..."
```
### Update plugin config
```bash
curl -X PUT http://localhost:20128/api/plugins/my-plugin/config \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{ "config": { "level": "debug", "maxItems": 100 } }'
```
`PUT .../config` validates each provided value against the plugin's
`configSchema` (declared in the manifest): `number` fields honor `min`/`max`,
`select` fields must match the declared `enum`. Keys not present in the schema
are allowed through.
## Configuration
### Plugin directory
Plugins live under the OmniRoute data directory:
```
~/.omniroute/plugins/<plugin-name>/
├─ plugin.json
└─ index.js # (or whatever manifest.main points to)
```
`getDefaultPluginDir()` (`src/lib/plugins/scanner.ts`) resolves this to
`<home>/.omniroute/plugins`, where `<home>` is taken from the `HOME` /
`USERPROFILE` environment variables. `POST /api/plugins/scan` discovers any
subdirectory there that holds a valid `plugin.json` and registers it.
### Custom marketplace registry URL
The marketplace catalog source is read from the `pluginMarketplaceUrl` setting
(`src/lib/plugins/marketplace.ts` reads `settings.pluginMarketplaceUrl`). When
set to an `http(s)` URL, `listMarketplacePlugins()` fetches that URL and accepts
either a top-level JSON array of entries or an object with a `plugins` array;
entries without a string `name` are filtered out. When unset (or when the fetch
fails the SSRF guard / returns a bad response), the built-in seed registry is
used.
The dashboard "Marketplace" tab exposes a field for this URL (read back from
`GET /api/settings`).
> Implementation note: the dashboard "Save" action sends
> `pluginMarketplaceUrl` to `PATCH /api/settings`. At the time of writing this
> key is not declared in `updateSettingsSchema`
> (`src/shared/validation/settingsSchemas.ts`), so verify persistence in your
> release before relying on it — the **read** path (`getSettings()` →
> `listMarketplacePlugins()`) honors the key once it is present in the settings
> store.
## Security
### Route tier — loopback only
Plugins execute code in spawned child processes, so the entire `/api/plugins`
surface is classified `LOCAL_ONLY` (Tier 1). Loopback enforcement runs
unconditionally **before** any auth check, so a leaked management token reaching
the box over a tunnel still cannot install, activate, or uninstall a plugin.
See [`docs/security/ROUTE_GUARD_TIERS.md`](../security/ROUTE_GUARD_TIERS.md) and
Hard Rules #15 / #17.
### Marketplace registry SSRF guard
A custom registry URL is attacker-influenceable configuration, so before
fetching it `listMarketplacePlugins()` runs it through two layers:
1. **`isSafeMarketplaceUrl(url)`** (`src/lib/plugins/marketplace.ts`):
- Rejects anything that is not `http:` / `https:`.
- Rejects literal private/loopback/link-local/ULA hosts (IPv4 **and** IPv6,
including IPv4-mapped) via the canonical `isPrivateHost`
(`src/shared/network/outboundUrlGuard.ts`).
- Resolves **both** `A` and `AAAA` records and rejects if **any** resolved
address is private — closing the public-hostname → private-IP bypass.
- **Fails closed**: a DNS resolution failure rejects the URL.
2. **`safeOutboundFetch(url, { guard: "public-only", timeoutMs: 5000 })`**
(`src/shared/network/safeOutboundFetch.ts`): re-applies the public-only URL
guard at fetch time and **blocks redirects** (no public → private `30x`
pivot).
A URL that fails either layer does not abort the request — the marketplace
silently falls back to the built-in seed registry and logs a warning.
> This guard was hardened in PR #3774 specifically to resolve A + AAAA and use
> the canonical `isPrivateHost` instead of an IPv4-only check.
### Plugin execution isolation
- **Process isolation** — `loadPlugin()` (`src/lib/plugins/loader.ts`) spawns
each plugin in a separate Node.js child process and communicates over IPC.
Hook calls have a timeout with `SIGTERM` → `SIGKILL` escalation.
- **Env allowlist** — the child receives only an allowlisted set of environment
variables; the broader set is only granted when the manifest requests the
`env` permission.
- **Path containment** — install/upgrade/uninstall assert that the plugin
directory and `manifest.main` resolve **within** the managed plugin root
before any copy or recursive delete (guards against tampered DB paths and
`../` traversal in `manifest.main`). Activation resolves symlinks via
`realpath` and refuses to load an entry point that escapes the plugin
directory.
- **Optional integrity pin** — a manifest may declare an `integrity`
(`sha256-<base64>`, SRI format) field. When present, the loader verifies the
entry file hash at load time and refuses to activate on mismatch. It is
opt-in tamper-detection, **not** a security boundary — loopback-only routing
and the permission model are the real boundaries.
## Manifest (`plugin.json`)
Validated by `PluginManifestSchema` (`src/lib/plugins/manifest.ts`):
| Field | Type | Notes |
| ------------------ | --------- | ----------------------------------------------------------- |
| `name` | string | Required; kebab-case (`^[a-z0-9-]+$`), 1–100 chars |
| `version` | string | Required; semver (`MAJOR.MINOR.PATCH`) |
| `description` | string? | ≤ 500 chars |
| `author` | string? | ≤ 200 chars |
| `license` | string? | Defaults to `MIT` |
| `main` | string? | Entry file; defaults to `index.js` |
| `source` | enum? | `local` \| `marketplace` (defaults to `local`) |
| `tags` | string[]? | Search tags |
| `requires` | object? | `{ omniroute?, permissions[] }` |
| `hooks` | object? | Booleans declaring which hooks the plugin implements |
| `skills` | object[]? | Optional skill definitions |
| `enabledByDefault` | boolean? | Auto-activate on install |
| `configSchema` | object? | Map of config fields (`string`/`number`/`boolean`/`select`) |
| `integrity` | string? | Optional `sha256-<base64>` entry-file pin |
Permissions are drawn from the enum
`network` / `file-read` / `file-write` / `env` / `exec`.
## Lifecycle Flow
```
install (POST /api/plugins, path)
→ scan/validate manifest → copy to staging → assert main within dir
→ atomic rename into ~/.omniroute/plugins/<name> → insert DB row
→ fire onInstall → if enabledByDefault: activate
activate (POST /api/plugins/{name}/activate)
→ realpath containment check → loadPlugin() (spawn child process)
→ register declared hooks → status = "active" → fire onActivate
deactivate (POST /api/plugins/{name}/deactivate)
→ fire onDeactivate (BEFORE unregister) → unregister hooks
→ kill child process → status = "inactive"
uninstall (DELETE /api/plugins/{name})
→ deactivate if active → fire onUninstall
→ containment-checked recursive delete of plugin dir → delete DB row
```
Re-running `install` against a directory whose manifest version is **strictly
newer** than the installed version auto-upgrades (clean reinstall; config resets
to defaults). A same-or-older version is rejected.
## Database
Table `plugins` (migration `076_create_plugins.sql`):
| Column | Type | Notes |
| --------------- | ------- | ------------------------------------------------ |
| `id` | TEXT PK | UUID |
| `name` | TEXT | Unique |
| `version` | TEXT | semver; default `1.0.0` |
| `description` | TEXT | Optional |
| `author` | TEXT | Optional |
| `license` | TEXT | Default `MIT` |
| `main` | TEXT | Entry file; default `index.js` |
| `source` | TEXT | Default `local` |
| `tags` | TEXT | JSON array; default `[]` |
| `status` | TEXT | `installed` \| `active` \| `inactive` \| `error` |
| `enabled` | INT | 0/1; default 0 |
| `manifest` | TEXT | Full manifest JSON |
| `config` | TEXT | JSON; default `{}` |
| `config_schema` | TEXT | JSON; default `{}` |
| `hooks` | TEXT | JSON array of declared hook names; default `[]` |
| `permissions` | TEXT | JSON array; default `[]` |
| `plugin_dir` | TEXT | Absolute install directory |
| `error_message` | TEXT | Set when `status = "error"` |
| `installed_at` | TEXT | `datetime('now')` |
| `updated_at` | TEXT | `datetime('now')` |
| `activated_at` | TEXT | Set on activation |
Plugin metrics/analytics are tracked in additional tables
(`090_plugin_metrics.sql`, `091_plugin_analytics.sql`).
## Dashboard
The dashboard page at `/dashboard/plugins`
(`src/app/(dashboard)/dashboard/plugins/page.tsx`) provides two tabs:
- **Installed** — lists installed plugins with their declared hooks, an
activate/deactivate toggle, an uninstall button, and a "Scan for plugins"
action (`POST /api/plugins/scan`).
- **Marketplace** — shows the catalog from `GET /api/plugins/marketplace` with a
field to set the custom registry URL.
A per-plugin config page lives at `/dashboard/plugins/[name]/config`
(`src/app/(dashboard)/dashboard/plugins/[name]/config/page.tsx`).
## See Also
- [`docs/security/ROUTE_GUARD_TIERS.md`](../security/ROUTE_GUARD_TIERS.md) —
why `/api/plugins` is loopback-only (Tier 1)
- [`docs/frameworks/SKILLS.md`](./SKILLS.md) — the related skills framework
(`src/lib/skills/`); plugins may declare skills in their manifest
- [`docs/frameworks/WEBHOOKS.md`](./WEBHOOKS.md) — event-driven outbound
integrations
- [`docs/security/ERROR_SANITIZATION.md`](../security/ERROR_SANITIZATION.md) —
the `buildErrorBody()` pattern every plugin route uses for error responses
docs/frameworks/PLUGIN_SDK.md
+253
View File
@@ -0,0 +1,253 @@
---
title: "OmniRoute Plugin SDK"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute Plugin SDK
## Quick Start
```ts
import { definePlugin } from "omniroute/plugins/sdk";
export default definePlugin({
name: "my-plugin",
priority: 50,
onRequest: async (ctx) => {
console.log(`Request ${ctx.requestId} for ${ctx.model}`);
},
onResponse: async (ctx, response) => {
console.log(`Response for ${ctx.requestId}`);
return response;
},
onError: async (ctx, error) => {
console.error(`Error: ${error.message}`);
},
});
```
## API Reference
### `definePlugin(def: PluginDefinition): Plugin`
Factory function that creates a Plugin object with defaults.
**Parameters:**
- `name` (string, required) — Plugin name in kebab-case
- `priority` (number, optional, default: 100) — Lower runs first
- `enabled` (boolean, optional, default: true) — Start enabled?
- `onRequest` (function, optional) — Runs before chat handler
- `onResponse` (function, optional) — Runs after chat handler
- `onError` (function, optional) — Runs on handler error
### `blockRequest(response?): BlockingHookResult`
Block the request and optionally return a custom response.
```ts
onRequest: (ctx) => {
if (!ctx.headers["authorization"]) {
return blockRequest({ error: "Unauthorized", status: 401 });
}
};
```
### `modifyBody(body): PluginResult`
Modify the request body before it reaches the provider.
```ts
onRequest: (ctx) => {
return modifyBody({ ...ctx.body, temperature: 0.7 });
};
```
### `addMetadata(metadata): PluginResult`
Attach metadata to the request context.
```ts
onRequest: (ctx) => {
return addMetadata({ source: "my-plugin", version: "1.0.0" });
};
```
## Plugin Context (`PluginContext`)
| Field | Type | Description |
| ----------- | ------------------------- | ------------------------- |
| `requestId` | `string` | Unique request identifier |
| `model` | `string` | Requested model name |
| `provider` | `string` | Target provider ID |
| `body` | `Record<string, unknown>` | Request body |
| `headers` | `Record<string, string>` | Request headers |
| `metadata` | `Record<string, unknown>` | Mutable metadata |
| `timestamp` | `number` | Request timestamp |
## Manifest (`plugin.json`)
```json
{
"name": "my-plugin",
"version": "1.0.0",
"description": "A sample plugin",
"author": "your-name",
"main": "index.js",
"hooks": {
"onRequest": { "enabled": true, "priority": 50 },
"onResponse": true,
"onError": false
},
"requires": {
"permissions": ["network", "file-read"]
},
"enabledByDefault": false,
"configSchema": {
"apiKey": { "type": "string", "description": "API key for external service" },
"maxRetries": { "type": "number", "min": 1, "max": 10, "default": 3 },
"debug": { "type": "boolean", "default": false },
"mode": { "type": "string", "enum": ["fast", "slow"], "default": "fast" }
}
}
```
### Hook Priority
Hooks can be configured with priority (lower = runs first):
```json
{
"hooks": {
"onRequest": { "enabled": true, "priority": 10 },
"onResponse": { "enabled": true, "priority": 100 }
}
}
```
Or as simple booleans (default priority 100):
```json
{
"hooks": {
"onRequest": true,
"onResponse": true
}
}
```
## Permission System
Plugins run in a sandboxed VM context. Access to external resources requires explicit permissions:
| Permission | Grants |
| ------------ | ------------------------------------------------------------ |
| `network` | `fetch`, `AbortController`, `Headers`, `Request`, `Response` |
| `file-read` | `fs.readFile`, `fs.readdir`, `fs.stat` |
| `file-write` | `fs.writeFile`, `fs.mkdir`, `fs.rm` |
| `env` | Read-only `process.env` proxy |
| `exec` | `child_process.exec`, `child_process.execSync` |
Without a permission, the corresponding globals are simply not available in the sandbox.
## Config Schema
Define configurable settings in `configSchema`:
```json
{
"configSchema": {
"apiKey": { "type": "string", "description": "External API key" },
"maxRetries": { "type": "number", "min": 1, "max": 10, "default": 3 },
"debug": { "type": "boolean", "default": false },
"mode": { "type": "string", "enum": ["fast", "slow"], "default": "fast" }
}
}
```
Field types: `string`, `number`, `boolean`, `select`
Field options: `default`, `min`, `max`, `enum`, `description`
Config values are persisted in the database and accessible via the dashboard config page.
## Built-in Events
| Event | When | Payload |
| ----------------- | ----------------------------------------- | ----------------------------- |
| `onRequest` | Before chat handler | Request context |
| `onResponse` | After chat handler | Response data |
| `onError` | On handler error | Error object |
| `onModelSelect` | Model selected for routing | Model info |
| `onComboResolve` | Combo routing resolved | Combo targets |
| `onRateLimit` | Rate limit hit | Limit info |
| `onQuotaExhaust` | Quota exhausted | Quota info |
| `onProviderError` | Provider returned error | Error details |
| `onStreamStart` | SSE stream started | Stream info |
| `onStreamEnd` | SSE stream ended | Stream stats |
| `onInstall` | Plugin installed | `{ name, version, manifest }` |
| `onActivate` | Plugin activated | `{ name, version, manifest }` |
| `onDeactivate` | Plugin deactivated | `{ name, version, manifest }` |
| `onUninstall` | Plugin uninstalled (before files deleted) | `{ name, version, manifest }` |
## Examples
### Request Logger
```ts
import { definePlugin } from "omniroute/plugins/sdk";
export default definePlugin({
name: "request-logger",
onRequest: async (ctx) => {
console.log(`[${new Date().toISOString()}] ${ctx.method} ${ctx.model} -> ${ctx.provider}`);
},
});
```
### Rate Limiter
```ts
import { definePlugin, blockRequest } from "omniroute/plugins/sdk";
const requests = new Map<string, number[]>();
export default definePlugin({
name: "rate-limiter",
priority: 10,
onRequest: async (ctx) => {
const key = ctx.headers["x-api-key"] || "anonymous";
const now = Date.now();
const window = 60000; // 1 minute
const maxRequests = 100;
const timestamps = (requests.get(key) || []).filter((t) => t > now - window);
timestamps.push(now);
requests.set(key, timestamps);
if (timestamps.length > maxRequests) {
return blockRequest({ error: "Rate limit exceeded", status: 429 });
}
},
});
```
### Response Transformer
```ts
import { definePlugin } from "omniroute/plugins/sdk";
export default definePlugin({
name: "response-transformer",
onResponse: async (ctx, response) => {
if (response.choices) {
response.choices = response.choices.map((c: any) => ({
...c,
message: { ...c.message, content: c.message.content.trim() },
}));
}
return response;
},
});
```
docs/frameworks/SEARCH_TOOLS_STUDIO.md
+183
View File
@@ -0,0 +1,183 @@
---
title: "Search Tools Studio"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Search Tools Studio
> **Feature:** Search Tools Studio — unified web tools workspace for `/dashboard/search-tools`.
> **Plans:** `18-search-tools-studio-redesign.plan.md` + `_orchestration/master-plan-group-C.md`
> **Status:** Released in v3.8.6
---
## Overview
Search Tools Studio transforms `/dashboard/search-tools` from a basic search playground into a
three-tab Studio unifying web search, web scraping, and side-by-side provider comparison.
```
┌ Search Tools ──────────────────────────────────────────────────────────┐
│ [🔍 Search] [📄 Scrape] [⚖ Compare] 142ms · $0.001 </> │
│ ⓘ [Modalities guide] │
├──────────────────────────────────────────┬─────────────────────────────┤
│ {active tab content} │ ─ Config │
│ │ Provider [auto ∨] │
│ │ 🟢 Serper $0.001 │
│ │ 🟢 Tavily $0.008 │
│ │ 🔥 Firecrawl (fetch) │
│ │ Type [web | news] │
│ │ Full page [ ] (scrape) │
│ │ Format [md|text|html] │
│ │ Rerank model [∨] │
└──────────────────────────────────────────┴─────────────────────────────┘
```
---
## Tabs
### Search Tab
Evolves the existing `SearchForm` + `ResultsPanel` + `RerankPanel` into a tab:
- Query → results (title, URL, snippet, relevance score).
- Provider metadata in Config pane (cost, quota, status).
- Rerank section: pick a rerank model, reorder results, show `positionDelta`.
- Empty state with CTA when no search providers are configured.
- Search history via `SearchHistory.tsx`.
- Calls `POST /v1/search` (existing endpoint, no changes).
### Scrape Tab
New tab for extracting content from a URL via `POST /v1/web/fetch` (created in plan 05):
- Input: URL + full-page toggle + format selector (markdown / text / HTML).
- Submit → fetch → render `ScrapeResult.tsx`.
- `ScrapeResult` renders markdown preview + raw toggle.
- Cap: if response body > **256 KB**, UI shows `(truncated, view raw)` and opens raw in a Monaco modal (D21).
- Metadata panel: provider (firecrawl/jina-reader/tavily-search/tinyfish), latency, cost, response size, links count.
- Uses `useScrapeFetch.ts` hook.
### Compare Tab
Runs the same query/URL across up to **4 providers in parallel** (D22):
- Side-by-side columns per provider.
- Metrics: latency, cost, result count, response size.
- URL overlap calculation for search (number of shared URLs vs initial result).
- Calls `POST /v1/search` (search) or `POST /v1/web/fetch` (scrape) per provider.
---
## Config Pane (Shared)
`SearchToolsConfigPane.tsx` — always visible, collapsible.
| Field | Notes |
| ------------ | ---------------------------------------------------------------- |
| Provider | Dropdown with status badge (configured / missing / rate limited) |
| Type | `web` or `news` (search only) |
| Full page | Toggle for scrape — fetches entire page vs first visible content |
| Format | `markdown`, `text`, or `html` (scrape only) |
| Rerank model | Optional model for post-search reranking |
| History | Collapsible search history section |
---
## SearchConceptCard
`SearchConceptCard.tsx` — always visible, collapsible accordion. Explains:
| Concept | One-liner |
| ------------------- | -------------------------------------------------------------------- |
| **Search** | Fetches a list of web results (title, URL, snippet, relevance score) |
| **Scrape** | Extracts the full content of a URL (markdown, text or HTML) |
| **Compare** | Runs the same query in N providers side-by-side |
| **Rerank** | Reorders results via LLM to improve query relevance |
| **Auto (cheapest)** | Picks the cheapest available provider automatically |
---
## Provider Catalog
`ProviderCatalog.tsx` exposes the full provider list from `GET /api/search/providers`
(extended in F4 to include fetch providers):
| Field | Source |
| ------------------------------ | ------------------------------------------------------------------------------------------ |
| `id`, `name` | `searchRegistry.ts` |
| `kind` | `"search"` (12 providers) or `"fetch"` (firecrawl, jina-reader, tavily-search, tinyfish) |
| `costPerQuery` | Registry data |
| `freeMonthlyQuota` | Registry data |
| `searchTypes` / `fetchFormats` | Registry data |
| `status` | `"configured"` / `"missing"` / `"rate_limited"` — derived at runtime from credential store |
| `configureHref` | `/dashboard/providers` |
The status is **derived at request time** by checking whether credentials exist and whether
all keys are currently in cooldown.
---
## Export Code
`ExportCodeModal` (imported from Playground Studio) + `codeExport.ts` generate
curl / Python / TypeScript snippets for both `/v1/search` and `/v1/web/fetch` calls.
API key placeholder is always `$OMNIROUTE_API_KEY` (D11, shared with Playground Studio).
---
## Backend Changes
Only one backend change was needed for this feature:
### Extended `GET /api/search/providers`
`src/app/api/search/providers/route.ts` was extended to:
- Include all 4 fetch providers (`firecrawl`, `jina-reader`, `tavily-search`, `tinyfish`) in the array.
- Add `kind: "search" | "fetch"` to every item.
- Add `status: "configured" | "missing" | "rate_limited"` derived from live credential state.
- Maintain backward compatibility — existing fields (`id`, `name`, etc.) unchanged.
---
## Key Files
| Path | Purpose |
| --------------------------------------------------------------------------------- | ------------------------------------------------- |
| `src/app/(dashboard)/dashboard/search-tools/SearchToolsClient.tsx` | Studio shell, tab orchestrator |
| `src/app/(dashboard)/dashboard/search-tools/components/SearchToolsTopBar.tsx` | Tabs + metrics + export button |
| `src/app/(dashboard)/dashboard/search-tools/components/SearchToolsConfigPane.tsx` | Shared config panel |
| `src/app/(dashboard)/dashboard/search-tools/components/SearchConceptCard.tsx` | Explainer cards (always visible) |
| `src/app/(dashboard)/dashboard/search-tools/components/ProviderCatalog.tsx` | Provider list with metadata |
| `src/app/(dashboard)/dashboard/search-tools/components/ScrapeResult.tsx` | Markdown preview + raw toggle |
| `src/app/(dashboard)/dashboard/search-tools/components/tabs/SearchTab.tsx` | Search + rerank tab |
| `src/app/(dashboard)/dashboard/search-tools/components/tabs/ScrapeTab.tsx` | Scrape tab |
| `src/app/(dashboard)/dashboard/search-tools/components/tabs/CompareTab.tsx` | Multi-provider compare tab |
| `src/app/(dashboard)/dashboard/search-tools/hooks/useScrapeFetch.ts` | Scrape fetch hook |
| `src/app/api/search/providers/route.ts` | Extended with `kind` + `status` + fetch providers |
| `open-sse/config/searchRegistry.ts` | Source of truth for search provider metadata |
---
## Troubleshooting
| Symptom | Cause | Fix |
| ----------------------------------------- | -------------------------- | ------------------------------------------------------------------------------- |
| Scrape tab shows "endpoint not available" | `/v1/web/fetch` not wired | Verify plan 05 is merged; check `src/app/api/v1/web/fetch/route.ts` exists |
| Provider catalog shows all as "missing" | Credentials not configured | Add credentials in `/dashboard/providers` |
| Scrape content is truncated | Response > 256 KB cap | Expected behavior (D21). Use "view raw" button for full content |
| Compare tab only shows 2 providers | Rate limit active | Two or more providers may be in cooldown — check provider status in Config pane |
| "Size" shown as raw key in table | Missing i18n key | Verify `search.size` exists in the locale file; rebuild i18n |
---
## References
- Master plan: `_tasks/features-v3.8.6/refactorpages/_orchestration/master-plan-group-C.md`
- Feature plan: `_tasks/features-v3.8.6/refactorpages/18-search-tools-studio-redesign.plan.md`
- Search provider registry: `open-sse/config/searchRegistry.ts`
- Playground Studio (shared `ExportCodeModal` + `codeExport.ts`): `docs/frameworks/PLAYGROUND_STUDIO.md`
- Web fetch backend: `src/app/api/v1/web/fetch/route.ts`
docs/frameworks/SKILLS.md
+487
View File
@@ -0,0 +1,487 @@
---
title: "Skills Framework"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Skills Framework
> **Source of truth:** `src/lib/skills/` and `src/app/api/skills/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute exposes an extensible Skills framework that lets language models (and operators) compose reusable capabilities — from filesystem reads and HTTP requests to sandboxed code execution and curated marketplace skills.
A skill is a versioned, schema-defined unit of work. OmniRoute can inject skills as tool definitions into outbound requests, intercept tool calls coming back from the model, run the matching handler, and feed the result back to the model so the conversation can continue. The model never sees the implementation — only the tool interface.
---
## Agent Skills vs Omni Skills
OmniRoute has two distinct but complementary skill systems:
| Dimension | **Omni Skills** (this doc) | **Agent Skills** |
| :-------------- | :------------------------------------------------------------ | :------------------------------------------------------------------------------------------ |
| Purpose | LLM tool injection + sandboxed execution | SKILL.md catalog for external agents to discover and consume |
| Source of truth | `src/lib/skills/` + marketplace | `src/lib/agentSkills/` + `skills/` directory |
| Runtime mode | Injected into outbound requests, executed on tool-call events | Static markdown catalog + REST/MCP/A2A discovery endpoints |
| Who uses it | OmniRoute itself (combo routing, inbound LLM calls) | External agents, MCP clients, A2A orchestrators |
| Count | Variable (marketplace-driven) | 42 canonical entries (22 API + 20 CLI) |
| Format | `SkillDefinition` with tool schema + handler | `SKILL.md` frontmatter + markdown body |
| Discovery | `/api/skills/*` REST + `omniroute_skills_*` MCP tools | `/api/agent-skills/*` REST + `omniroute_agent_skills_*` MCP tools + A2A `list-capabilities` |
**Omni Skills** are the execution engine — they define what OmniRoute _can do_ when an LLM invokes a tool.
**Agent Skills** are the documentation catalog — they explain to external agents _how to use_ OmniRoute's REST API and CLI, with structured SKILL.md files that can be fed directly into agent prompts.
For the Agent Skills catalog, generator, MCP tools, and A2A skill, see [docs/frameworks/AGENT-SKILLS.md](./AGENT-SKILLS.md).
---
## Concepts
### Skill Sources
Three sources of skills coexist in the same registry:
1. **Built-in skills** (`src/lib/skills/builtins.ts`) — shipped with OmniRoute. Cover the common cases:
- `file_read`, `file_write` — per-API-key sandbox workspace under `<DATA_DIR>/skills/workspaces/<hashed-key>/`
- `http_request` — outbound HTTP through `safeOutboundFetch` with `guard: "public-only"`
- `web_search` — pluggable search provider with caching (`executeWebSearch`)
- `eval_code` — Docker-sandboxed `node` or `python` execution
- `execute_command` — Docker-sandboxed shell command
- `browser` — Playwright-backed scaffolding, disabled by default (`builtin/browser.ts`)
2. **SkillsMP** (the OmniRoute Marketplace) — fetched from `https://skillsmp.com/api/v1/skills/search`. Requires `skillsmpApiKey` in Settings.
3. **SkillsSH** (`skills.sh` community catalog) — fetched from `https://skills.sh/api/search`. No auth needed; SKILL.md content pulled from GitHub raw.
A single "active provider" controls which catalog the dashboard installs from (`src/lib/skills/providerSettings.ts`). Switch it under **Settings → Memory & Skills**. Default: `skillsmp`.
### Skill Identity
Skills are keyed by `name@version` in the in-memory registry (`src/lib/skills/registry.ts`). Version must be semver (`^\d+\.\d+\.\d+$`). `resolveVersion()` understands `^`, `~`, `>`, `>=`, `<`, `<=`, `==`, and exact-match constraints.
### Skill Mode
Each skill has a runtime mode that controls when it is injected:
| Mode | Behavior |
| ------ | ------------------------------------------------------------------------------------------ |
| `on` | Always injected as a tool definition |
| `off` | Never injected, never executable |
| `auto` | Scored against the incoming request; injected only if score ≥ `AUTO_MIN_SCORE` (default 3) |
`auto` is the default for marketplace-installed skills. `enabled=true` and `mode="off"` together mean "registered but inactive" — toggling `enabled` via the legacy column also bumps `mode` so older codepaths stay consistent (`src/app/api/skills/[id]/route.ts`).
### Status (executions)
Skill executions are tracked in the `skill_executions` table with the following statuses (`src/lib/skills/types.ts`):
```ts
enum SkillStatus {
PENDING = "pending",
RUNNING = "running",
SUCCESS = "success",
ERROR = "error",
TIMEOUT = "timeout",
}
```
### Registry Cache
`SkillRegistry` is a singleton with a 60-second TTL cache (`registry.ts:14`). `loadFromDatabase()` is idempotent and dedupes concurrent calls via `pendingLoad`. Any write (`register`/`unregister`/`unregisterById`) invalidates the cache. Look up versions via `getSkillVersions(name)` and `resolveVersion(name, constraint)`.
### Provider-Aware Injection
`injectSkills()` in `src/lib/skills/injection.ts` is the entry point that turns registered skills into provider-specific tool definitions:
- **OpenAI** — `{ type: "function", function: { name, description, parameters } }`
- **Anthropic** — `{ name, description, input_schema }`
- **Google (Gemini)** — `{ name, description, parameters }`
The tool name is encoded as `name@version` so the handler can pick the right version when the model calls it back.
### AUTO Scoring
When `mode="auto"`, each candidate skill is scored against the request context (`scoreAutoSkill()` in `injection.ts`):
| Signal | Points |
| ---------------------------------------------- | ------------ |
| Skill name appears verbatim in context | +6 |
| Each name token matches a context token | +2 |
| Each tag substring matches context | +3 |
| Each description token matches context | +1 |
| Background reason matches a name token | +2 per token |
| Background reason matches a tag | +2 per token |
| Provider hint in tags matches request provider | +2 / −2 |
Top `AUTO_MAX_SKILLS = 5` skills with `score >= AUTO_MIN_SCORE = 3` are injected. Ties are broken by `installCount` (desc), then alphabetical name (`injection.ts:225-235`).
### Tool Call Interception
`handleToolCallExecution()` in `src/lib/skills/interception.ts` is invoked by the chat handler after the upstream returns a tool-calling response:
1. `extractToolCalls()` reads provider-specific shapes (OpenAI `tool_calls` / Responses `function_call`, Anthropic `tool_use`, Gemini `functionCalls`).
2. Built-in tool aliases (e.g. `omniroute_web_search` → `web_search`) are resolved first. Built-in handlers run inline.
3. Anything else routes through `skillExecutor.execute(name@version, args, { apiKeyId, sessionId })`.
4. Results are spliced back into the response — `tool_results`, `function_call_output` items, or Anthropic `tool_result` blocks as appropriate.
`customSkillExecutionEnabled` in the execution context can be set to `false` to allow only built-in interception (used by request paths that explicitly disable user-defined handlers).
---
## Docker Sandbox
Non-builtin code paths (`eval_code`, `execute_command`) run inside Docker via `SandboxRunner` (`src/lib/skills/sandbox.ts`). Every container is launched with:
```
--rm --network none|bridge --cap-drop ALL
--security-opt no-new-privileges --pids-limit 100
--cpus <cpuLimit/1000> --memory <memoryLimit>m
--tmpfs /tmp:rw,noexec,nosuid,size=64m
--tmpfs /workspace:rw,noexec,nosuid,size=64m
--read-only (when readOnly=true)
```
Defaults (`SandboxRunner.DEFAULT_CONFIG`):
| Field | Default | Notes |
| ---------------- | --------------- | ---------------------------------------------------- |
| `cpuLimit` | 100 (= 0.1 CPU) | Divided by 1000 before passing to `--cpus` |
| `memoryLimit` | 256 MB | Hard limit |
| `timeout` | 30000 ms | Soft kill via `SIGTERM` + `docker kill` |
| `networkEnabled` | `false` | Becomes `--network none` |
| `readOnly` | `true` | Root FS read-only; `/tmp` and `/workspace` are tmpfs |
`SandboxRunner.kill(id)` and `killAll()` are exposed for shutdown; running containers are tracked in `runningContainers: Map<string, ChildProcess>`.
### Sandbox Env Vars
Configured via `process.env` in `src/lib/skills/builtins.ts`:
| Env Var | Default | Purpose |
| --------------------------------- | ---------------- | ------------------------------------------------------------------ |
| `SKILLS_MAX_FILE_BYTES` | `1048576` (1 MB) | Cap for `file_read` and `file_write` |
| `SKILLS_MAX_HTTP_RESPONSE_BYTES` | `256000` | Cap for `http_request` response body |
| `SKILLS_MAX_SANDBOX_OUTPUT_CHARS` | `100000` | Cap for stdout/stderr returned to the caller |
| `SKILLS_SANDBOX_TIMEOUT_MS` | `10000` | Default timeout for sandboxed commands; capped at 60 s |
| `SKILLS_SANDBOX_NETWORK_ENABLED` | `false` | Master gate for egress. Set `1` or `true` to allow per-call opt-in |
| `SKILLS_ALLOWED_SANDBOX_IMAGES` | (see below) | Comma-separated allowlist of Docker images |
Default allowed images: `alpine:3.20`, `node:22-alpine`, `python:3.12-alpine`. Any additions via `SKILLS_ALLOWED_SANDBOX_IMAGES` are merged with the defaults; unknown images are rejected by `normalizeImage()`.
> Note: there is no separate `SKILLS_EXECUTION_TIMEOUT_MS` env var. The non-sandbox handler timeout is hard-coded to 30 s in `SkillExecutor` (`executor.ts:13`) but can be overridden at runtime via `skillExecutor.setTimeout(ms)`.
### Workspace Isolation
`file_read` and `file_write` resolve every path relative to a per-API-key workspace at `<DATA_DIR>/skills/workspaces/<sha256(apiKeyId).slice(0,24)>/`. Path traversal (`..`) and forbidden segments (`.env`, `.git`, `.ssh`, `.omniroute`, `.codex`, `secrets`) are rejected before any disk I/O.
### HTTP Hardening
`http_request` (`builtins.ts:257`):
- Method allowlist: `GET, HEAD, POST, PUT, PATCH, DELETE`
- Blocked outbound headers: `host, connection, content-length, cookie, set-cookie, authorization, proxy-authorization`
- Redirects disabled (`allowRedirect: false`)
- Routed through `safeOutboundFetch` with `guard: "public-only"` (private/loopback ranges blocked)
- Response truncated at `SKILLS_MAX_HTTP_RESPONSE_BYTES`; client sees `truncated: true`
---
## Hybrid Executor (preview)
`src/lib/skills/hybrid.ts` defines a `HybridExecutor` that decides between `direct` (in-process) and `sandbox` execution per call, with an `autoUpgrade` retry path on timeout/memory errors. The wired-in `directExecutor` / `sandboxRunner` implementations are stubs (`executeDirect`, `executeInSandbox` return placeholder objects) — treat this module as a contract under construction. Real execution still goes through `skillExecutor` + `SandboxRunner`.
---
## Storage
Schema lives in two migrations:
- `src/lib/db/migrations/016_create_skills.sql` — base `skills` and `skill_executions` tables, with indexes on `(api_key_id, name)` and `(skill_id, status, created_at)`.
- `src/lib/db/migrations/027_skill_mode_and_metadata.sql` — adds `mode`, `source_provider`, `tags` (JSON), `install_count` to `skills`.
`skill_executions.status` is constrained at the database level: `CHECK(status IN ('pending', 'running', 'success', 'error', 'timeout'))`.
---
## REST API
All endpoints live under `src/app/api/skills/`. Management endpoints (`/api/skills`, `/api/skills/[id]`, `/api/skills/install`) require **management auth** via `requireManagementAuth()`. The marketplace/install flows use the lighter `isAuthenticated()` (session or API key).
| Endpoint | Method | Purpose |
| --------------------------------- | ------ | ------------------------------------------------------------------------ | --- | ------------------------ | -------- | ------------------ |
| `/api/skills` | GET | List registered skills. Supports `?q=`, `?mode=on | off | auto`, `?source=skillsmp | skillssh | local`, pagination |
| `/api/skills/[id]` | PUT | Update `enabled` or `mode` |
| `/api/skills/[id]` | DELETE | Unregister by id |
| `/api/skills/install` | POST | Install a custom skill (handler code + schema) |
| `/api/skills/marketplace` | GET | Search the SkillsMP catalog (returns popular defaults when `q` is empty) |
| `/api/skills/marketplace/install` | POST | Install a SkillsMP skill (requires active provider = `skillsmp`) |
| `/api/skills/skillssh` | GET | Search the skills.sh catalog (`?q=&limit=`, capped at 100) |
| `/api/skills/skillssh/install` | POST | Install a skills.sh skill (requires active provider = `skillssh`) |
| `/api/skills/executions` | GET | Paginated execution history (`?apiKeyId=`) |
| `/api/skills/executions` | POST | Execute a registered skill ad-hoc |
The `POST /api/skills/executions` endpoint returns HTTP `503` with `{ error: "Skills execution is disabled..." }` when `settings.skillsEnabled === false` (`executor.ts:42-45`). Operators can flip the master switch from **Settings → AI**.
### Example: install a custom skill
```bash
curl -X POST http://localhost:20128/api/skills/install \
-H "Authorization: Bearer $OMNIROUTE_MGMT_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "reverse-text",
"version": "1.0.0",
"description": "Reverses a string",
"schema": {
"input": { "type": "object", "properties": { "text": { "type": "string" } }, "required": ["text"] },
"output": { "type": "object", "properties": { "reversed": { "type": "string" } } }
},
"handlerCode": "echo-handler",
"apiKeyId": "your-api-key-id"
}'
```
The `handlerCode` string is a **handler name lookup** — not executable code. The executor maps it via `skillExecutor.registerHandler(name, fn)` (`executor.ts:25`). Marketplace installs store the SKILL.md text in this field as documentation and route execution through model-generated tool calls. Arbitrary user-supplied source is not eval'd.
---
## MCP Tools
Four MCP tools wrap the skills surface (`open-sse/mcp-server/tools/skillTools.ts`). They are auto-registered when the MCP server boots.
| Tool | Description |
| ----------------------------- | ------------------------------------------------------------ |
| `omniroute_skills_list` | List skills, optional filters: `apiKeyId`, `name`, `enabled` |
| `omniroute_skills_enable` | Enable/disable a skill by `skillId` |
| `omniroute_skills_execute` | Execute a skill with an input payload |
| `omniroute_skills_executions` | Recent execution history (default 50, max 100) |
See [MCP-SERVER.md](./MCP-SERVER.md) for transport setup and scope assignments.
---
## A2A Integration
`src/lib/skills/a2a.ts` exports the `memory_aware_routing` A2A skill descriptor and a `registerA2ASkill(registry)` helper. Custom A2A skills live in `src/lib/a2a/skills/` and are dispatched via `A2A_SKILL_HANDLERS` (`src/lib/a2a/taskExecution.ts`). See [A2A-SERVER.md](./A2A-SERVER.md) for the full task lifecycle.
---
## Adding a New Built-in Skill
1. **Define the handler** in `src/lib/skills/builtins.ts` (or a sibling file under `src/lib/skills/builtin/`). Signature: `(input, { apiKeyId, sessionId }) => Promise<output>`.
2. **Sandboxed code path?** Call `sandboxRunner.run(image, command, env, sandboxConfig({...}))`. Use `normalizeImage()` against the allowlist.
3. **Filesystem path?** Always pass through `resolveWorkspacePath(input, context)` before touching disk.
4. **Network call?** Use `safeOutboundFetch` with `guard: "public-only"`; sanitize headers via `sanitizeHeaders()`.
5. **Register** by adding the entry to `builtinSkills` (or calling `registerBrowserSkill(executor)`-style at boot).
6. **Wire built-in tool aliases** (optional) in `BUILTIN_TOOL_ALIASES` (`interception.ts:23`) if the upstream model emits a different name.
7. **Tests** in `src/lib/skills/__tests__/` (Vitest).
---
## Adding a Custom (Non-Builtin) Skill
1. Register the handler at process startup:
```ts
skillExecutor.registerHandler("my-handler", async (input, ctx) => { ... });
```
2. Insert the skill via `POST /api/skills/install` (the `handlerCode` field must match the registered handler name).
3. Toggle `mode` to `on` or `auto` via `PUT /api/skills/[id]`.
---
## Operational Tips
- **Master switch:** `settings.skillsEnabled = false` blocks all execution and returns HTTP `503` on `/api/skills/executions`. The registry continues to load.
- **Lock down egress:** keep `SKILLS_SANDBOX_NETWORK_ENABLED` unset (default) for fully air-gapped sandboxing. Per-call `networkEnabled: true` still requires the master gate.
- **Allow specific images:** set `SKILLS_ALLOWED_SANDBOX_IMAGES="myorg/sandbox:1.0,node:22-alpine"` to extend the allowlist.
- **Audit executions:** `/dashboard/skills/executions` and `omniroute_skills_executions` both query `skill_executions`. Successful runs include `durationMs`; failures include `errorMessage`.
- **Cache invalidation:** call `skillRegistry.invalidateCache()` after manual DB edits; otherwise wait 60 s.
- **Anonymous workspace:** when `apiKeyId` is empty, all calls hash to the same `"anonymous"` workspace — share-aware code should always pass a real key.
---
## Execution Lifecycle (v3.8.16+)
The `SkillExecutor` (`src/lib/skills/executor.ts`) is a **singleton** that manages every skill invocation. Understanding its lifecycle is critical for debugging timeouts, retries, and execution state.
### The 5-Stage Lifecycle
```
execute() called
│
▼
┌─────────────┐
│ PENDING │ ← queued, not yet started (DB row created)
└──────┬──────┘
│ start handler
▼
┌─────────────┐
│ RUNNING │ ← handler invoked with timeout
└──────┬──────┘
│
┌────┴────┬──────────┬──────────┐
│ │ │ │
▼ ▼ ▼ ▼
SUCCESS ERROR TIMEOUT (no other path — killed by parent)
│ │ │
└────┬────┴──────────┘
│
▼
DB row updated with status, output, durationMs
```
### Default Configuration
| Setting | Default | Configurable via |
| ------------ | ------------- | ------------------------------------ |
| `timeout` | `30000` (30s) | `skillExecutor.setTimeout(ms)` |
| `maxRetries` | `3` | `skillExecutor.setMaxRetries(count)` |
> **Important**: The executor is a singleton — calling `setTimeout()` affects all subsequent invocations globally. Per-skill timeouts are not currently supported; if you need different timeouts per skill, submit separate processes or fork the executor.
### Status Values
From `src/lib/skills/types.ts`:
```ts
enum SkillStatus {
PENDING = "pending", // Queued, not yet started
RUNNING = "running", // Handler invoked
SUCCESS = "success", // Handler returned valid output
ERROR = "error", // Handler threw an exception
TIMEOUT = "timeout", // Exceeded the executor's timeout
}
```
> **Note**: The `TIMEOUT` status is defined in the enum but is **not actually written to the DB** by the current executor implementation — timeouts surface as `ERROR` with the message `"Skill execution timed out"`. The status enum is reserved for future use.
### Inspecting Executions
```ts
import { skillExecutor } from "omniroute/skills/executor";
// Get a specific execution by ID
const exec = skillExecutor.getExecution("exec-uuid-123");
if (exec) {
console.log(`${exec.skillName}: ${exec.status} in ${exec.durationMs}ms`);
}
// List recent executions for an API key
const recent = skillExecutor.listExecutions("api-key-id", 50, 0);
for (const e of recent) {
console.log(`${e.skillName} → ${e.status} (${e.durationMs}ms)`);
}
// Count total executions
const total = skillExecutor.countExecutions("api-key-id");
```
### Retry Behavior
The `maxRetries` setting is stored but **not currently used** by the executor's `execute()` method — it only performs a single attempt. The `maxRetries` value is exposed for future implementation and for hooks that want to read it.
For now, retries must be implemented inside the skill handler itself. Built-in
skills are registered against the executor (e.g. `registerBuiltinSkills(executor)`
/ `registerBrowserSkill(executor)` in `src/lib/skills/builtin/`); whichever handler
you register can wrap its own retry loop:
```ts
// inside a skill handler
async function handler(input, ctx) {
const maxRetries = 3;
let lastError: Error | null = null;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await fetchSomething(input);
} catch (err) {
lastError = err as Error;
if (attempt < maxRetries) {
await new Promise((r) => setTimeout(r, 1000 * attempt));
}
}
}
throw lastError;
}
```
---
## SkillMode in Detail
The `SkillMode` enum (`src/lib/skills/types.ts`) controls **when and how** skills are invoked:
```ts
enum SkillMode {
AUTO = "auto", // LLM decides when to call the skill
MANUAL = "manual", // Only invoked by explicit user request
HYBRID = "hybrid", // AUTO scoring + manual override
}
```
> **Note**: The codebase defines `SkillMode` (AUTO/MANUAL/HYBRID), while the `Skill.mode` field uses a different shape (`"on" | "off" | "auto"`). They are related but not identical — `SkillMode` is for executor policy, `Skill.mode` is for per-skill enablement.
### When to Use Each Mode
| Mode | LLM behavior | Use case |
| -------- | ------------------------------------------------------------------------------ | -------------------------------------------------- |
| `AUTO` | LLM can call the skill when it deems necessary | General-purpose skills (file reads, HTTP requests) |
| `MANUAL` | LLM cannot call the skill; only an explicit `executeSkill` API call invokes it | Sensitive operations (database writes, payments) |
| `HYBRID` | LLM can suggest the skill; user must confirm | Skills that have side effects but aren't dangerous |
### AUTO Scoring
When `AUTO` mode is active, each candidate skill is scored against the request
context by `scoreAutoSkill()` in `src/lib/skills/injection.ts` — an additive,
integer point system (skill-name match, name/tag/description token overlap,
background-reason hints, provider-hint bonus/penalty). The top
`AUTO_MAX_SKILLS = 5` skills with `score >= AUTO_MIN_SCORE = 3` are injected as
callable tools, ties broken by `installCount` then name. See the full point table
in [**Tool Schema Generation → AUTO Scoring**](#auto-scoring) earlier in this
document; there is no float `0.6`-style threshold and no `registry.ts` scoring.
---
## Built-in Skills Catalog
OmniRoute ships with a curated set of built-in skills in `src/lib/skills/builtin/`. The most common ones:
### Browser Automation Skill
The browser skill (`src/lib/skills/builtin/browser.ts`) provides headless browser automation via Playwright/Puppeteer. **It is implemented but not in the default skills catalog** — to use it, install the browser extension plugin separately.
```ts
// Enable in your config
const config: SkillConfig = {
enabled: true,
mode: SkillMode.MANUAL, // Always require explicit invocation
allowedSkills: ["browser"],
timeout: 60000, // 60s for page loads
maxRetries: 1,
};
```
### Other Built-in Categories
| Category | Skills | Mode |
| --------- | ------------------------------------------- | ------ |
| File I/O | `file_read`, `file_write` | AUTO |
| HTTP | `http_request` | AUTO |
| Search | `web_search` | AUTO |
| Code Exec | `eval_code` (sandboxed JavaScript/Python) | HYBRID |
| System | `execute_command` (sandboxed CLI execution) | MANUAL |
### Adding a Custom Skill
See the [Plugin SDK & Skills Integration](./PLUGIN_SDK.md) for how to add a custom skill via the plugin system.
---
## See Also
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP tool registration and transports
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A task lifecycle and skill dispatch
- [USER_GUIDE.md](../guides/USER_GUIDE.md#-skills-system) — user-facing introduction
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — request pipeline and component map
- Source: `src/lib/skills/`, `src/app/api/skills/`, `open-sse/mcp-server/tools/skillTools.ts`
- Tests: `src/lib/skills/__tests__/integration.test.ts`
docs/frameworks/TRAFFIC_INSPECTOR.md
+491
View File
@@ -0,0 +1,491 @@
---
title: "Traffic Inspector"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Traffic Inspector
Traffic Inspector is OmniRoute's built-in HTTPS traffic debugger — a Charles Proxy / mitmweb / HTTP Toolkit-like tool that is **LLM-aware** and **agent-aware**. It lives at `/dashboard/tools/traffic-inspector` and receives live traffic from up to 5 simultaneous capture sources.
**Dashboard location:** `/dashboard/tools/traffic-inspector`
**Sidebar group:** Tools (after AgentBridge)
**See also:** [`AGENTBRIDGE.md`](./AGENTBRIDGE.md) — AgentBridge is capture mode 1.
---
## §1 Overview
### What makes Traffic Inspector unique
| Feature | mitmweb | Charles | Fiddler | **OmniRoute Traffic Inspector** |
| ------------------------------------------------------------------- | :-----: | :-----: | :-----: | :-----------------------------: |
| Web-based | ✓ | ✗ | ✗ | ✓ |
| Open-source | ✓ | ✗ | partial | ✓ |
| **Agent-aware** (knows if request is from Antigravity/Copilot/etc.) | ✗ | ✗ | ✗ | ✓ |
| **LLM-aware** (parses OpenAI/Anthropic/Gemini shape, tokens, model) | ✗ | ✗ | ✗ | ✓ |
| **Model mapping visible** (gemini-3-flash → claude-sonnet-4.7) | ✗ | ✗ | ✗ | ✓ |
| **Proxy/upstream latency split** | partial | ✗ | ✗ | ✓ |
| **Integrated with OmniRoute** routing, fallback, cost | ✗ | ✗ | ✗ | ✓ |
| **System-wide proxy debug** (any app on the machine) | ✓ | ✓ | ✓ | ✓ |
| **Custom host capture** (per-host DNS redirect) | ✓ | ✓ | ✓ | ✓ |
| **HTTP_PROXY env mode** | ✓ | ✓ | ✓ | ✓ |
| **Conversation view** (multi-turn bubbles, tool_use/tool_result) | ✗ | ✗ | ✗ | ✓ |
| **SSE stream merger** (reconstruct from delta events) | ✗ | ✗ | ✗ | ✓ |
| **Session recording** (named, exportable .har/.jsonl) | ✗ | ✓ | ✓ | ✓ |
### Architecture in one paragraph
The `TrafficBuffer` (`src/mitm/inspector/buffer.ts`) is a shared in-memory ring buffer (default 1000 entries, configurable via `INSPECTOR_BUFFER_SIZE`). All capture sources write to it via `push()`. The buffer classifies each entry using `kindDetector.ts` (determines if it's an LLM request), computes a `contextKey` (SHA-256 fingerprint of the system prompt), and broadcasts to all WebSocket subscribers via `globalTrafficBuffer.subscribe()`. The dashboard connects via `GET /api/tools/traffic-inspector/ws` and receives a snapshot on connect, followed by `new`/`update`/`clear` events.
---
## §2 Capture modes
Traffic Inspector supports **5 simultaneous capture sources**. Each is independently toggleable. The `source` field on every `InterceptedRequest` (`src/mitm/inspector/types.ts`) is one of `"agent-bridge"`, `"custom-host"`, `"http-proxy"`, `"system-proxy"`, or `"tproxy"`.
### Mode 1 — AgentBridge (default, always on)
**Source:** AgentBridge handlers (`src/mitm/handlers/base.ts`)
**Mechanism:** Every `intercept()` call in `MitmHandlerBase` calls `hookBufferStart()` before forwarding and `hookBufferUpdate()` on completion. Zero extra config — works as soon as AgentBridge is running.
**Reach:** The 9 IDE agents configured in AgentBridge
**Note:** `source` field in `InterceptedRequest` = `"agent-bridge"`
### Mode 2 — Custom Hosts (DNS redirect)
**Source:** User-defined host list (`inspector_custom_hosts` table)
**Mechanism:** Adding a host via the UI adds `127.0.0.1 <host>` to `/etc/hosts` (requires sudo). The existing AgentBridge MITM server (port 443) generates a SNI cert dynamically for the new host.
**Reach:** Any application using the added host — no app config change needed
**Note:** `source` = `"custom-host"`
Example use cases:
- Monitor `api.openai.com` from Python scripts
- Debug `my-internal-llm.company.com`
- Capture traffic from mobile devices on the same network (via ARP spoofing — advanced)
### Mode 3 — HTTP_PROXY listener (port 8080)
**Source:** Applications using `HTTP_PROXY`/`HTTPS_PROXY` environment variables
**Mechanism:** Secondary listener at port 8080 (`src/mitm/inspector/httpProxyServer.ts`) that acts as a standard explicit HTTP/HTTPS proxy. Accepts `CONNECT` tunnels (HTTPS) and direct HTTP requests.
**Reach:** Any application that respects `HTTP_PROXY` env — no DNS change, no sudo
**Note:** `source` = `"http-proxy"`
```bash
# Quick capture for a single command:
HTTPS_PROXY=http://127.0.0.1:8080 curl https://api.openai.com/v1/models
# Persistent capture in a shell session:
export HTTP_PROXY=http://127.0.0.1:8080
export HTTPS_PROXY=http://127.0.0.1:8080
```
**TLS limitation:** HTTPS `CONNECT` tunnels are captured as metadata only (host, port, timing) — TLS body is not decrypted by default. Enable "Decrypt HTTPS in proxy mode" toggle (opt-in, requires AgentBridge cert to be trusted) for full body inspection.
**Port conflict:** If port 8080 is in use, AgentBridge returns a 409 with a structured error. Change the port via `INSPECTOR_HTTP_PROXY_PORT` env var.
### Mode 4 — System-wide proxy (advanced, opt-in)
**Source:** OS-level proxy settings (applies to all apps on the machine)
**Mechanism:** Uses OS APIs to redirect all HTTP/HTTPS traffic through the HTTP_PROXY listener:
- **macOS:** `networksetup -setwebproxy / -setsecurewebproxy`
- **Linux:** `gsettings set org.gnome.system.proxy` + `/etc/environment`
- **Windows:** `netsh winhttp set proxy 127.0.0.1:8080`
**Reach:** Every application on the machine that respects system proxy settings
**Note:** `source` = `"system-proxy"`
**Safety mechanisms:**
- Auto-disable timer (default 30 min, configurable via `INSPECTOR_SYSTEM_PROXY_GUARD_MINUTES`)
- Previous system proxy state is saved in DB and restored on revert
- Dashboard shows "Reverting system proxy" prompt if user navigates away while active
- UI shows `⚠ Advanced` badge + explicit confirmation checkbox
### Mode 5 — TPROXY transparent decrypt (Linux, root, opt-in)
**Source:** Kernel TPROXY + policy routing (`src/mitm/tproxy/`)
**Mechanism:** Marks new local outbound TCP connections to a target port (default `443`) in `mangle OUTPUT`, an `ip rule` reroutes the marked packets to local delivery, and `mangle PREROUTING`'s `TPROXY` target hands them to a transparent (**IP_TRANSPARENT**) listener (default port `8443`). The listener terminates TLS with a leaf certificate issued **per SNI hostname on demand** by a dynamic CA, captures the decrypted exchange, and forwards the request re-encrypted to the original destination.
**Reach:** **Arbitrary** destination hosts on the target port — no `/etc/hosts` spoof, no `HTTP_PROXY` env, no system-wide proxy mutation. The intercepted process needs no config change, but must trust the dynamic CA.
**Note:** `source` = `"tproxy"`
**Requirements:** Linux only (**IP_TRANSPARENT** is Linux-only), the **CAP_NET_ADMIN** capability (root), and a native N-API addon that must be built with a C toolchain (`npm run build:native:tproxy`). When unavailable, the dashboard toggle is disabled with the tooltip "TPROXY decrypt requires Linux + root + the native addon". The firewall rules apply/revert transactionally (a crash never leaves a `mangle` rule behind) and flush on reboot. An SO_MARK-based anti-loop keeps the proxy's own re-encrypted forward from being re-intercepted.
This is a substantial subsystem with its own dedicated operator guide — see **[`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md)** for the full firewall recipe, the per-SNI dynamic CA + trust-store installer, the local-only route, anti-loop details, and the configuration schema. The toggle is driven by `GET / POST / DELETE /api/tools/agent-bridge/tproxy` (note: the route lives under the AgentBridge prefix, not the Traffic Inspector prefix).
### Capture mode comparison
| Mode | Setup | Sudo? | Reach | Notes |
| ----------------- | ----------------------------- | :---------------------: | --------------------------- | ----------------------------------------------------------------------------------------------------------- |
| 1. AgentBridge | Automatic | Once (cert+hosts) | 9 IDE agents | Default on |
| 2. Custom Hosts | Per-host input | Yes (hosts file) | Any app using that host | Persisted in DB |
| 3. HTTP_PROXY | `export HTTPS_PROXY=...` | No | Apps respecting env | Port 8080, no TLS decrypt by default |
| 4. System-wide | Toggle + confirm | Yes | All apps on machine | Auto-disable in 30 min |
| 5. TPROXY decrypt | Toggle (Linux + native addon) | Yes (root + CA install) | Any host on the target port | Decrypts arbitrary hosts; off by default — see [MITM-TPROXY-DECRYPT.md](../security/MITM-TPROXY-DECRYPT.md) |
---
## §3 UI
### 3.1 Layout
```
┌─ Traffic Inspector ─────────────────────────────────────────────────────┐
│ ┌─ Capture sources toolbar ─────────────────────────────────────────┐ │
│ │ [✓ AgentBridge] [✓ Custom hosts (3)] [○ HTTP_PROXY] [○ System]│ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ ┌─ Filter/control bar ──────────────────────────────────────────────┐ │
│ │ Profile: (●) LLM only (○) Custom (○) All │ │
│ │ [⎉ Pause] [🗑 Clear] [⬇ .har] [● REC session] ● live 482/1k │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
├══◀▶══════════════════════════════╬══════════════════════════════════════╤╡
│ REQUEST LIST (resizable) ║ DETAIL PANE ▲ │
│ ────────────────────────────── │ ║ [Conversation][Headers][Request] │ │
│ ▎ 14:32 POST 200 12k AG openai ║ [Response][Timing][LLM][Stats] │ │
│ ▎ 14:31 POST 200 8k CP openai ║ ▼ │
│ ▎ 14:31 POST 503 ⚠ KR ... ║ │
│ ▎ 14:30 GET 200 3k 🌐 custom ║ │
└══════════════════════════════════╝══════════════════════════════════════╝
```
### 3.2 Request list (left panel)
- **Virtualized** (`useVirtualList` + `ResizeObserver`): handles 1000 items without freezing
- **Auto-scroll** with toggle to pause while inspecting
- **Color-coded status**: green (2xx), yellow (3xx), red (4xx/5xx), gray (in-flight)
- **Agent emoji**: 🔵 Antigravity, 🟢 Copilot, 🟠 Kiro, 🟣 Codex, 🔷 Cursor, 🟤 Zed, 🟡 Claude Code, ⚫ Open Code, 🌐 custom host
- **Context color bar**: 1px left border colored by `contextKey` (SHA-256 of system prompt) — visually groups related conversations
- **Lazy body**: only the selected request's body is materialized in the detail tabs (avoids rendering 1000 × 1MB bodies)
### 3.3 Detail pane — 7 tabs
| Tab | Content | Notes |
| ---------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| **Conversation** | Multi-turn chat bubbles (system/user/assistant + tool_use/tool_result) | Normalized from any provider format; only shown for `detectedKind === "llm"` |
| **Headers** | Request + response header tables | Sensitive headers (Authorization, Cookie, api-key) masked by default; "Show secrets" toggle |
| **Request** | Raw body, JSON tree view, model field badge | Pretty-printed JSON or raw text |
| **Response** | Raw body or SSE event list; toggle "Raw ↔ Merged" | SSE merger reconstructs final message from delta events |
| **Timing** | Waterfall: proxy overhead vs upstream latency | Total, TTFB, and size |
| **LLM Details** | Provider, model, messages count, tokens in/out, cost estimate, mapped target | Only shown for LLM requests |
| **Stats** | Recharts: latency timeline, token bar chart, tool call scatter | Only shown when a recorded session is loaded |
### 3.4 Toolbar controls
| Control | Action |
| ---------------- | --------------------------------------------------------------------- |
| ⎉ Pause | Stops rendering new requests; "X new" badge accumulates |
| 🗑 Clear | Clears the UI list (server buffer is not affected) |
| ⬇ Export .har | Downloads current filtered list as HAR file |
| ● Record session | Starts a named recording session |
| Profile selector | LLM only / Custom hosts / All |
| Host filter | Substring match on `host` field |
| Agent filter | Dropdown: All / per-agent |
| Status filter | All / 2xx / 3xx / 4xx / 5xx / error |
| Source filter | All / agent-bridge / custom-host / http-proxy / system-proxy / tproxy |
| **Live** filter | Show only in-flight (open) requests — `liveOnly` toggle (see §4.6) |
### 3.5 Resizable panels
- List and detail pane separated by a drag handle
- List width: min 280px, max 720px, persisted in `localStorage` (`inspector.listWidth`)
- Collapsible to a 48px rail (icon-only); click a row in the rail to expand
---
## §4 LLM-aware features
### 4.1 Kind detector (`src/mitm/inspector/kindDetector.ts`)
Classifies each request as `"llm"`, `"app"`, or `"unknown"` using 4 signals:
1. **Host registry** — ~18 known LLM API hostnames (OpenAI, Anthropic, Gemini, Groq, Mistral, Together, Fireworks, Cohere, Perplexity, Hugging Face, OpenRouter, xAI, Moonshot, etc.)
2. **Path patterns** — `/v1/chat/completions`, `/v1/messages`, `/generateContent`, `/v1/responses`, etc.
3. **Body shape** — detects `messages[]` (OpenAI/Claude), `contents[]` (Gemini), `prompt`, `input` fields
4. **User-agent hints** — `codex`, `claude`, `gemini`, `antigravity`, `kiro`, `copilot`, `cursor` in UA string
Custom hosts added via Mode 2 inherit their `kind` from the form input (defaults to `"custom"`).
### 4.2 SSE merger (`src/mitm/inspector/sseMerger.ts`)
**MIT port from [chouzz/llm-interceptor](https://github.com/chouzz/llm-interceptor)**
Reconstructs the final assistant message from raw SSE delta events:
- **Anthropic**: accumulates `content_block_delta` by index; handles `text_delta`, `input_json_delta` (tool calls), `thinking_delta`
- **OpenAI**: accumulates `choices[i].delta.content` and `tool_calls` by index
- **Gemini**: accumulates `candidates[i].content.parts`
- **Unknown**: returns raw events as-is
The Response tab shows a toggle: **"Raw events ↔ Merged"**.
### 4.3 Conversation normalizer (`src/mitm/inspector/conversationNormalizer.ts`)
**MIT port from [chouzz/llm-interceptor](https://github.com/chouzz/llm-interceptor)**
Converts OpenAI, Anthropic, and Gemini message formats to a single `NormalizedConversation` before rendering:
```ts
interface NormalizedConversation {
request: NormalizedTurn[]; // messages / contents / prompt from request body
response: NormalizedTurn[]; // assistant response (merged via sseMerger)
contextKey: string | null; // SHA-256 system-prompt fingerprint
}
```
Block types: `text`, `tool_use`, `tool_result`. The Conversation tab uses this shape regardless of provider.
### 4.4 Context key colorization (`src/mitm/inspector/contextKey.ts`)
- Computes `SHA-256` of the system prompt (first `role:system` message, or `system` field, or Gemini `systemInstruction`)
- Returns a 12-character hex prefix (`"a3f9c2..."`)
- Frontend maps the key to a deterministic HSL color for the left-border bar
- **Filtro "same context"**: clicking the `ctx #a3f` chip adds a filter to show only requests with the same fingerprint
This makes it easy to visually distinguish different "personas" or tasks running in the same agent session.
### 4.5 LLM metadata extraction
For LLM requests, the LLM Details tab extracts:
```ts
interface LlmMetadata {
provider: string | null; // "openai" | "anthropic" | "gemini" | ...
apiKind: string | null; // "chat.completions" | "messages" | "embeddings" | ...
model: string | null; // from request body or response
messages: number; // turn count
tokensIn: number | null; // usage.prompt_tokens / usage.input_tokens
tokensOut: number | null; // usage.completion_tokens / usage.output_tokens
streamed: boolean; // true if SSE response
mappedTo: string | null; // x-omniroute-mapped header
costEstimateUsd: number | null; // estimated cost based on OmniRoute pricing
}
```
### 4.6 Live in-flight request filter
The request `status` field is `number | "in-flight" | "error"` — an entry is
pushed as `"in-flight"` the moment the request starts and **updated in place**
when the response (or error) arrives. The toolbar's **"Live"** toggle
(`liveOnly`, i18n key `trafficInspector.liveOnly`) restricts the list to entries
whose `status === "in-flight"`, letting you watch open connections in real time.
The filter is a pure, client-side predicate in
`src/lib/inspector/matchesTrafficFilter.ts`:
```ts
if (f.liveOnly && req.status !== "in-flight") return false;
```
The toggle state lives in `useTrafficFilters` (the inspector dashboard hooks) and
combines with the other filters (profile, host, agent, source, status, context).
### 4.7 Process attribution (Linux)
On Linux, each intercepted request can be attributed to the **originating local
process**. Two optional fields are added to `InterceptedRequest`:
```ts
pid?: number; // originating process id (Linux only)
processName?: string; // originating process name (Linux only)
```
`src/mitm/inspector/processAttribution.ts` maps the connection's _client_
ephemeral port to a PID + name by:
1. Reading `/proc/net/tcp` and `/proc/net/tcp6` to find the socket inode for the
port (`parseProcNetTcpForInode`, a pure fixture-testable parser).
2. Scanning `/proc/<pid>/fd/` for a symlink to `socket:[<inode>]`.
3. Reading the process name from `/proc/<pid>/comm`.
A 1-second TTL cache bounds the procfs scan cost under load. Attribution is
**best-effort** — any failure resolves to `null` and never blocks capture. On
macOS/Windows the function returns `null` (stub; `lsof`/`GetExtendedTcpTable`
support is a follow-up).
---
## §5 Sessions
### 5.1 Recording a session
1. Click **"● Record session"** in the toolbar → enter a name (optional)
2. Live tail continues normally; a red pulsing indicator shows `◉ REC · <name> · 00:42 · 23 reqs`
3. Click **"⏹ Stop"** → the session snapshot is saved to `inspector_sessions` + `inspector_session_requests`
### 5.2 Viewing a recorded session
The **Sessions** dropdown in the toolbar lists saved sessions. Selecting one:
- Loads the session's snapshot (frozen state)
- A banner shows: `Viewing recorded session "<name>" — [Back to live]`
- The Stats tab becomes available with Recharts aggregates
### 5.3 Export formats
Each session can be exported as:
| Format | Use |
| -------------------------- | ------------------------------------------------------------------------------- |
| **HAR** (HTTP Archive 1.2) | Compatible with Chrome DevTools, Charles, Fiddler — import for offline analysis |
| **JSONL** | One `InterceptedRequest` per line — compatible with `llm-interceptor` format |
Export via `GET /api/tools/traffic-inspector/sessions/{id}/export.har` or the ⬇ button in the Sessions dropdown.
---
## §6 Security
Traffic Inspector shows **all intercepted HTTPS traffic**, including authorization headers and request bodies. The following controls are in place:
| Control | Details |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| **LOCAL_ONLY** | All routes and the WebSocket endpoint are loopback-only (enforced in `routeGuard.ts` before auth) |
| **Secret masking** | `maskSecrets()` applied to all headers and bodies before `TrafficBuffer.push()` — enabled by default (`INSPECTOR_MASK_SECRETS=true`) |
| **Body size cap** | Bodies > `INSPECTOR_MAX_BODY_KB` (default 1024 KB) are truncated with `"(truncated for performance)"` notice |
| **Sensitive header masking** | `authorization`, `cookie`, `api-key`, `x-api-key`, `proxy-authorization` → `Bearer ***` in Headers tab; "Show secrets" toggle |
| **CSP** | Strict Content Security Policy on Traffic Inspector pages to prevent XSS via injected response bodies |
| **No persistence by default** | The `TrafficBuffer` is in-memory and lost on server restart. Sessions are persisted only when explicitly recorded |
### Hard Rules applied
| Rule | Application |
| --------------------------------- | ------------------------------------------------------------------------------------- |
| **#12** `sanitizeErrorMessage` | All HTTP error responses from Traffic Inspector routes are sanitized |
| **#15 + #17** `isLocalOnlyPath()` | `/api/tools/traffic-inspector/` is LOCAL_ONLY + SPAWN_CAPABLE (system proxy commands) |
### Known limitations
- **System-wide proxy mode** affects all applications on the machine, including VPN clients and SSO. Always use with the auto-disable timer. Do not use on shared machines.
- **CONNECT tunnel HTTPS**: Mode 3 (HTTP_PROXY) captures only tunnel metadata for HTTPS destinations unless TLS interception is enabled. This is by design — transparent capture without the AgentBridge cert being trusted would break TLS verification for those apps.
- **Hardcoded strings in some components**: Some UI components (F7/F8) have a small number of hardcoded strings not yet covered by i18n keys. These are documented as a Known Limitation in the i18n gap report; they will be migrated in a follow-up pass. Affected strings are UI decorative labels that don't require translation for functional use.
---
## §7 Troubleshooting
### WebSocket disconnection
If the live tail shows "Disconnected":
1. Check the server is still running: `GET /api/tools/traffic-inspector/capture-modes`
2. Reload the page — the WebSocket reconnects and receives a fresh snapshot
3. If the server was restarted, the in-memory buffer was cleared — old entries are gone unless a session was recorded
### Port 8080 conflict
If HTTP_PROXY mode fails to start:
```bash
lsof -i :8080 # find the process
```
Change the port:
```bash
# .env
INSPECTOR_HTTP_PROXY_PORT=8888
```
### System proxy not reverted
If OmniRoute crashes while system-wide proxy mode is active:
**macOS:**
```bash
networksetup -setwebproxystate Wi-Fi off
networksetup -setsecurewebproxystate Wi-Fi off
```
**Linux (GNOME):**
```bash
gsettings set org.gnome.system.proxy mode 'none'
```
**Windows:**
```cmd
netsh winhttp reset proxy
```
The dashboard will also offer "Revert system proxy" on next load if it detects the DB state indicates proxy was active.
### Buffer full
When the buffer reaches `INSPECTOR_BUFFER_SIZE` (default 1000), new entries rotate out the oldest. If important requests are being lost:
- Increase `INSPECTOR_BUFFER_SIZE` (e.g., 5000) — trades memory for retention
- Record a session to persist the relevant window to DB
---
## §8 API reference
All routes are `LOCAL_ONLY` (loopback-only) and `SPAWN_CAPABLE` (system proxy commands). See `src/server/authz/routeGuard.ts`.
Base path: `/api/tools/traffic-inspector/`
### Request management
| Method | Path | Description |
| ------ | --------------------------- | ---------------------------------------------------------------------------------- |
| GET | `/requests` | List requests (filterable: `?profile=llm&host=&agent=&status=&source=&sessionId=`) |
| GET | `/requests/{id}` | Single request details |
| DELETE | `/requests` | Clear the in-memory buffer |
| POST | `/requests/{id}/replay` | Re-execute the same request through OmniRoute router |
| PUT | `/requests/{id}/annotation` | Save or update a note on a request |
### WebSocket
| Method | Path | Description |
| ------ | ----- | -------------------------------------------------------------------------------------- |
| GET | `/ws` | Live WebSocket stream. Sends `snapshot` on connect, then `new`/`update`/`clear` events |
### Export
| Method | Path | Description |
| ------ | ------------- | --------------------------------------- |
| GET | `/export.har` | Export current filtered list as HAR 1.2 |
### Custom hosts
| Method | Path | Description |
| ------ | --------------- | ---------------------------------- |
| GET | `/hosts` | List custom hosts |
| POST | `/hosts` | Add host (auto-edits `/etc/hosts`) |
| DELETE | `/hosts/{host}` | Remove host |
| PATCH | `/hosts/{host}` | Toggle `enabled` |
### Capture modes
| Method | Path | Description |
| ------ | ------------------------------ | ------------------------------------------------------------------------------------------------------ |
| GET | `/capture-modes` | State of the AgentBridge / custom-hosts / HTTP_PROXY / system-proxy modes + the `tls-intercept` toggle |
| POST | `/capture-modes/http-proxy` | Start/stop HTTP_PROXY listener (`{action: "start"\|"stop"}`) |
| POST | `/capture-modes/system-proxy` | Apply/revert system-wide proxy (`{action: "apply"\|"revert"}`) |
| POST | `/capture-modes/tls-intercept` | Toggle HTTPS body decryption in proxy mode (`{enabled: boolean}`) |
> **TPROXY decrypt** (capture mode 5) is driven by a **separate** route under the
> AgentBridge prefix — `GET / POST / DELETE /api/tools/agent-bridge/tproxy` — not
> under `/api/tools/traffic-inspector/`. See
> [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md).
### Sessions
| Method | Path | Description |
| ------ | --------------------------- | ------------------------------------------------------------ |
| POST | `/sessions` | Start recording (`{name?: string}`) |
| PATCH | `/sessions/{id}` | Stop or rename (`{action: "stop"\|"rename", name?: string}`) |
| GET | `/sessions` | List all saved sessions |
| GET | `/sessions/{id}` | Session snapshot (all requests) |
| DELETE | `/sessions/{id}` | Delete session |
| GET | `/sessions/{id}/export.har` | Export session as HAR 1.2 |
### Internal ingest (D4 fallback)
| Method | Path | Description |
| ------ | ------------------ | ----------------------------------------------------------------------------------------------------------------- |
| POST | `/internal/ingest` | Accepts intercepted request from `server.cjs` passthrough path; requires `INSPECTOR_INTERNAL_INGEST_TOKEN` header |
Full OpenAPI schemas: `docs/openapi.yaml` → tag `Traffic Inspector`.
docs/frameworks/WEBHOOKS.md
+259
View File
@@ -0,0 +1,259 @@
---
title: "Webhooks"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Webhooks
> **Source of truth:** `src/lib/webhookDispatcher.ts`, `src/lib/db/webhooks.ts`, `src/app/api/webhooks/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute can fire HTTP webhooks on platform events. Use them to integrate with
Slack, PagerDuty, Datadog, internal alerting services, or any HTTP receiver.
The dispatcher signs each delivery with HMAC-SHA256, retries on transient
failures, tracks delivery health per webhook, and auto-disables endpoints that
keep failing.
## Supported Events
The `WebhookEvent` type (`src/lib/webhookDispatcher.ts`) currently models:
| Event | Fires when |
| -------------------- | --------------------------------------------------------- |
| `request.completed` | A proxied request completes successfully |
| `request.failed` | A proxied request fails after all retries/fallback |
| `provider.error` | A provider returns an error eligible for circuit-breaking |
| `provider.recovered` | A previously failing provider returns to a healthy state |
| `quota.exceeded` | An API key crosses a budget/quota threshold |
| `combo.switched` | A combo strategy switches its primary target |
| `test.ping` | Synthetic event used by the test endpoint |
Subscriptions accept the literal `"*"` to receive every event. Unknown event
names in `events` are ignored at dispatch time.
> Note: the dispatcher API is wired, but production call sites for some of the
> non-`test.ping` events are still landing. Check `grep dispatchEvent` to see
> which paths currently invoke the dispatcher in your release.
## Architecture
```
Caller (handler, service, monitor)
dispatchEvent(event, data) [src/lib/webhookDispatcher.ts]
-> getEnabledWebhooks() [src/lib/db/webhooks.ts]
-> filter by webhook.events
-> for each match (in parallel):
deliverWebhook(url, payload, secret)
build payload { event, timestamp, data }
sign body with HMAC-SHA256 (if secret present)
POST with 10s timeout
retry up to 3 times on 5xx / network error
recordWebhookDelivery(id, status, success)
-> disableWebhooksWithHighFailures(10)
```
Dispatch is fire-and-forget for the caller: `Promise.allSettled` swallows
per-webhook errors so one bad receiver cannot block the others.
## HMAC Signing
When a webhook has a `secret`, OmniRoute signs the JSON body and sends:
```
Content-Type: application/json
User-Agent: OmniRoute-Webhook/1.0
X-Webhook-Event: <event>
X-Webhook-Timestamp: <ISO-8601>
X-Webhook-Signature: sha256=<hex HMAC-SHA256(secret, body)>
```
> Header names use the `X-Webhook-*` prefix (not `X-OmniRoute-*`). The signature
> value is `sha256=<hex>` — verify the full prefix.
If `createWebhook` is called without a secret, the DB module generates one
(`whsec_<48 hex>`) so all webhooks are signed by default.
### Verifying on the receiver
```typescript
import { createHmac, timingSafeEqual } from "node:crypto";
function verify(rawBody: string, signature: string, secret: string) {
const expected = "sha256=" + createHmac("sha256", secret).update(rawBody).digest("hex");
const a = Buffer.from(expected);
const b = Buffer.from(signature);
return a.length === b.length && timingSafeEqual(a, b);
}
```
Always verify against the **raw** request body, before any JSON parsing.
## Retry & Failure Policy
`deliverWebhook(url, payload, secret, maxRetries = 3)`:
- 10 second timeout per attempt (`AbortController`).
- HTTP 2xx counts as success.
- HTTP 3xx/4xx counts as a non-retryable final status — recorded as delivered
with `success = res.ok`.
- HTTP 5xx and network errors are retried with exponential backoff:
`2^attempt * 1000 ms` (1s, 2s, 4s).
- After `maxRetries`, the delivery is recorded as failed.
- Each delivery updates `last_triggered_at`, `last_status`, and either resets
or increments `failure_count`.
- The dispatcher calls `disableWebhooksWithHighFailures(10)` after each fan-out,
so any webhook with `failure_count >= 10` is automatically disabled.
## Database
Table `webhooks` (migration `011_webhooks.sql`):
| Column | Type | Notes |
| ------------------- | ------- | --------------------------------------------- |
| `id` | TEXT PK | UUID |
| `url` | TEXT | Destination URL |
| `events` | TEXT | JSON array; default `["*"]` |
| `secret` | TEXT | HMAC secret (auto-generated if not given) |
| `enabled` | INT | 0/1; defaults to 1 |
| `description` | TEXT | Optional human label |
| `created_at` | TEXT | `datetime('now')` |
| `last_triggered_at` | TEXT | Updated on every delivery attempt |
| `last_status` | INT | HTTP status of the last attempt (0 = network) |
| `failure_count` | INT | Resets to 0 on success, +1 on failure |
There is **no separate `webhook_deliveries` table** in the current schema —
delivery history is aggregated on the `webhooks` row. If you need full audit
history, consume `request.completed` / `audit` style events from a downstream
log store.
## REST API
All endpoints require management auth (`requireManagementAuth`).
| Endpoint | Method | Description |
| ------------------------- | ------ | ------------------------------- |
| `/api/webhooks` | GET | List webhooks (secrets masked) |
| `/api/webhooks` | POST | Create webhook |
| `/api/webhooks/[id]` | GET | Webhook detail (full secret) |
| `/api/webhooks/[id]` | PUT | Update fields |
| `/api/webhooks/[id]` | DELETE | Remove |
| `/api/webhooks/[id]/test` | POST | Fire a `test.ping` (no retries) |
`GET /api/webhooks` masks the secret to `<first 10 chars>...` to avoid leaking
on listing pages. Use the `[id]` GET when you actually need the secret.
### Create webhook
```bash
curl -X POST http://localhost:20128/api/webhooks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"url": "https://hooks.slack.com/services/...",
"secret": "whsec_my_shared_secret",
"events": ["quota.exceeded", "provider.error"],
"description": "Slack alerts"
}'
```
If `secret` is omitted, the server generates a `whsec_<hex>` secret and returns
it in the response.
### Test webhook
```bash
curl -X POST http://localhost:20128/api/webhooks/<id>/test \
-H "Cookie: auth_token=..."
```
Returns `{ delivered, status, error }`. No retries are attempted — useful for
quickly validating that the receiver accepts the payload and signature.
## Dashboard
The dashboard page at `/dashboard/webhooks` (see
`src/app/(dashboard)/dashboard/webhooks/page.tsx`) provides:
- Create/edit webhooks with an event picker
- Status indicator (active / inactive / errored) based on `enabled`,
`failure_count`, and `last_status`
- One-click test delivery
- Manual enable/disable toggle
## Payload Examples
### request.completed
```json
{
"event": "request.completed",
"timestamp": "2026-05-13T20:30:00.123Z",
"data": {
"trace_id": "...",
"api_key_id": "...",
"provider": "openai",
"model": "gpt-5",
"status": 200,
"tokens_in": 142,
"tokens_out": 350,
"cost_usd": 0.0042
}
}
```
### provider.error
```json
{
"event": "provider.error",
"timestamp": "2026-05-13T20:31:00.000Z",
"data": {
"provider": "anthropic",
"status": 503,
"consecutive_failures": 5,
"circuit_state": "open"
}
}
```
### test.ping
```json
{
"event": "test.ping",
"timestamp": "2026-05-13T20:32:00.000Z",
"data": {
"message": "Test webhook delivery from OmniRoute",
"webhookId": "<uuid>"
}
}
```
Field shapes for non-`test.ping` events are defined by the call sites that emit
them; treat the `data` object as forward-compatible (add fields, don't depend on
absence).
## Best Practices
- **Verify the signature on every delivery** against the raw body — prevents
spoofed POSTs from anyone who guesses your webhook URL.
- **Respond 2xx within ~5 seconds** — the dispatcher times out at 10 s. Slow
receivers will eat retries and inflate `failure_count`.
- **Make handlers idempotent** — retries and at-least-once delivery semantics
mean duplicates are possible.
- **Subscribe minimally** — list only events you actually consume; `"*"` will
add cost on receivers you do not control.
- **Watch `failure_count`** — endpoints are auto-disabled at 10 consecutive
failures; reset by calling `PUT /api/webhooks/[id]` with `enabled: true`
after fixing the receiver.
- **Rotate secrets periodically** — `PUT` a new `secret`, deploy the new value
to the receiver, and confirm via the test endpoint.
## See Also
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — full management API surface
- [RESILIENCE_GUIDE.md](../architecture/RESILIENCE_GUIDE.md) — circuit breaker / cooldown
semantics that drive `provider.error` / `provider.recovered`
- Source: `src/lib/webhookDispatcher.ts`, `src/lib/db/webhooks.ts`
docs/frameworks/meta.json
+21
View File
@@ -0,0 +1,21 @@
{
"title": "Frameworks",
"pages": [
"MCP-SERVER",
"A2A-SERVER",
"AGENT_PROTOCOLS_GUIDE",
"ACP",
"CLOUD_AGENT",
"EVALS",
"GAMIFICATION",
"MEMORY",
"NOTION_CONTEXT",
"OBSIDIAN_CONTEXT",
"OPENCODE",
"PLUGIN_MARKETPLACE",
"PLUGINS",
"PLUGIN_SDK",
"SKILLS",
"WEBHOOKS"
]
}
docs/getting-started/AUTO-COMBO-GUIDE.md
+210
View File
@@ -0,0 +1,210 @@
# Auto-Combo: Let OmniRoute Pick the Best AI for You
> **TL;DR**: Set your model to `auto` and OmniRoute automatically picks the best AI provider for each request. No configuration needed.
---
## What It Does
Instead of choosing a specific AI model (like GPT-4o or Claude), you can let OmniRoute **automatically pick the best one** for each request. It considers:
- **Health** — Is the provider working right now?
- **Speed** — How fast is it?
- **Cost** — How much does it cost?
- **Quality** — Is it good at this type of task?
- **Capacity** — Does it have quota remaining?
OmniRoute scores all your connected providers and picks the best one. If it fails, it automatically tries the next one.
---
## Quick Start
**Step 1**: Set your model to `auto` in your IDE or CLI:
```
model: "auto"
```
**Step 2**: That's it! OmniRoute handles the rest.
**Step 3** (optional): Use a variant for specific tasks:
```
model: "auto/coding" # Best for code
model: "auto/fast" # Fastest response
model: "auto/cheap" # Cheapest option
```
---
## Which "auto" Should I Use?
| If you want... | Use this | Best for | How it works |
|----------------|----------|----------|--------------|
| **Best overall** | `auto` | General questions, chat | Balances speed, cost, and quality |
| **Best code** | `auto/coding` | Writing code, debugging | Picks models good at coding tasks |
| **Fastest response** | `auto/fast` | Quick answers, low latency | Prioritizes speed over everything |
| **Cheapest option** | `auto/cheap` | Saving money | Picks the cheapest provider |
| **Smartest model** | `auto/smart` | Complex tasks | Quality-first + explores new models |
| **Most available** | `auto/offline` | When providers are busy | Picks providers with most capacity |
### Examples
```bash
# General chat — balanced
curl http://localhost:20128/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'
# Code generation — quality-first
curl http://localhost:20128/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"auto/coding","messages":[{"role":"user","content":"Write a Python function"}]}'
# Quick answer — speed-first
curl http://localhost:20128/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"auto/fast","messages":[{"role":"user","content":"What is 2+2?"}]}'
```
---
## How It Works (Simple Version)
When you send a request with `model: "auto"`, OmniRoute:
1. **Looks at all your connected providers** — Every provider you've added (OpenAI, Anthropic, Google, etc.)
2. **Scores each one** on 5 factors:
- Is it working? (health)
- Does it have capacity? (quota)
- How much does it cost? (price)
- How fast is it? (speed)
- Is it good at this task? (quality)
3. **Picks the best one** — The highest-scoring provider gets your request
4. **Auto-recovers** — If it fails, OmniRoute tries the next one automatically
### The Scoring System
Each provider gets a score from 0 to 1. The higher the score, the better the fit.
| Factor | Weight | What it means |
|--------|--------|---------------|
| Health | 20% | Is the provider working? (circuit breaker state) |
| Quota | 15% | Does it have capacity remaining? |
| Cost | 15% | How expensive is it? (cheaper = higher score) |
| Speed | 12% | How fast is it? (lower latency = higher score) |
| Task Fit | 8% | Is it good at this type of task? |
| Stability | 5% | Is it consistent? (low error rate) |
| Tier | 5% | Account tier (Ultra > Pro > Free) |
| Other | 20% | Context affinity, connection density, etc. |
### How Variants Change the Scoring
Each variant uses different weights:
| Variant | Prioritizes | Key Weights |
|---------|-------------|-------------|
| `auto` | Balanced | health=20%, quota=15%, cost=15% |
| `auto/coding` | Quality | taskFit=37%, stability=15% |
| `auto/fast` | Speed | latency=32%, health=28% |
| `auto/cheap` | Cost | cost=37% |
| `auto/smart` | Quality + Explore | taskFit=37%, exploration=10% |
| `auto/offline` | Capacity | quota=37%, health=28% |
---
## How It Handles Failures
OmniRoute has **three layers of protection**:
### 1. Auto-Fallback
If the best provider fails, OmniRoute automatically tries the next one. You don't need to do anything.
### 2. Self-Healing
If a provider keeps failing:
- **Score < 0.2** → Excluded for 5 minutes
- **Circuit breaker open** → Auto-excluded
- **More than 50% providers down** → Incident mode (no exploration)
### 3. Emergency Fallback
If all providers fail, OmniRoute routes to stable free providers (like Kiro or Qoder) as a last resort.
---
## Multi-Account Support
If you have multiple accounts for the same provider (e.g., two OpenAI keys), OmniRoute treats each as a **separate candidate**. This means:
- Account A has quota remaining → use it
- Account B is rate-limited → skip it
- Account C is cheaper → prefer it
Each account is scored independently based on its own health, quota, and speed.
---
## Bandit Exploration
OmniRoute occasionally **explores** new providers to discover better options:
- **Default**: 5% of requests go to random providers
- **Auto/smart**: 10% exploration rate
- **Disabled** when more than 50% of providers are unhealthy
This helps OmniRoute learn which providers work best for your usage patterns.
---
## Common Questions
### "Will it always pick the most expensive model?"
**No.** Cost is only 15% of the score by default. A cheap, fast, healthy provider can beat an expensive one. Use `auto/cheap` if you want to prioritize cost even more.
### "What if a provider goes down?"
OmniRoute automatically skips it and tries the next one. If a provider keeps failing, it's excluded temporarily (5-30 minutes). You don't need to do anything.
### "Can I see which provider was used?"
Check the response headers — OmniRoute includes the provider and model used in each response.
### "Does it learn from my usage?"
Yes! The scoring system uses historical data (latency, error rates, success rates) to make better decisions over time.
### "What's the difference between `auto` and `auto/smart`?"
- `auto` — Balanced, 5% exploration
- `auto/smart` — Quality-first (same weights as `auto/coding`), 10% exploration
Use `auto/smart` when you want the best quality and are okay with occasional exploration.
### "Can I force a specific provider?"
Yes! Use a combo with `priority` strategy instead of `auto`. See the [Technical Reference](../routing/AUTO-COMBO.md) for details.
### "How is this different from round-robin?"
Round-robin cycles through providers in order. Auto-combo **scores each provider** and picks the best one. It's smarter — it considers health, speed, cost, and quality.
---
## What's Next?
- **[Connect a Provider](./PROVIDERS-GUIDE.md)** — Add your first AI provider
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
- **[Technical Reference](../routing/AUTO-COMBO.md)** — Deep dive into the scoring algorithm
---
## Learn More
For developers and contributors, see the [Auto-Combo Technical Reference](../routing/AUTO-COMBO.md) for:
- Full 12-factor scoring algorithm
- Mode pack weight tables
- Implementation file paths
- API endpoints
- Self-healing algorithm details
docs/getting-started/FREE-TIERS-GUIDE.md
+270
View File
@@ -0,0 +1,270 @@
# Free Tiers Guide: Get Free AI Without a Credit Card
> **TL;DR**: OmniRoute aggregates free tiers from 50+ providers. Connect multiple free providers for unlimited free AI with automatic fallback.
---
## What Are Free Tiers?
Many AI providers offer **free usage** — no credit card required. Think of it like free samples at a grocery store. You can try the product without paying.
OmniRoute **aggregates** these free tiers into one endpoint. Instead of signing up for 10 different services, you connect them all to OmniRoute and use `model: "auto"` to automatically pick the best free option for each request.
---
## Best Free Providers (No Credit Card)
### Tier 1: Free Forever (Unlimited)
These providers are **always free** with no limits:
| Provider | Models | Quota | How to Connect |
|----------|--------|-------|----------------|
| **Kiro AI** | Claude Sonnet 4.5, Haiku 4.5, Opus 4.6 | 50 credits/month | No auth needed |
| **OpenCode Free** | GPT-4o, Claude, Gemini | Unlimited | No auth needed |
| **Pollinations** | GPT-5, Claude, Gemini, DeepSeek, Llama 4 | No key needed | No auth needed |
| **LongCat** | LongCat-2.0 | 10M tokens (one-time) | API key + KYC |
| **Cloudflare AI** | 50+ models | 10K neurons/day | No auth needed |
| **Qwen** | Qwen3-coder-plus/flash/next | Unlimited | No auth needed |
| **Qoder** | Kimi-K2, DeepSeek-R1, Qwen3-coder | Unlimited | No auth needed |
### Tier 2: Free with Signup (Generous)
These providers give you **free credits** when you sign up:
| Provider | Free Credits | Models | How to Get |
|----------|-------------|--------|------------|
| **NVIDIA NIM** | ~40 RPM | 129 models | Sign up at build.nvidia.com |
| **Cerebras** | 1M tokens/day | Qwen3 235B, GPT-OSS 120B | Sign up at cerebras.ai |
| **DeepSeek** | 5M free tokens | DeepSeek V4 | Sign up at platform.deepseek.com |
| **Groq** | 30 RPM free | Llama 4, Mixtral | Sign up at console.groq.com |
| **OpenAI** | $5 free credits | GPT-5, GPT-4o | Sign up at platform.openai.com |
| **Anthropic** | $5 free credits | Claude Opus 4.6, Sonnet 4.6 | Sign up at console.anthropic.com |
| **Google** | 1,500 req/day | Gemini 2.5 Pro, Flash | Sign up at aistudio.google.com |
### Tier 3: Free with Limits (Specific Use Cases)
These providers have **free tiers** with specific limits:
| Provider | Free Limit | Models | Best For |
|----------|-----------|--------|----------|
| **Cerebras** | 1M tokens/day | Qwen3 235B | Fast inference |
| **NVIDIA NIM** | ~40 RPM | 129 models | Variety |
| **Groq** | 30 RPM | Llama 4, Mixtral | Speed |
| **Cloudflare AI** | 10K neurons/day | 50+ models | Variety |
---
## How to Stack Free Tiers
The magic of OmniRoute is **stacking free tiers**. Instead of relying on one provider, you connect multiple free providers and let OmniRoute automatically pick the best one for each request.
### Example: Unlimited Free AI
Connect these 4 providers for **unlimited free AI**:
1. **Kiro AI** — 50 credits/month (Claude models)
2. **OpenCode Free** — Unlimited (GPT models)
3. **Pollinations** — No key needed (multiple models)
4. **LongCat** — 10M tokens one-time (backup, requires KYC)
Then use `model: "auto"` and OmniRoute will:
- Try Kiro first (best quality)
- If Kiro is busy → try OpenCode Free
- If OpenCode Free is slow → try Pollinations
- If all fail → use LongCat as backup
**Result**: Unlimited free AI with automatic fallback!
---
## How to Connect Free Providers
### Step 1: Open the Dashboard
Go to `http://localhost:20128` in your browser.
### Step 2: Go to Providers
Click **Providers** in the sidebar.
### Step 3: Click Add Provider
Click the **+ Add Provider** button.
### Step 4: Select a Free Provider
Browse the list and select one of these free providers:
- **Kiro AI** — Free Claude models
- **OpenCode Free** — Free GPT models
- **Pollinations** — Free GPT-5, Claude, Gemini
- **LongCat** — 10M tokens free (one-time, requires KYC)
- **Cloudflare AI** — 50+ models, 10K neurons/day
### Step 5: Click Connect
No API key needed — just click **Connect**.
### Step 6: Repeat
Connect 3-4 free providers for the best experience.
---
## Free Provider Details
### Kiro AI
- **Models**: Claude Sonnet 4.5, Haiku 4.5, Opus 4.6
- **Quota**: 50 credits/month
- **Auth**: No auth needed
- **Best for**: High-quality Claude models
### OpenCode Free
- **Models**: GPT-4o, Claude, Gemini
- **Quota**: Unlimited
- **Auth**: No auth needed
- **Best for**: General-purpose AI
### Pollinations
- **Models**: GPT-5, Claude, Gemini, DeepSeek, Llama 4
- **Quota**: No key needed
- **Auth**: No auth needed
- **Best for**: Variety of models
### LongCat
- **Models**: LongCat-2.0
- **Quota**: 10M tokens, one-time grant on signup (not recurring daily/monthly)
- **Auth**: API key + KYC verification required to unlock the free grant
- **Best for**: A one-off free allowance; pay-as-you-go beyond it
### Cloudflare AI
- **Models**: 50+ models
- **Quota**: 10K neurons/day
- **Auth**: No auth needed
- **Best for**: Variety and reliability
### NVIDIA NIM
- **Models**: 129 models
- **Quota**: ~40 RPM
- **Auth**: Sign up at build.nvidia.com
- **Best for**: Variety and speed
### Cerebras
- **Models**: Qwen3 235B, GPT-OSS 120B
- **Quota**: 1M tokens/day
- **Auth**: Sign up at cerebras.ai
- **Best for**: Fast inference
### Qwen
- **Models**: Qwen3-coder-plus/flash/next
- **Quota**: Unlimited
- **Auth**: No auth needed
- **Best for**: Coding tasks
### Qoder
- **Models**: Kimi-K2, DeepSeek-R1, Qwen3-coder
- **Quota**: Unlimited
- **Auth**: No auth needed
- **Best for**: Coding tasks
---
## How OmniRoute Makes Free Tiers Better
### 1. Automatic Fallback
If one free provider is busy or down, OmniRoute automatically tries the next one. You don't need to do anything.
### 2. Smart Routing
OmniRoute picks the **best free provider** for each request based on:
- Speed — Which provider is fastest right now?
- Quality — Which provider is best for this task?
- Capacity — Which provider has quota remaining?
### 3. Token Savings
OmniRoute's **compression** feature saves 15-95% of tokens. This means your free quota lasts **5-20x longer**.
### 4. Multi-Account Support
If you have multiple accounts for the same provider, OmniRoute treats each as a separate candidate. This doubles or triples your free quota.
---
## Free Tier Math
Let's calculate how much free AI you can get:
### Conservative Estimate (3 providers)
| Provider | Daily Quota | Monthly Quota |
|----------|-------------|---------------|
| Kiro AI | ~1.7 credits | 50 credits |
| OpenCode Free | Unlimited | Unlimited |
| Pollinations | Unlimited | Unlimited |
**Total**: Unlimited free AI
### Aggressive Estimate (7 providers)
| Provider | Daily Quota | Monthly Quota |
|----------|-------------|---------------|
| Kiro AI | ~1.7 credits | 50 credits |
| OpenCode Free | Unlimited | Unlimited |
| Pollinations | Unlimited | Unlimited |
| LongCat | — (one-time) | 10M tokens (one-time, KYC) |
| Cloudflare AI | 10K neurons | 300K neurons |
| NVIDIA NIM | ~40 RPM | ~1.7M requests |
| Cerebras | 1M tokens | 30M tokens |
**Total**: ~1.6B documented free tokens/month — up to ~2.1B in your first month with signup credits (with compression: ~7.5B+ effective tokens)
---
## Common Questions
### "Is this really free?"
**Yes!** These are official free tiers from the providers. OmniRoute just makes it easier to use them all at once.
### "Will the free tier run out?"
Some providers have limits (like Kiro's 50 credits/month), but others are unlimited (like OpenCode Free and Pollinations). By connecting multiple providers, you always have a backup.
### "Can I use free providers for production?"
**Yes!** Many free providers are production-ready. However, for critical applications, consider adding a paid provider as a backup.
### "What's the catch?"
No catch! Providers offer free tiers to attract users. OmniRoute just makes it easier to use them all at once.
### "How do I get more free quota?"
1. Connect more free providers
2. Use compression to save tokens (15-95% savings)
3. Use `auto/cheap` to prioritize free/cheap providers
4. Create multiple accounts for the same provider
### "Do free providers have worse quality?"
**Not necessarily!** Many free providers offer the same models as paid providers. For example, Kiro gives you access to Claude Sonnet 4.5 — the same model you'd get with a paid Anthropic subscription.
---
## What's Next?
- **[Auto-Combo Guide](./AUTO-COMBO-GUIDE.md)** — Let OmniRoute pick the best AI for you
- **[Providers Guide](./PROVIDERS-GUIDE.md)** — Connect more providers
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
- **[Free Tiers Reference](../reference/FREE_TIERS.md)** — Full list of free tiers
docs/getting-started/PROVIDERS-GUIDE.md
+220
View File
@@ -0,0 +1,220 @@
# Providers Guide: Connect AI Models to OmniRoute
> **TL;DR**: A provider is a connection to an AI service (like OpenAI, Anthropic, Google). You need at least one provider to use OmniRoute.
---
## What Is a Provider?
Think of a provider like a **phone carrier**. Just as you need a phone carrier to make calls, you need an AI provider to use AI models. OmniRoute is like a phone that works with **all carriers** — you can switch between them automatically.
### Types of Providers
| Type | What It Is | Examples | Cost |
| -------------- | ------------------------- | --------------------------------- | ---------------------- |
| **Free** | No payment required | Kiro, OpenCode Free, Pollinations | $0 |
| **API Key** | You need an API key | OpenAI, Anthropic, Google | Pay per use |
| **OAuth** | Login with your account | Claude Code, GitHub Copilot | Subscription |
| **Web Cookie** | Uses your browser session | ChatGPT Web, Gemini Web | $0 (uses your account) |
---
## Quick Start: Connect Your First Provider
### Option A: Free Provider (No Credit Card)
1. Open the dashboard at `http://localhost:20128`
2. Go to **Providers** → **Add Provider**
3. Select one of these free providers:
- **Kiro AI** — Free Claude models (no auth needed)
- **OpenCode Free** — Free GPT models (no auth needed)
- **Pollinations** — Free GPT-5, Claude, Gemini (no key needed)
- **LongCat** — 10M tokens free (one-time grant, requires account + KYC)
- **Cloudflare AI** — 50+ models, 10K neurons/day
4. Click **Connect**
5. Done! You now have free AI access.
### Option B: API Key Provider (Paid)
1. Get an API key from the provider's website:
- **OpenAI**: https://platform.openai.com/api-keys
- **Anthropic**: https://console.anthropic.com/
- **Google**: https://aistudio.google.com/apikey
- **DeepSeek**: https://platform.deepseek.com/
- **Groq**: https://console.groq.com/
2. Open the dashboard at `http://localhost:20128`
3. Go to **Providers** → **Add Provider**
4. Select your provider
5. Paste your API key
6. Click **Connect**
7. Done! You now have access to that provider's models.
### Option C: OAuth Provider (Subscription)
1. Open the dashboard at `http://localhost:20128`
2. Go to **Providers** → **Add Provider**
3. Select your provider (e.g., Claude Code, GitHub Copilot)
4. Click **Connect with OAuth**
5. Login with your account
6. Done! You now have access to your subscription models.
---
## Best Free Providers
These providers offer **free access** with no credit card:
| Provider | Free Quota | Models | How to Connect |
| ----------------- | ---------------- | ---------------------------------------- | -------------- |
| **Kiro AI** | 50 credits/month | Claude Sonnet 4.5, Haiku 4.5, Opus 4.6 | No auth needed |
| **OpenCode Free** | Unlimited | GPT-4o, Claude, Gemini | No auth needed |
| **Pollinations** | No key needed | GPT-5, Claude, Gemini, DeepSeek, Llama 4 | No auth needed |
| **LongCat** | 10M one-time | LongCat-2.0 | API key + KYC |
| **Cloudflare AI** | 10K neurons/day | 50+ models | No auth needed |
| **NVIDIA NIM** | ~40 RPM | 129 models | API key needed |
| **Cerebras** | 1M tokens/day | Qwen3 235B, GPT-OSS 120B | API key needed |
| **Qwen** | Unlimited | Qwen3-coder-plus/flash/next | No auth needed |
| **Qoder** | Unlimited | Kimi-K2, DeepSeek-R1, Qwen3-coder | No auth needed |
**Tip**: Connect multiple free providers for **unlimited free AI** with automatic fallback!
---
## Best Paid Providers
These providers offer **high-quality models** with API keys:
| Provider | Best Models | Cost | Free Tier |
| ------------- | --------------------------- | ---------------------- | ------------------ |
| **OpenAI** | GPT-5, GPT-4o | $2.50-$10/1M tokens | $5 free credits |
| **Anthropic** | Claude Opus 4.6, Sonnet 4.6 | $3-$15/1M tokens | $5 free credits |
| **Google** | Gemini 2.5 Pro, Flash | $0.075-$1.25/1M tokens | 1,500 req/day free |
| **DeepSeek** | DeepSeek V4 | $0.14-$0.28/1M tokens | 5M free tokens |
| **Groq** | Llama 4, Mixtral | $0.05-$0.27/1M tokens | 30 RPM free |
| **xAI** | Grok 3 | $0.30-$0.60/1M tokens | — |
---
## How to Connect a Provider (Step-by-Step)
### Step 1: Open the Dashboard
Go to `http://localhost:20128` in your browser.
### Step 2: Go to Providers
Click **Providers** in the sidebar.
### Step 3: Click Add Provider
Click the **+ Add Provider** button.
### Step 4: Select Your Provider
Browse the list or search for your provider. Click on it.
### Step 5: Enter Credentials
- **Free providers**: No credentials needed — just click **Connect**
- **API key providers**: Paste your API key
- **OAuth providers**: Click **Connect with OAuth** and login
### Step 6: Test the Connection
Click **Test Connection** to verify it works.
### Step 7: Done!
Your provider is now connected. You can use it with `model: "auto"` or specify the provider directly.
---
## Using Multiple Providers
OmniRoute works best with **multiple providers**. This gives you:
- **Automatic fallback** — If one provider fails, OmniRoute tries the next
- **Cost optimization** — OmniRoute picks the cheapest provider for each request
- **Speed optimization** — OmniRoute picks the fastest provider for each request
- **Quality optimization** — OmniRoute picks the best provider for each task
### Recommended Setup
Connect at least **3 providers** for the best experience:
1. **One free provider** (Kiro, OpenCode Free, or Pollinations) — Always available
2. **One fast provider** (Groq, Cerebras) — For quick responses
3. **One quality provider** (OpenAI, Anthropic, Google) — For complex tasks
Then use `model: "auto"` and OmniRoute will automatically pick the best one for each request.
---
## Provider-Specific Setup
### OpenAI
1. Get API key: https://platform.openai.com/api-keys
2. In OmniRoute: Providers → Add Provider → OpenAI
3. Paste API key → Connect
### Anthropic
1. Get API key: https://console.anthropic.com/
2. In OmniRoute: Providers → Add Provider → Anthropic
3. Paste API key → Connect
### Google (Gemini)
1. Get API key: https://aistudio.google.com/apikey
2. In OmniRoute: Providers → Add Provider → Gemini
3. Paste API key → Connect
### DeepSeek
1. Get API key: https://platform.deepseek.com/
2. In OmniRoute: Providers → Add Provider → DeepSeek
3. Paste API key → Connect
### Groq
1. Get API key: https://console.groq.com/
2. In OmniRoute: Providers → Add Provider → Groq
3. Paste API key → Connect
---
## Common Questions
### "Do I need to pay to use OmniRoute?"
**No!** OmniRoute is free and open-source. You can use free providers (Kiro, OpenCode Free, Pollinations) without paying anything. You only pay if you choose to use paid providers.
### "Which provider should I start with?"
Start with **Kiro AI** — it's free, requires no API key, and gives you access to Claude models. Then add more providers as needed.
### "Can I use multiple providers at once?"
**Yes!** That's the whole point of OmniRoute. Connect multiple providers and use `model: "auto"` to let OmniRoute pick the best one for each request.
### "What if a provider goes down?"
OmniRoute automatically skips failed providers and tries the next one. You don't need to do anything.
### "How do I disconnect a provider?"
Go to Providers → click on the provider → click **Disconnect**.
### "Can I use my existing API keys?"
**Yes!** If you already have API keys for OpenAI, Anthropic, Google, etc., you can use them in OmniRoute. Just paste them when connecting the provider.
---
## What's Next?
- **[Auto-Combo Guide](./AUTO-COMBO-GUIDE.md)** — Let OmniRoute pick the best AI for you
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — Full list of 226 providers
docs/getting-started/QUICK-START.md
+135
View File
@@ -0,0 +1,135 @@
# Quick Start: Get OmniRoute Running in 3 Minutes
> **TL;DR**: Install → Connect a free provider → Point your IDE to OmniRoute. Done.
---
## Step 1: Install OmniRoute
Choose your preferred method:
### Option A: npm (Recommended)
```bash
npm install -g omniroute
```
### Option B: Docker
```bash
docker run -d --name omniroute -p 20128:20128 diegosouzapw/omniroute:latest
```
### Option C: From Source
```bash
git clone https://github.com/diegosouzapw/OmniRoute.git
cd OmniRoute
npm install
npm run dev
```
---
## Step 2: Start OmniRoute
```bash
omniroute
```
OmniRoute starts at `http://localhost:20128`. The dashboard opens automatically.
---
## Step 3: Connect a Free Provider
You can use OmniRoute **without paying anything** by connecting a free provider.
### Option A: Kiro (Free Claude — No Credit Card)
1. Open the dashboard at `http://localhost:20128`
2. Go to **Providers** → **Add Provider**
3. Select **Kiro AI**
4. Click **Connect** (no API key needed!)
5. Done! You now have free access to Claude models.
### Option B: OpenCode Free (No Auth)
1. Open the dashboard at `http://localhost:20128`
2. Go to **Providers** → **Add Provider**
3. Select **OpenCode Free**
4. Click **Connect** (no API key needed!)
5. Done! You now have free access to multiple models.
### Option C: Pollinations (No Key Needed)
1. Open the dashboard at `http://localhost:20128`
2. Go to **Providers** → **Add Provider**
3. Select **Pollinations**
4. Click **Connect** (no API key needed!)
5. Done! You now have free access to GPT-5, Claude, Gemini, and more.
---
## Step 4: Point Your IDE to OmniRoute
In your IDE or CLI tool, set:
```
Base URL: http://localhost:20128/v1
API Key: [copy from Dashboard → Endpoints]
Model: auto
```
That's it! Your IDE now uses OmniRoute with automatic provider selection.
---
## Step 5: Verify It Works
```bash
curl http://localhost:20128/v1/models -H "Authorization: Bearer YOUR_KEY"
```
You should see your connected models listed.
---
## What's Next?
- **[Auto-Combo Guide](./AUTO-COMBO-GUIDE.md)** — Let OmniRoute pick the best AI for you
- **[Providers Guide](./PROVIDERS-GUIDE.md)** — Connect more providers (free and paid)
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
---
## Common Questions
### "Do I need an API key?"
**No!** You can use free providers (Kiro, OpenCode Free, Pollinations) without any API key. Just connect them in the dashboard.
### "What is `auto`?"
`auto` tells OmniRoute to automatically pick the best provider for each request. It considers speed, cost, quality, and availability. See the [Auto-Combo Guide](./AUTO-COMBO-GUIDE.md) for details.
### "How much does it cost?"
OmniRoute itself is **free and open-source**. You only pay for the providers you use. Many providers have free tiers — see the [Free Tiers Guide](./FREE-TIERS-GUIDE.md).
### "Can I use it with Claude Code / Cursor / Copilot?"
**Yes!** OmniRoute works with any tool that supports OpenAI format. Just set the base URL to `http://localhost:20128/v1`. See the [CLI Tools Guide](../reference/CLI-TOOLS.md) for specific setup instructions.
### "What if a provider goes down?"
OmniRoute automatically skips failed providers and tries the next one. You don't need to do anything. See the [Auto-Combo Guide](./AUTO-COMBO-GUIDE.md) for details.
---
## Need Help?
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Common issues and fixes
- **[Discord](https://discord.gg/EkzRkpzKYt)** — Community support
- **[GitHub Issues](https://github.com/diegosouzapw/OmniRoute/issues)** — Report bugs
docs/getting-started/TROUBLESHOOTING.md
+504
View File
@@ -0,0 +1,504 @@
---
title: "Troubleshooting"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Troubleshooting
> **For Users**: Looking for quick fixes? See the [Quick Reference](#quick-reference) below.
🌐 **Languages:** 🇺🇸 [English](./TROUBLESHOOTING.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/TROUBLESHOOTING.md) | 🇪🇸 [Español](../i18n/es/docs/guides/TROUBLESHOOTING.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/TROUBLESHOOTING.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/TROUBLESHOOTING.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/TROUBLESHOOTING.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/TROUBLESHOOTING.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/TROUBLESHOOTING.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/guides/TROUBLESHOOTING.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/TROUBLESHOOTING.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/TROUBLESHOOTING.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/TROUBLESHOOTING.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/TROUBLESHOOTING.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/TROUBLESHOOTING.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/TROUBLESHOOTING.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/TROUBLESHOOTING.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/TROUBLESHOOTING.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/TROUBLESHOOTING.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/TROUBLESHOOTING.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/TROUBLESHOOTING.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/TROUBLESHOOTING.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/TROUBLESHOOTING.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/TROUBLESHOOTING.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/TROUBLESHOOTING.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/TROUBLESHOOTING.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/TROUBLESHOOTING.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/TROUBLESHOOTING.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/TROUBLESHOOTING.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/TROUBLESHOOTING.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/TROUBLESHOOTING.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/TROUBLESHOOTING.md)
Common problems and solutions for OmniRoute.
---
## Quick Reference
**New to OmniRoute?** Start here — these solve 90% of problems:
| I see this | What it means | What to do |
| ----------------------- | ----------------------------------- | ------------------------------------------------------------------------------------------------- |
| "Can't connect" | OmniRoute isn't running | Run `omniroute` or `docker restart omniroute` |
| "Invalid API key" | Your key is wrong or expired | Re-copy the key from the provider's website |
| "Rate limit exceeded" | You're sending too many requests | Wait 1 minute, or use `model: "auto"` for automatic fallback |
| "Quota exceeded" | You've used up your free/paid quota | Connect more providers, or use free providers (Kiro, Pollinations) |
| "Slow responses" | Provider is busy or far away | Use `model: "auto/fast"` or connect a faster provider (Groq, Cerebras) |
| "Wrong provider used" | `auto` picked a different provider | That's normal! `auto` picks the best one. Force a specific provider with `model: "openai/gpt-4o"` |
| "502 Bad Gateway" | Provider is down | Wait and retry, or use `model: "auto"` to switch providers |
| "401 Unauthorized" | Your credentials are wrong | Check your API key or re-authenticate with OAuth |
| "429 Too Many Requests" | Rate limited | Wait 1 minute, or connect more providers |
**Still stuck?** See the [Quick Fixes](#quick-fixes) below, or ask on [Discord](https://discord.gg/EkzRkpzKYt).
---
## Quick Fixes
| Problem | Solution |
| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| First login not working | Set `INITIAL_PASSWORD` in `.env` (no hardcoded default) |
| Dashboard opens on wrong port | Set `PORT=20128` and `NEXT_PUBLIC_BASE_URL=http://localhost:20128` |
| No logs written to disk | Set `APP_LOG_TO_FILE=true` and verify call log capture is enabled |
| EACCES: permission denied | Set `DATA_DIR=/path/to/writable/dir` to override `~/.omniroute` |
| Routing strategy not saving | Update to the latest v3.x release (Zod schema fix for settings persistence shipped in earlier versions) |
| Login crash / blank page | Check Node.js version — see [Node.js Compatibility](#nodejs-compatibility) below |
| `dlopen` / `slice is not valid mach-o file` (macOS) | Run `cd $(npm root -g)/omniroute/app && npm rebuild better-sqlite3 && omniroute` — see [macOS native module rebuild](#macos-native-module-rebuild) below |
| Proxy "fetch failed" | Ensure proxy config is set at the correct level — see [Proxy Issues](#proxy-issues) below |
---
## Node.js Compatibility
<a name="nodejs-compatibility"></a>
### Login page crashes or shows "Module self-registration" error
**Cause:** You are running a Node.js version outside OmniRoute's approved secure runtime floor. The most common case is running an older Node 22 or 24 patch level that falls below the patched security floor OmniRoute requires.
**Symptoms:**
- Login page shows a blank screen or a server error
- Console shows `Error: Module did not self-register` or similar native binding errors
- The login page shows an **orange warning banner** with your Node version if the runtime is outside the supported secure policy
**Fix:**
1. Install a supported Node.js LTS release (recommended: Node.js 24.x):
```bash
nvm install 24
nvm use 24
```
2. Verify your version: `node --version` should show `v24.0.0` or newer on the 24.x LTS line
3. Reinstall OmniRoute: `npm install -g omniroute`
4. Restart: `omniroute`
> **Supported secure versions:** `>=22.22.2 <23` or `>=24.0.0 <27`. Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported.
### macOS: `dlopen` / "slice is not valid mach-o file"
<a name="macos-native-module-rebuild"></a>
**Cause:** After a global `npm install -g omniroute`, the `better-sqlite3` native binary inside the package may have been compiled for a different architecture or Node.js ABI than what is running locally. This is common on macOS (both Apple Silicon and Intel) when the pre-built binary does not match your environment.
**Symptoms:**
- Server fails immediately on startup with a `dlopen` error
- Error contains `slice is not valid mach-o file`
- Full example:
```
dlopen(/Users/<user>/.nvm/versions/node/v24.14.1/lib/node_modules/omniroute/app/node_modules/better-sqlite3/build/Release/better_sqlite3.node, 0x0001): tried: '...' (slice is not valid mach-o file)
```
**Fix — rebuild for your local environment (no Node.js downgrade required):**
```bash
cd $(npm root -g)/omniroute/app
npm rebuild better-sqlite3
omniroute
```
> **Note:** This recompiles the native binding against your local Node.js version and CPU architecture, resolving the binary mismatch. The officially supported runtime range is **`>=22.22.2 <23` or `>=24.0.0 <27`** (`SUPPORTED_NODE_RANGE` in `src/shared/utils/nodeRuntimeSupport.ts`, aligned with the `package.json` `engines` field). Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported with `better-sqlite3` v12.x.
---
## Proxy Issues
<a name="proxy-issues"></a>
### Provider validation shows "fetch failed"
**Cause:** The API key validation endpoint (`POST /api/providers/validate`) was previously bypassing proxy configuration, causing failures in environments that require proxy routing.
**Fix (v3.5.5+):** This is now fixed. Provider validation routes through `runWithProxyContext`, honoring provider-level and global proxy settings automatically.
### Token health check fails with "fetch failed"
**Cause:** Background OAuth token refresh was not resolving proxy configuration per connection.
**Fix (v3.5.5+):** The token health check scheduler now resolves proxy config per connection before attempting refresh. Update to v3.5.5+.
### SOCKS5 proxy returns "invalid onRequestStart method"
**Cause:** On Node.js 22, the undici@8 dispatcher is incompatible with Node's built-in `fetch()` implementation.
**Fix (v3.5.5+):** OmniRoute now uses undici's own `fetch()` function when a proxy dispatcher is active, ensuring consistent behavior. Update to v3.5.5+.
---
## Provider Issues
### "Language model did not provide messages"
**Cause:** Provider quota exhausted.
**Fix:**
1. Check dashboard quota tracker
2. Use a combo with fallback tiers
3. Switch to cheaper/free tier
### Rate Limiting
**Cause:** Subscription quota exhausted.
**Fix:**
- Add fallback: `cc/claude-opus-4-6 → glm/glm-4.7 → if/kimi-k2-thinking`
- Use GLM/MiniMax as cheap backup
### OAuth Token Expired
OmniRoute auto-refreshes tokens. If issues persist:
1. Dashboard → Provider → Reconnect
2. Delete and re-add the provider connection
### Kiro multi-account: second account invalidates the first
**Cause:** Kiro's backend enforces a single active session per OIDC client registration.
When two accounts share the same registered client (connections imported before v3.8.0),
refreshing one account's token invalidates the other's refresh token.
**Fix (v3.8.0+):** Re-import affected connections.
Starting with v3.8.0, every new Kiro connection created via **Import Token**,
**Google/GitHub social login**, or **Auto-Import** automatically registers its own
dedicated OIDC client. The connection is therefore fully isolated and refreshing one
account has no effect on any other account.
Connections that were imported _before_ v3.8.0 do not carry a per-connection client
registration. Those connections continue to use the shared social-auth refresh endpoint.
To gain isolation, delete the old connection from Dashboard → Providers and re-add it
via any of the three import flows.
For full details and step-by-step instructions for adding two Kiro accounts side by side,
see [`docs/guides/KIRO_SETUP.md`](../guides/KIRO_SETUP.md).
---
## Cloud Issues
### Cloud Sync Errors
1. Verify `BASE_URL` points to your running instance (e.g., `http://localhost:20128`)
2. Verify `CLOUD_URL` points to your cloud endpoint (e.g., `https://omniroute.dev`)
3. Keep `NEXT_PUBLIC_*` values aligned with server-side values
### Cloud `stream=false` Returns 500
**Symptom:** `Unexpected token 'd'...` on cloud endpoint for non-streaming calls.
**Cause:** Upstream returns SSE payload while client expects JSON.
**Workaround:** Use `stream=true` for cloud direct calls. Local runtime includes SSE→JSON fallback.
### Cloud Says Connected but "Invalid API key"
1. Create a fresh key from local dashboard (`/api/keys`)
2. Run cloud sync: Enable Cloud → Sync Now
3. Old/non-synced keys can still return `401` on cloud
---
## Docker Issues
### CLI Tool Shows Not Installed
1. Check runtime fields: `curl http://localhost:20128/api/cli-tools/runtime/codex | jq`
2. For portable mode: use image target `runner-cli` (bundled CLIs)
3. For host mount mode: set `CLI_EXTRA_PATHS` and mount host bin directory as read-only
4. If `installed=true` and `runnable=false`: binary was found but failed healthcheck
### Quick Runtime Validation
```bash
curl -s http://localhost:20128/api/cli-tools/codex-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
curl -s http://localhost:20128/api/cli-tools/claude-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
curl -s http://localhost:20128/api/cli-tools/openclaw-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
```
---
## Cost Issues
### High Costs
1. Check usage stats in Dashboard → Usage
2. Switch primary model to GLM/MiniMax
3. Use free tier (Qoder, Kiro) for non-critical tasks
4. Set cost budgets per API key: Dashboard → API Keys → Budget
---
## Debugging
### Enable Log Files
Set `APP_LOG_TO_FILE=true` in your `.env` file. Application logs are written under `logs/`.
Request artifacts are stored under `${DATA_DIR}/call_logs/` when the call log pipeline is
enabled in settings.
When pipeline capture is enabled, set `CALL_LOG_PIPELINE_CAPTURE_STREAM_CHUNKS=false` to omit
stream chunk payloads, or tune `CALL_LOG_PIPELINE_MAX_SIZE_KB` to change the artifact cap in KB.
### Check Provider Health
```bash
# Health dashboard
http://localhost:20128/dashboard/health
# API health check
curl http://localhost:20128/api/monitoring/health
```
### Runtime Storage
- Main state: `${DATA_DIR}/storage.sqlite` (providers, combos, aliases, keys, settings)
- Usage: SQLite tables in `storage.sqlite` (`usage_history`, `call_logs`, `proxy_logs`) + optional `${DATA_DIR}/call_logs/`
- Application logs: `<repo>/logs/...` (when `APP_LOG_TO_FILE=true`)
- Call log artifacts: `${DATA_DIR}/call_logs/YYYY-MM-DD/...` when the call log pipeline is enabled
The Request Logs page's **Clean history** action clears `call_logs`, legacy
`request_detail_logs`, and the local `${DATA_DIR}/call_logs/` artifact directory.
---
## Circuit Breaker Issues
### Provider stuck in OPEN state
When a provider's circuit breaker is OPEN, requests are blocked until the cooldown expires.
**Fix:**
1. Go to **Dashboard → Settings → Resilience**
2. Check the circuit breaker card for the affected provider
3. Click **Reset All** to clear all breakers, or wait for the cooldown to expire
4. Verify the provider is actually available before resetting
### Provider keeps tripping the circuit breaker
If a provider repeatedly enters OPEN state:
1. Check **Dashboard → Health → Provider Health** for the failure pattern
2. Go to **Settings → Resilience → Provider Profiles** and increase the failure threshold
3. Check if the provider has changed API limits or requires re-authentication
4. Review latency telemetry — high latency may cause timeout-based failures
---
## Audio Transcription Issues
### "Unsupported model" error
- Ensure you're using the correct prefix: `deepgram/nova-3` or `assemblyai/best`
- Verify the provider is connected in **Dashboard → Providers**
### Transcription returns empty or fails
- Check supported audio formats: `mp3`, `wav`, `m4a`, `flac`, `ogg`, `webm`
- Verify file size is within provider limits (typically < 25MB)
- Check provider API key validity in the provider card
---
## Translator Debugging
Use **Dashboard → Translator** to debug format translation issues:
| Mode | When to Use |
| ---------------- | -------------------------------------------------------------------------------------------- |
| **Playground** | Compare input/output formats side by side — paste a failing request to see how it translates |
| **Chat Tester** | Send live messages and inspect the full request/response payload including headers |
| **Test Bench** | Run batch tests across format combinations to find which translations are broken |
| **Live Monitor** | Watch real-time request flow to catch intermittent translation issues |
### Common format issues
- **Thinking tags not appearing** — Check if the target provider supports thinking and the thinking budget setting
- **Tool calls dropping** — Some format translations may strip unsupported fields; verify in Playground mode
- **System prompt missing** — Claude and Gemini handle system prompts differently; check translation output
- **SDK returns raw string instead of object** — Resolved in v1.x; response sanitizer strips non-standard fields (`x_groq`, `usage_breakdown`, etc.) that cause OpenAI SDK Pydantic validation failures. If you still see this on v3.x+, please file an issue.
- **GLM/ERNIE rejects `system` role** — Resolved in v1.x; role normalizer automatically merges system messages into user messages for incompatible models. If you still see this on v3.x+, please file an issue.
- **`developer` role not recognized** — Resolved in v1.x; automatically converted to `system` for non-OpenAI providers. If you still see this on v3.x+, please file an issue.
- **`json_schema` not working with Gemini** — Resolved in v1.x; `response_format` is now converted to Gemini's `responseMimeType` + `responseSchema`. If you still see this on v3.x+, please file an issue.
---
## Resilience Settings
### Auto rate-limit not triggering
- Auto rate-limit only applies to API key providers (not OAuth/subscription)
- Verify **Settings → Resilience → Provider Profiles** has auto-rate-limit enabled
- Check if the provider returns `429` status codes or `Retry-After` headers
### Tuning exponential backoff
Provider profiles support these settings:
- **Base delay** — Initial wait time after first failure (default: 1s)
- **Max delay** — Maximum wait time cap (default: 30s)
- **Multiplier** — How much to increase delay per consecutive failure (default: 2x)
### Anti-thundering herd
When many concurrent requests hit a rate-limited provider, OmniRoute uses mutex + auto rate-limiting to serialize requests and prevent cascading failures. This is automatic for API key providers.
---
## Optional RAG / LLM failure taxonomy (16 problems)
Some OmniRoute users place the gateway in front of RAG or agent stacks. In those setups it is common to see a strange pattern: OmniRoute looks healthy (providers up, routing profiles ok, no rate limit alerts) but the final answer is still wrong.
In practice these incidents usually come from the downstream RAG pipeline, not from the gateway itself.
If you want a shared vocabulary to describe those failures you can use the WFGY ProblemMap, an external MIT license text resource that defines sixteen recurring RAG / LLM failure patterns. At a high level it covers:
- retrieval drift and broken context boundaries
- empty or stale indexes and vector stores
- embedding versus semantic mismatch
- prompt assembly and context window issues
- logic collapse and overconfident answers
- long chain and agent coordination failures
- multi agent memory and role drift
- deployment and bootstrap ordering problems
The idea is simple:
1. When you investigate a bad response, capture:
- user task and request
- route or provider combo in OmniRoute
- any RAG context used downstream (retrieved documents, tool calls, etc)
2. Map the incident to one or two WFGY ProblemMap numbers (`No.1` … `No.16`).
3. Store the number in your own dashboard, runbook, or incident tracker next to the OmniRoute logs.
4. Use the corresponding WFGY page to decide whether you need to change your RAG stack, retriever, or routing strategy.
Full text and concrete recipes live here (MIT license, text only):
[WFGY ProblemMap README](https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md)
You can ignore this section if you do not run RAG or agent pipelines behind OmniRoute.
---
## v3.8.0 Known Issues
Issues specific to the v3.8.0 release and their current workarounds. If a fix lands in a later patch, the entry will be updated or removed.
### Windsurf OAuth flow fails with 401
**Symptoms:**
- "401 unauthorized" while completing the Windsurf OAuth flow from the dashboard
- Windsurf provider card stays in "needs reconnection" state after the callback
**Causes:**
- `WINDSURF_FIREBASE_API_KEY` env var missing or empty
- `WINDSURF_API_KEY` misconfigured or pointing at a stale token
- Local firewall/proxy blocking the OAuth callback
**Fix:**
1. Verify both `WINDSURF_FIREBASE_API_KEY` and `WINDSURF_API_KEY` are set in `.env`
2. Restart OmniRoute so the new env values are picked up
3. Re-run the OAuth flow from **Dashboard → Providers → Windsurf → Reconnect**
### Devin CLI auth failures
**Symptoms:**
- "Devin CLI not found" or "auth failed" when invoking Devin-backed tools
- CLI runtime check reports `installed=false`
**Causes:**
- `CLI_DEVIN_BIN` points to a path that does not exist
- Devin CLI is not installed on the host
**Fix:**
1. Install the Devin CLI for your platform
2. Set `CLI_DEVIN_BIN=/usr/local/bin/devin` (or the real path) in `.env`
3. Restart OmniRoute and re-test from **Dashboard → CLI Tools**
### Model cooldown stuck (manual reset)
**Symptoms:**
- A model stays listed in cooldown even after the expiration time has passed
- Requests still skip the model in combo routing despite the timestamp being in the past
**Manual reset:**
- **Dashboard:** **Settings → Model Cooldowns** → click **Re-enable** on the affected card
- **API:** `DELETE /api/resilience/model-cooldowns` with management auth headers
### Command Code provider connection fails with 403
**Symptoms:**
- 403 when testing the Command Code provider connection
- The provider card shows "unauthorized" after a fresh add
**Cause:** The OAuth flow did not complete (callback not received or token not persisted).
**Fix:**
- Run `omniroute providers` from the CLI to re-trigger the OAuth flow, or
- Re-run OAuth from **Dashboard → Providers → Command Code → Reconnect**
### ModelScope returns aggressive 429 cooldowns
**Symptoms:**
- Very short or immediate cooldowns on ModelScope after a small burst of requests
- Combo routing skips ModelScope earlier than expected
**Cause:** ModelScope emits provider-specific `Retry-After` headers. v3.8.0 ships dedicated handling for those headers, so older versions misread them as generic rate-limit hints.
**Fix:**
- Ensure you are on v3.8.0 or later
- Verify the `useUpstream429BreakerHints` toggle is enabled under **Settings → Resilience**
### OMNIROUTE_WS_BRIDGE_SECRET missing in production
**Symptoms:**
- 401 on every Codex/Responses WebSocket bridge request when running on a remote production host
- WebSocket bridge handshake closes immediately after connect
**Cause:** The `OMNIROUTE_WS_BRIDGE_SECRET` env var is missing from the production environment.
**Fix:**
1. Generate a random secret: `openssl rand -hex 32`
2. Set `OMNIROUTE_WS_BRIDGE_SECRET=<random-secret>` in the production server env (and any client that talks to the bridge)
3. Restart OmniRoute
### Responses API: background mode degraded to synchronous
**Symptoms:**
- Warning logged: `background mode degraded to synchronous`
- A `background: true` request returns a normal synchronous response instead of a background job handle
**Cause:** v3.8.0 intentionally degrades `background: true` on the Responses API to synchronous execution while emitting a warning. Full async background execution is a future deliverable.
**Fix:**
- Adjust the client to call without `background`, or
- Wait for a later release that ships full async background mode (track the changelog)
---
## Still Stuck?
- **GitHub Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
- **Architecture**: See [`docs/architecture/ARCHITECTURE.md`](../architecture/ARCHITECTURE.md) for internal details
- **API Reference**: See [`docs/reference/API_REFERENCE.md`](../reference/API_REFERENCE.md) for all endpoints
- **Health Dashboard**: Check **Dashboard → Health** for real-time system status
- **Translator**: Use **Dashboard → Translator** to debug format issues
docs/getting-started/meta.json
+11
View File
@@ -0,0 +1,11 @@
{
"title": "Getting Started",
"description": "Get started with OmniRoute in minutes — no technical background needed",
"pages": [
"QUICK-START",
"AUTO-COMBO-GUIDE",
"PROVIDERS-GUIDE",
"FREE-TIERS-GUIDE",
"TROUBLESHOOTING"
]
}
docs/guides/CLAUDE-CODE-CONFIGURATION.md
+152
View File
@@ -0,0 +1,152 @@
---
title: "Claude Code CLI — Configuration with OmniRoute"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Claude Code CLI — Configuration with OmniRoute
Point the **Claude Code** CLI (`claude`) at OmniRoute — local or a remote VPS —
with per-model profiles, mirroring the Codex setup.
---
## Quick start
```bash
# Launch Claude Code against a local OmniRoute (auto-detects the active context)
omniroute launch
# Against a remote OmniRoute (after `omniroute connect <host>`, this is automatic)
omniroute launch --remote http://192.168.0.15:20128 --api-key oma_live_xxx
# Generate per-model profiles, then launch one
omniroute setup-claude # writes ~/.claude/profiles/<name>/settings.json
omniroute launch --profile glm52 # Claude Code using glm/glm-5.2 via OmniRoute
```
---
## How Claude Code connects to a gateway
Claude Code talks the **Anthropic Messages API** and is pointed at a custom
endpoint with environment variables (it has no `--base-url` flag):
| Variable | Purpose |
| -------------------------------------------- | -------------------------------------------------------------------------------------- |
| `ANTHROPIC_BASE_URL` | Gateway root URL (Claude Code appends `/v1/messages`). **No `/v1` suffix.** |
| `ANTHROPIC_AUTH_TOKEN` | Sent as `Authorization: Bearer …` — use your OmniRoute access token / API key |
| `ANTHROPIC_API_KEY` | Alternative: sent as `x-api-key`. If both set, `ANTHROPIC_AUTH_TOKEN` wins |
| `ANTHROPIC_MODEL` | Force a specific model (overrides the `/model` picker default) |
| `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY` | `1` → the native `/model` picker lists `claude*`/`anthropic*` models from `/v1/models` |
| `CLAUDE_CODE_MAX_OUTPUT_TOKENS` | Cap output tokens per response (e.g. `65536`) |
| `CLAUDE_CODE_AUTO_COMPACT_WINDOW` | Token threshold for auto-compaction |
> Env vars are read **once at startup** — restart Claude Code after changing them.
`omniroute launch` sets all of these for you: it resolves the base URL + token
from the active context (so `omniroute connect <vps>` then `omniroute launch`
just works), health-checks the server, and execs `claude`.
---
## Profiles (`CLAUDE_CONFIG_DIR`)
Claude Code has **no native profile files** (unlike Codex's `~/.codex/<name>.config.toml`).
The idiomatic mechanism is `CLAUDE_CONFIG_DIR` — a separate config directory per
profile, each with its own `settings.json`, credentials, history and cache.
`omniroute setup-claude` fetches the live `/v1/models` catalog and writes one
profile per model at `~/.claude/profiles/<name>/settings.json`, reusing the
**same names as `setup-codex`** (`glm52`, `kimi-k27`, `deepseek-pro`, …):
```jsonc
// ~/.claude/profiles/glm52/settings.json
{
"$schema": "https://json.schemastore.org/claude-code-settings.json",
"model": "glm/glm-5.2",
"effortLevel": "xhigh",
"env": {
"ANTHROPIC_BASE_URL": "http://192.168.0.15:20128",
"ANTHROPIC_MODEL": "glm/glm-5.2",
"CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY": "1",
"CLAUDE_CODE_AUTO_COMPACT_WINDOW": "190000",
},
}
```
> **The auth token is never written to the profile.** Launch with
> `omniroute launch --profile <name>` (it injects `ANTHROPIC_AUTH_TOKEN` from the
> active context), or export `ANTHROPIC_AUTH_TOKEN` yourself and run
> `CLAUDE_CONFIG_DIR=~/.claude/profiles/<name> claude`.
**Auto-sync after model discovery (opt-in).** OmniRoute can regenerate these same
`~/.claude/profiles/<name>/settings.json` files automatically whenever a provider model
sync changes the live catalog — so new/renamed models get profiles without re-running the
command. It is **off by default**: toggle it from the **CLI Code dashboard** ("CLI profile
auto-sync" → Claude Code), or set `OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES=true` (it also honors
`CLI_ALLOW_CONFIG_WRITES`, on by default). When enabled it only writes profile files; it never
changes your active/default Claude config, auth, or the `~/.claude/settings.json`.
### Generating + using profiles
```bash
# Local OmniRoute
omniroute setup-claude
# Remote VPS (bakes the VPS URL into every profile)
omniroute setup-claude --remote http://192.168.0.15:20128 --api-key oma_live_xxx
# Only some providers
omniroute setup-claude --only glm,kimi
# Preview without writing
omniroute setup-claude --dry-run
# Launch a profile
omniroute launch --profile kimi-k27
```
---
## Model tiers (optional)
Claude Code routes to capability tiers. Map each to an OmniRoute model via env /
settings if you want different providers per tier:
```bash
export ANTHROPIC_DEFAULT_OPUS_MODEL="glm/glm-5.2"
export ANTHROPIC_DEFAULT_SONNET_MODEL="kmc/kimi-k2.6"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="glm/glm-4.7-flash"
```
Otherwise a single `ANTHROPIC_MODEL` (what profiles set) is used for everything.
---
## Remote mode
Once you've run `omniroute connect <host>` (see
[Remote Mode](./REMOTE-MODE.md)), `omniroute launch` and `omniroute setup-claude`
automatically target that remote server and use its scoped access token — no
extra flags needed. Override per-invocation with `--remote` / `--api-key`.
---
## Troubleshooting
**Claude Code ignores the gateway** — confirm `ANTHROPIC_BASE_URL` has **no
`/v1`** and restart `claude` (env is read once at startup). `omniroute launch`
handles this for you.
**`/model` picker is empty / missing gateway models** — needs Claude Code
v2.1.129+ and `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1`. Only `claude*` /
`anthropic*` model IDs appear in the picker; force any other model with
`ANTHROPIC_MODEL=<id>` (this is what profiles do).
**Auth errors** — the profile holds no token. Use `omniroute launch --profile`
(injects it) or export `ANTHROPIC_AUTH_TOKEN`.
**Profiles don't isolate** — each profile is a distinct `CLAUDE_CONFIG_DIR`;
verify `echo $CLAUDE_CONFIG_DIR` inside the session points at
`~/.claude/profiles/<name>`.
docs/guides/CLI-INTEGRATIONS.md
+202
View File
@@ -0,0 +1,202 @@
---
title: "CLI Integrations — point any coding CLI at OmniRoute"
version: 3.8.40
lastUpdated: 2026-06-28
---
# CLI Integrations
OmniRoute ships a family of `setup-*` commands that configure a coding
CLI (Codex, Claude Code, OpenCode, Cline, …) to use OmniRoute as its backend — so
the tool talks to **one** endpoint and OmniRoute routes to the right provider with
auto-fallback. Each command reads the **live** model catalog from a running
OmniRoute (local or remote) and writes the tool's own config file on **your**
machine. The API key is referenced by env var wherever the tool supports it, so the
secret is never written to disk (the exceptions are noted below).
There are also two launchers — `omniroute launch` (Claude Code) and
`omniroute launch-codex` (Codex) — that spawn the CLI with the right env injected,
without writing any config at all.
For the one-time, hand-written base setup of the two richest integrations, see the
per-tool deep dives:
- [Claude Code configuration](./CLAUDE-CODE-CONFIGURATION.md)
- [Codex CLI configuration](./CODEX-CLI-CONFIGURATION.md)
- [Remote Mode](./REMOTE-MODE.md) — drive a remote OmniRoute (VPS / Tailnet) from your laptop
---
## Master table
Every command honours the **active context** (set with `omniroute connect`, see
[Remote Mode](./REMOTE-MODE.md)) or explicit `--remote <url> --api-key <key>` flags.
"Local vs remote" below means: with no flags it targets `http://localhost:20128`;
with `--remote` (or an active remote context) it fetches the catalog from that
server and writes the config locally.
| Command | Tool | What it writes | Key flags | Local vs remote |
|---------|------|----------------|-----------|-----------------|
| `omniroute setup-codex` | OpenAI Codex CLI | `~/.codex/<name>.config.toml` — one profile per compatible text model (`codex --profile <name>`) | `--remote` `--api-key` `--only` `--dry-run` `--port` `--codex-home` | Both |
| `omniroute setup-claude` | Claude Code | `~/.claude/profiles/<name>/settings.json` — one profile per matched model (`CLAUDE_CONFIG_DIR`) | `--remote` `--api-key` `--only` `--dry-run` `--port` `--claude-home` | Both |
| `omniroute setup-opencode` | OpenCode (openai-compatible) | `~/.config/opencode/opencode.json` — `omniroute` provider with every catalog model (`opencode -m omniroute/<model>`) | `--remote` `--api-key` `--only` `--model` `--dry-run` `--port` | Both |
| `omniroute setup-cline` | Cline | `~/.cline/data/{globalState,secrets}.json` (CLI mode) + prints VS Code extension settings | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--cline-dir` | Both |
| `omniroute setup-kilo` | Kilo Code | `~/.local/share/kilo/auth.json` (CLI) + merges `kilocode.*` into VS Code `settings.json` if present | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--auth-path` `--vscode-settings` | Both |
| `omniroute setup-continue` | Continue / `cn` CLI | `~/.continue/config.yaml` — `provider: openai` models, key via `${{ secrets.OMNIROUTE_API_KEY }}` | `--remote` `--api-key` `--only` `--dry-run` `--port` `--config-path` | Both |
| `omniroute setup-cursor` | Cursor | Nothing — prints the in-app steps (Cursor config is opaque SQLite) | `--remote` `--api-key` `--only` `--port` | Both |
| `omniroute setup-roo` | Roo Code | `~/.omniroute/roo-settings.json` (import doc) + sets `roo-cline.autoImportSettingsPath` if a VS Code `settings.json` exists | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--import-path` `--vscode-settings` | Both |
| `omniroute setup-crush` | Crush | `~/.config/crush/crush.json` — `openai-compat` provider, key via `$OMNIROUTE_API_KEY` | `--remote` `--api-key` `--only` `--dry-run` `--port` `--config-path` | Both |
| `omniroute setup-goose` | Goose | `~/.config/goose/config.yaml` (`GOOSE_PROVIDER`/`OPENAI_HOST`/`GOOSE_MODEL`) + prints env recipe | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--config-path` | Both |
| `omniroute setup-qwen` | Qwen Code | `~/.qwen/settings.json` — openai `modelProvider`, key via `envKey` (`OMNIROUTE_API_KEY`) | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--config-path` | Both |
| `omniroute setup-aider` | Aider | `~/.aider.conf.yml` (`openai-api-base` + `model: openai/<id>`) + prints env recipe | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--config-path` | Both |
| `omniroute launch` | Claude Code | Nothing — spawns `claude` with `ANTHROPIC_BASE_URL`/`ANTHROPIC_AUTH_TOKEN` injected | `--remote` `--api-key` `--token` `--profile` `--port` | Both |
| `omniroute launch-codex` | OpenAI Codex CLI | Nothing — spawns `codex` with the `omniroute` provider injected via `-c` flags | `--remote` `--api-key` `--profile` (`-p`) `--port` | Both |
Notes on flags (verified in the command source):
- `--remote <url>` — fetch the catalog from a remote OmniRoute (overrides `--port`
and the active context). `--api-key <key>` supplies the credential for that
server (defaults to the `OMNIROUTE_API_KEY` env var, or the active context's token).
- `--only <patterns>` — comma-separated substrings; keep only model IDs that match
(e.g. `--only glm,kimi`). Available on `setup-codex`, `setup-claude`,
`setup-opencode`, `setup-continue`, `setup-cursor`, `setup-crush`.
- `--dry-run` — print exactly what would be written without touching the
filesystem. Available on every `setup-*` command **except** `setup-cursor`
(which never writes a file).
- `--model <id>` — required (or picked interactively) for the tools that have no
model auto-discovery: Cline, Kilo, Roo, Goose, Qwen, Aider. Those tools
also accept `--yes` for non-interactive runs (which then requires `--model`).
`setup-opencode` takes `--model` to set the default top-level model.
- `--port <port>` — local OmniRoute port (default `20128`, ignored when `--remote`
is set). Present on all `setup-*` and both launchers.
- The two launchers (`launch`, `launch-codex`) accept `--profile <name>` to select
a profile written by `setup-claude` / `setup-codex`, plus pass-through args for
the underlying `claude` / `codex` binary.
> `setup-opencode` is the **lightweight openai-compatible** OpenCode integration.
> There is also a richer plugin integration — `omniroute setup opencode` — which
> installs `@omniroute/opencode-plugin`. They are different commands; the table
> above documents `setup-opencode`.
---
## Local usage
With OmniRoute running on `localhost:20128`, just run the setup command for your
tool. The catalog is fetched from the local server.
```bash
# Codex: write a profile per matched model into ~/.codex/
omniroute setup-codex
codex --profile glm52 # use a generated profile
# Claude Code: write per-model profiles, then launch one
omniroute setup-claude
omniroute launch --profile glm52
# OpenCode: write the openai-compatible provider with all catalog models
omniroute setup-opencode
export OMNIROUTE_API_KEY=sk-... # referenced via {env:OMNIROUTE_API_KEY}, never on disk
opencode -m omniroute/glm/glm-5.2 "..."
# Tools without auto-discovery need an explicit model:
omniroute setup-aider --model glm/glm-5.2
omniroute setup-qwen --model kmc/kimi-k2.7
# Preview without writing anything:
omniroute setup-continue --dry-run
```
Launch without writing any config at all (env-injection only):
```bash
omniroute launch # Claude Code → local OmniRoute
omniroute launch-codex # Codex CLI → local OmniRoute
omniroute launch-codex --profile glm52
```
---
## Remote usage
Point any setup command at a remote OmniRoute with `--remote` + `--api-key`. The
catalog is fetched from the remote; the config is written on your local machine.
```bash
# OpenCode against a remote VPS, keep only glm/kimi models
omniroute setup-opencode --remote http://192.168.0.15:20128 --api-key oma_live_xxx \
--only glm,kimi
opencode -m omniroute/glm/glm-5.2 "..." # export OMNIROUTE_API_KEY first
# Codex profiles from a remote catalog
omniroute setup-codex --remote http://192.168.0.15:20128 --api-key oma_live_xxx
# Launch a CLI straight against the remote
omniroute launch --remote http://192.168.0.15:20128 --api-key oma_live_xxx
omniroute launch-codex --remote http://192.168.0.15:20128 --api-key oma_live_xxx
```
Instead of passing `--remote`/`--api-key` every time, log in once and let the
**active context** supply them automatically:
```bash
omniroute connect 192.168.0.15 # mints a scoped token, stores the context
omniroute setup-codex # ← now uses the remote catalog
omniroute setup-opencode # ← same
omniroute launch # ← Claude Code against the remote
```
See [Remote Mode](./REMOTE-MODE.md) for contexts, scopes, and token management.
---
## Base URL conventions (which tools want `/v1`)
OmniRoute exposes the OpenAI surface at `/v1`, the Anthropic surface at the root,
and a native Gemini surface at `/v1beta`. Each integration is wired to the form its
tool expects (verified in the command source):
| Integration | Base URL written | `/v1`? |
|-------------|------------------|--------|
| `setup-cline` (`openAiBaseUrl`) | root | No — Cline appends `/v1/chat/completions` |
| `setup-goose` (`OPENAI_HOST`) | root | No — Goose appends the path |
| `setup-aider` (`OPENAI_API_BASE`) | root | No — LiteLLM appends `/v1/chat/completions` |
| `setup-kilo`, `setup-roo`, `setup-continue`, `setup-crush`, `setup-qwen`, `setup-cursor` | with `/v1` | Yes |
| `setup-claude` (`ANTHROPIC_BASE_URL`), `launch` | root | No — Claude Code appends `/v1/messages` |
| `setup-codex`, `launch-codex` (`model_providers.omniroute.base_url`) | with `/v1` | Yes |
---
## Keeping native deps on update: `--include=optional`
When you update with `omniroute update` (after confirming, or with `--apply`),
OmniRoute runs the install with `--include=optional` baked in:
```bash
npm install -g omniroute@latest --include=optional
```
This is **not** a flag you pass to `omniroute update` — it is always applied by the
updater. It guarantees the `optionalDependencies` (`better-sqlite3`, `keytar`,
`tls-client`, the LLMLingua SLM stack) survive the update even if your npm config
has `omit=optional` set, which would otherwise silently drop the native SQLite
driver and OS-keyring binding. To preview the exact command without applying:
```bash
omniroute update --dry-run
# [DRY RUN] Would run: npm install -g omniroute@latest --include=optional
```
Other `omniroute update` flags (verified in source): `--check` (exit 1 if
outdated), `--apply` (install without prompting), `--changelog`, `--no-backup`,
`--yes`.
---
## See also
- [Claude Code configuration](./CLAUDE-CODE-CONFIGURATION.md) — the deeper Claude Code guide
- [Codex CLI configuration](./CODEX-CLI-CONFIGURATION.md) — the one-time `[model_providers.omniroute]` base setup
- [Remote Mode](./REMOTE-MODE.md) — contexts, scoped access tokens, driving a remote server
- [CLI Tools reference](../reference/CLI-TOOLS.md) — the full catalog of supported tools + dashboard pages
- [Setup Guide](./SETUP_GUIDE.md) — install methods and first-run onboarding
docs/guides/CODEX-CLI-CONFIGURATION.md
+413
View File
@@ -0,0 +1,413 @@
---
title: "Codex CLI — Configuration with OmniRoute"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Codex CLI — Configuration with OmniRoute
Complete guide for using the Codex CLI pointed at OmniRoute as an OpenAI-compatible backend.
---
## Ready-to-paste config.toml
Replace `<YOUR_HOST>` and `<YOUR_KEY>` with your values:
```toml
# ~/.codex/config.toml
model = "cx/gpt-5.5"
model_provider = "omniroute"
model_reasoning_effort = "xhigh"
model_context_window = 400000
model_auto_compact_token_limit = 350000
tool_output_token_limit = 32768 # history storage cap per tool call
[model_providers.omniroute]
name = "OmniRoute"
base_url = "http://<YOUR_HOST>:20128/v1"
env_key = "OMNIROUTE_API_KEY"
requires_openai_auth = false
wire_api = "responses"
```
```bash
# ~/.bashrc or ~/.zshrc — actual key value, never in config.toml
export OMNIROUTE_API_KEY="<YOUR_KEY>"
```
> **Common host options**
>
> | Access | URL |
> | ------------- | ----------------------------- |
> | Local network | `http://192.168.0.1:20128/v1` |
> | Tailscale | `http://100.x.x.x:20128/v1` |
> | Loopback | `http://localhost:20128/v1` |
---
## `wire_api = "responses"` — why it works for all models
Codex CLI deprecated `wire_api = "chat"` (Chat Completions) in February 2026 and now **requires** `wire_api = "responses"` (OpenAI Responses API). Setting `wire_api = "chat"` causes an immediate startup crash since v0.138.
DeepSeek, GLM, Kimi and others only expose a Chat Completions endpoint — not the Responses API. If you pointed Codex directly at them, it would fail.
**OmniRoute solves this transparently:**
```
Codex CLI
→ wire_api = "responses"
→ POST /v1/responses (OmniRoute)
→ OmniRoute Responses ↔ Chat Completions transformer
→ POST /chat/completions (DeepSeek / Mistral / GLM / Kimi / any provider)
```
You never need a separate translation proxy when using OmniRoute. **All models use `wire_api = "responses"`** — OmniRoute handles the rest.
> **`wire_api` is the default** — the field defaults to `"responses"` and can be omitted entirely from `config.toml`. Only ever set it explicitly if you're documenting intent.
---
## Context window and compaction
### Token configuration fields
| Field | Description |
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model_context_window` | Total token budget for the active model. Set to the model's advertised limit. |
| `model_auto_compact_token_limit` | Threshold that triggers automatic history compaction. **Maximum: 90% of `model_context_window`** — values above 90% are silently ignored. |
| `tool_output_token_limit` | Cap on tokens stored per tool call output in history. Prevents a single large tool response from filling the window. **This is not the max output** — it is a history storage cap. |
| `compact_prompt` | Inline override for the system prompt used during compaction (v0.138+). |
> **Note on `model_max_output_tokens`**: This field is **not part of the Codex CLI config schema** (absent from the Codex Rust codebase). It is silently ignored if set. Do not rely on it — use `tool_output_token_limit` to control how much tool output is stored in history.
### Context windows by model
| Model | OmniRoute ID | Context window | `auto_compact` | `tool_output_limit` |
| ------------------------------------ | ------------------------------------ | ---------------------- | -------------- | ------------------- |
| GPT-5.5 | `cx/gpt-5.5` | 400k reliable (1M max) | 350,000 | 32,768 |
| Kimi K2.7 (thinking) | `kmc/kimi-k2.7` | 131,072 | 112,000 | 32,768 |
| Kimi K2.6 | `kmc/kimi-k2.6` | 131,072 | 112,000 | 32,768 |
| GLM-5.2 / 5.2-max (thinking) | `glm/glm-5.2` | 131,072 | 112,000 | 32,768 |
| MiMo V2.5 Pro (thinking) | `opencode-go/mimo-v2.5-pro` | 131,072 | 112,000 | 32,768 |
| Qwen 3.7 Plus (thinking) | `opencode-go/qwen3.7-plus` | 32,768 | 28,000 | 16,384 |
| DeepSeek V4 Pro (OllamaCloud) | `ollamacloud/deepseek-v4-pro` | 131,072 | 112,000 | 32,768 |
| DeepSeek V4 Pro | `ds/deepseek-v4-pro` | 1,000,000 | 900,000 | 65,536 |
| MiMo V2.5 | `opencode-go/mimo-v2.5` | 131,072 | 112,000 | 32,768 |
| Gemma 4 31B (OllamaCloud) | `ollamacloud/gemma4:31b` | 32,768 | 28,000 | 16,384 |
| Nemotron 3 Super (OllamaCloud) | `ollamacloud/nemotron-3-super` | 32,768 | 28,000 | 16,384 |
| GPT-OSS 20B (OllamaCloud) | `ollamacloud/gpt-oss:20b` | 32,768 | 28,000 | 16,384 |
| DeepSeek V4 Flash (OllamaCloud) | `ollamacloud/deepseek-v4-flash` | 65,536 | 56,000 | 16,384 |
| Gemini 3 Flash Preview (OllamaCloud) | `ollamacloud/gemini-3-flash-preview` | 1,000,000 | 850,000 | 32,768 |
| GLM-5 Turbo | `glm/glm-5-turbo` | 131,072 | 112,000 | 16,384 |
| GLM-4.7 Flash | `glm/glm-4.7-flash` | 131,072 | 112,000 | 16,384 |
| Mistral Large Latest | `mistral/mistral-large-latest` | 262,144 | 220,000 | 16,384 |
> **Compaction formula:** `effective_window = model_context_window - min(tool_output_token_limit, 20000)`. Values above 20k do not change the compaction trigger.
> **Rule of thumb:** set `model_auto_compact_token_limit` to 85–88% of `model_context_window`. Never go above 90% — silently ignored.
---
## Model prefix: `cx/`
All Codex models in OmniRoute use the `cx/` prefix:
| Codex CLI name | OmniRoute model |
| ----------------------- | ------------------ |
| `cx/gpt-5.5` | GPT-5.5 standard |
| `cx/gpt-5.4` | GPT-5.4 standard |
| `cx/gpt-5.4-mini` | GPT-5.4 mini |
| `cx/gpt-5.1-codex-mini` | GPT-5.1 Codex mini |
Other providers use their own prefix (`kmc/`, `glm/`, `ds/`, `ollamacloud/`, `opencode-go/`, `mistral/`) — the prefix matches the OmniRoute provider alias.
---
## Reasoning Effort
Controls how much the model "thinks" before responding.
| Value | Use for |
| -------- | --------------------------------------------- |
| `none` | No reasoning — direct response |
| `low` | Trivial tasks (rename, format) |
| `medium` | **Server default** when not specified |
| `high` | Intermediate tasks (refactoring, debug) |
| `xhigh` | Architecture, deep analysis, complex problems |
```bash
# Per invocation override
codex -c model_reasoning_effort=low "rename variable x to count"
codex -c model_reasoning_effort=xhigh "design the auth module"
```
---
## Profiles — named configurations per model/workflow
Profiles let you switch model + context window with a single flag. Each profile is a flat
`~/.codex/<name>.config.toml` that overlays on top of the base `config.toml`.
> **Naming rule (Codex CLI v0.137+):** file must be `~/.codex/<name>.config.toml` — **no `profile-` prefix**.
> The CLI resolves `-p kimi-k27` → `~/.codex/kimi-k27.config.toml`. If the file is not found, the default applies silently.
```bash
codex --profile kimi-k27 "analyze 10k lines of this codebase"
codex -p glm52 "architecture review"
codex --profile deepseek-flash "rename variable" # fast, cheap
```
### Effort profiles (same model, different effort)
```bash
codex -p low # cx/gpt-5.5, effort=low
codex -p medium # cx/gpt-5.5, effort=medium
codex -p high # cx/gpt-5.5, effort=high
codex -p xhigh # cx/gpt-5.5, effort=xhigh (default)
codex -p chat # cx/gpt-5.5, no effort set (server default)
```
### Thinking models (alto pensamento) — xhigh + detailed summary
| Profile | Model | Context | Use for |
| ------------ | --------------------------- | ------- | ---------------------------- |
| `kimi-k27` | `kmc/kimi-k2.7` | 128k | Best thinking quality (Kimi) |
| `glm52` | `glm/glm-5.2` | 128k | GLM thinking |
| `glm52max` | `glm/glm-5.2-max` | 128k | GLM thinking max |
| `mimo-pro` | `opencode-go/mimo-v2.5-pro` | 128k | MiMo thinking |
| `qwen37plus` | `opencode-go/qwen3.7-plus` | 32k | Qwen thinking |
### Good models (bons) — high effort
| Profile | Model | Context | Use for |
| -------------- | ----------------------------- | ------- | --------------------------------- |
| `kimi-k26` | `kmc/kimi-k2.6` | 128k | General purpose (Kimi) |
| `deepseek-pro` | `ollamacloud/deepseek-v4-pro` | 128k | DeepSeek Pro via OllamaCloud |
| `deepseek` | `ds/deepseek-v4-pro` | 1M | DeepSeek Pro direct, huge context |
| `mimo` | `opencode-go/mimo-v2.5` | 128k | MiMo general |
### Simple models (simples) — no reasoning effort
| Profile | Model | Context | Use for |
| ---------- | ------------------------------ | ------- | ----------------------- |
| `gemma4` | `ollamacloud/gemma4:31b` | 32k | Cost-effective, capable |
| `nemotron` | `ollamacloud/nemotron-3-super` | 32k | NVIDIA Nemotron |
| `gptoss` | `ollamacloud/gpt-oss:20b` | 32k | Open-source GPT |
### Fast models — low effort
| Profile | Model | Context | Use for |
| ---------------- | ------------------------------------ | ------- | ----------------------- |
| `deepseek-flash` | `ollamacloud/deepseek-v4-flash` | 64k | Quick tasks |
| `gemini-flash` | `ollamacloud/gemini-3-flash-preview` | 1M | Very fast, huge context |
| `glm5turbo` | `glm/glm-5-turbo` | 128k | GLM Turbo |
| `glm47flash` | `glm/glm-4.7-flash` | 128k | GLM Flash |
| `mistral` | `mistral/mistral-large-latest` | 256k | Mistral Large |
### Quick decision table
| Task | Recommended profile |
| -------------------------------- | ------------------------------------------------ |
| Rename, format, boilerplate | `--profile deepseek-flash` or `-p low` |
| Explain, light review | `-p chat` or `-p gemini-flash` |
| Debug, moderate refactor | `-p medium` or `-p kimi-k26` |
| New feature, complex tests | `-p high` or `-p mimo` |
| Architecture, deep analysis | `-p kimi-k27` or `-p glm52` or `-p xhigh` |
| Codebase analysis (needs 1M ctx) | `--profile deepseek` or `--profile gemini-flash` |
| Maximum thinking quality | `-p glm52max` or `-p mimo-pro` |
| Cost-conscious | `-p gemma4` or `-p gptoss` |
---
## Generating profiles automatically with `omniroute setup-codex`
If you run OmniRoute on a VPS, you can auto-generate profile files from the live model catalog:
```bash
# From a VPS (uses local OmniRoute on port 20128)
omniroute setup-codex
# From any machine — point at your VPS
omniroute setup-codex --remote http://100.x.x.x:20128 --api-key sk-xxx
# Preview without writing files
omniroute setup-codex --remote http://100.x.x.x:20128 --dry-run
# Only generate GLM and Kimi profiles
omniroute setup-codex --only glm,kimi
# Write to a custom directory
omniroute setup-codex --codex-home /path/to/.codex
```
The command fetches `/v1/models`, uses tuned profiles for known models, falls back to catalog metadata for other compatible text models, and writes `~/.codex/<name>.config.toml` for each. Idempotent — safe to re-run.
OmniRoute can also **auto-sync** these same profile files after a successful provider model discovery/import changes the live catalog. This is **opt-in and off by default**: toggle it from the **CLI Code dashboard** ("CLI profile auto-sync" → Codex), or set `OMNIROUTE_AUTO_SYNC_CODEX_PROFILES=true` (it also honors `CLI_ALLOW_CONFIG_WRITES`, on by default). When enabled it only writes separate `~/.codex/*.config.toml` profile files; it never changes the active/default `~/.codex/config.toml`, Codex-lb settings, auth, or provider selection.
---
## Launching Codex with `omniroute launch-codex`
Health-checks your OmniRoute instance before launching Codex:
```bash
# Launch against local OmniRoute (default port 20128)
omniroute launch-codex
# Launch with a specific profile
omniroute launch-codex --profile kimi-k27
# Launch against a remote VPS
omniroute launch-codex --remote http://100.x.x.x:20128/v1 --api-key sk-xxx
# Pass extra args to codex
omniroute launch-codex --profile glm52 -- --yolo "fix this bug"
```
---
## New Codex CLI features (v0.138–v0.141)
| Version | Feature |
| ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| v0.138 | Desktop app handoff (`/app`), v2 personal access tokens, `--profile` as the exclusive profile selector (legacy in-file `[profiles]` tables crash on startup) |
| v0.139 | `web_search = "live"` — native web search from code mode; `oneOf`/`allOf` in MCP tool schemas; `codex doctor` env diagnostics |
| v0.140 | `/usage` token view in-session; `/import` from Claude Code sessions; `codex delete <SESSION_ID>` subcommand; Amazon Bedrock auth via `aws` object in provider config |
| v0.141 | E2E encrypted Noise relay for remote executors; SQLite WAL fix; P-521 TLS support |
### New `config.toml` fields (post-v0.137)
```toml
# Native web search (v0.139)
web_search = "live" # "disabled" | "cached" | "live"
# Separate developer system prompt (v0.138)
developer_instructions = "Always prefer functional style."
# Custom compaction prompt
compact_prompt = "Summarise the above as bullet points."
# Route /review to a cheaper model
review_model = "glm/glm-5-turbo"
# OpenAI service tier
service_tier = "fast" # "fast" | "flex"
```
### New `[model_providers.<id>]` fields
```toml
[model_providers.omniroute]
base_url = "http://100.x.x.x:20128/v1"
env_key = "OMNIROUTE_API_KEY"
requires_openai_auth = false
# Static extra headers on every request
[model_providers.omniroute.http_headers]
"X-Custom-Header" = "value"
# Headers read from env vars
[model_providers.omniroute.env_http_headers]
"X-Trace-Id" = "TRACE_ID"
# Extra URL query params (useful for Azure api-version)
[model_providers.omniroute.query_params]
"api-version" = "2024-12-01-preview"
```
### Amazon Bedrock auth (v0.140)
```toml
[model_providers.bedrock]
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
[model_providers.bedrock.aws]
profile = "default" # ~/.aws/credentials profile
region = "us-east-1"
```
---
## Multiple servers
```toml
[model_providers.omniroute-main]
base_url = "http://192.168.0.1:20128/v1"
env_key = "OMNIROUTE_API_KEY"
[model_providers.omniroute-tailscale]
base_url = "http://100.x.x.x:20128/v1"
env_key = "OMNIROUTE_API_KEY"
```
---
## Claude Code — equivalent configuration
| Codex CLI (`config.toml`) | Claude Code (env var) | Effect |
| --------------------------------- | ------------------------------------- | ----------------------- |
| `tool_output_token_limit = 32768` | _(not directly exposed)_ | Per-tool history cap |
| `model_context_window = 400000` | _(determined by the model)_ | Context window |
| — | `CLAUDE_CODE_MAX_OUTPUT_TOKENS=65536` | Max tokens per response |
```bash
# ~/.bashrc — Claude Code token cap
export CLAUDE_CODE_MAX_OUTPUT_TOKENS=65536
```
---
## Quick reference — CLI flags
| Flag | Short | Effect |
| --------------------- | ----- | -------------------------------------------- |
| `--model <id>` | `-m` | Overrides `model` for this invocation |
| `--profile <name>` | `-p` | Loads `~/.codex/<name>.config.toml` |
| `--config key=value` | `-c` | Overrides any config.toml field (repeatable) |
| `--enable <feature>` | — | Force-enables a feature flag |
| `--disable <feature>` | — | Force-disables a feature flag |
| `--search` | — | Enable live web search for this invocation |
New in v0.140:
```bash
codex delete <SESSION_ID> # delete a session
codex delete <SESSION_ID> --force # skip confirmation
codex debug models --bundled # list bundled model catalog as JSON
```
Inside an interactive session:
| Command | Effect |
| --------- | ------------------------------------------- |
| `/model` | Opens the model picker |
| `/usage` | Shows token usage for this session (v0.140) |
| `/app` | Hands off to the desktop app (v0.138) |
| `/import` | Import a Claude Code session (v0.140) |
| `/help` | Lists all slash commands |
---
## Troubleshooting
**`Error: wire_api = "chat" is no longer supported`**
Remove `wire_api = "chat"` from your config. Set `wire_api = "responses"` or omit the field (defaults to `"responses"` since v0.138).
**`Error: model not found`**
Verify the model exists in OmniRoute with the correct prefix. Use `omniroute models list` or open `/dashboard/providers/<provider>`.
**`Authentication error`**
Confirm `OMNIROUTE_API_KEY` is exported: `echo $OMNIROUTE_API_KEY`.
**`Connection refused`**
Verify OmniRoute is running and the `base_url` host/port is correct for your network (local vs Tailscale vs VPS).
**Session crashes near context limit**
Set `model_context_window` and `model_auto_compact_token_limit` explicitly. See the context window table above.
**Compaction fires too late**
Lower `model_auto_compact_token_limit` to 80–85% of the window. Never set above 90%.
**Profile not loading (`-p <name>` silently ignored)**
Confirm the file exists at `~/.codex/<name>.config.toml` (no `profile-` prefix). Run `ls ~/.codex/*.config.toml`.
docs/guides/COST_TRACKING.md
+261
View File
@@ -0,0 +1,261 @@
---
title: "Cost & Spend Tracking"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Cost & Spend Tracking
How OmniRoute estimates, records, and reports the cost of every request — and why the
dashboard number is a **savings tracker**, not a bill.
See also: [User Guide](./USER_GUIDE.md) · [Features Gallery](./FEATURES.md)
---
## What it is (and what it is not)
OmniRoute attributes a per-request USD cost to every completion by multiplying token
counts by a model's pricing rates. These numbers power the **Costs** dashboard, the
`omniroute cost` / `omniroute usage` CLI, CSV/JSON exports, and per-API-key budgets.
> **The dashboard "cost" is a savings tracker, not a bill.** OmniRoute never charges you
> — it routes your requests to providers you have already connected (your own
> subscriptions, free tiers, and API keys). A "$290 total cost" accrued entirely on free
> models means roughly **$290 you did _not_ pay** a paid API. The figure is an _estimate_
> of what the same traffic would have cost at standard list prices, so you can see where
> your usage is concentrated and how much routing to cheaper/free providers is saving you.
This framing is stated directly in the project [README](../../README.md) ("the dashboard
'cost' is a savings tracker, not a bill").
Because the number is an estimate:
- It depends on the pricing table OmniRoute has for each model. A model with no pricing
entry contributes `0` cost (it shows as a "Legacy / Free" row in the explorer).
- Free-tier and subscription traffic still accrues an _estimated_ cost — that is the
amount you are saving, not an amount owed.
---
## How costs are estimated
### The pricing source
Costs come from a pricing table resolved in this precedence order
([`src/lib/pricingSync.ts`](../../src/lib/pricingSync.ts)):
1. **User overrides** — prices you set in the dashboard / via `PATCH /api/pricing`.
2. **Synced external pricing** — fetched from LiteLLM's public
`model_prices_and_context_window.json` when sync is enabled (stored in a separate
`pricing_synced` namespace so it never clobbers your overrides).
3. **Hardcoded defaults** — shipped with OmniRoute.
External pricing sync is **opt-in**, disabled by default. Relevant env vars
(see [`.env.example`](../../.env.example)):
| Env var | Default | Purpose |
| ----------------------- | --------- | ---------------------------------------------------------------- |
| `PRICING_SYNC_ENABLED` | `false` | Enable the background LiteLLM pricing sync at startup. |
| `PRICING_SYNC_INTERVAL` | `86400` | Sync interval in **seconds** (default daily). |
| `PRICING_SYNC_SOURCES` | `litellm` | Comma-separated source list (only `litellm` is supported today). |
### The cost formula
Cost is computed per request from token counts and per-million-token rates in
[`src/lib/usage/costCalculator.ts`](../../src/lib/usage/costCalculator.ts)
(`computeCostFromPricing` / `calculateCost`):
- **Input tokens** (minus cache reads and cache-creation tokens) × `input` rate.
- **Cache-read tokens** × `cached` rate (falls back to the input rate).
- **Cache-creation tokens** × `cache_creation` rate (falls back to the input rate).
- **Output tokens** × `output` rate.
- **Reasoning tokens** × `reasoning` rate (falls back to the output rate).
All rates are interpreted as USD per 1,000,000 tokens. A Codex "fast"/"priority" or
"flex" service tier applies a cost multiplier (`getCodexFastCostMultiplier`) — e.g. flex
is billed at a 50% token discount, surfaced as **flex savings** in the dashboard.
Model names are normalized first (provider-path prefixes such as `openai/` or
`accounts/fireworks/models/` are stripped) so historical rows still match a price.
### How spend is recorded
- The per-request cost is computed after the response and recorded fire-and-forget so it
never adds latency to the client. Shared-quota consumption is scheduled on the next
event-loop tick via [`src/lib/quota/spendRecorder.ts`](../../src/lib/quota/spendRecorder.ts).
- API-key spend is buffered and flushed in batches by the
[`SpendBatchWriter`](../../src/lib/spend/batchWriter.ts) (default 60s flush interval,
1,000-entry buffer). Tunable via:
| Env var | Default | Purpose |
| ----------------------------------- | ------- | ---------------------------------- |
| `OMNIROUTE_SPEND_FLUSH_INTERVAL_MS` | `60000` | Flush interval in milliseconds. |
| `OMNIROUTE_SPEND_MAX_BUFFER_SIZE` | `1000` | Max buffered entries before flush. |
The dashboard's cost figures are **not** read from a stored per-row dollar amount — they
are recomputed on the fly from token counts and the current pricing table each time the
analytics endpoint runs. That means correcting a wrong price (and re-syncing) updates
historical cost estimates retroactively.
---
## Dashboard: the Costs page
The **Costs** page lives at `/dashboard/costs`
(`src/app/(dashboard)/dashboard/costs/`).
Its main view is the **Cost Overview** tab
(`src/app/(dashboard)/dashboard/costs/CostOverviewTab.tsx`),
which loads everything from `GET /api/usage/analytics`.
What it shows:
- **Spend tiles** — estimated spend for _Today (1d)_, _7d_, _30d_, and the selected
window. Range selector: `7d`, `30d`, `90d`, `all`.
- **Headline metrics** — requests in window, active providers, active models, average
cost per request.
- **Cost Explorer** — a sortable/filterable table grouped by **provider**, **model**,
**API key**, **account**, or **service tier**, with cost, requests, tokens, avg
cost/request, and share-of-total %.
- **Token usage** — total / input / output tokens and input:output ratio.
- **Routing efficiency** — fallback count, fallback rate, and requested-model coverage.
- **Monthly forecast** — projects month-end spend from the recent daily average.
- **Period comparison** — % change between the first and second half of the window.
- **Charts** — daily cost trend, provider share (pie), top providers, top models, cost
by API key, cost by account, weekly usage pattern, and an activity heatmap.
- **Export** — download the current window as **CSV** or **JSON** (the buttons appear
once there is non-zero cost data).
When there is no priced traffic, rows render as a "Legacy / Free" label instead of `$0`,
reflecting the savings-tracker model.
### Related Costs sub-pages
The Costs area also hosts (all under `/dashboard/costs/`):
- **Pricing** (`/dashboard/costs/pricing`) — view and override per-model prices (renders
the shared Pricing tab).
- **Budget** (`/dashboard/costs/budget`) — set per-scope spend limits (renders the shared
Budget tab).
- **Quota Share** (`/dashboard/costs/quota-share`) — shared-quota pools and burn-rate
views.
---
## API endpoints
All of these require management auth (loopback/JWT, via `requireManagementAuth`) unless
noted.
### Usage & cost analytics
| Method | Endpoint | Purpose |
| ------ | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET` | `/api/usage/analytics` | Full cost/usage analytics: summary, daily trend, by provider/model/API key/account/tier. Query: `range`, `startDate`, `endDate`, `apiKeyIds`, `presets`. |
| `GET` | `/api/usage/utilization` | Per-provider quota utilization over time. Query: `range` (`1h`/`24h`/`7d`/`30d`), `provider`. |
| `GET` | `/api/usage/history` | Raw usage history rows. |
| `GET` | `/api/usage/call-logs` | Per-request call logs (model, tokens, cost, latency, status). |
| `GET` | `/api/usage/quota` | Provider quota status. |
| `GET` | `/api/usage/proxy-logs` | Proxy request logs. |
### Budgets
| Method | Endpoint | Purpose |
| ------ | ------------------------ | ------------------------------------------------------------------------------ |
| `GET` | `/api/usage/budget` | Cost summary + budget check for one API key (`apiKeyId` query param required). |
| `POST` | `/api/usage/budget` | Set daily/weekly/monthly USD limits + warning threshold for an API key. |
| `GET` | `/api/usage/budget/bulk` | Bulk budget summaries across API keys. |
> The budget API is scoped per **API key** (`apiKeyId`). Limits returned by
> `GET /api/usage/budget` include `dailyLimitUsd`, `weeklyLimitUsd`, `monthlyLimitUsd`,
> a `warningThreshold`, and the running totals (`totalCostToday`, `totalCostMonth`, …).
### Pricing
| Method | Endpoint | Purpose |
| -------- | ----------------------- | ----------------------------------------------------------------------------------------------- |
| `GET` | `/api/pricing` | Current merged pricing (user + synced + defaults). `?includeSources=1` to see source per entry. |
| `PATCH` | `/api/pricing` | Override pricing for `{ provider: { model: { input, output, cached, … } } }`. |
| `DELETE` | `/api/pricing` | Reset pricing to defaults (optionally scoped by `?provider=&model=`). |
| `GET` | `/api/pricing/defaults` | Show default per-1M fallback rates. |
| `GET` | `/api/pricing/models` | Pricing keyed by model. |
| `POST` | `/api/pricing/sync` | Trigger a manual sync from external sources (LiteLLM). |
| `GET` | `/api/pricing/sync` | Current sync status. |
| `DELETE` | `/api/pricing/sync` | Clear all synced pricing data. |
### Other cost-relevant endpoints
| Method | Endpoint | Purpose |
| ------ | ----------------------------- | ----------------------------------------------------------------------- |
| `GET` | `/api/free-tier/summary` | Free-model token totals, used-this-month, and remaining free allowance. |
| `GET` | `/api/quota/pools/[id]/usage` | Usage for a shared-quota pool. |
---
## CLI
OmniRoute's CLI exposes cost, usage, and pricing commands (registered in
[`bin/cli/commands/registry.mjs`](../../bin/cli/commands/registry.mjs)).
### `omniroute cost`
A cost report aggregated from `/api/usage/analytics`.
```bash
omniroute cost # last 30d, grouped by provider
omniroute cost --period 7d # last 7 days
omniroute cost --group-by model # group by provider | model | combo | api-key | day
omniroute cost --since 2026-06-01 --until 2026-06-13
omniroute cost --api-key <key> --limit 50
```
Columns: group, requests, tokens in/out, cost (USD), and % of total. A grand total line
is printed at the end (suppressed with `--quiet` or `--output json`).
### `omniroute usage`
```bash
omniroute usage analytics --period 30d [--provider <id>] # per-provider cost summary
omniroute usage logs [--limit 100] [--follow] [--api-key <k>] [--search <q>]
omniroute usage quota [--provider <id>] [--check]
omniroute usage utilization [--api-key <k>]
omniroute usage history [--limit 100]
omniroute usage proxy-logs [--limit 100]
# Budgets
omniroute usage budget list
omniroute usage budget get [scope]
omniroute usage budget set <amount> [--scope global] [--period monthly]
omniroute usage budget reset [scope]
```
### `omniroute pricing`
```bash
omniroute pricing list [--provider <p>] [--model <m>] [--limit 200]
omniroute pricing get <model>
omniroute pricing sync [--provider <p>] [--force] # POST /api/pricing/sync
omniroute pricing diff [--model <m>]
omniroute pricing defaults show
omniroute pricing defaults set [--input <p>] [--output <p>] [--cache-read <p>] [--cache-write <p>]
```
> `pricing defaults show` reads `GET /api/pricing/defaults`. To edit individual model
> prices instead, use the **Pricing** dashboard page or `PATCH /api/pricing`.
---
## Troubleshooting
- **All costs show $0 / "Legacy / Free".** The models in use have no pricing entry.
Enable external sync (`PRICING_SYNC_ENABLED=true`) and run `omniroute pricing sync`, or
set prices manually via the Pricing page / `PATCH /api/pricing`.
- **A historical model is mispriced.** Fix the price (override or re-sync) — cost is
recomputed from token counts on every analytics read, so estimates update retroactively.
- **Spend lags behind real time.** Per-key spend is batched; lower
`OMNIROUTE_SPEND_FLUSH_INTERVAL_MS` if you need fresher numbers.
---
For where this fits in the broader dashboard, see the [User Guide](./USER_GUIDE.md) and
the [Features Gallery](./FEATURES.md).
docs/guides/DOCKER_GUIDE.md
+254
View File
@@ -0,0 +1,254 @@
---
title: "🐳 Docker Guide — OmniRoute"
version: 3.8.40
lastUpdated: 2026-06-28
---
# 🐳 Docker Guide — OmniRoute
> Complete Docker deployment reference. For a quick start, see the [README Docker section](../README.md#-docker).
## Table of Contents
- [Quick Run](#quick-run)
- [With Environment File](#with-environment-file)
- [Docker Compose](#docker-compose)
- [Available Profiles](#available-profiles)
- [Redis Sidecar](#redis-sidecar)
- [Production Compose](#production-compose)
- [Dockerfile Stages](#dockerfile-stages)
- [Critical Environment Variables](#critical-environment-variables)
- [Docker Compose with Caddy (HTTPS)](#docker-compose-with-caddy-https-auto-tls)
- [Cloudflare Quick Tunnel](#cloudflare-quick-tunnel)
- [Image Tags](#image-tags)
- [Important Notes](#important-notes)
---
## Quick Run
```bash
docker run -d \
--name omniroute \
--restart unless-stopped \
--stop-timeout 40 \
-p 20128:20128 \
-v omniroute-data:/app/data \
diegosouzapw/omniroute:latest
```
## With Environment File
```bash
# Copy and edit .env first
cp .env.example .env
docker run -d \
--name omniroute \
--restart unless-stopped \
--stop-timeout 40 \
--env-file .env \
-p 20128:20128 \
-v omniroute-data:/app/data \
diegosouzapw/omniroute:latest
```
## Docker Compose
```bash
# Base profile (no CLI tools)
docker compose --profile base up -d
# CLI profile (Claude Code, Codex, OpenClaw built-in)
docker compose --profile cli up -d
# Host profile (Linux-first; mounts host CLI binaries read-only)
docker compose --profile host up -d
# Combine CLI + CLIProxyAPI sidecar
docker compose --profile cli --profile cliproxyapi up -d
```
## Available Profiles
OmniRoute ships four Compose profiles. Pick the one that matches your environment.
| Profile | Service | When to use | Command |
| ---------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| `base` (default) | `omniroute-base` | Headless server / minimal runtime, no provider CLIs bundled | `docker compose --profile base up -d` |
| `cli` | `omniroute-cli` | Agentic workflows that call `omniroute providers/setup/doctor` and bundled CLIs (Codex, Claude Code, Droid, OpenClaw) | `docker compose --profile cli up -d` |
| `host` | `omniroute-host` | Linux hosts that want `network_mode`-like access to host CLIs by mounting `~/.local/bin`, `~/.codex`, `~/.claude`, etc. read-only | `docker compose --profile host up -d` |
| `cliproxyapi` | `cliproxyapi` | Run the [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) sidecar on port `8317` for upstream CLI proxying | `docker compose --profile cliproxyapi up -d` |
> Multiple profiles can be combined: `docker compose --profile cli --profile cliproxyapi up -d`.
## Redis Sidecar
OmniRoute relies on Redis to back the distributed rate limiter and shared cache. The `redis` service is **always defined** in `docker-compose.yml` (it has no profile gate) and starts alongside any other profile.
| Detail | Value |
| -------------------- | --------------------------------- |
| Image | `redis:7-alpine` |
| Container name | `omniroute-redis` |
| Internal port | `6379` |
| Host port (override) | `REDIS_PORT` (defaults to `6379`) |
| Volume | `omniroute-redis-data` → `/data` |
| Healthcheck | `redis-cli ping` (10s interval) |
Related environment variables:
- `REDIS_URL` — connection string injected into the app (`redis://redis:6379` by default).
- `REDIS_PORT` — host-side port mapping for the Redis container.
**Disabling Redis** is not recommended (rate limiter will degrade to in-memory fallback). If you must, either remove/comment the `redis:` service block in `docker-compose.yml` or scale it to zero:
```bash
docker compose up -d --scale redis=0
```
## Production Compose
For an isolated production snapshot running alongside dev, use `docker-compose.prod.yml`.
| Detail | Value |
| ---------------------- | ---------------------------------------------------------------------------------- |
| File | `docker-compose.prod.yml` |
| Default dashboard port | `PROD_DASHBOARD_PORT=20130` (mapped to internal `${DASHBOARD_PORT:-20128}`) |
| Default API port | `PROD_API_PORT=20131` |
| Image | `omniroute:prod` (built from `runner-cli` target) |
| Redis container | `omniroute-redis-prod` (`redis:8.6.2`, dedicated `redis-prod-data` volume) |
| Data volume | `omniroute-prod-data` (named, persisted across rebuilds) |
| Healthchecks | `node healthcheck.mjs` + `redis-cli ping`, with `depends_on` gated on Redis health |
How to use:
```bash
# Build & start the production stack
docker compose -f docker-compose.prod.yml up -d --build
# Stream logs
docker compose -f docker-compose.prod.yml logs -f
# Tear down (keep volumes)
docker compose -f docker-compose.prod.yml down
```
The prod stack runs in parallel with the dev compose (different container names, ports, and volumes), so you can keep iterating locally while production stays up.
## Dockerfile Stages
The repository ships a multi-stage Dockerfile (`Dockerfile`). Three stages are exposed; pick the right `target` for your use case.
| Stage | Base image | Purpose |
| ------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `builder` | `node:24.15.0-trixie-slim` | Installs deps (`npm ci --legacy-peer-deps`) and runs `npm run build -- --webpack` |
| `runner-base` | `node:24.15.0-trixie-slim` | Production runtime with the Next.js standalone output. **No provider CLIs bundled.** |
| `runner-cli` | `runner-base` | Adds `git`, `docker.io`, `docker-compose` and global CLIs: `@openai/codex`, `@anthropic-ai/claude-code`, `droid`, `openclaw`. **Pick this for agentic workflows.** |
Build a specific target manually:
```bash
docker build --target runner-base -t omniroute:base .
docker build --target runner-cli -t omniroute:cli .
```
Defaults exported by `runner-base`: `PORT=20128`, `HOSTNAME=0.0.0.0`, `NODE_OPTIONS=--max-old-space-size=512`, `DATA_DIR=/app/data`, `OMNIROUTE_MIGRATIONS_DIR=/app/migrations`.
Memory behavior in Docker:
- `NODE_OPTIONS=--max-old-space-size=512` is baked into the image as a fallback.
- The actual server process is started by the standalone launcher, which reads `OMNIROUTE_MEMORY_MB` and appends `--max-old-space-size=<OMNIROUTE_MEMORY_MB>`.
- Node uses the last repeated `--max-old-space-size` value, so setting `OMNIROUTE_MEMORY_MB` controls the effective Docker heap limit.
- If `OMNIROUTE_MEMORY_MB` is unset, the launcher uses `512`.
## Critical Environment Variables
Beyond the defaults documented in [ENVIRONMENT.md](../reference/ENVIRONMENT.md), the following variables matter most when running under Docker:
| Variable | Purpose | Default |
| ----------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------ |
| `OMNIROUTE_WS_BRIDGE_SECRET` | Shared secret for the WebSocket bridge. **Required in production** — set to a strong random string. | unset (must be provided) |
| `REDIS_URL` | Connection string for the rate limiter / cache backend | `redis://redis:6379` |
| `REDIS_PORT` | Host-side port for the bundled Redis container | `6379` |
| `AUTO_UPDATE_HOST_REPO_DIR` | Host path mounted into `cli` profile at `/workspace/omniroute` for self-update workflows | `.` (current directory) |
| `OMNIROUTE_MEMORY_MB` | Runtime Node heap ceiling for the Docker standalone server; overrides the image fallback above | `512` |
| `DASHBOARD_PORT` / `API_PORT` | Override exposed ports for dashboard (20128) and API (20129) | `20128` / `20129` |
| `PROD_DASHBOARD_PORT` | Host-side dashboard port for `docker-compose.prod.yml` | `20130` |
| `CLIPROXYAPI_PORT` | Host-side port for the `cliproxyapi` sidecar | `8317` |
## Docker Compose with Caddy (HTTPS Auto-TLS)
OmniRoute can be securely exposed using Caddy's automatic SSL provisioning. Ensure your domain's DNS A record points to your server's IP.
```yaml
services:
omniroute:
image: diegosouzapw/omniroute:latest
container_name: omniroute
restart: unless-stopped
volumes:
- omniroute-data:/app/data
environment:
- PORT=20128
# Browser-facing origin for OAuth callbacks, dashboard links, and generated public URLs.
- NEXT_PUBLIC_BASE_URL=https://your-domain.com
# Internal server-to-server URL for scheduled jobs / self-fetches.
- BASE_URL=http://omniroute:20128
- AUTH_COOKIE_SECURE=true
caddy:
image: caddy:latest
container_name: caddy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
command: caddy reverse-proxy --from https://your-domain.com --to http://omniroute:20128
volumes:
omniroute-data:
```
Caddy sets the standard forwarding headers for the upstream container. OmniRoute uses
`NEXT_PUBLIC_BASE_URL` as the canonical public origin for OAuth callbacks and generated public
links; authenticated dashboard writes use same-origin requests plus session-bound CSRF
protection. Only enable `OMNIROUTE_TRUST_PROXY` for advanced deployments where you intentionally
want OmniRoute to derive the public origin from trusted forwarded headers instead of explicit
configuration.
## Cloudflare Quick Tunnel
Dashboard support for Docker deployments includes a one-click **Cloudflare Quick Tunnel** on `Dashboard → Endpoints`. The first enable downloads `cloudflared` only when needed, starts a temporary tunnel to your current `/v1` endpoint, and shows the generated `https://*.trycloudflare.com/v1` URL directly below your normal public URL.
Endpoint tunnel panels (Cloudflare, Tailscale, ngrok) can be shown or hidden from `Settings → Appearance` without changing active tunnel state.
### Tunnel Notes
- Quick Tunnel URLs are temporary and change after every restart.
- Quick Tunnels are not auto-restored after an OmniRoute or container restart. Re-enable them from the dashboard when needed.
- Managed install currently supports Linux, macOS, and Windows on `x64` / `arm64`.
- Managed Quick Tunnels default to HTTP/2 transport to avoid noisy QUIC UDP buffer warnings in constrained container environments. Set `CLOUDFLARED_PROTOCOL=quic` or `auto` if you want a different transport.
- Docker images bundle system CA roots and pass them to managed `cloudflared`, which avoids TLS trust failures when the tunnel bootstraps inside the container.
- Set `CLOUDFLARED_BIN=/absolute/path/to/cloudflared` if you want OmniRoute to use an existing binary instead of downloading one.
## Image Tags
| Image | Tag | Size | Description |
| ------------------------ | -------- | ------ | --------------------- |
| `diegosouzapw/omniroute` | `latest` | ~250MB | Latest stable release |
| `diegosouzapw/omniroute` | `3.8.0` | ~250MB | Current version |
Multi-platform manifest: `linux/amd64` + `linux/arm64` native (Apple Silicon, AWS Graviton, Raspberry Pi). Docker selects the matching architecture automatically; pass `--platform linux/amd64` if you need to force AMD64 emulation on ARM hosts.
## Important Notes
- **SQLite WAL Mode:** `docker stop` should be allowed to finish so OmniRoute can checkpoint the latest changes back into `storage.sqlite`. The bundled Compose files already set a 40s stop grace period. If you run the image directly, keep `--stop-timeout 40`.
- **`DISABLE_SQLITE_AUTO_BACKUP`:** Set to `true` if backups are managed externally.
- **Data Persistence:** Always mount a volume to `/app/data` to persist your database, keys, and configurations across container restarts.
- **Port Configuration:** Override `PORT` environment variable to change the default `20128` port.
## See Also
- [VM Deployment Guide](../ops/VM_DEPLOYMENT_GUIDE.md) — VM + nginx + Cloudflare setup
- [Fly.io Deployment Guide](../ops/FLY_IO_DEPLOYMENT_GUIDE.md) — Deploy to Fly.io
- [Environment Config](../reference/ENVIRONMENT.md) — Complete `.env` reference
docs/guides/ELECTRON_GUIDE.md
+277
View File
@@ -0,0 +1,277 @@
---
title: "Electron Desktop Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Electron Desktop Guide
> **Source of truth:** `electron/` workspace
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute ships a cross-platform desktop app (Windows / macOS / Linux) built on
**Electron 41** + **electron-builder 26.10**. The desktop app spawns the Next.js
standalone server as a child process, points a `BrowserWindow` at it, and adds a
system tray, auto-updater, IPC bridge, and zero-config secret bootstrap.
## Architecture
```
┌──────────────────────────────────────────────┐
│ Electron main process (electron/main.js) │
│ ├─ Single-instance lock │
│ ├─ Child process: Next.js standalone server │
│ │ (spawned with Electron's Node runtime) │
│ ├─ BrowserWindow → http://localhost:PORT │
│ ├─ System tray + context menu │
│ ├─ Auto-update via electron-updater │
│ ├─ Content Security Policy (session headers) │
│ └─ Secret bootstrap (JWT / API_KEY_SECRET) │
└──────────────────────────────────────────────┘
↕ IPC bridge (electron/preload.js)
┌──────────────────────────────────────────────┐
│ Renderer (Next.js dashboard) │
│ window.electronAPI.* (contextIsolation) │
└──────────────────────────────────────────────┘
```
## Versions
Confirmed from `electron/package.json`:
| Package | Version |
| ------------------ | -------------------------- |
| `electron` | `^41.5.1` |
| `electron-builder` | `^26.10.0` |
| `electron-updater` | `^6.8.5` |
| `better-sqlite3` | `^12.9.0` |
| App version | `3.8.0` |
| App id | `online.omniroute.desktop` |
| Product name | `OmniRoute` |
## Scripts (root `package.json`)
| Script | Purpose |
| --------------------------------- | -------------------------------------------------------------------------- |
| `npm run electron:dev` | Starts `npm run dev` + waits for `localhost:20128` + launches Electron |
| `npm run electron:build` | Builds Next.js then runs `electron-builder` for the current OS |
| `npm run electron:build:win` | Builds Windows NSIS installer + portable (x64) |
| `npm run electron:build:mac` | Builds macOS DMG (Intel + Apple Silicon) |
| `npm run electron:build:linux` | Builds Linux AppImage + DEB (x64 + arm64) |
| `npm run electron:smoke:packaged` | Launches packaged binary and probes `/login` for HTTP 200, then shuts down |
The `electron/` workspace also exposes:
- `npm run prepare:bundle` — runs `scripts/build/prepare-electron-standalone.mjs`
- `npm run build:mac-x64` / `build:mac-arm64` — single-arch macOS builds
- `npm run pack` — directory-only build for local testing (no installer)
## Directory Layout
```
electron/
├── package.json # Electron deps + electron-builder config
├── main.js # Main process (24 KB — see annotations below)
├── preload.js # contextBridge IPC bridge
├── types.d.ts # AppInfo / ServerStatus / ElectronAPI types
├── README.md # In-workspace notes
├── assets/ # icon.png, icon.ico, icon.icns, tray-icon.png
└── dist-electron/ # electron-builder output (gitignored)
scripts/
├── build/
│ └── prepare-electron-standalone.mjs # Stages .next/electron-standalone bundle
└── dev/
└── smoke-electron-packaged.mjs # Post-build smoke test
```
Both `main.js` and `preload.js` are **CommonJS `.js` files**, not TypeScript. The
renderer-side typings live in `electron/types.d.ts`.
## IPC Bridge (`preload.js`)
The preload exposes a whitelisted API on `window.electronAPI` using `contextBridge`
with `contextIsolation: true` and `nodeIntegration: false`.
```javascript
const VALID_CHANNELS = {
invoke: [
"get-app-info",
"open-external",
"get-data-dir",
"restart-server",
"check-for-updates",
"download-update",
"install-update",
"get-app-version",
],
send: ["window-minimize", "window-maximize", "window-close"],
receive: ["server-status", "port-changed", "update-status"],
};
```
Exposed methods:
| Renderer call | Type |
| ----------------------------------------------------------------- | -------------------------- |
| `getAppInfo()` → `{ name, version, platform, isDev, port }` | invoke |
| `openExternal(url)` | invoke |
| `getDataDir()` | invoke |
| `restartServer()` | invoke |
| `getAppVersion()` | invoke |
| `checkForUpdates()` / `downloadUpdate()` / `installUpdate()` | invoke |
| `minimizeWindow()` / `maximizeWindow()` / `closeWindow()` | send |
| `onServerStatus(cb)` / `onPortChanged(cb)` / `onUpdateStatus(cb)` | receive (returns disposer) |
The receive helpers return a **disposer function** rather than relying on
`removeAllListeners` — this prevents listener accumulation when React components
remount.
## Server Lifecycle
`main.js` spawns the Next.js standalone bundle directly with the Electron Node
runtime to avoid native-module ABI mismatch with system Node:
```js
spawn(process.execPath, [serverScript], {
cwd: NEXT_SERVER_PATH,
env: { ...serverEnv, PORT, NODE_ENV: "production", ELECTRON_RUN_AS_NODE: "1", NODE_PATH },
stdio: "pipe",
});
```
Highlights:
- `waitForServer()` polls the URL up to 30 s before showing the window (no blank screen on cold start).
- `stdio: "pipe"` captures stdout/stderr; ready phrases (`Ready` / `listening`) emit `server-status: running` over IPC.
- `before-quit` waits up to 5 s for graceful SIGTERM (WAL checkpoint) then sends SIGKILL.
- Port switcher in the tray (`20128`, `3000`, `8080`) stops and restarts the server, then reloads the BrowserWindow.
## Zero-config Secret Bootstrap
On first launch, the main process auto-generates and persists missing secrets:
| Secret | Source |
| ------------------------ | ----------------------------------------------------------------------------------- |
| `JWT_SECRET` | `crypto.randomBytes(64).toString("hex")` |
| `STORAGE_ENCRYPTION_KEY` | `crypto.randomBytes(32).toString("hex")` (refuses if encrypted creds already exist) |
| `API_KEY_SECRET` | `crypto.randomBytes(32).toString("hex")` |
Persisted to `<DATA_DIR>/server.env`. `DATA_DIR` resolves to:
- Windows: `%APPDATA%\omniroute`
- Linux: `$XDG_CONFIG_HOME/omniroute` or `~/.omniroute`
- macOS: `~/.omniroute`
## Window & Tray
- `BrowserWindow`: 1400×900 (min 1024×700), `backgroundColor: "#0a0a0a"`.
- macOS: `titleBarStyle: "hiddenInset"`, traffic-light at `{ x: 16, y: 16 }`.
- Windows/Linux: native title bar.
- Close button minimizes to tray; the tray menu has **Open OmniRoute**, **Open Dashboard** (external browser), **Server Port** submenu, **Check for Updates**, **Quit**.
## Content Security Policy
Set via `session.defaultSession.webRequest.onHeadersReceived`. Notable directives:
- `frame-ancestors 'none'`, `object-src 'none'`, `child-src 'none'`
- `connect-src 'self' http://localhost:* http://127.0.0.1:* ws://localhost:* ws://127.0.0.1:* https://*.omniroute.online https://*.omniroute.dev`
- Dev mode adds `'unsafe-eval'` to `script-src` only
## Auto-update
Uses `electron-updater` with the GitHub provider (`diegosouzapw/OmniRoute`).
- `autoDownload = false`, `autoInstallOnAppQuit = true`
- Events forwarded to renderer via `update-status` IPC:
`checking`, `available`, `not-available`, `downloading` (with `percent`), `downloaded`, `error`
- `installUpdate()` kills the server then calls `autoUpdater.quitAndInstall()`
- Skipped in dev mode (`!app.isPackaged`)
## Build Pipeline
1. `npm run build` → Next.js standalone in `.next/standalone`.
2. `prepare-electron-standalone.mjs` → re-stages into `.next/electron-standalone` and rewrites absolute paths inside `server.js` + `required-server-files.json` so the bundle is relocatable.
3. `electron-builder` packages `main.js`, `preload.js`, `node_modules`, and `extraResources: { ../.next/electron-standalone → app }`.
### Build targets
| OS | Targets |
| ------- | ----------------------------------------- |
| Windows | NSIS installer + portable (x64) |
| macOS | DMG (Intel + arm64, drag-to-Applications) |
| Linux | AppImage + DEB (x64 + arm64) |
NSIS settings: `oneClick: false`, lets the user choose the install directory, creates Desktop and Start-Menu shortcuts.
## Smoke Testing Packaged Build
```bash
npm run electron:smoke:packaged
```
`scripts/dev/smoke-electron-packaged.mjs`:
- Auto-discovers the packaged binary in `electron/dist-electron/` for the current platform.
- Launches with isolated `HOME`/`APPDATA`/`XDG_*` directories so it doesn't touch developer data.
- Polls `http://127.0.0.1:20128/login` for HTTP 200 within 45 s.
- Watches stderr/stdout for fatal patterns (`Cannot find module`, `MODULE_NOT_FOUND`, `ERR_DLOPEN_FAILED`, `Failed to start server`, etc.).
- Waits 2 s of stable runtime after readiness, then issues SIGTERM and waits for the port to free.
- In CI, automatically passes `--no-sandbox --disable-gpu` (and `--disable-dev-shm-usage` on Linux).
Env overrides: `ELECTRON_SMOKE_APP_EXECUTABLE`, `ELECTRON_SMOKE_URL`, `ELECTRON_SMOKE_TIMEOUT_MS`, `ELECTRON_SMOKE_SETTLE_MS`, `ELECTRON_SMOKE_DATA_DIR`, `ELECTRON_SMOKE_KEEP_DATA`, `ELECTRON_SMOKE_STREAM_LOGS`.
## Code Signing
`electron/package.json` does **not** wire signing credentials directly. Pass them via env vars to `electron-builder`:
### macOS
```bash
export APPLE_ID=<email>
export APPLE_APP_SPECIFIC_PASSWORD=<password>
export APPLE_TEAM_ID=<id>
export CSC_LINK=path/to/cert.p12
export CSC_KEY_PASSWORD=<cert-password>
npm run electron:build:mac
```
### Windows
```bash
export CSC_LINK=path/to/cert.pfx
export CSC_KEY_PASSWORD=<cert-password>
npm run electron:build:win
```
### Linux
AppImage signing is optional — set `LINUX_GPG_KEY` if signing.
## Distribution
Artifacts land in `electron/dist-electron/`:
- `OmniRoute Setup X.Y.Z.exe`, `OmniRoute-X.Y.Z-portable.exe` (Windows)
- `OmniRoute-X.Y.Z-mac.dmg`, `OmniRoute-X.Y.Z-arm64-mac.dmg` (macOS)
- `OmniRoute-X.Y.Z.AppImage`, `omniroute-desktop_X.Y.Z_amd64.deb` (Linux)
Releases are published to GitHub Releases (`diegosouzapw/OmniRoute`), which is also where `electron-updater` checks for new versions.
## Troubleshooting
| Symptom | Fix |
| --------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `Cannot find module 'better-sqlite3'` after Electron major bump | `cd electron && npm rebuild` |
| `ERR_DLOPEN_FAILED` for native module | Re-run `prepare:bundle` and verify ABI matches Electron's Node |
| Window appears blank on Linux | Confirm Next.js server actually bound to PORT (check `[Server]` logs) |
| macOS notarization stalls | Ensure `APPLE_*` vars are exported, not just in `.env` |
| Windows SmartScreen warning | Sign with EV cert, or users right-click → "Run anyway" |
| Smoke test fails with port-in-use | Stop any local dev server on 20128 before running `electron:smoke:packaged` |
## See Also
- [SETUP_GUIDE.md](./SETUP_GUIDE.md)
- [RELEASE_CHECKLIST.md](../ops/RELEASE_CHECKLIST.md)
- Source: `electron/main.js`, `electron/preload.js`, `electron/package.json`
- Helpers: `scripts/build/prepare-electron-standalone.mjs`, `scripts/dev/smoke-electron-packaged.mjs`
docs/guides/FEATURES.md
+329
View File
@@ -0,0 +1,329 @@
---
title: "OmniRoute — Dashboard Features Gallery"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute — Dashboard Features Gallery
🌐 **Main README translations:** 🇺🇸 [English](../README.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/README.md) | 🇪🇸 [Español](../i18n/es/README.md) | 🇫🇷 [Français](../i18n/fr/README.md) | 🇮🇹 [Italiano](../i18n/it/README.md) | 🇷🇺 [Русский](../i18n/ru/README.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/README.md) | 🇩🇪 [Deutsch](../i18n/de/README.md) | 🇮🇳 [हिन्दी](../i18n/in/README.md) | 🇹🇭 [ไทย](../i18n/th/README.md) | 🇺🇦 [Українська](../i18n/uk-UA/README.md) | 🇸🇦 [العربية](../i18n/ar/README.md) | 🇯🇵 [日本語](../i18n/ja/README.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/README.md) | 🇧🇬 [Български](../i18n/bg/README.md) | 🇩🇰 [Dansk](../i18n/da/README.md) | 🇫🇮 [Suomi](../i18n/fi/README.md) | 🇮🇱 [עברית](../i18n/he/README.md) | 🇭🇺 [Magyar](../i18n/hu/README.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/README.md) | 🇰🇷 [한국어](../i18n/ko/README.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/README.md) | 🇳🇱 [Nederlands](../i18n/nl/README.md) | 🇳🇴 [Norsk](../i18n/no/README.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/README.md) | 🇷🇴 [Română](../i18n/ro/README.md) | 🇵🇱 [Polski](../i18n/pl/README.md) | 🇸🇰 [Slovenčina](../i18n/sk/README.md) | 🇸🇪 [Svenska](../i18n/sv/README.md) | 🇵🇭 [Filipino](../i18n/phi/README.md) | 🇨🇿 [Čeština](../i18n/cs/README.md)
Visual guide to every section of the OmniRoute dashboard.
> 📅 **Last updated:** 2026-06-28 — **v3.8.40**
---
## ✨ v3.8.0 Highlights
The v3.7.x → v3.8.0 cycle added zero-config auto routing, new providers, OAuth flows, deeper resilience, and a much richer CLI experience. Headline features below — full details further in the document and in linked specs.
- 🤖 **Auto Combo / Zero-config auto-routing** — use prefixes `auto/coding`, `auto/fast`, `auto/cheap`, `auto/offline`, `auto/smart`, `auto/lkgp`. Backed by a 9-factor scoring engine and 4 curated **mode packs** (ship-fast, cost-saver, quality-first, offline-friendly)
- 🆕 **Command Code provider** (#2199) — first-class registration with model catalog and quota tracking
- 🆕 **Z.AI provider** — new free-tier provider with quota labels
- 🎬 **KIE media expansion** — extended catalog including video generation models
- 🔐 **Windsurf + Devin CLI OAuth flows** (#2168) — end-to-end browser-based login
- 🆓 **9 new free providers** — LLM7, Lepton, Kluster, UncloseAI, BazaarLink, Completions, Enally, FreeTheAi, Command Code
- 🎯 **Manifest-aware tier routing W1–W4** — provider manifests drive weighted tier selection
- 🎨 **Cursor full OpenAI parity** — tool calls, streaming, session management end-to-end
- 📊 **Cursor Pro plan usage** — quota & cycle data surfaced in the provider-limits dashboard
- ⚡ **Service tier breakdown / Codex fast tier analytics** — per-tier consumption visibility
- 📌 **Per-session sticky routing** — Codex sessions pin to the same account between turns
- 🔊 **Inworld TTS enhancements** — voice catalogs, streaming, and latency improvements
- 🔑 **Kiro headless auth** — login via local `kiro-cli` SQLite store, no browser required
- 📉 **DeepSeek quota and limit monitoring** — daily/monthly usage exposed via dashboard
- 🔄 **Reset-aware routing strategy** — combos now prefer accounts whose quota window resets soonest
- ⏱️ **`fallbackDelayMs`** and **dynamic tool limit detection** — finer fallback timing + per-provider tool-count limits
- 🔧 **Background mode degradation (Responses API)** — falls back to synchronous mode with a structured warning when an upstream lacks background polling
- 🚦 **Per-provider 429 classification** + `useUpstream429BreakerHints` toggle — finer breaker behavior using upstream rate-limit hints
- 🩺 **Model cooldowns dashboard** — observe per-model lockouts and manually re-enable from the UI
- 🔒 **MITM dynamic Linux cert detection** — works across Debian/Ubuntu, Fedora/RHEL, Arch, and other distros
- 💻 **CLI enhancement suite** — 20+ commands including `omniroute providers`, `omniroute combos`, `omniroute doctor`, `omniroute setup`
- 🔍 **Qdrant embedding model discovery** — automatic vector-store model probe
- 🔑 **API Keys / Bearer keys with `manage` scope** — perform admin operations programmatically via API
- 🏥 **Combo target health analytics** + **structured combo builder** — per-target health & UI builder for assembling `(provider, model, connection)` steps
- 🤝 **GitLab Duo OAuth provider** — login with GitLab credentials
- 🧠 **Reasoning Replay Cache** — hybrid in-memory + SQLite persistence of reasoning traces
📚 **Related docs:** [Skills Framework](../frameworks/SKILLS.md) · [Memory System](../frameworks/MEMORY.md) · [Cloud Agents](../frameworks/CLOUD_AGENT.md) · [Webhooks](../frameworks/WEBHOOKS.md) · [Reasoning Replay Cache](../routing/REASONING_REPLAY.md)
---
## 🔌 Providers
Manage AI provider connections: OAuth providers (Claude Code, Codex), API key providers (Groq, DeepSeek, OpenRouter), and free providers (Qoder, Qwen, Kiro). Kiro accounts include credit balance tracking — remaining credits, total allowance, and renewal date visible in Dashboard → Usage.
OpenRouter connections can store a per-connection `preset` in Advanced Settings. When set, OmniRoute sends it as the OpenRouter top-level request field, for example `"preset": "email-copywriter"`, unless the client request already supplied its own `preset`.
![Providers Dashboard](../screenshots/01-providers.png)
---
## 🎨 Combos
Create model routing combos with 17 strategies: priority, weighted, fill-first, round-robin, p2c (power-of-two-choices), random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp (last-known-good-provider), context-optimized, context-relay, and **fusion** (fan out to a panel of models in parallel, then synthesize one answer via a judge). Each combo chains multiple models with automatic fallback and includes quick templates and readiness checks.
Recent combo improvements:
- **Structured combo builder** — create each step by selecting provider, model, and exact account/connection
- **Repeated provider support** — reuse the same provider many times in one combo as long as the `(provider, model, connection)` tuple is unique
- **Combo target health** — analytics and health surfaces now distinguish individual combo targets/steps instead of collapsing everything into model strings
- **Composite tier ordering** — `defaultTier -> fallbackTier` now influences runtime execution/fallback order for top-level combo steps
![Combos Dashboard](../screenshots/02-combos.png)
---
## 📊 Analytics
Comprehensive usage analytics with token consumption, cost estimates, activity heatmaps, weekly distribution charts, and per-provider breakdowns.
![Analytics Dashboard](../screenshots/03-analytics.png)
---
## 🏥 System Health
Real-time monitoring: uptime, memory, version, latency percentiles (p50/p95/p99), cache statistics, provider circuit breaker states, active quota-monitored sessions, and combo target health.
![Health Dashboard](../screenshots/04-health.png)
---
## 🔧 Translator Playground
Four modes for debugging API translations: **Playground** (format converter), **Chat Tester** (live requests), **Test Bench** (batch tests), and **Live Monitor** (real-time stream).
![Translator Playground](../screenshots/05-translator.png)
---
## 🎮 Model Playground _(v2.0.9+)_
Test any model directly from the dashboard. Select provider, model, and endpoint, write prompts with Monaco Editor, stream responses in real-time, abort mid-stream, and view timing metrics.
---
## 🎨 Themes _(v2.0.5+)_
Customizable color themes for the entire dashboard. Choose from 7 preset colors (Coral, Blue, Red, Green, Violet, Orange, Cyan) or create a custom theme by picking any hex color. Supports light, dark, and system mode.
---
## ⚙️ Settings
Comprehensive settings panel with **7 tabs**:
- **General** — System storage, backup management (export/import database)
- **Appearance** — Theme selector (dark/light/system), color theme presets and custom colors, health log visibility, sidebar item and group separator visibility controls, Endpoint tunnel visibility controls
- **AI** — AI assistant features, default routing presets (Auto Combo `auto/coding`, `auto/fast`, `auto/cheap`, `auto/smart`), reasoning replay cache, and skill/memory toggles
- **Security** — API endpoint protection, custom provider blocking, IP filtering, session info
- **Routing** — Model aliases, background task degradation, manifest-aware tier routing (W1–W4), `fallbackDelayMs`, per-session sticky routing
- **Resilience** — Rate limit persistence, circuit breaker tuning, auto-disable banned accounts, provider expiration monitoring, **Context Relay** handoff threshold and summary model configuration, per-provider 429 classification & `useUpstream429BreakerHints` toggle, model cooldowns
- **Advanced** — Configuration overrides, configuration audit trail, fallback degradation mode, background mode degradation for Responses API
![Settings Dashboard](../screenshots/06-settings.png)
---
## 🔧 CLI Tools
One-click configuration for AI coding tools: Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline, Continue, Cursor, and Factory Droid. Features automated config apply/reset, connection profiles, and model mapping.
![CLI Tools Dashboard](../screenshots/07-cli-tools.png)
---
## 🤖 CLI Agents _(v2.0.11+)_
Dashboard for discovering and managing CLI agents. Shows a grid of 17 built-in agents (Codex, Claude, Goose, OpenClaw, Aider, OpenCode, Cline, Qwen Code, ForgeCode, Amazon Q, Open Interpreter, Cursor CLI, Warp, **Windsurf**, **Devin CLI**, **Kimi Coding**, **Command Code**) with:
- **Installation status** — Installed / Not Found with version detection
- **Protocol badges** — stdio, HTTP, etc.
- **Custom agents** — Register any CLI tool via form (name, binary, version command, spawn args)
- **CLI Fingerprint Matching** — Per-provider toggle to match native CLI request signatures, reducing ban risk while preserving proxy IP
- **OAuth-backed agents** — Windsurf & Devin CLI now use browser OAuth flows for authentication (v3.8.0+)
---
## 🔗 Context Relay _(v3.5.5+)_
A combo strategy that preserves session continuity when account rotation happens mid-conversation. Before the active account is exhausted, OmniRoute generates a structured handoff summary in the background. After the next request resolves to a different account, the summary is injected as a system message so the new account continues with full context.
Configurable via combo-level or global settings:
- **Handoff Threshold** — Quota usage percentage that triggers summary generation (default 85%)
- **Max Messages For Summary** — How much recent history to condense
- **Summary Model** — Optional override model for generating the handoff summary
Currently supports Codex account rotation. See [Context Relay documentation](../architecture/ARCHITECTURE.md).
---
## 🗜️ Prompt Compression _(v3.7.9+)_
Context & Cache now exposes dedicated pages for Caveman, RTK, and Compression Combos:
- **Caveman** — language-aware rule packs, preview, output-mode controls, and analytics
- **RTK** — command-aware compression for shell, git, test, build, package, Docker, infra, JSON, and stack-trace output
- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos; the default stacked math reaches `~89%` average and `78-95%` eligible-context savings when both engines apply
- **Raw-output recovery** — optional redacted RTK raw-output pointers for debugging compressed failures
See [Compression Guide](../compression/COMPRESSION_GUIDE.md), [RTK Compression](../compression/RTK_COMPRESSION.md), and
[Compression Engines](../compression/COMPRESSION_ENGINES.md).
---
## 🛡️ Proxy Hardening _(v3.5.5+)_
Comprehensive proxy configuration enforcement across the entire request pipeline:
- **Token Health Check** — Background OAuth refresh now resolves proxy config per connection, preventing failures in proxy-required environments
- **API Key Validation** — Provider key validation (`POST /api/providers/validate`) routes through `runWithProxyContext`, honoring provider-level and global proxy settings
- **undici Dispatcher Fix** — Proxy dispatchers use undici's own fetch implementation instead of Node's built-in fetch, resolving `invalid onRequestStart method` errors on Node.js 22
- **Node.js Version Detection** — Login page proactively detects incompatible Node.js versions (24+) and displays a warning banner with instructions to use Node 22 LTS
---
## 📧 Email Privacy Masking _(v3.5.6+)_
OAuth account emails are masked by default (e.g. `di*****@g****.com`) to prevent accidental exposure when sharing screenshots or recording demos. Use Settings → Appearance → Account email visibility to reveal or mask full account emails globally across providers, combos, logs, quota, and playground screens.
---
## 👁️ Model Visibility Toggle _(v3.5.6+)_
The provider page model list now includes:
- **Real-time search/filter bar** — Quickly find specific models
- **Per-model visibility toggle** (👁 icon) — Hidden models are grayed out and excluded from the `/v1/models` catalog
- **Active-count badge** (`N/M active`) — Shows at a glance how many models are enabled vs total
---
## 🔧 OAuth Env Repair _(v3.6.1+)_
One-click "Repair env" action for OAuth providers that restores missing environment variables and fixes broken auth state. Accessible from `Dashboard → Providers → [OAuth Provider] → Repair env`. Automatically detects and repairs:
- Missing OAuth client credentials
- Corrupted env file entries
- Backup path sanitization
---
## 🗑️ Uninstall / Full Uninstall _(v3.6.2+)_
Clean removal scripts for all installation methods:
| Command | Action |
| ------------------------ | ----------------------------------------------------------------------------------- |
| `npm run uninstall` | Removes the system app but **keeps your DB and configurations** in `~/.omniroute`. |
| `npm run uninstall:full` | Removes the app AND permanently **erases all configurations, keys, and databases**. |
---
## 🖼️ Media _(v2.0.3+)_
Generate images, videos, and music from the dashboard. Supports OpenAI, xAI, Together, Hyperbolic, SD WebUI, ComfyUI, AnimateDiff, Stable Audio Open, and MusicGen.
---
## 📝 Request Logs
Real-time request logging with filtering by provider, model, account, and API key. Shows status codes, token usage, latency, and response details.
![Usage Logs](../screenshots/08-usage.png)
---
## 🌐 API Endpoint
Your unified API endpoint with capability breakdown: Chat Completions, Responses API, Embeddings, Image Generation, Reranking, Audio Transcription, Text-to-Speech, Moderations, and registered API keys. Cloudflare Quick Tunnel, Tailscale Funnel, ngrok Tunnel, and cloud proxy support are available for remote access.
![Endpoint Dashboard](../screenshots/09-endpoint.png)
---
## 🔑 API Key Management
Create, scope, and revoke API keys. Each key can be restricted to specific models/providers with full access or read-only permissions. Visual key management with usage tracking.
---
## 📋 Audit Log
Administrative action tracking with filtering by action type, actor, target, IP address, and timestamp. Full security event history.
---
## 🖥️ Desktop Application
Native Electron desktop app for Windows, macOS, and Linux. Run OmniRoute as a standalone application with system tray integration, offline support, auto-update, and one-click install.
Key features:
- Server readiness polling (no blank screen on cold start)
- System tray with port management
- Content Security Policy
- Single-instance lock
- Auto-update on restart
- Platform-conditional UI (macOS traffic lights, Windows/Linux default titlebar)
- Hardened Electron build packaging — symlinked `node_modules` in the standalone bundle is detected and rejected before packaging, preventing runtime dependency on the build machine (v2.5.5+)
- **Graceful shutdown** — Electron `before-quit` shuts down Next.js cleanly, preventing SQLite WAL database locks (v3.6.2+)
📖 See [`electron/README.md`](../../electron/README.md) for full documentation.
---
## 🌐 V1 WebSocket Bridge _(v3.6.6+)_
OmniRoute now supports **OpenAI-compatible WebSocket clients** via the `/v1/ws` upgrade endpoint. The custom `scripts/dev/v1-ws-bridge.mjs` server wraps Next.js and upgrades WS connections to full bidirectional streaming sessions. Authentication uses the same API key or session cookie as HTTP requests.
Key behaviours:
- WS upgrade validated by `src/lib/ws/handshake.ts` before the connection is established
- Streams terminated cleanly on session close or upstream error
- Works alongside the existing HTTP+SSE streaming path simultaneously
---
## 🔑 Sync Tokens & Config Bundle _(v3.6.6+)_
Multi-device and external operator access is now possible via **scoped sync tokens**:
- **`POST /api/sync/tokens`** — Issue a new sync token (scoped, with optional expiry)
- **`DELETE /api/sync/tokens/:id`** — Revoke a token
- **`GET /api/sync/bundle`** — Download a versioned, ETag-keyed JSON snapshot of all non-sensitive settings (passwords redacted)
The config bundle is built by `src/lib/sync/bundle.ts`. Consumers compare the `ETag` response header to detect changes without re-downloading the full payload.
---
## 🧠 GLM Thinking Preset _(v3.6.6+)_
**GLM Thinking (`glmt`)** is now a registered first-class provider: 65 536 max output tokens, 24 576 thinking budget, 900 s default timeout, Claude-compatible API format, and shared usage sync with the GLM family.
**Hybrid token counting** also lands in v3.6.6: when a Claude-compatible provider exposes `/messages/count_tokens`, OmniRoute calls it before large requests with graceful estimation fallback.
---
## 🛡️ Safe Outbound Fetch & SSRF Guard _(v3.6.6+)_
All provider validation and model discovery calls now go through a two-layer outbound guard:
1. **URL guard** (`src/shared/network/outboundUrlGuard.ts`) — Blocks private/loopback/link-local IP ranges before the socket is opened.
2. **Safe fetch wrapper** (`src/shared/network/safeOutboundFetch.ts`) — Applies the URL guard, normalises timeouts, and retries transient errors with exponential backoff.
Guard violations surface as HTTP 422 (`URL_GUARD_BLOCKED`) and are written to the compliance audit log via `providerAudit.ts`.
---
## 🔄 Cooldown-Aware Retries _(v3.6.6+)_
Chat requests now **automatically retry** when an upstream provider returns a model-scoped cooldown. Configurable via `REQUEST_RETRY` (default: 2) and `MAX_RETRY_INTERVAL_SEC` (default: 30 s). Rate-limit header learning improved across `x-ratelimit-reset-requests`, `x-ratelimit-reset-tokens`, and `Retry-After` — per-model cooldown state is visible in the Resilience dashboard.
---
## 📋 Compliance Audit v2 _(v3.6.6+)_
The audit log has been expanded with cursor-based pagination, request context enrichment (request ID, user agent, IP), structured auth events, provider CRUD events with diff context, and SSRF-blocked validation logging. New events emitted by `src/lib/compliance/providerAudit.ts`.
docs/guides/FREE_PROVIDER_RANKINGS.md
+253
View File
@@ -0,0 +1,253 @@
---
title: "Free Provider Rankings (Arena ELO)"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Free Provider Rankings (Arena ELO)
> **TL;DR**: OmniRoute ranks its **free** providers by model quality using **Arena AI
> (LMArena-style) ELO scores**. Open the **Free Provider Rankings** page in the
> dashboard to see which free providers ship the strongest models for your task —
> overall, or filtered by category (coding, review, documentation, debugging).
---
## What It Is
OmniRoute aggregates 160+ providers, many of which expose a **free tier** (no-auth,
free-tier OAuth, or free-tier API key — see the
[Free Tiers Guide](../getting-started/FREE-TIERS-GUIDE.md) and the full
[Free Tiers directory](../reference/FREE_TIERS.md)). The catch: free providers vary
wildly in model quality. A no-auth provider serving a frontier model is far more useful
than one serving a small legacy model.
**Free Provider Rankings** answers "**which free provider gives me the best model?**" by
joining each free provider's catalog with **crowd-sourced quality scores** from the
**Arena AI leaderboard** (human-preference ELO, the same idea behind the LMArena
chatbot arena). Providers are then ranked by the strength of their **best free model**.
The ranking is computed from three real sources:
1. The free-provider lists — `NOAUTH_PROVIDERS`, plus `OAUTH_PROVIDERS` /
`APIKEY_PROVIDERS` entries flagged `hasFree`
(`src/shared/constants/providers.ts`).
2. Each provider's model catalog from the provider registry
(`open-sse/config/providerRegistry.ts`).
3. ELO-derived task-fit scores stored in the `model_intelligence` DB table by the
Arena ELO sync engine (`src/lib/arenaEloSync.ts`).
The join logic lives in `src/lib/freeProviderRankings.ts`.
---
## How to Access
### Dashboard page
Open the dashboard and go to **Costs → Free Provider Rankings**, or navigate directly to:
```
/dashboard/free-provider-rankings
```
The page (`src/app/(dashboard)/dashboard/free-provider-rankings/page.tsx`) shows:
- A **top-3 podium** (🥇 🥈 🥉) of the best-ranked free providers.
- A full **ranking table** with columns: **Rank**, **Provider**, **Top Model**,
**Score**, **Avg Score**, **Models**, **Type**.
- **Category filter buttons**: _All Categories_, _Default_, _Coding_, _Review_,
_Documentation_, _Debugging_.
Each provider's **Type** badge tells you how it is free:
| Badge | Meaning |
| -------- | --------------------------------------------- |
| `NOAUTH` | Always free, no credentials needed |
| `OAUTH` | OAuth provider with a free tier (`hasFree`) |
| `APIKEY` | API-key provider with a free tier (`hasFree`) |
Scores are shown as human-readable labels (e.g. _Elite_, _Excellent_, _Very Good_,
_Good_, _Average_) rather than raw numbers, because the underlying value is a relative
ranking quality, not a percentage.
### API endpoint
The page is backed by a public read endpoint
(`src/app/api/free-provider-rankings/route.ts`):
```
GET /api/free-provider-rankings
GET /api/free-provider-rankings?category=coding
GET /api/free-provider-rankings?category=coding&limit=20
```
Query parameters (validated with Zod):
| Param | Type | Default | Notes |
| ---------- | ------ | ------- | -------------------------------------------------------------------------------------------------- |
| `category` | string | (none) | One of `default`, `coding`, `review`, `documentation`, `debugging`. Omit for the combined ranking. |
| `limit` | number | `50` | Clamped to the range `1–100`. |
Response shape:
```json
{
"rankings": [
{
"id": "<provider-id>",
"name": "<provider name>",
"icon": "<icon>",
"color": "<hex color>",
"textIcon": "<short label>",
"category": "noauth | oauth | apikey",
"topModel": {
"modelId": "<registry model id>",
"modelName": "<model display name>",
"score": 0.0,
"eloRaw": 0,
"confidence": "high | medium | low",
"category": "<task category>"
},
"averageScore": 0.0,
"modelCount": 0
}
]
}
```
`eloRaw` is the original Arena ELO value; `score` is the normalized task-fit value
(see below). Providers with no scored models are omitted from the response.
---
## How the Scores Work
### Source: Arena AI leaderboard
The Arena ELO sync engine (`src/lib/arenaEloSync.ts`) fetches two leaderboards — `text`
and `code` — from the Arena AI leaderboard API
(`https://api.wulong.dev/arena-ai-leaderboards/v1/leaderboard`). Each leaderboard entry
carries a model name, vendor, ELO `score`, confidence interval, and vote count.
Leaderboard categories map to OmniRoute task categories:
| Arena leaderboard | OmniRoute task categories |
| ----------------- | ------------------------------------------------- |
| `text` | `default`, `review`, `documentation`, `debugging` |
| `code` | `coding` |
### Normalization (task-fit score)
Raw ELO scores are normalized per leaderboard into a **task-fit value in `[0.4, 0.98]`**:
```
taskFit = 0.4 + 0.58 * ((elo - minElo) / (maxElo - minElo))
```
The score never reaches `0` or `1`, leaving headroom for user overrides. This is the
`score` field you see in the API response and the label shown on the dashboard.
### Confidence
Each entry gets a confidence level based on Arena vote count:
| Confidence | Votes |
| ---------- | ------- |
| `high` | ≥ 5,000 |
| `medium` | ≥ 1,000 |
| `low` | < 1,000 |
### Storage and freshness
Normalized entries are written to the `model_intelligence` DB table with
`source = "arena_elo"` (`src/lib/db/modelIntelligence.ts`). Entries **expire after
7 days**, so a provider that stops syncing eventually drops out rather than serving
stale data.
The sync runs **on by default**:
- It runs once at server startup and then on a periodic timer
(`src/lib/arenaEloSync.ts`, wired from `src/server-init.ts`).
- It is **non-blocking and never fatal** — if the upstream fetch fails, OmniRoute keeps
running and the rankings simply show the last good data (or an empty state).
Two environment variables control it (documented in
[`docs/reference/ENVIRONMENT.md`](../reference/ENVIRONMENT.md)):
| Variable | Default | Purpose |
| ------------------------- | ------------- | ----------------------------------------------- |
| `ARENA_ELO_SYNC_ENABLED` | `true` | Set to `false` to opt out of the outbound sync. |
| `ARENA_ELO_SYNC_INTERVAL` | `86400` (24h) | Sync interval, in seconds. |
### Manual sync / status / clear
For operators, an authenticated management endpoint exposes manual control
(`src/app/api/intelligence/sync/route.ts` — requires management auth):
```
GET /api/intelligence/sync # current sync status (enabled, lastSync, nextSync, intervalMs)
POST /api/intelligence/sync # trigger a manual sync; body: { "dryRun": true } to preview without writing
DELETE /api/intelligence/sync # clear all synced arena_elo intelligence entries
```
If the rankings page is empty, a manual `POST /api/intelligence/sync` (or simply
restarting the server) repopulates it.
### Matching models to the leaderboard
Registry model IDs and Arena model names don't always match exactly. The ranking uses
flexible matching (`findMatchingIntelligence` in `src/lib/freeProviderRankings.ts`):
1. Exact match on the normalized model ID.
2. Match after stripping a trailing version suffix (e.g. `kimi-k2.6` → `kimi-k2`).
3. Prefix match (a leaderboard model name is a prefix of the registry ID).
On the sync side, known vendor prefixes (`anthropic/`, `openai/`, `google/`, …) are
stripped and a small alias map expands canonical names into the variants OmniRoute uses
internally, so models stay findable under any name.
### How a provider is ranked
For each free provider, the engine scores every model in its catalog, then:
- **Top Model** = the provider's highest-scoring model.
- **Avg Score** = the mean score across all of that provider's scored models.
- **Models** = how many of the provider's models had an Arena score.
Providers are sorted by **top-model score** first, then by average score. This rewards a
provider that ships at least one strong free model.
---
## Using It to Choose Free Providers
1. **Pick the right category.** Use the **Coding** filter for agentic/code workloads, or
leave it on **All Categories** / **Default** for general chat. The same provider can
rank differently across categories because its top model differs per leaderboard.
2. **Prefer the podium for one-shot setups.** If you only want to connect one or two free
providers, start with the top-ranked ones for your category.
3. **Check the Type badge.** `NOAUTH` providers are the fastest to connect (no
credentials). `OAUTH` / `APIKEY` free tiers need a quick sign-up but often expose
stronger models. See [Free Tiers Guide](../getting-started/FREE-TIERS-GUIDE.md) for
connection steps.
4. **Connect several and let Auto-Combo decide.** The same Arena ELO data that powers
this page also feeds the **task-fitness factor** of the Auto-Combo scoring engine
(`open-sse/services/autoCombo/taskFitness.ts`, resolution order
`user_override → arena_elo → models_dev_tier → static table`). So after you connect
the top free providers, routing with `model: "auto"` (e.g. `auto/coding`) will
automatically prefer the higher-quality free models per request. See
[Auto-Combo](../routing/AUTO-COMBO.md) for the full 9-factor scoring.
---
## Related Documentation
- [Free Tiers Guide](../getting-started/FREE-TIERS-GUIDE.md) — how to connect free
providers, no credit card required.
- [Free Tiers directory](../reference/FREE_TIERS.md) — full catalog of free providers
and their limits.
- [Auto-Combo](../routing/AUTO-COMBO.md) — the 9-factor routing engine that consumes the
same Arena ELO task-fitness data.
- [Environment variables](../reference/ENVIRONMENT.md) — `ARENA_ELO_SYNC_ENABLED` /
`ARENA_ELO_SYNC_INTERVAL` reference.
docs/guides/I18N.md
+578
View File
@@ -0,0 +1,578 @@
---
title: "i18n — Internationalization Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# i18n — Internationalization Guide
OmniRoute supports **42 languages** with full dashboard UI translation, translated documentation, and RTL support for Arabic and Hebrew.
🌐 **Languages:** 🇺🇸 [English](./I18N.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/I18N.md) | 🇪🇸 [Español](../i18n/es/docs/guides/I18N.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/I18N.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/I18N.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/I18N.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/I18N.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/I18N.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/I18N.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/I18N.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/I18N.md) | 🇮🇳 [हिन्दी](../i18n/hi/docs/guides/I18N.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/I18N.md) | 🇹🇷 [Türkçe](../i18n/tr/docs/guides/I18N.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/I18N.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/I18N.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/I18N.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/I18N.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/I18N.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/I18N.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/I18N.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/I18N.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/I18N.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/I18N.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/I18N.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/I18N.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/I18N.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/I18N.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/I18N.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/I18N.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/I18N.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/I18N.md)
## Translation pipeline (recommended — v3.8.0)
OmniRoute uses a hash-based incremental translator for docs, backed by an
OpenAI-compatible LLM endpoint (typically `cx/gpt-5.4-mini` through OmniRoute
Cloud):
```bash
# Run translations (incremental — only touches changed sources)
npm run i18n:run
# Limit to one locale
npm run i18n:run -- --locale=pt-BR
# Specific files (comma-separated, repo-relative paths)
npm run i18n:run -- --files=CLAUDE.md,docs/architecture/ARCHITECTURE.md
# Force retranslate everything (expensive)
npm run i18n:run -- --force
# Preview what would happen (no API calls, no writes)
npm run i18n:run:dry
# CI gate — exits non-zero if state is drifting
npm run i18n:check
```
**Source of truth.** `config/i18n.json` lists every locale (UI + docs) plus
the RTL set and the `docsExcluded` codes. The runtime config in
`src/i18n/config.ts` is a thin adapter over that JSON.
**Backend.** Configured via env (set in `.env`, never committed):
| Variable | Purpose |
| ----------------------------------- | --------------------------------------- |
| `OMNIROUTE_TRANSLATION_API_URL` | OpenAI-compatible base URL, e.g. `…/v1` |
| `OMNIROUTE_TRANSLATION_API_KEY` | bearer token (kept out of logs) |
| `OMNIROUTE_TRANSLATION_MODEL` | model id, e.g. `cx/gpt-5.4-mini` |
| `OMNIROUTE_TRANSLATION_TIMEOUT_MS` | optional, default `60000` |
| `OMNIROUTE_TRANSLATION_CONCURRENCY` | optional, default `4` |
**State tracking.** `.i18n-state.json` (committed) keeps SHA-256 hashes per
source + per locale. Drift detection is automatic and deterministic — no API
calls in `i18n:check`.
**Output shape.** Each translated file gets a top-level `# <heading>
(<native>)` line, a `🌐 Languages: …` bar, an `---` separator, and the
translated body. That layout matches what `scripts/check/check-docs-sync.mjs`
already enforces for `llm.txt` and `CHANGELOG.md` mirrors.
### Legacy scripts (deprecated)
The older Python script (`scripts/i18n/i18n_autotranslate.py`) and the
Google-Translate-backed generator (`scripts/i18n/generate-multilang.mjs`)
still exist with a deprecation banner. They will be removed in v3.10. The
`messages` and `readme` modes of `generate-multilang.mjs` (UI strings + root
README variants) are not yet handled by the new pipeline and are still used.
## Quick Reference
| Task | Command |
| ----------------------- | ---------------------------------------------------------- |
| Translate docs (LLM) | `npm run i18n:run` (preferred — incremental, hash-based) |
| Translate UI strings | `node scripts/i18n/generate-multilang.mjs messages` |
| Check translation drift | `npm run i18n:check` |
| Validate a locale | `python3 scripts/i18n/validate_translation.py quick -l cs` |
| Check code keys | `python3 scripts/i18n/check_translations.py` |
| Generate QA report | `node scripts/i18n/generate-qa-checklist.mjs` |
| Visual QA (Playwright) | `node scripts/i18n/run-visual-qa.mjs` |
## Architecture
![Incremental hash-based i18n pipeline](../diagrams/exported/i18n-flow.svg)
> Source: [diagrams/i18n-flow.mmd](../diagrams/i18n-flow.mmd)
### Source of Truth
- **UI strings**: `src/i18n/messages/en.json` (English source, ~2800 keys)
- **Locale files**: `src/i18n/messages/{locale}.json` (30 translations)
- **Framework**: `next-intl` with cookie-based locale resolution
- **Config**: `src/i18n/config.ts` — defines all 30 locales, language names, flags
### Runtime Flow
1. User selects language → `NEXT_LOCALE` cookie set
2. `src/i18n/request.ts` resolves locale: cookie → `Accept-Language` header → fallback `en`
3. Dynamic import loads `messages/{locale}.json`
4. Components use `useTranslations("namespace")` and `t("key")`
### Supported Locales
| Code | Language | RTL | Google Translate Code |
| ------- | -------------------- | --- | --------------------- |
| `ar` | العربية | Yes | `ar` |
| `bg` | Български | No | `bg` |
| `cs` | Čeština | No | `cs` |
| `da` | Dansk | No | `da` |
| `de` | Deutsch | No | `de` |
| `es` | Español | No | `es` |
| `fi` | Suomi | No | `fi` |
| `fr` | Français | No | `fr` |
| `he` | עברית | Yes | `iw` |
| `hi` | हिन्दी | No | `hi` |
| `hu` | Magyar | No | `hu` |
| `id` | Bahasa Indonesia | No | `id` |
| `it` | Italiano | No | `it` |
| `ja` | 日本語 | No | `ja` |
| `ko` | 한국어 | No | `ko` |
| `ms` | Bahasa Melayu | No | `ms` |
| `nl` | Nederlands | No | `nl` |
| `no` | Norsk | No | `no` |
| `phi` | Filipino | No | `tl` |
| `pl` | Polski | No | `pl` |
| `pt` | Português (Portugal) | No | `pt` |
| `pt-BR` | Português (Brasil) | No | `pt` |
| `ro` | Română | No | `ro` |
| `ru` | Русский | No | `ru` |
| `sk` | Slovenčina | No | `sk` |
| `sv` | Svenska | No | `sv` |
| `th` | ไทย | No | `th` |
| `tr` | Türkçe | No | `tr` |
| `uk-UA` | Українська | No | `uk` |
| `vi` | Tiếng Việt | No | `vi` |
| `zh-CN` | 中文 (简体) | No | `zh-CN` |
## Adding a New Language
### 1. Register the Locale
Edit `src/i18n/config.ts`:
```ts
// Add to LOCALES array
"xx",
// Add to LANGUAGES array
{ code: "xx", label: "XX", name: "Language Name", flag: "🏳️" },
```
### 2. Add to Generator
Edit `scripts/i18n/generate-multilang.mjs` — add entry to `LOCALE_SPECS`:
```js
{
code: "xx",
googleTl: "xx",
label: "XX",
flag: "🏳️",
languageName: "Language Name",
readmeName: "Language Name",
docsName: "Language Name",
},
```
### 3. Generate Initial Translation
```bash
node scripts/i18n/generate-multilang.mjs messages
```
This creates `src/i18n/messages/xx.json` auto-translated from `en.json` via Google Translate.
### 4. Review & Fix Auto-Translations
Auto-translations are a starting point. Review manually for:
- Technical accuracy
- Context-appropriate terminology
- Proper handling of placeholders (`{count}`, `{value}`, etc.)
### 5. Validate
```bash
python3 scripts/i18n/validate_translation.py quick -l xx
python3 scripts/i18n/validate_translation.py diff common -l xx
```
### 6. Generate Translated Documentation
```bash
node scripts/i18n/generate-multilang.mjs docs
```
## Auto-Translation Pipeline
### generate-multilang.mjs (Google Translate)
**Primary auto-translation engine** — uses Google Translate free API to generate translations for UI strings, READMEs, and documentation.
```bash
node scripts/i18n/generate-multilang.mjs [messages|readme|docs|all]
```
| Mode | What it does |
| ---------- | ----------------------------------------------------------------------------- |
| `messages` | Translates missing keys in `src/i18n/messages/{locale}.json` from `en.json` |
| `readme` | Translates `README.md` into all locales as `README.{code}.md` in project root |
| `docs` | Translates `DOC_SOURCE_FILES` into `docs/i18n/{locale}/{docName}` |
| `all` | Runs all three modes |
**Features:**
- **Text protection**: Masks code blocks (` ``` `), inline code (`` ` ``), markdown links/images (`[text](url)`), HTML tags, tables, and ICU placeholders (`{count}`, `{value}`, `{total}`, etc.) before translation, then restores them
- **Chunked batching**: Joins multiple strings with `__OMNIROUTE_I18N_SEPARATOR__` delimiters to minimize API calls (max 1800 chars per request)
- **In-memory cache**: Avoids redundant API calls for repeated strings within a session
- **Retry logic**: Exponential backoff (up to 5 attempts with 300ms × attempt delay) for 429/5xx errors
- **Timeout**: 20 seconds per request
- **Skip existing**: If target file already exists, it is NOT overwritten
**Important behaviors:**
- `docs/i18n/README.md` is **regenerated** each run — it's an auto-generated index of all docs
- Root `README.{code}.md` files are only created if they don't exist (skips locales in `EXISTING_README_CODES`)
- Language bars (`🌐 **Languages:** ...`) are automatically inserted/updated in all translated docs
### i18n_autotranslate.py (LLM-based)
**Secondary translator** — uses any OpenAI-compatible LLM API (including OmniRoute itself) to translate existing `docs/i18n/` markdown files. Best for polishing or re-translating docs with better quality than Google Translate.
```bash
python3 scripts/i18n/i18n_autotranslate.py \
--api-url http://localhost:20128/v1 \
--api-key sk-your-key \
--model gpt-4o
```
**Features:**
- Scans `docs/i18n/` markdown files for English paragraphs
- Skips code blocks, tables, and already-translated content
- Sends paragraphs to LLM with technical translation system prompt
- Supports all 42 languages
## CLI i18n
The `omniroute` CLI has its own i18n layer separate from the Next.js dashboard.
### How it works
- Every user-facing string in CLI commands goes through `t("module.key", vars)` from `bin/cli/i18n.mjs`.
- Catalogs are JSON files in `bin/cli/locales/` — 42 ship out-of-the-box.
- Locale falls back to `en` for any missing key, so partial translations are valid.
- The source of truth for available locales is `config/i18n.json` (shared with the dashboard).
### Locale selection
Detection order (first match wins):
| Priority | Source | Example |
| -------- | ------------------------ | --------------------------------------- |
| 1 | `--lang` flag | `omniroute --lang de status` |
| 2 | `OMNIROUTE_LANG` env var | `OMNIROUTE_LANG=ja omniroute providers` |
| 3 | `LC_ALL` system env | auto-detected from terminal locale |
| 4 | `LC_MESSAGES` system env | auto-detected from terminal locale |
| 5 | `LANG` system env | auto-detected from terminal locale |
| 6 | Fallback | `en` |
Locale codes with underscores (`pt_BR`) are normalized to hyphen form (`pt-BR`).
Locale codes are validated against `/^[a-zA-Z0-9-]+$/` — path traversal is rejected.
### Saving a language preference
```bash
# Set language and save to ~/.omniroute/.env (persists across sessions)
omniroute config lang set pt-BR
# View current language
omniroute config lang get
# List all 42 available languages
omniroute config lang list
# JSON output
omniroute config lang list --output json
```
The saved preference is written atomically to `~/.omniroute/.env` and is loaded by the
CLI bootstrap before any command runs.
### One-time override
```bash
# Override for one command only (not persisted)
omniroute --lang de providers list
```
Note: the `--lang` flag does NOT write to the env file — it only affects the current
invocation. Use `config lang set` to persist.
### Available locales
42 locale files ship in `bin/cli/locales/`. Full translations: `en`, `pt-BR`.
Scaffold-only (all keys fall back to `en`): `bn`, `gu`, `he`, `in`, `mr`, `ms`, `phi`, `sw`, `ta`, `te`, `ur`.
All other 29 locales have `common` + `program` keys translated.
### Adding a new CLI locale
1. Add the locale entry to `config/i18n.json`.
2. Run `node bin/cli/scripts/generate-locales.mjs` — creates the locale file.
3. Translate the keys (or leave as `{}` for en-fallback scaffold).
4. PRs must add strings to `en.json` and `pt-BR.json`; other files are best-effort.
## Validation & QA
### validate_translation.py
**Translation validator** — compares any locale JSON against `en.json` and reports issues.
```bash
# Quick check (counts only)
python3 scripts/i18n/validate_translation.py quick -l cs
# Output:
# Missing: 0
# Untranslated: 0
# Ignored (UNTRANSLATABLE_KEYS): 236
# Detailed diff by category
python3 scripts/i18n/validate_translation.py diff common -l cs
python3 scripts/i18n/validate_translation.py diff settings -l cs
# Export to CSV
python3 scripts/i18n/validate_translation.py csv -l cs > report.csv
# Export to Markdown
python3 scripts/i18n/validate_translation.py md -l cs > report.md
# Full report (default)
python3 scripts/i18n/validate_translation.py -l cs
```
**Detects:**
- **Missing keys** — keys in `en.json` but not in locale file
- **Extra keys** — keys in locale file but not in `en.json`
- **Untranslated keys** — keys where locale value equals English source (excluding allowlist)
- **Placeholder mismatches** — ICU placeholders that don't match between source and translation
**Exit codes:**
| Code | Meaning |
|------|---------|
| 0 | OK |
| 1 | Generic error |
| 2 | Missing strings (hard error) |
| 3 | Untranslated warning (soft) |
**Environment:** Set `TRANSLATION_LANG=cs` or use `-l cs` flag.
### check_translations.py
**Code-to-JSON key checker** — scans `src/**/*.tsx` and `src/**/*.ts` for `useTranslations()` calls and verifies all referenced keys exist in `en.json`.
```bash
# Basic check
python3 scripts/i18n/check_translations.py
# Verbose output
python3 scripts/i18n/check_translations.py --verbose
# Auto-fix (adds missing keys to en.json)
python3 scripts/i18n/check_translations.py --fix
```
### generate-qa-checklist.mjs
**Static analysis QA** — scans Next.js page files for i18n risk metrics and generates a Markdown report.
```bash
node scripts/i18n/generate-qa-checklist.mjs
```
**Checks:**
- Fixed-width class usage (overflow risk)
- Directional left/right classes (RTL risk)
- Clipping-prone patterns
- Locale parity (missing/extra keys vs `en.json`)
- README language selector bars in priority locales (`es`, `fr`, `de`, `ja`, `ar`)
**Output:** `docs/reports/i18n-qa-checklist-{date}.md`
### run-visual-qa.mjs
**Visual QA via Playwright** — takes screenshots of all dashboard routes in multiple locales and viewports, then evaluates page health.
```bash
# Default: es, fr, de, ja, ar on localhost:20128
node scripts/i18n/run-visual-qa.mjs
# Custom base URL and locales
QA_BASE_URL=http://staging.example.com QA_LOCALES=de,fr node scripts/i18n/run-visual-qa.mjs
# Custom routes
QA_ROUTES=/dashboard/settings,/dashboard/providers node scripts/i18n/run-visual-qa.mjs
```
**Detects:**
- Text overflow
- Element clipping
- RTL layout mismatches
**Output:** `docs/reports/i18n-visual-qa-{date}.md` + JSON report
## Managing Untranslatable Keys
### untranslatable-keys.json
**File:** `scripts/i18n/untranslatable-keys.json`
Allowlist of keys that should remain identical to English source. Used by `validate_translation.py` to avoid false-positive "untranslated" warnings.
```json
{
"description": "Keys that should remain untranslated...",
"keys": [
"common.model",
"common.oauth",
"health.cpu",
...
]
}
```
**What belongs here:**
- Brand/product names: `landing.brandName`, `common.social-github`
- Technical terms/acronyms: `health.cpu`, `mcpDashboard.pid`, `settings.ai`
- ICU/format strings: `apiManager.modelsCount`, `health.millisecondsShort`
- Placeholder values: `providers.openaiBaseUrlPlaceholder`, `cliTools.baseUrlPlaceholder`
- Protocol names: `common.http`, `common.oauth`, `providers.oauth2Label`
- Navigation sections: `sidebar.primarySection`, `sidebar.cliSection`
**To add a key:** Edit the `keys` array in `scripts/i18n/untranslatable-keys.json` and re-run validation.
## CI Integration
### GitHub Actions (`.github/workflows/ci.yml`)
The CI pipeline validates all locales on every push and PR:
1. **`i18n-matrix` job** — dynamically discovers all locale files (excluding `en.json`)
2. **`i18n` job** — runs `validate_translation.py quick -l '<lang>'` for each locale in parallel
3. **`ci-summary` job** — aggregates results into a dashboard summary
```yaml
# i18n-matrix: discovers languages
LANGS=$(ls src/i18n/messages/*.json | xargs -n1 basename | sed 's/.json$//' | grep -v '^en$')
# i18n: validates each language
python3 scripts/i18n/validate_translation.py quick -l '${{ matrix.lang }}'
```
**Dashboard output:**
```
## 🌍 Translations
| Metric | Value |
|--------|------|
| Languages checked | 30 |
| Total untranslated | 0 |
✅ All translations complete
```
## File Structure
```
src/i18n/
├── config.ts # Locale definitions (30 locales, RTL config)
├── request.ts # Runtime locale resolution
└── messages/
├── en.json # Source of truth (~2800 keys)
├── cs.json # Czech translation
├── de.json # German translation
└── ... # 30 locale files total
scripts/
├── i18n/
│ ├── generate-multilang.mjs # Auto-translation engine (Google Translate, 888 lines)
│ ├── generate-qa-checklist.mjs # Static analysis QA
│ ├── run-visual-qa.mjs # Playwright visual QA
│ └── untranslatable-keys.json # Allowlist for validation (236 keys)
├── validate_translation.py # Translation validator
├── check_translations.py # Code-to-JSON key checker
└── i18n_autotranslate.py # LLM-based doc translator
.github/workflows/
└── ci.yml # i18n validation in CI matrix
docs/
├── I18N.md # This file — i18n toolchain documentation
├── i18n/
│ ├── README.md # Auto-generated language index
│ ├── cs/ # Czech docs
│ │ └── docs/
│ │ ├── I18N.md # Czech translation of this file
│ │ └── ...
│ ├── de/ # German docs
│ └── ... # 30 locale directories
└── reports/
├── i18n-qa-checklist-*.md # Static analysis reports
└── i18n-visual-qa-*.md # Visual QA reports
```
## Best Practices
### When Editing Translations
1. **Always edit `en.json` first** — it's the source of truth
2. **Run `generate-multilang.mjs messages`** to propagate new keys to all locales
3. **Review auto-translations** — Google Translate is a starting point, not final
4. **Validate before committing** — `python3 scripts/i18n/validate_translation.py quick -l <lang>`
5. **Update `untranslatable-keys.json`** if a key should remain in English
### Placeholder Safety
- ICU placeholders (`{count}`, `{value}`, `{total}`, `{seconds}`) must be preserved exactly
- Plural formats (`{count, plural, one {# model} other {# models}}`) must maintain structure
- The validator detects placeholder mismatches automatically
### Adding New Translation Keys in Code
```tsx
// Use namespaced keys
const t = useTranslations("settings");
t("cacheSettings"); // maps to settings.cacheSettings in JSON
// Run check_translations.py to verify keys exist
python3 scripts/i18n/check_translations.py --verbose
```
### RTL Considerations
- Arabic (`ar`) and Hebrew (`he`) are RTL locales
- Avoid hardcoded `left`/`right` CSS — use `start`/`end` logical properties
- Visual QA catches RTL layout mismatches via `run-visual-qa.mjs`
## Known Issues & History
### `in.json` → `hi.json` Fix
The generator originally used `code: "in"` (deprecated Google Translate code) for Hindi instead of the correct ISO 639-1 `hi`. This created an orphaned `in.json` duplicate of `hi.json`. Fixed by changing `code: "in"` to `code: "hi"` in `generate-multilang.mjs` and removing the orphaned file.
> ⚠️ **Audit (2026-05-13):** The `docs/i18n/in/` directory still exists on disk (full duplicate of `hi/`). Translation generator no longer writes to it, but the historical tree was not pruned. Safe to delete with `rm -rf docs/i18n/in/` after confirming no external links reference the old path.
### `docs/i18n/README.md` Is Auto-Generated
The `docs/i18n/README.md` file is completely regenerated by `generate-multilang.mjs docs`. Any manual edits will be lost. Use `docs/guides/I18N.md` (this file) for hand-written documentation that should persist.
### External Untranslatable Keys List
The `untranslatable-keys.json` allowlist was moved from an inline Python set in `validate_translation.py` to an external JSON file for easier maintenance. The validator loads it at runtime.
### `generate-multilang.mjs` Hindi Code Fix
The generator originally used `code: "in"` (deprecated Google Translate code) for Hindi instead of the correct ISO 639-1 `hi`. This was introduced in upstream commit `952b0b22c` by `diegosouzapw`. Fixed by changing `code: "in"` to `code: "hi"` in the `LOCALE_SPECS` array and removing the orphaned `in.json` file.
### `validate_translation.py` Ignored Count Output
The `quick` check now displays the count of ignored keys from `untranslatable-keys.json`:
```
Missing: 0
Untranslated: 0
Ignored (UNTRANSLATABLE_KEYS): <varies per release>
```
docs/guides/KIRO_SETUP.md
+140
View File
@@ -0,0 +1,140 @@
---
title: "Kiro Setup Guide"
---
# Kiro Setup Guide
This guide covers adding Kiro (AWS-hosted AI coding assistant) accounts to OmniRoute,
with a focus on running multiple accounts simultaneously without session conflicts.
---
## Background: Why Kiro Accounts Can Conflict
Kiro's backend uses AWS SSO OIDC client registrations to track active sessions.
The critical constraint: **each OIDC client registration supports only one active
session at a time**. When a second device or user authenticates using the same
registered client, the backend invalidates the first account's refresh token.
This is the same mechanism that causes problems when running `kiro-cli login` on a
machine where another Kiro account is already signed in — the new login revokes the
first account's token.
---
## How OmniRoute Solves This (v3.8.0+)
Starting with v3.8.0, OmniRoute calls `registerClient()` (AWS SSO OIDC) during every
Kiro connection import. This gives each OmniRoute connection its own dedicated OIDC
client registration. Because each client registration is independent, refreshing or
re-authenticating one account does not affect any other account's refresh token.
The isolation applies to all three import methods:
| Import method | Isolation status |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| AWS Builder ID / IDC device-code flow | Isolated since the device-code flow was introduced |
| **Import Token** (manual refresh token paste) | Isolated from v3.8.0 |
| **Google / GitHub social login** | Isolated from v3.8.0 |
| **Auto-Import** (kiro-cli SQLite) | Isolated from v3.8.0 (SQLite path was already isolated; SSO-cache fallback is now also isolated) |
---
## Migration Note for Connections Created Before v3.8.0
Connections imported before v3.8.0 do not have a dedicated OIDC client registration
stored in `providerSpecificData`. These connections continue to work but use the shared
social-auth refresh endpoint, which means two such connections can still invalidate each
other.
**To gain isolation:** delete the old connection from **Dashboard → Providers** and
re-import it using any of the supported import flows. All newly created connections will
receive their own client registration automatically.
---
## Adding Two Kiro Accounts Side by Side
### Prerequisites
- OmniRoute v3.8.0 or later.
- A working Kiro account (email + password, Google, or GitHub login).
- Optionally a second Kiro account.
### Step 1: Import the first account
1. Open **Dashboard → Providers → Add Provider → Kiro**.
2. Choose one of:
- **Import Token** — paste a refresh token starting with `aorAAAAAG`.
- **Google / GitHub login** — complete the OAuth flow in the browser.
- **Auto-Import** — click the button; OmniRoute reads credentials from the
local kiro-cli database or `~/.aws/sso/cache`.
3. The connection is saved. OmniRoute automatically registers a dedicated OIDC client for it.
### Step 2: Import the second account
Repeat step 1 for the second account. Because each import creates a separate OIDC
client registration, the two connections are fully isolated.
### Step 3: Verify both connections are active
1. **Dashboard → Providers** — both Kiro connections should show **Active** status.
2. **Dashboard → Health** — both connections should pass their token health check.
### Step 4: Use a combo to route between accounts
Create a combo with both connections as targets to load-balance or fall back between them:
```
kiro/kiro-dev → kiro/kiro-pro
```
See [FEATURES.md](./FEATURES.md) and the routing documentation for combo configuration.
---
## Enterprise / IDC Users
For AWS IAM Identity Center (IDC) accounts, use the **AWS Builder ID / IDC device-code**
flow from **Dashboard → Providers → Kiro → Device Code**. The device-code flow has
always been fully isolated. No re-import is needed for these connections.
Enterprise users who operate in a non-default AWS region can specify the region when
importing via the Import Token API:
```bash
curl -X POST http://localhost:20128/api/oauth/kiro/import \
-H "Content-Type: application/json" \
-d '{"refreshToken": "aorAAAAAG...", "region": "eu-west-1"}'
```
The `region` field defaults to `us-east-1` when omitted.
---
## OIDC Client Expiry
AWS SSO OIDC public clients typically expire after 90 days
(`clientSecretExpiresAt`). OmniRoute stores this timestamp in `providerSpecificData`
for observability. If a connection stops refreshing after ~90 days, re-import the
connection to obtain a fresh OIDC client registration. Automatic re-registration on
expiry is tracked as a future improvement.
---
## Troubleshooting
### Second account keeps getting logged out
- Check both connections in **Dashboard → Providers** and confirm each shows a non-null
`clientId` in its raw JSON (visible via the info icon). If either connection is missing
`clientId`, it was imported before v3.8.0 — re-import it.
### Import fails with "Token validation failed"
- Ensure the refresh token starts with `aorAAAAAG`.
- Ensure OmniRoute can reach `https://oidc.us-east-1.amazonaws.com` (or the configured
region). If you are behind a corporate proxy, set a provider-level proxy in
**Dashboard → Settings → Proxies**.
For other issues, see the main [TROUBLESHOOTING.md](./TROUBLESHOOTING.md).
docs/guides/PWA_GUIDE.md
+192
View File
@@ -0,0 +1,192 @@
---
title: "Progressive Web App (PWA) Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Progressive Web App (PWA) Guide
OmniRoute ships as a fully installable Progressive Web App. When you access the dashboard from any mobile browser — Android (Chrome) or iOS (Safari) — you can "Add to Home Screen" and get a native app-like experience with no app store required.
## What Is a PWA?
A Progressive Web App turns the OmniRoute web dashboard into something that looks and feels like a native mobile app. Once installed, it:
- Launches from your home screen with its own icon
- Opens fullscreen — no browser address bar or tab UI
- Works offline with a dedicated connectivity page
- Caches static assets for faster loading
- Supports both portrait and landscape orientations
## Installation
### Android (Chrome)
1. Open the OmniRoute dashboard in Chrome: `http://YOUR_IP:20128`
2. Chrome will show an **"Add OmniRoute to Home screen"** banner automatically, or:
- Tap the **⋮** menu (three dots) → **"Add to Home screen"** or **"Install app"**
3. Confirm the prompt
4. OmniRoute appears on your home screen as a standalone app
### iOS (Safari)
1. Open the OmniRoute dashboard in Safari: `http://YOUR_IP:20128`
2. Tap the **Share** button (box with arrow)
3. Scroll down and tap **"Add to Home Screen"**
4. Name it (defaults to "OmniRoute") and tap **Add**
5. OmniRoute appears on your home screen with the app icon
### Desktop (Chrome / Edge)
1. Open the OmniRoute dashboard
2. Click the **install icon** in the address bar (or ⋮ → "Install OmniRoute...")
3. Confirm the prompt
4. OmniRoute opens as a standalone window — no tabs, no address bar
## Features
### Fullscreen Experience
The manifest is configured with `display: "fullscreen"`, which means the installed app uses the entire screen — no browser chrome, no status bar overlap. This makes the dashboard feel truly native.
### Offline Support
OmniRoute includes a service worker (`sw.js`) that provides intelligent caching:
| Asset Type | Strategy | Behavior |
| ------------------------------------------------------- | ---------------------------------- | ---------------------------------------------------------------------------- |
| **App Shell** | Cache-first | `/`, `/offline`, manifest, and icons are pre-cached on install |
| **Static assets** (CSS, JS, images, fonts) | Network-first with cache fallback | Fetches fresh from the network; falls back to cache if offline |
| **Next.js bundles** (`/_next/`) | Network-first with cache update | Fetches from network and updates cache; serves cached version if offline |
| **Navigation requests** | Network-only with offline fallback | Always fetches from network; shows `/offline` page if network is unavailable |
| **API routes** (`/api/`, `/a2a`, `/dashboard/endpoint`) | Bypass (never cached) | Always goes directly to the server — never intercepted by the service worker |
### Offline Page
When the network is unavailable and a user navigates to a new page, the service worker serves a dedicated `/offline` page that:
- Displays a clear **"Connectivity Issue"** message
- Shows a live **online/offline status indicator** that updates in real time
- Provides a **"Retry Connection"** button to reload when connectivity returns
- Links to the **Status Page** for diagnostics
### App Icons
OmniRoute provides icons optimized for each platform:
| File | Size | Used By |
| ---------------------- | ---------------- | ------------------------------------- |
| `icon-512.png` | 512×512 | Android install prompt, splash screen |
| `apple-touch-icon.png` | 180×180 | iOS home screen icon |
| `icon-192.svg` | 192×192 (vector) | Android adaptive icon |
| `apple-touch-icon.svg` | 180×180 (vector) | Apple fallback |
| `favicon.svg` | Vector | Browser tabs |
| `favicon.ico` | Multi-size | Legacy browsers |
### Automatic Registration
The service worker is registered automatically via the `<PwaRegister />` component in the root layout. No user action is needed — the app becomes installable as soon as the browser detects the valid manifest and service worker.
## Technical Architecture
### Web App Manifest (`manifest.webmanifest`)
Generated by Next.js via `src/app/manifest.ts`:
```json
{
"name": "OmniRoute",
"short_name": "OmniRoute",
"description": "OmniRoute is an AI gateway for multi-provider LLMs. One endpoint for all your AI providers.",
"start_url": "/",
"scope": "/",
"display": "fullscreen",
"orientation": "any",
"background_color": "#0b0f1a",
"theme_color": "#0b0f1a",
"icons": [
{ "src": "/icon-512.png", "sizes": "512x512", "type": "image/png", "purpose": "any maskable" },
{ "src": "/apple-touch-icon.png", "sizes": "180x180", "type": "image/png" }
]
}
```
### Service Worker (`public/sw.js`)
A vanilla service worker (no framework dependencies) with:
- **Install phase**: Pre-caches the app shell (root, offline page, manifest, icons)
- **Activate phase**: Cleans up old cache versions and claims all clients
- **Fetch phase**: Intelligent routing based on request type (navigation, static asset, API)
- **Cache versioning**: `omniroute-pwa-v2` — bump this to force a fresh cache on update
### Layout Metadata (`src/app/layout.tsx`)
The root layout provides all the meta tags required for PWA compliance:
- `manifest` link to `/manifest.webmanifest`
- `apple-web-app-capable: true` for iOS standalone mode
- `apple-web-app-status-bar-style: black-translucent`
- `mobile-web-app-capable: yes` for Android Chrome
- `theme-color: #0b0f1a`
- `viewport-fit: cover` for edge-to-edge rendering
### Component: `PwaRegister`
Located at `src/shared/components/PwaRegister.tsx`, this client component:
1. Runs on mount (client-side only)
2. Checks for `serviceWorker` support in the browser
3. Registers `/sw.js` silently (errors are swallowed to avoid blocking the app)
4. Renders nothing (`return null`) — it's a side-effect-only component
## Use With Termux (Android)
When running OmniRoute on Android via Termux, the PWA works seamlessly:
1. Start OmniRoute in Termux: `npx omniroute`
2. Open Chrome on the same phone: `http://localhost:20128`
3. Install the PWA via "Add to Home Screen"
4. The PWA connects to the local Termux server — everything runs on-device
This combination means your Android phone is both the **server** (Termux) and the **client** (PWA) — a complete self-contained AI gateway.
## Use From Other Devices
Install the PWA on any device that has browser access to your OmniRoute server:
- **Another phone/tablet**: Navigate to `http://PHONE_IP:20128` and install the PWA
- **Laptop**: Open Chrome/Edge and install it as a desktop PWA
- **Smart TV with browser**: Access the dashboard fullscreen
## Customization
### Instance Name
The PWA title respects the **Instance Name** setting from `Dashboard → Settings`. If you rename your instance to "My AI Gateway", the installed PWA will show that name.
### Custom Favicon
If you upload a custom favicon via `Dashboard → Settings`, the PWA icon on desktop will reflect the custom icon. Mobile home screen icons use the pre-built `icon-512.png` and `apple-touch-icon.png` files.
## Limitations
- **No push notifications** — The service worker does not implement the Push API. Notifications are handled by the Electron app instead.
- **No background sync** — Offline actions are not queued for replay. The PWA is primarily a dashboard viewer.
- **iOS restrictions** — Safari on iOS does not support all PWA features (e.g., install prompts are manual, and background service workers are limited).
- **Cache size** — The service worker caches static assets only. Large response payloads from `/api/` routes are never cached.
- **Custom icons on mobile** — Changing the favicon in settings does not update the home screen icon on mobile (this requires regenerating the PWA icons).
## Files Reference
| File | Purpose |
| --------------------------------------- | ---------------------------------------------------------------- |
| `src/app/manifest.ts` | Next.js manifest route (generates `manifest.webmanifest`) |
| `public/sw.js` | Service worker with caching logic |
| `src/shared/components/PwaRegister.tsx` | Client component that registers the service worker |
| `src/app/offline/page.tsx` | Offline fallback page with live status indicator |
| `src/app/layout.tsx` | Root layout with PWA metadata (apple-web-app, theme-color, etc.) |
| `public/icon-512.png` | 512×512 PNG icon (Android, splash screen) |
| `public/apple-touch-icon.png` | 180×180 PNG icon (iOS home screen) |
| `public/icon-192.svg` | 192×192 SVG icon (Android adaptive) |
| `public/apple-touch-icon.svg` | 180×180 SVG icon (Apple fallback) |
docs/guides/REMOTE-MODE.md
+347
View File
@@ -0,0 +1,347 @@
---
title: "Remote Mode — Drive a remote OmniRoute from your laptop"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Remote Mode
Run the `omniroute` CLI on your laptop while OmniRoute itself runs somewhere else
(a VPS, a home server, another machine on your Tailnet). You log in once with
`omniroute connect`, and from then on **every** CLI command targets that remote
server — same commands, same output, just executed against the remote.
There is no second tool to install: remote mode is the regular `omniroute` CLI
plus scoped **access tokens**.
```bash
npm install -g omniroute # the normal CLI
omniroute connect 192.168.0.15 # log in (password → scoped token)
omniroute models list # ← now lists the REMOTE server's models
omniroute configure codex # ← writes a local Codex profile from the remote catalog
```
---
## How it works
```
your laptop remote OmniRoute (VPS)
┌────────────────────┐ ┌───────────────────────────────┐
│ omniroute CLI │ POST /api/cli/connect (password → token) │
│ context: vps │ ───────────────► │ mints a scoped access token │
│ baseUrl, token │ Authorization: Bearer oma_live_… │
│ │ ───────────────► │ every management route, scope- │
│ writes configs │ ◄─────────────── │ checked per the token's scope │
│ LOCALLY │ └───────────────────────────────┘
└────────────────────┘
```
- **Contexts** store one server each (`~/.omniroute/config.json`, `chmod 600`).
`omniroute contexts use <name>` switches the active server; `default` is local.
- **Access tokens** (`oma_live_…`) authorize management commands. They are
distinct from inference API keys (`sk-…`, used for `/v1/chat/completions`).
- Only the SHA-256 hash of a token is stored server-side. The plaintext is shown
**once**, at creation.
---
## Connecting
### With the management password (bootstrap)
```bash
omniroute connect 192.168.0.15
# Management password for http://192.168.0.15:20128: ********
# ✔ Connected to http://192.168.0.15:20128 — context '192.168.0.15' (scope: admin)
```
The password flow mints an **admin** token by default (you hold the password, so
you already have full control). Downscope with `--scope`:
```bash
omniroute connect 192.168.0.15 --scope write
```
Options: `--port <p>` (when the host has none), `--name <ctx>` (context name),
`--scope read|write|admin`. A full URL is honoured as-is:
`omniroute connect https://omni.example.com`.
### With a pre-generated token
Generate a scoped token in the dashboard (or with `omniroute tokens create`) and
paste it — no password needed:
```bash
omniroute connect 192.168.0.15 --key oma_live_xxxxxxxx
```
The CLI validates it via `GET /api/cli/whoami` and saves it as the active context.
---
## Scopes
Three levels, hierarchical (`admin ⊃ write ⊃ read`):
| Scope | Can do |
| ------- | ---------------------------------------------------------------------------- |
| `read` | list/inspect — `models list`, `providers status`, `logs`, `usage`, `cost` |
| `write` | read **+** configure/apply — `setup-codex`, `keys add`, `config set`, combos |
| `admin` | write **+** manage — `tokens` CRUD, add providers, services, policy, oauth |
The server infers the scope each route requires from the HTTP method
(`GET`→read, mutations→write) plus an admin allowlist for sensitive surfaces
(`/api/cli/tokens`, `/api/providers` mutations, `/api/oauth`, `/api/services`, …).
A token with insufficient scope gets `403` with a clear message.
> Routes that spawn processes (`/api/services/*`, `/api/mcp/*`, …) stay
> **loopback-only** — a remote token can never reach them, regardless of scope.
---
## Connecting Antigravity on a remote install
Antigravity uses Google's firstparty/nativeapp consent screen. Google only
releases the authorization code when the **loopback redirect**
(`http://127.0.0.1:<port>/callback`) is **reachable from the browser that
approves the sign-in**. On a remote VPS install that loopback lives on the
server, not on your machine, so the consent screen **hangs forever and never
emits a code** — the normal "paste the callback URL" fallback has nothing to
paste. (This is a Google-side constraint: the same hang happens in any proxy
that uses the bundled Antigravity desktop client, not just OmniRoute.)
There are two supported ways to connect Antigravity to a remote OmniRoute.
### Option A — local login helper (recommended)
Run the OAuth on **your own computer**, where `127.0.0.1` is reachable, and paste
the result into the remote dashboard. The helper talks only to Google — it does
**not** need network access to your VPS, so it works even behind firewalls.
```bash
# On your LOCAL machine (needs Node.js + a browser):
npx omniroute login antigravity
# ↳ opens the Google consent in your browser, captures the callback on a local
# loopback port, exchanges it, and prints a one-line credential blob:
#
# omniroute-cred-v1.eyJ2IjoxLCJ...
```
Then, in the **remote** dashboard: **Providers → Antigravity → Connect**, and
paste the `omniroute-cred-v1.…` blob into the **Step 2** field (it accepts either
a callback URL or a credential blob). OmniRoute decodes it, runs the Cloud Code
onboarding server-side, and persists the connection.
> The blob contains a refresh token — treat it like a password. It is sent once
> over your dashboard connection and stored encrypted at rest.
Flags: `--no-browser` (print the URL instead of auto-opening), `--port <n>`
(pin the loopback port), `--timeout <ms>`.
### Option B — SSH local-forward tunnel
If you have SSH access to the VPS, forward the dashboard port so that the
loopback callback resolves back to the server through the tunnel:
```bash
# On your LOCAL machine:
ssh -L 20128:localhost:20128 user@your-vps
# then open http://localhost:20128 in your LOCAL browser and connect Antigravity
# normally — the 127.0.0.1:20128/callback redirect now reaches the VPS via SSH.
```
Because you reach the dashboard as `localhost:20128`, the Google consent
completes and the callback is delivered to the server through the same tunnel —
no blob needed. Keep the tunnel open until the connection shows as active.
> A fully headless alternative (no helper, no tunnel) is to configure your **own**
> Google OAuth web credentials + a public base URL; see the provider's OAuth
> environment variables. The two options above need no extra Google setup.
---
## Managing tokens
```bash
omniroute tokens create --name "laptop" --scope write [--expires 30]
# ↳ prints the secret ONCE — copy it now
omniroute tokens list # masked: id, name, scope, prefix, status, expiry
omniroute tokens revoke <id|prefix> # revoke immediately
omniroute tokens scopes # explain the three scopes
```
`tokens` commands require an **admin** credential. You can also manage tokens in
the dashboard under **Settings → Access Tokens** (create, revoke, copy-once).
---
## Configuring a coding CLI from the remote catalog
`omniroute configure` reads the **active server's** live model catalog and writes
a config on **your** machine.
```bash
omniroute configure codex
# Providers: glm, kmc, ollamacloud, opencode-go, …
# Provider: glm
# Model id: glm/glm-5.2
# ✔ Wrote ~/.codex/glm52.config.toml
# Use it: codex --profile glm52
# non-interactive
omniroute configure codex --provider glm --model glm/glm-5.2 --name glm52
```
The written profile references the inference key by env var
(`OMNIROUTE_API_KEY`) — the secret is never written to disk. For the one-time
base Codex setup (the `[model_providers.omniroute]` block), see
[CODEX-CLI-CONFIGURATION.md](./CODEX-CLI-CONFIGURATION.md).
### Per-CLI setup commands
Each supported CLI has a remote-aware setup command (all honour the active
context, or `--remote <url> --api-key <key>`):
| CLI | Command | What it writes |
|-----|---------|----------------|
| Codex | `omniroute setup-codex` | `~/.codex/<name>.config.toml` profiles (per model) |
| Claude Code | `omniroute setup-claude` | `~/.claude/profiles/<name>/settings.json` (per model) |
| OpenCode | `omniroute setup-opencode` | `~/.config/opencode/opencode.json` — the `omniroute` openai-compatible provider with every catalog model (run `opencode -m omniroute/<model>`) |
| Cline | `omniroute setup-cline` | `~/.cline/data/{globalState,secrets}.json` (CLI mode) + prints the VS Code extension settings to paste (OpenAI-compatible, Base URL **without** `/v1`) |
| Kilo Code | `omniroute setup-kilo` | `~/.local/share/kilo/auth.json` (CLI) + VS Code `kilocode.*` settings — OpenAI-compatible, Base URL **with** `/v1` |
| Continue | `omniroute setup-continue` | `~/.continue/config.yaml` (VS Code/JetBrains + `cn` CLI) — `provider: openai`, `apiBase` **with** `/v1`, key via `${{ secrets.OMNIROUTE_API_KEY }}` |
| Cursor | `omniroute setup-cursor` | prints the in-app steps (Settings → Models → Override OpenAI Base URL **with** `/v1` + key + model). Cursor config is opaque SQLite — chat panel only |
| Roo Code | `omniroute setup-roo` | writes a Roo import JSON (`~/.omniroute/roo-settings.json`) + sets `roo-cline.autoImportSettingsPath` + prints UI steps (OpenAI-compatible, Base URL **with** `/v1`) |
| Crush | `omniroute setup-crush` | `~/.config/crush/crush.json` — `openai-compat` provider, `base_url` **with** `/v1`, key via `$OMNIROUTE_API_KEY` |
| Goose | `omniroute setup-goose` | `~/.config/goose/config.yaml` (`GOOSE_PROVIDER=openai` + `OPENAI_HOST` **without** `/v1` + `GOOSE_MODEL`) + env recipe |
| Qwen Code | `omniroute setup-qwen` | `~/.qwen/settings.json` — openai `modelProvider`, `baseUrl` **with** `/v1`, key via `envKey` (OMNIROUTE_API_KEY) |
| Aider | `omniroute setup-aider` | `~/.aider.conf.yml` (`openai-api-base` **without** `/v1` + `model: openai/<id>`) + env recipe (`aider --message --yes`) |
```bash
# OpenCode (openai-compatible provider, all catalog models, remote VPS)
omniroute setup-opencode --remote http://192.168.0.15:20128 --api-key oma_live_xxx
omniroute setup-opencode --only glm,kimi # keep only matching models
opencode -m omniroute/glm/glm-5.2 "..." # export OMNIROUTE_API_KEY first
```
> OpenCode also has a richer **plugin** integration: `omniroute setup opencode`
> (now remote-aware via `--remote`) installs `@omniroute/opencode-plugin`.
> `setup-opencode` is the lightweight openai-compatible alternative. The API key
> is referenced via `{env:OMNIROUTE_API_KEY}` — never written to disk.
---
## Managing contexts (switch between servers)
A **context** is a saved server (baseUrl + credential + scope). `omniroute connect`
creates one and makes it active; from then on every command targets it. Manage and
switch between them with `omniroute contexts`:
```bash
omniroute contexts list # all contexts; the active one is marked ●
omniroute contexts current # the active server, auth status, scope
```
```text
| Name | Base URL | Auth | Scope | Description
● | vps | http://100.67.86.91:20128 | token | admin | Remote OmniRoute (…)
| default | http://localhost:20128 | ✗ | |
```
**Switch servers** — every subsequent command follows the active context:
```bash
omniroute contexts use vps # → all commands now hit the remote VPS
omniroute tokens list # (runs against the VPS)
omniroute contexts use default # → back to localhost
omniroute tokens list # (runs against the local server)
```
**Add a context manually** (instead of `connect`), inspect, or rename:
```bash
omniroute contexts add staging --url https://staging.example.com:20128 \
--access-token oma_live_xxxx --scope write --description "staging box"
omniroute contexts show staging # full details for one context
omniroute contexts rename staging stg
```
**Remove a context** — prompts for confirmation; pass `--yes` to skip it
(required for scripts / non-interactive shells, which otherwise decline safely):
```bash
omniroute contexts remove stg --yes
```
> `default` (localhost) cannot be removed. Removing the active context falls back
> to `default`. Tip: removing a context only drops the **local** saved credential —
> revoke the token on the server with `omniroute tokens revoke <id>` to actually
> kill access.
**Export / import** contexts (e.g. to move them between machines — secrets included,
so handle the file carefully):
```bash
omniroute contexts export --out contexts.json # default: stdout
omniroute contexts import contexts.json # overwrite; --merge to keep existing
```
---
## Quick end-to-end check
A copy-paste lifecycle to verify a remote setup from scratch — connect, mint a
scoped token, route a command, switch back, and tear down. Replace
`192.168.0.15` with your server's host/IP (Tailscale, LAN, or a public
`https://…` URL).
```bash
# 1. Connect (password → admin token, saved as a context that becomes active)
omniroute connect 192.168.0.15 # or: --key oma_live_xxxx (no password)
omniroute contexts current # shows the remote server + scope
# 2. Use it — management commands now run against the remote
omniroute tokens create --name laptop --scope read # mint a narrower token
omniroute tokens list # masked list, from the remote
# 3. Switch back and forth
omniroute contexts use default # → local
omniroute contexts use 192-168-0-15 # → remote again (name from `contexts list`)
# 4. Tear down. NOTE: `contexts remove` only deletes the LOCAL credential —
# it does NOT revoke the token on the server. Revoke server-side first if you
# want to actually kill access.
omniroute tokens revoke <id|prefix> # kills access on the server
omniroute contexts remove 192-168-0-15 --yes # drop the local context (even if active → falls back to default), no prompt
```
> `--yes` makes `contexts remove` non-interactive (required in scripts/CI; without
> it, a non-interactive shell declines safely instead of hanging). Removing the
> **active** context falls back to `default` automatically.
---
## Security notes
- Token plaintext is shown once; only the SHA-256 hash is persisted (same as API keys).
- `omniroute connect` reuses the login brute-force lockout + audit logging.
- Prefer HTTPS or a Tailnet for the transport; a bare host defaults to `http://`
for LAN/Tailscale convenience — pass a full `https://…` URL for TLS.
- The local context file is `~/.omniroute/config.json` (`chmod 600`); tokens are
never printed in logs (masked to a prefix).
---
## API endpoints (reference)
| Method | Route | Auth | Scope |
| ------ | --------------------- | ------------------- | -------------------------- |
| POST | `/api/cli/connect` | management password | — (public, password-gated) |
| GET | `/api/cli/whoami` | access token | read |
| GET | `/api/cli/tokens` | access token | admin |
| POST | `/api/cli/tokens` | access token | admin |
| DELETE | `/api/cli/tokens/:id` | access token | admin |
See [openapi.yaml](../openapi.yaml) for full schemas.
docs/guides/SETUP_GUIDE.md
+418
View File
@@ -0,0 +1,418 @@
---
title: "📖 Setup Guide — OmniRoute"
version: 3.8.40
lastUpdated: 2026-06-28
---
# 📖 Setup Guide — OmniRoute
> Complete setup reference for OmniRoute. For the quick version, see the [Quick Start in README](../README.md#-quick-start).
## Table of Contents
- [Install Methods](#install-methods)
- [CLI Tool Configuration](#cli-tool-configuration)
- [Protocol Setup (MCP + A2A)](#protocol-setup-mcp--a2a)
- [Timeout Configuration](#timeout-configuration)
- [Split-Port Mode](#split-port-mode)
- [Void Linux (xbps-src)](#void-linux-xbps-src-template)
- [Uninstalling](#uninstalling)
---
## Install Methods
### npm (recommended)
```bash
npm install -g omniroute
omniroute
```
Dashboard opens at `http://localhost:20128` and API base URL is `http://localhost:20128/v1`.
### pnpm
```bash
pnpm add -g omniroute@latest --allow-build=better-sqlite3 --allow-build=@swc/core
omniroute
```
> **pnpm users:** the `--allow-build` flag is required to enable native build scripts for `better-sqlite3` and `@swc/core`. The `pnpm approve-builds -g` command is not supported for global installs on pnpm v11.
### Arch Linux (AUR)
```bash
yay -S omniroute-bin
systemctl --user enable --now omniroute.service
```
The [AUR package](https://aur.archlinux.org/packages/omniroute-bin) installs OmniRoute and provides a systemd user service.
### From Source
```bash
npm install
PORT=20128 DASHBOARD_PORT=20129 NEXT_PUBLIC_BASE_URL=http://localhost:20129 npm run dev
```
> **Note:** `npm install` auto-generates `.env` from `.env.example` on first run. Subsequent installs will not overwrite an existing `.env`, so customizations are preserved. To re-seed, delete `.env` before re-running.
### Docker
See the [Docker Guide](./DOCKER_GUIDE.md) for complete Docker setup including Compose profiles and Caddy HTTPS.
### Desktop App (Electron)
OmniRoute ships a desktop wrapper built on Electron 41 + electron-builder 26.10. Available scripts (workspace root):
```bash
npm run electron:dev # Run desktop with hot-reload
npm run electron:build # Build for current OS (auto-detected)
npm run electron:build:win # Windows installer (NSIS + portable)
npm run electron:build:mac # macOS (dmg + zip, arm64+x64)
npm run electron:build:linux # Linux (AppImage + deb + rpm)
npm run electron:smoke:packaged # Smoke-test packaged build
```
Releases of the desktop installers are attached to GitHub Releases. For the full Electron deep-dive (signing, IPC bridge, distros), see [`ELECTRON_GUIDE.md`](./ELECTRON_GUIDE.md) _(criado em fase posterior)_.
### Headless server (CI/automation)
For unattended setups (Docker, Kubernetes, CI), use:
```bash
omniroute setup --non-interactive
omniroute providers test-batch
```
Combined with env vars (`INITIAL_PASSWORD`, `OMNIROUTE_WS_BRIDGE_SECRET`, etc.), this lets you spin up an OmniRoute instance fully scriptable.
### CLI Options
| Command | Description |
| ----------------------- | -------------------------------------------------------------- |
| `omniroute` | Start server (`PORT=20128`, API and dashboard on same port) |
| `omniroute setup` | Guided CLI onboarding for password and first provider |
| `omniroute doctor` | Run local health checks without starting the server |
| `omniroute providers` | Discover, list, validate, and test providers from CLI |
| `omniroute config` | CLI tool configuration — list, get, set, validate configs |
| `omniroute status` | Offline status dashboard — version, DB, tools, config |
| `omniroute logs` | Stream usage logs from the API (supports `--follow`) |
| `omniroute update` | Check for or apply OmniRoute updates |
| `omniroute provider` | Manage provider connections — add, list, remove, test, default |
| `omniroute --port 3000` | Set canonical/API port to 3000 |
| `omniroute --mcp` | Start MCP server (stdio transport) |
| `omniroute --no-open` | Don't auto-open browser |
| `omniroute --help` | Show help |
Headless setup can be scripted with flags or environment variables:
```bash
omniroute setup --non-interactive --password "$OMNIROUTE_PASSWORD"
omniroute setup --non-interactive --add-provider --provider openai --api-key "$OPENAI_API_KEY"
omniroute setup --non-interactive --add-provider --provider openai --api-key "$OPENAI_API_KEY" --test-provider
```
Run local diagnostics without opening the dashboard:
```bash
omniroute doctor
omniroute doctor --json
omniroute doctor --no-liveness
```
Manage providers from SSH or scripts without opening the dashboard:
```bash
omniroute providers available
omniroute providers available --search openai
omniroute providers available --category api-key
omniroute providers list
omniroute providers test <id-or-name>
omniroute providers test-all
omniroute providers validate
```
---
## CLI Tool Configuration
### 1) Connect Providers and Create API Key
1. Open Dashboard → `Providers` and connect at least one provider (OAuth or API key).
2. Open Dashboard → `Endpoints` and create an API key.
3. (Optional) Open Dashboard → `Combos` and set your fallback chain.
### 2) Point Your Coding Tool
```txt
Base URL: http://localhost:20128/v1
API Key: [copy from Endpoint page]
Model: if/kimi-k2-thinking (or any provider/model prefix)
```
If your editor cannot send `Authorization: Bearer ...`, use the tokenized compatibility base instead:
```txt
Base URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/
Models URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/models
Chat URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/chat/completions
Ollama Tags URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/api/tags
```
Works with Claude Code, Codex CLI, Cursor, Cline, OpenClaw, OpenCode, and OpenAI-compatible SDKs.
#### Auto-configure with `setup-*`
Instead of pasting the base URL and key by hand, let OmniRoute write each tool's
own config from the live model catalog. One command per tool:
```bash
omniroute setup-codex # ~/.codex/<name>.config.toml profiles
omniroute setup-claude # ~/.claude/profiles/<name>/settings.json
omniroute setup-opencode # ~/.config/opencode/opencode.json (openai-compatible)
omniroute setup-cline # Cline CLI + VS Code extension settings
omniroute setup-kilo # Kilo Code
omniroute setup-continue # ~/.continue/config.yaml (Continue / cn)
omniroute setup-cursor # prints Cursor's in-app steps
omniroute setup-roo # Roo Code import + autoImport pointer
omniroute setup-crush # ~/.config/crush/crush.json
omniroute setup-goose # ~/.config/goose/config.yaml
omniroute setup-qwen # ~/.qwen/settings.json
omniroute setup-aider # ~/.aider.conf.yml
```
Each accepts `--remote <url> --api-key <key>` to configure a local tool against a
**remote** OmniRoute, plus `--dry-run` to preview. The launchers
`omniroute launch` (Claude Code) and `omniroute launch-codex` (Codex) spawn the CLI
with the right env injected, writing no config at all.
For the full table (what each command writes, every flag, local vs remote, base-URL
`/v1` conventions), see **[CLI Integrations](./CLI-INTEGRATIONS.md)**.
For detailed per-tool configuration (Claude Code, Codex CLI, Cursor, Cline, OpenClaw, Kilo Code, Copilot, and more), see the dedicated **[CLI Tools Guide](../reference/CLI-TOOLS.md)**.
---
## Protocol Setup (MCP + A2A)
### MCP Setup (Model Context Protocol)
Start MCP transport in stdio mode:
```bash
omniroute --mcp
```
Recommended validation flow:
```bash
# 1. Start MCP server
omniroute --mcp
# 2. From your MCP client, call:
omniroute_get_health # Should return system health
omniroute_list_combos # Should return active combos
# 3. Or run the full E2E suite:
npm run test:protocols:e2e
```
#### MCP Client Configuration
**Claude Code:**
```bash
claude mcp add-server omniroute --type http --url http://localhost:20128/api/mcp/stream
```
**Cursor / Cline:**
Add to your MCP settings:
```json
{
"mcpServers": {
"omniroute": {
"command": "omniroute",
"args": ["--mcp"],
"env": {}
}
}
}
```
**Full MCP documentation:** [MCP Server README](../../open-sse/mcp-server/README.md) — 87 tools, IDE configs, Python/TS/Go clients.
### A2A Setup (Agent-to-Agent Protocol)
Verify the Agent Card:
```bash
curl http://localhost:20128/.well-known/agent.json
```
Send a task:
```bash
curl -X POST http://localhost:20128/a2a \
-H 'content-type: application/json' \
-d '{"jsonrpc":"2.0","id":"quickstart","method":"message/send","params":{"skill":"quota-management","messages":[{"role":"user","content":"Give me a short quota summary."}]}}'
```
**Full A2A documentation:** [A2A Server README](../../src/lib/a2a/README.md) — JSON-RPC 2.0, skills, streaming, task lifecycle.
---
## Timeout Configuration
### Basic Timeouts
For most deployments, you only need these two variables:
| Variable | Default | Purpose |
| ------------------------ | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `REQUEST_TIMEOUT_MS` | `600000` | Shared baseline for upstream response-start timeout, hidden Undici timeouts, TLS fingerprint requests, and API bridge request/proxy timeouts |
| `STREAM_IDLE_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` | Maximum gap between streaming chunks before OmniRoute aborts the SSE stream |
Backward compatibility is preserved: existing `FETCH_TIMEOUT_MS`, `API_BRIDGE_PROXY_TIMEOUT_MS`, and other per-layer timeout vars still work and override the shared baseline.
### Provider-Specific Notes
For Claude Code-compatible upstreams (`anthropic-compatible-cc-*`), OmniRoute derives the outbound `X-Stainless-Timeout` header from the resolved fetch timeout so provider-side read timeouts stay aligned with your env configuration.
For third-party Claude Code-compatible reverse proxies, OmniRoute keeps the default `anthropic-beta` set conservative and, when `Client Cache Control` is left on `Auto`, only forwards client-provided `cache_control` markers. Enable the per-connection "Enable redact-thinking beta" toggle only when the upstream specifically requires redacted Claude thinking streams.
### Advanced Timeout Overrides
| Variable | Default | Purpose |
| ---------------------------------------- | ------------------------------------------ | -------------------------------------------------------------------- |
| `FETCH_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` | Upstream response-start timeout used until response headers arrive |
| `FETCH_HEADERS_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Undici time limit for receiving upstream response headers |
| `FETCH_BODY_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Undici time limit between upstream body chunks (`0` disables it) |
| `FETCH_CONNECT_TIMEOUT_MS` | `30000` | Undici TCP connect timeout |
| `FETCH_KEEPALIVE_TIMEOUT_MS` | `4000` | Undici idle keep-alive socket timeout |
| `TLS_CLIENT_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Timeout for TLS fingerprint requests made through `wreq-js` |
| `API_BRIDGE_PROXY_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` or `600000` | Timeout for `/v1` proxy forwarding from API port to dashboard port |
| `API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS` | `max(API_BRIDGE_PROXY_TIMEOUT_MS, 300000)` | Incoming request timeout on the API bridge server |
| `API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS` | `60000` | Incoming header timeout on the API bridge server |
| `API_BRIDGE_SERVER_KEEPALIVE_TIMEOUT_MS` | `5000` | Keep-alive timeout on the API bridge server |
| `API_BRIDGE_SERVER_SOCKET_TIMEOUT_MS` | `0` | Socket inactivity timeout on the API bridge server (`0` disables it) |
> **Note:** For streaming requests, `FETCH_TIMEOUT_MS` only covers connection setup / waiting for the first upstream response. Once the stream is active, OmniRoute will only abort on an actual stall (`STREAM_IDLE_TIMEOUT_MS`) or Undici body inactivity (`FETCH_BODY_TIMEOUT_MS`).
### Reverse Proxy Compatibility
If you run OmniRoute behind Nginx, Caddy, Cloudflare, or another reverse proxy, make sure the proxy timeouts are also higher than your OmniRoute stream/fetch timeouts.
---
## Split-Port Mode
Run API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking):
```bash
PORT=20128 DASHBOARD_PORT=20129 omniroute
# API: http://localhost:20128/v1
# Dashboard: http://localhost:20129
```
---
## Void Linux (xbps-src) Template
For Void Linux users, you can build a native package using `xbps-src`. Save this block as `srcpkgs/omniroute/template`:
```bash
# Template file for 'omniroute'
pkgname=omniroute
version=3.8.0
revision=1
hostmakedepends="nodejs python3 make"
depends="openssl"
short_desc="Universal AI gateway with smart routing for multiple LLM providers"
maintainer="zenobit <zenobit@disroot.org>"
license="MIT"
homepage="https://github.com/diegosouzapw/OmniRoute"
distfiles="https://github.com/diegosouzapw/OmniRoute/archive/refs/tags/v${version}.tar.gz"
# Regenerate the checksum for each release with:
# curl -L -o /tmp/omniroute.tar.gz "https://github.com/diegosouzapw/OmniRoute/archive/refs/tags/v${version}.tar.gz" && sha256sum /tmp/omniroute.tar.gz
checksum=PLACEHOLDER_REGENERATE_PER_RELEASE
system_accounts="_omniroute"
omniroute_homedir="/var/lib/omniroute"
export NODE_ENV=production
export npm_config_engine_strict=false
export npm_config_loglevel=error
export npm_config_fund=false
export npm_config_audit=false
do_build() {
local _gyp_arch
case "$XBPS_TARGET_MACHINE" in
aarch64*) _gyp_arch=arm64 ;;
armv7*|armv6*) _gyp_arch=arm ;;
i686*) _gyp_arch=ia32 ;;
*) _gyp_arch=x64 ;;
esac
NODE_ENV=development npm ci --ignore-scripts
npm run build
cp -r .next/static .next/standalone/.next/static
[ -d public ] && cp -r public .next/standalone/public || true
local _node_gyp=/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js
(cd node_modules/better-sqlite3 && node "$_node_gyp" rebuild --arch="$_gyp_arch")
local _bs3_release=.next/standalone/node_modules/better-sqlite3/build/Release
mkdir -p "$_bs3_release"
cp node_modules/better-sqlite3/build/Release/better_sqlite3.node "$_bs3_release/"
rm -rf .next/standalone/node_modules/@img
for _mod in pino-abstract-transport split2 process-warning; do
cp -r "node_modules/$_mod" .next/standalone/node_modules/
done
}
do_check() {
npm run test:unit
}
do_install() {
vmkdir usr/lib/omniroute/.next
vcopy .next/standalone/. usr/lib/omniroute/.next/standalone
for _d in \
.next/standalone/.next/server/app/dashboard \
.next/standalone/.next/server/app/dashboard/settings \
.next/standalone/.next/server/app/dashboard/providers; do
touch "${DESTDIR}/usr/lib/omniroute/${_d}/.keep"
done
cat > "${WRKDIR}/omniroute" <<'EOF'
#!/bin/sh
export PORT="${PORT:-20128}"
export DATA_DIR="${DATA_DIR:-${XDG_DATA_HOME:-${HOME}/.local/share}/omniroute}"
export APP_LOG_TO_FILE="${APP_LOG_TO_FILE:-false}"
mkdir -p "${DATA_DIR}"
exec node /usr/lib/omniroute/.next/standalone/server.js "$@"
EOF
vbin "${WRKDIR}/omniroute"
}
post_install() {
vlicense LICENSE
}
```
---
## Uninstalling
| Command | Action |
| ------------------------ | ----------------------------------------------------------------------------------- |
| `npm run uninstall` | Removes the system app but **keeps your DB and configurations** in `~/.omniroute`. |
| `npm run uninstall:full` | Removes the app AND permanently **erases all configurations, keys, and databases**. |
> For detailed uninstall instructions across all methods, see [UNINSTALL.md](./UNINSTALL.md).
docs/guides/TERMUX_GUIDE.md
+168
View File
@@ -0,0 +1,168 @@
---
title: "Termux Headless Setup"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Termux Headless Setup
OmniRoute can run as a headless server on Android through Termux. The Electron desktop app is not supported in Termux, but the web dashboard and OpenAI-compatible API work from the local browser or from other devices on the same network.
## Prerequisites
Install Termux from F-Droid or GitHub releases, then update packages and install the build tools required by native dependencies such as `better-sqlite3`.
```bash
pkg update
pkg upgrade
pkg install nodejs python build-essential git
```
> **Node.js version:** OmniRoute requires Node `>=22.22.2 <23 || >=24.0.0 <27` (matches `engines` in `package.json` / `SUPPORTED_NODE_RANGE`). Termux's `nodejs-lts` typically ships Node 20 LTS, which is **no longer supported** — install `pkg install nodejs` (current) instead and verify `node --version` reports a 22.x/24.x+ line.
If native package compilation fails, rerun the `pkg install` command above and then retry the OmniRoute install.
## Install
Run the latest published package directly:
```bash
npx -y omniroute@latest
```
You can also install it globally:
```bash
npm install -g omniroute
omniroute
```
## Run
Start OmniRoute in headless server mode:
```bash
omniroute
```
or:
```bash
npx omniroute
```
The dashboard listens on:
```text
http://localhost:20128
```
Open that URL in the Android browser. If you run clients inside Termux, use the same host and port as the OpenAI-compatible base URL.
## Background Execution
For a simple background process:
```bash
nohup omniroute > omniroute.log 2>&1 &
```
To stop it:
```bash
pkill -f omniroute
```
For automatic startup after device boot, install the Termux:Boot add-on and create a boot script:
```bash
mkdir -p ~/.termux/boot
cat > ~/.termux/boot/omniroute.sh <<'EOF'
#!/data/data/com.termux/files/usr/bin/sh
cd "$HOME"
nohup omniroute > "$HOME/omniroute.log" 2>&1 &
EOF
chmod +x ~/.termux/boot/omniroute.sh
```
Android battery optimization can stop long-running background processes. Disable battery optimization for Termux if the server is expected to stay online.
## Access From Other Devices
Find the phone IP address on the WiFi network:
```bash
ip addr show wlan0
```
Then open the dashboard from another device:
```text
http://PHONE_IP:20128
```
For example:
```text
http://192.168.1.50:20128
```
Keep the phone and client on the same trusted network. If you expose OmniRoute outside the phone, enable API keys and dashboard authentication.
## Data Directory
By default OmniRoute stores data under the Termux home directory, following the same server-side data path behavior used on Linux. To place the database somewhere explicit:
```bash
export DATA_DIR="$HOME/.omniroute"
omniroute
```
## Limitations
- Electron does not run in Termux.
- There is no system tray or desktop integration.
- This setup is server-only: use the browser dashboard.
- Native dependencies may need local compilation.
- Low-memory Android devices may need fewer concurrent requests.
- MITM/system certificate features may require Android-level trust-store work outside Termux.
## Troubleshooting
### better-sqlite3 Build Errors
Install the Termux build toolchain:
```bash
pkg install nodejs python build-essential
```
Then rerun:
```bash
npx -y omniroute@latest
```
### Port Already In Use
Check what is listening on the default port:
```bash
ss -ltnp | grep 20128
```
Stop the old process:
```bash
pkill -f omniroute
```
### Dashboard Not Reachable From Another Device
Verify both devices are on the same WiFi network, then test from Termux:
```bash
curl http://localhost:20128
```
If local access works but LAN access does not, check Android hotspot/WiFi isolation and any firewall or VPN profile on the phone.
docs/guides/TIERS.md
+106
View File
@@ -0,0 +1,106 @@
---
title: "OmniRoute Tiers — User Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute Tiers — User Guide
OmniRoute organizes the 207+ supported providers into 3 economic tiers. Each
request travels through them in order until one returns successfully — you
get the cheapest viable response without ever writing fallback code.
## Tier 1 — Subscription
**Providers you already pay for.** OmniRoute uses every drop of quota before
it expires.
| Provider | Why Tier 1 |
| ----------------------------------- | -------------------------------------------- |
| Claude Code OAuth | Anthropic Pro/Team — flat-rate, often unused |
| OpenAI Codex (ChatGPT subscription) | Plus/Team includes Codex quota |
| GitHub Copilot | Per-seat — quota resets monthly |
| Cursor IDE | Pro plan quota |
| Antigravity / Windsurf | Built-in quotas |
**Strategy**: route here first for every request that fits the model's
strengths. Quota tracker monitors approaching reset; combo strategies
`reset-aware` and `subscription` prioritize accordingly.
## Tier 2 — Cheap
**Pay-per-token providers under $1/1M tokens.** Reserved for high-volume work
or after Tier 1 quotas hit limits.
| Provider | Price (input/output) | Strengths |
| ---------------------------- | -------------------- | -------------------- |
| DeepSeek V4 Pro | $0.27 / $1.10 per 1M | Code, reasoning |
| GLM-4.5 | $0.60 / $2.20 per 1M | Long context |
| MiniMax M1 | $0.20 / $1.10 per 1M | Speed |
| Qwen Coder | $0.30 / $1.20 per 1M | Code |
| OpenRouter (price-optimized) | varies | 100+ models, dynamic |
**Strategy**: combo `cost-optimized` picks lowest $/token model that meets
the task's capability filter (vision, JSON mode, tools, max-context).
## Tier 3 — Free
**Zero-cost providers** — free tiers, credit programs, OAuth daily quotas.
| Provider | Free quota / credits |
| ---------------- | ------------------------------------ |
| Kiro AI | Free Claude tier (generous fair-use) |
| OpenCode Free | No auth, generous rate limits |
| Qoder | Free OAuth |
| Google Vertex AI | $300 new-account credits |
| Amazon Q | Free tier for AWS users |
| Pollinations | Open public API |
| Cloudflare AI | Workers AI free tier |
**Strategy**: combo `auto` with budget cap routes here when Tier 1+2 fail
or when `useFreeOnly=true` is set. Free providers often have weaker
rate limits — circuit breaker recovers them on backoff.
## Configuring tiers
Dashboard → **Tiers** → assign your providers. Defaults (from `tierDefaults.json`) are
sensible; edit when you have specific subscriptions to prioritize or providers to exclude.
Auto-Combo's 9-factor scoring also considers tier. See
[`docs/routing/AUTO-COMBO.md`](../routing/AUTO-COMBO.md).
## Telemetry
Dashboard → **Usage** shows tokens spent per tier per day. Use this to:
- Confirm Tier 1 is utilized fully (otherwise you're wasting subscription value)
- Identify which Tier 2 models are picked most (consolidate to 1-2)
- Verify Tier 3 saves money on test/exploration workloads
## Common patterns
### Pure-free workload
```json
{
"strategy": "auto",
"config": { "auto": { "weights": { "costInv": 0.5, "tierPriority": 0.3 } } }
}
```
Forces strongly towards Tier 3; only uses Tier 2 if Tier 3 is unavailable.
### Subscription-first with cheap fallback
```json
{
"strategy": "priority",
"targets": [
{ "provider": "claude-code-oauth", "weight": 1 },
{ "provider": "deepseek", "weight": 1 },
{ "provider": "kiro", "weight": 1 }
]
}
```
Explicit ordered list matching Tier 1 → Tier 2 → Tier 3.
docs/guides/TROUBLESHOOTING.md
+514
View File
@@ -0,0 +1,514 @@
---
title: "Troubleshooting"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Troubleshooting
> **For Users**: Looking for quick fixes? See the [Quick Reference](#quick-reference) below.
🌐 **Languages:** 🇺🇸 [English](./TROUBLESHOOTING.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/TROUBLESHOOTING.md) | 🇪🇸 [Español](../i18n/es/docs/guides/TROUBLESHOOTING.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/TROUBLESHOOTING.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/TROUBLESHOOTING.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/TROUBLESHOOTING.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/TROUBLESHOOTING.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/TROUBLESHOOTING.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/guides/TROUBLESHOOTING.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/TROUBLESHOOTING.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/TROUBLESHOOTING.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/TROUBLESHOOTING.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/TROUBLESHOOTING.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/TROUBLESHOOTING.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/TROUBLESHOOTING.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/TROUBLESHOOTING.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/TROUBLESHOOTING.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/TROUBLESHOOTING.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/TROUBLESHOOTING.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/TROUBLESHOOTING.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/TROUBLESHOOTING.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/TROUBLESHOOTING.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/TROUBLESHOOTING.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/TROUBLESHOOTING.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/TROUBLESHOOTING.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/TROUBLESHOOTING.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/TROUBLESHOOTING.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/TROUBLESHOOTING.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/TROUBLESHOOTING.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/TROUBLESHOOTING.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/TROUBLESHOOTING.md)
Common problems and solutions for OmniRoute.
---
## Quick Reference
**New to OmniRoute?** Start here — these solve 90% of problems:
| I see this | What it means | What to do |
| ----------------------- | ----------------------------------- | ------------------------------------------------------------------------------------------------- |
| "Can't connect" | OmniRoute isn't running | Run `omniroute` or `docker restart omniroute` |
| "Invalid API key" | Your key is wrong or expired | Re-copy the key from the provider's website |
| "Rate limit exceeded" | You're sending too many requests | Wait 1 minute, or use `model: "auto"` for automatic fallback |
| "Quota exceeded" | You've used up your free/paid quota | Connect more providers, or use free providers (Kiro, Pollinations) |
| "Slow responses" | Provider is busy or far away | Use `model: "auto/fast"` or connect a faster provider (Groq, Cerebras) |
| "Wrong provider used" | `auto` picked a different provider | That's normal! `auto` picks the best one. Force a specific provider with `model: "openai/gpt-4o"` |
| "502 Bad Gateway" | Provider is down | Wait and retry, or use `model: "auto"` to switch providers |
| "401 Unauthorized" | Your credentials are wrong | Check your API key or re-authenticate with OAuth |
| "429 Too Many Requests" | Rate limited | Wait 1 minute, or connect more providers |
**Still stuck?** See the [detailed troubleshooting](#detailed-troubleshooting) below, or ask on [Discord](https://discord.gg/EkzRkpzKYt).
---
## Detailed Troubleshooting
---
## Quick Fixes
| Problem | Solution |
| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| First login not working | Set `INITIAL_PASSWORD` in `.env` (no hardcoded default) |
| Dashboard opens on wrong port | Set `PORT=20128` and `NEXT_PUBLIC_BASE_URL=http://localhost:20128` |
| No logs written to disk | Set `APP_LOG_TO_FILE=true` and verify call log capture is enabled |
| EACCES: permission denied | Set `DATA_DIR=/path/to/writable/dir` to override `~/.omniroute` |
| Routing strategy not saving | Update to the latest v3.x release (Zod schema fix for settings persistence shipped in earlier versions) |
| Login crash / blank page | Check Node.js version — see [Node.js Compatibility](#nodejs-compatibility) below |
| `dlopen` / `slice is not valid mach-o file` (macOS) | Run `cd $(npm root -g)/omniroute/app && npm rebuild better-sqlite3 && omniroute` — see [macOS native module rebuild](#macos-native-module-rebuild) below |
| Proxy "fetch failed" | Ensure proxy config is set at the correct level — see [Proxy Issues](#proxy-issues) below |
---
## Node.js Compatibility
<a name="nodejs-compatibility"></a>
### Login page crashes or shows "Module self-registration" error
**Cause:** You are running a Node.js version outside OmniRoute's approved secure runtime floor. The most common case is running an older Node 22 or 24 patch level that falls below the patched security floor OmniRoute requires.
**Symptoms:**
- Login page shows a blank screen or a server error
- Console shows `Error: Module did not self-register` or similar native binding errors
- The login page shows an **orange warning banner** with your Node version if the runtime is outside the supported secure policy
**Fix:**
1. Install a supported Node.js LTS release (recommended: Node.js 24.x):
```bash
nvm install 24
nvm use 24
```
2. Verify your version: `node --version` should show `v24.0.0` or newer on the 24.x LTS line
3. Reinstall OmniRoute: `npm install -g omniroute`
4. Restart: `omniroute`
> **Supported secure versions:** `>=22.22.2 <23` or `>=24.0.0 <27`. Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported.
### macOS: `dlopen` / "slice is not valid mach-o file"
<a name="macos-native-module-rebuild"></a>
**Cause:** After a global `npm install -g omniroute`, the `better-sqlite3` native binary inside the package may have been compiled for a different architecture or Node.js ABI than what is running locally. This is common on macOS (both Apple Silicon and Intel) when the pre-built binary does not match your environment.
**Symptoms:**
- Server fails immediately on startup with a `dlopen` error
- Error contains `slice is not valid mach-o file`
- Full example:
```
dlopen(/Users/<user>/.nvm/versions/node/v24.14.1/lib/node_modules/omniroute/app/node_modules/better-sqlite3/build/Release/better_sqlite3.node, 0x0001): tried: '...' (slice is not valid mach-o file)
```
**Fix — rebuild for your local environment (no Node.js downgrade required):**
```bash
cd $(npm root -g)/omniroute/app
npm rebuild better-sqlite3
omniroute
```
> **Note:** This recompiles the native binding against your local Node.js version and CPU architecture, resolving the binary mismatch. The officially supported runtime range is **`>=22.22.2 <23` or `>=24.0.0 <27`** (`SUPPORTED_NODE_RANGE` in `src/shared/utils/nodeRuntimeSupport.ts`, aligned with the `package.json` `engines` field). Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported with `better-sqlite3` v12.x.
---
## Proxy Issues
<a name="proxy-issues"></a>
### Provider validation shows "fetch failed"
**Cause:** The API key validation endpoint (`POST /api/providers/validate`) was previously bypassing proxy configuration, causing failures in environments that require proxy routing.
**Fix (v3.5.5+):** This is now fixed. Provider validation routes through `runWithProxyContext`, honoring provider-level and global proxy settings automatically.
### Token health check fails with "fetch failed"
**Cause:** Background OAuth token refresh was not resolving proxy configuration per connection.
**Fix (v3.5.5+):** The token health check scheduler now resolves proxy config per connection before attempting refresh. Update to v3.5.5+.
### SOCKS5 proxy returns "invalid onRequestStart method"
**Cause:** On Node.js 22, the undici@8 dispatcher is incompatible with Node's built-in `fetch()` implementation.
**Fix (v3.5.5+):** OmniRoute now uses undici's own `fetch()` function when a proxy dispatcher is active, ensuring consistent behavior. Update to v3.5.5+.
### MITM proxy under WSL: desktop apps on the Windows host are not intercepted
**Cause:** The MITM proxy and its CA certificate install into the environment where OmniRoute runs. Under WSL that environment is the Linux guest, while the AI desktop apps (Kiro, Trae, Copilot, Zed, …) run on the Windows host. The host apps do not trust the guest's certificate store and do not route through the guest's system proxy, so desktop interception does not engage there.
**Recommendation:** Run OmniRoute natively on the same OS as the desktop apps you want to intercept (Windows for Windows apps; macOS/Linux likewise). Keeping OmniRoute inside WSL while targeting host apps requires manually trusting the generated CA certificate on the Windows host and pointing each host app's network/proxy settings at the WSL proxy endpoint — an unsupported, fragile setup.
---
## Provider Issues
### "Language model did not provide messages"
**Cause:** Provider quota exhausted.
**Fix:**
1. Check dashboard quota tracker
2. Use a combo with fallback tiers
3. Switch to cheaper/free tier
### Rate Limiting
**Cause:** Subscription quota exhausted.
**Fix:**
- Add fallback: `cc/claude-opus-4-6 → glm/glm-4.7 → if/kimi-k2-thinking`
- Use GLM/MiniMax as cheap backup
### OAuth Token Expired
OmniRoute auto-refreshes tokens. If issues persist:
1. Dashboard → Provider → Reconnect
2. Delete and re-add the provider connection
### Kiro multi-account: second account invalidates the first
**Cause:** Kiro's backend enforces a single active session per OIDC client registration.
When two accounts share the same registered client (connections imported before v3.8.0),
refreshing one account's token invalidates the other's refresh token.
**Fix (v3.8.0+):** Re-import affected connections.
Starting with v3.8.0, every new Kiro connection created via **Import Token**,
**Google/GitHub social login**, or **Auto-Import** automatically registers its own
dedicated OIDC client. The connection is therefore fully isolated and refreshing one
account has no effect on any other account.
Connections that were imported _before_ v3.8.0 do not carry a per-connection client
registration. Those connections continue to use the shared social-auth refresh endpoint.
To gain isolation, delete the old connection from Dashboard → Providers and re-add it
via any of the three import flows.
For full details and step-by-step instructions for adding two Kiro accounts side by side,
see [`docs/guides/KIRO_SETUP.md`](./KIRO_SETUP.md).
---
## Cloud Issues
### Cloud Sync Errors
1. Verify `BASE_URL` points to your running instance (e.g., `http://localhost:20128`)
2. Verify `CLOUD_URL` points to your cloud endpoint (e.g., `https://omniroute.dev`)
3. Keep `NEXT_PUBLIC_*` values aligned with server-side values
### Cloud `stream=false` Returns 500
**Symptom:** `Unexpected token 'd'...` on cloud endpoint for non-streaming calls.
**Cause:** Upstream returns SSE payload while client expects JSON.
**Workaround:** Use `stream=true` for cloud direct calls. Local runtime includes SSE→JSON fallback.
### Cloud Says Connected but "Invalid API key"
1. Create a fresh key from local dashboard (`/api/keys`)
2. Run cloud sync: Enable Cloud → Sync Now
3. Old/non-synced keys can still return `401` on cloud
---
## Docker Issues
### CLI Tool Shows Not Installed
1. Check runtime fields: `curl http://localhost:20128/api/cli-tools/runtime/codex | jq`
2. For portable mode: use image target `runner-cli` (bundled CLIs)
3. For host mount mode: set `CLI_EXTRA_PATHS` and mount host bin directory as read-only
4. If `installed=true` and `runnable=false`: binary was found but failed healthcheck
### Quick Runtime Validation
```bash
curl -s http://localhost:20128/api/cli-tools/codex-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
curl -s http://localhost:20128/api/cli-tools/claude-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
curl -s http://localhost:20128/api/cli-tools/openclaw-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
```
---
## Cost Issues
### High Costs
1. Check usage stats in Dashboard → Usage
2. Switch primary model to GLM/MiniMax
3. Use free tier (Qoder, Kiro) for non-critical tasks
4. Set cost budgets per API key: Dashboard → API Keys → Budget
---
## Debugging
### Enable Log Files
Set `APP_LOG_TO_FILE=true` in your `.env` file. Application logs are written under `logs/`.
Request artifacts are stored under `${DATA_DIR}/call_logs/` when the call log pipeline is
enabled in settings.
When pipeline capture is enabled, set `CALL_LOG_PIPELINE_CAPTURE_STREAM_CHUNKS=false` to omit
stream chunk payloads, or tune `CALL_LOG_PIPELINE_MAX_SIZE_KB` to change the artifact cap in KB.
### Check Provider Health
```bash
# Health dashboard
http://localhost:20128/dashboard/health
# API health check
curl http://localhost:20128/api/monitoring/health
```
### Runtime Storage
- Main state: `${DATA_DIR}/storage.sqlite` (providers, combos, aliases, keys, settings)
- Usage: SQLite tables in `storage.sqlite` (`usage_history`, `call_logs`, `proxy_logs`) + optional `${DATA_DIR}/call_logs/`
- Application logs: `<repo>/logs/...` (when `APP_LOG_TO_FILE=true`)
- Call log artifacts: `${DATA_DIR}/call_logs/YYYY-MM-DD/...` when the call log pipeline is enabled
The Request Logs page's **Clean history** action clears `call_logs`, legacy
`request_detail_logs`, and the local `${DATA_DIR}/call_logs/` artifact directory.
---
## Circuit Breaker Issues
### Provider stuck in OPEN state
When a provider's circuit breaker is OPEN, requests are blocked until the cooldown expires.
**Fix:**
1. Go to **Dashboard → Settings → Resilience**
2. Check the circuit breaker card for the affected provider
3. Click **Reset All** to clear all breakers, or wait for the cooldown to expire
4. Verify the provider is actually available before resetting
### Provider keeps tripping the circuit breaker
If a provider repeatedly enters OPEN state:
1. Check **Dashboard → Health → Provider Health** for the failure pattern
2. Go to **Settings → Resilience → Provider Profiles** and increase the failure threshold
3. Check if the provider has changed API limits or requires re-authentication
4. Review latency telemetry — high latency may cause timeout-based failures
---
## Audio Transcription Issues
### "Unsupported model" error
- Ensure you're using the correct prefix: `deepgram/nova-3` or `assemblyai/best`
- Verify the provider is connected in **Dashboard → Providers**
### Transcription returns empty or fails
- Check supported audio formats: `mp3`, `wav`, `m4a`, `flac`, `ogg`, `webm`
- Verify file size is within provider limits (typically < 25MB)
- Check provider API key validity in the provider card
---
## Translator Debugging
Use **Dashboard → Translator** to debug format translation issues:
| Mode | When to Use |
| ---------------- | -------------------------------------------------------------------------------------------- |
| **Playground** | Compare input/output formats side by side — paste a failing request to see how it translates |
| **Chat Tester** | Send live messages and inspect the full request/response payload including headers |
| **Test Bench** | Run batch tests across format combinations to find which translations are broken |
| **Live Monitor** | Watch real-time request flow to catch intermittent translation issues |
### Common format issues
- **Thinking tags not appearing** — Check if the target provider supports thinking and the thinking budget setting
- **Tool calls dropping** — Some format translations may strip unsupported fields; verify in Playground mode
- **System prompt missing** — Claude and Gemini handle system prompts differently; check translation output
- **SDK returns raw string instead of object** — Resolved in v1.x; response sanitizer strips non-standard fields (`x_groq`, `usage_breakdown`, etc.) that cause OpenAI SDK Pydantic validation failures. If you still see this on v3.x+, please file an issue.
- **GLM/ERNIE rejects `system` role** — Resolved in v1.x; role normalizer automatically merges system messages into user messages for incompatible models. If you still see this on v3.x+, please file an issue.
- **`developer` role not recognized** — Resolved in v1.x; automatically converted to `system` for non-OpenAI providers. If you still see this on v3.x+, please file an issue.
- **`json_schema` not working with Gemini** — Resolved in v1.x; `response_format` is now converted to Gemini's `responseMimeType` + `responseSchema`. If you still see this on v3.x+, please file an issue.
---
## Resilience Settings
### Auto rate-limit not triggering
- Auto rate-limit only applies to API key providers (not OAuth/subscription)
- Verify **Settings → Resilience → Provider Profiles** has auto-rate-limit enabled
- Check if the provider returns `429` status codes or `Retry-After` headers
### Tuning exponential backoff
Provider profiles support these settings:
- **Base delay** — Initial wait time after first failure (default: 1s)
- **Max delay** — Maximum wait time cap (default: 30s)
- **Multiplier** — How much to increase delay per consecutive failure (default: 2x)
### Anti-thundering herd
When many concurrent requests hit a rate-limited provider, OmniRoute uses mutex + auto rate-limiting to serialize requests and prevent cascading failures. This is automatic for API key providers.
---
## Optional RAG / LLM failure taxonomy (16 problems)
Some OmniRoute users place the gateway in front of RAG or agent stacks. In those setups it is common to see a strange pattern: OmniRoute looks healthy (providers up, routing profiles ok, no rate limit alerts) but the final answer is still wrong.
In practice these incidents usually come from the downstream RAG pipeline, not from the gateway itself.
If you want a shared vocabulary to describe those failures you can use the WFGY ProblemMap, an external MIT license text resource that defines sixteen recurring RAG / LLM failure patterns. At a high level it covers:
- retrieval drift and broken context boundaries
- empty or stale indexes and vector stores
- embedding versus semantic mismatch
- prompt assembly and context window issues
- logic collapse and overconfident answers
- long chain and agent coordination failures
- multi agent memory and role drift
- deployment and bootstrap ordering problems
The idea is simple:
1. When you investigate a bad response, capture:
- user task and request
- route or provider combo in OmniRoute
- any RAG context used downstream (retrieved documents, tool calls, etc)
2. Map the incident to one or two WFGY ProblemMap numbers (`No.1` … `No.16`).
3. Store the number in your own dashboard, runbook, or incident tracker next to the OmniRoute logs.
4. Use the corresponding WFGY page to decide whether you need to change your RAG stack, retriever, or routing strategy.
Full text and concrete recipes live here (MIT license, text only):
[WFGY ProblemMap README](https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md)
You can ignore this section if you do not run RAG or agent pipelines behind OmniRoute.
---
## v3.8.0 Known Issues
Issues specific to the v3.8.0 release and their current workarounds. If a fix lands in a later patch, the entry will be updated or removed.
### Windsurf OAuth flow fails with 401
**Symptoms:**
- "401 unauthorized" while completing the Windsurf OAuth flow from the dashboard
- Windsurf provider card stays in "needs reconnection" state after the callback
**Causes:**
- `WINDSURF_FIREBASE_API_KEY` env var missing or empty
- `WINDSURF_API_KEY` misconfigured or pointing at a stale token
- Local firewall/proxy blocking the OAuth callback
**Fix:**
1. Verify both `WINDSURF_FIREBASE_API_KEY` and `WINDSURF_API_KEY` are set in `.env`
2. Restart OmniRoute so the new env values are picked up
3. Re-run the OAuth flow from **Dashboard → Providers → Windsurf → Reconnect**
### Devin CLI auth failures
**Symptoms:**
- "Devin CLI not found" or "auth failed" when invoking Devin-backed tools
- CLI runtime check reports `installed=false`
**Causes:**
- `CLI_DEVIN_BIN` points to a path that does not exist
- Devin CLI is not installed on the host
**Fix:**
1. Install the Devin CLI for your platform
2. Set `CLI_DEVIN_BIN=/usr/local/bin/devin` (or the real path) in `.env`
3. Restart OmniRoute and re-test from **Dashboard → CLI Tools**
### Model cooldown stuck (manual reset)
**Symptoms:**
- A model stays listed in cooldown even after the expiration time has passed
- Requests still skip the model in combo routing despite the timestamp being in the past
**Manual reset:**
- **Dashboard:** **Settings → Model Cooldowns** → click **Re-enable** on the affected card
- **API:** `DELETE /api/resilience/model-cooldowns` with management auth headers
### Command Code provider connection fails with 403
**Symptoms:**
- 403 when testing the Command Code provider connection
- The provider card shows "unauthorized" after a fresh add
**Cause:** The OAuth flow did not complete (callback not received or token not persisted).
**Fix:**
- Run `omniroute providers` from the CLI to re-trigger the OAuth flow, or
- Re-run OAuth from **Dashboard → Providers → Command Code → Reconnect**
### ModelScope returns aggressive 429 cooldowns
**Symptoms:**
- Very short or immediate cooldowns on ModelScope after a small burst of requests
- Combo routing skips ModelScope earlier than expected
**Cause:** ModelScope emits provider-specific `Retry-After` headers. v3.8.0 ships dedicated handling for those headers, so older versions misread them as generic rate-limit hints.
**Fix:**
- Ensure you are on v3.8.0 or later
- Verify the `useUpstream429BreakerHints` toggle is enabled under **Settings → Resilience**
### OMNIROUTE_WS_BRIDGE_SECRET missing in production
**Symptoms:**
- 401 on every Codex/Responses WebSocket bridge request when running on a remote production host
- WebSocket bridge handshake closes immediately after connect
**Cause:** The `OMNIROUTE_WS_BRIDGE_SECRET` env var is missing from the production environment.
**Fix:**
1. Generate a random secret: `openssl rand -hex 32`
2. Set `OMNIROUTE_WS_BRIDGE_SECRET=<random-secret>` in the production server env (and any client that talks to the bridge)
3. Restart OmniRoute
### Responses API: background mode degraded to synchronous
**Symptoms:**
- Warning logged: `background mode degraded to synchronous`
- A `background: true` request returns a normal synchronous response instead of a background job handle
**Cause:** v3.8.0 intentionally degrades `background: true` on the Responses API to synchronous execution while emitting a warning. Full async background execution is a future deliverable.
**Fix:**
- Adjust the client to call without `background`, or
- Wait for a later release that ships full async background mode (track the changelog)
---
## Still Stuck?
- **GitHub Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
- **Architecture**: See [`docs/architecture/ARCHITECTURE.md`](../architecture/ARCHITECTURE.md) for internal details
- **API Reference**: See [`docs/reference/API_REFERENCE.md`](../reference/API_REFERENCE.md) for all endpoints
- **Health Dashboard**: Check **Dashboard → Health** for real-time system status
- **Translator**: Use **Dashboard → Translator** to debug format issues
docs/guides/UNINSTALL.md
+161
View File
@@ -0,0 +1,161 @@
---
title: "OmniRoute — Uninstall Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute — Uninstall Guide
🌐 **Languages:** 🇺🇸 [English](./UNINSTALL.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/UNINSTALL.md) | 🇪🇸 [Español](../i18n/es/docs/guides/UNINSTALL.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/UNINSTALL.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/UNINSTALL.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/UNINSTALL.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/UNINSTALL.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/UNINSTALL.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/guides/UNINSTALL.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/UNINSTALL.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/UNINSTALL.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/UNINSTALL.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/UNINSTALL.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/UNINSTALL.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/UNINSTALL.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/UNINSTALL.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/UNINSTALL.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/UNINSTALL.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/UNINSTALL.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/UNINSTALL.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/UNINSTALL.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/UNINSTALL.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/UNINSTALL.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/UNINSTALL.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/UNINSTALL.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/UNINSTALL.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/UNINSTALL.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/UNINSTALL.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/UNINSTALL.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/UNINSTALL.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/UNINSTALL.md)
This guide covers how to cleanly remove OmniRoute from your system.
---
## Quick Uninstall (v3.6.2+)
OmniRoute provides two built-in scripts for clean removal:
### Keep Your Data
```bash
npm run uninstall
```
This removes the OmniRoute application but **preserves** your database, configurations, API keys, and provider settings in `~/.omniroute/`. Use this if you plan to reinstall later and want to keep your setup.
### Full Removal
```bash
npm run uninstall:full
```
This removes the application **and permanently erases** all data:
- Database (`storage.sqlite`)
- Provider configurations and API keys
- Backup files
- Log files
- All files in the `~/.omniroute/` directory
> ⚠️ **Warning:** `npm run uninstall:full` is irreversible. All your provider connections, combos, API keys, and usage history will be permanently deleted.
---
## Manual Uninstall
### NPM Global Install
```bash
# Remove the global package
npm uninstall -g omniroute
# (Optional) Remove data directory
rm -rf ~/.omniroute
```
### pnpm Global Install
```bash
pnpm uninstall -g omniroute
rm -rf ~/.omniroute
```
### Docker
```bash
# Stop and remove the container
docker stop omniroute
docker rm omniroute
# Remove the volume (deletes all data)
docker volume rm omniroute-data
# (Optional) Remove the image
docker rmi diegosouzapw/omniroute:latest
```
### Docker Compose
```bash
# Stop and remove containers
docker compose down
# Also remove volumes (deletes all data)
docker compose down -v
```
### Electron Desktop App
**Windows:**
- Open `Settings → Apps → OmniRoute → Uninstall`
- Or run the NSIS uninstaller from the install directory
**macOS:**
- Drag `OmniRoute.app` from `/Applications` to Trash
- Remove data: `rm -rf ~/Library/Application Support/omniroute`
**Linux:**
- Remove the AppImage file
- Remove data: `rm -rf ~/.omniroute`
### Source Install (git clone)
```bash
# Remove the cloned directory
rm -rf /path/to/omniroute
# (Optional) Remove data directory
rm -rf ~/.omniroute
```
---
## Data Directories
OmniRoute stores data in the following locations by default:
| Platform | Default Path | Override |
| ------------- | ----------------------------- | ------------------------- |
| Linux | `~/.omniroute/` | `DATA_DIR` env var |
| macOS | `~/.omniroute/` | `DATA_DIR` env var |
| Windows | `%APPDATA%/omniroute/` | `DATA_DIR` env var |
| Docker | `/app/data/` (mounted volume) | `DATA_DIR` env var |
| XDG-compliant | `$XDG_CONFIG_HOME/omniroute/` | `XDG_CONFIG_HOME` env var |
### Files in the data directory
| File/Directory | Description |
| -------------------- | ------------------------------------------------- |
| `storage.sqlite` | Main database (providers, combos, settings, keys) |
| `storage.sqlite-wal` | SQLite write-ahead log (temporary) |
| `storage.sqlite-shm` | SQLite shared memory (temporary) |
| `call_logs/` | Request payload archives |
| `backups/` | Automatic database backups |
| `log.txt` | Legacy request log (optional) |
---
## Verify Complete Removal
After uninstalling, verify there are no remaining files:
```bash
# Check for global npm package
npm list -g omniroute 2>/dev/null
# Check for data directory
ls -la ~/.omniroute/ 2>/dev/null
# Check for running processes
pgrep -f omniroute
```
If any process is still running, stop it:
```bash
pkill -f omniroute
```
docs/guides/USAGE_QUOTA_GUIDE.md
+433
View File
@@ -0,0 +1,433 @@
---
title: "Usage, Quota & Spend Tracking"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Usage, Quota & Spend Tracking
> **TL;DR**: OmniRoute tracks every request's token usage, computes cost, enforces per-API-key quota, and surfaces analytics in the dashboard. This guide explains how it all works.
**Sources:**
- `open-sse/services/usage.ts` (~70KB) — main usage tracking
- `src/lib/usageAnalytics.ts` (~10KB) — aggregation for dashboard
- `src/lib/db/quotaSnapshots.ts` — historical quota data
- `src/lib/db/usage*.ts` — multiple usage-related DB modules
---
## Overview
Every request that flows through OmniRoute generates a **usage record** that captures:
- **Identity**: which API key, provider, model, combo
- **Tokens**: prompt tokens, completion tokens, cached tokens, total
- **Cost**: USD amount (computed from pricing data)
- **Timing**: latency, start/end timestamps
- **Status**: success, error, rate-limited, etc.
These records are aggregated into **analytics**, persisted as **quota snapshots**, and used to enforce **per-key budget limits**.
```
Request ──▶ chatCore ──▶ usage.record() ──▶ SQLite
│
┌───────┼───────┐
▼ ▼ ▼
analytics quota billing
(dashboard) (enforce) (export)
```
---
## What Gets Recorded
The `usage.ts` service captures a **usage event** for every request:
| Field | Type | Source |
| ------------------ | ------- | ---------------------------------------------------------- |
| `id` | string | UUID generated on record |
| `apiKeyId` | string | The API key that initiated the request |
| `provider` | string | Provider ID (openai, anthropic, etc.) |
| `model` | string | Model ID (gpt-5, claude-opus-4-6, etc.) |
| `comboId` | string? | Combo ID if routed through a combo |
| `promptTokens` | number | From upstream response |
| `completionTokens` | number | From upstream response |
| `cachedTokens` | number | Cache hit tokens (Anthropic prompt caching, etc.) |
| `totalTokens` | number | prompt + completion |
| `costUsd` | number | Computed from pricing data |
| `latencyMs` | number | End-to-end request duration |
| `status` | enum | `success`, `error`, `rate_limited`, `timeout`, `cancelled` |
| `errorClass` | string? | Error class if status != success |
| `timestamp` | string | ISO 8601 UTC |
| `metadata` | object | Custom plugin-injected data |
### Where Tokens Come From
Tokens are extracted from the upstream provider's response in the **response handler**:
```ts
// From open-sse/handlers/chatCore.ts
const response = await providerExecutor.execute(provider, request);
const usage = response.usage || {
prompt_tokens: 0,
completion_tokens: 0,
cached_tokens: 0,
};
```
For providers that don't return usage (some web-cookie providers), OmniRoute **estimates** tokens using a `~4 chars per token` heuristic (see `open-sse/services/autoCombo/pipelineRouter.ts`).
### Cached Tokens
OmniRoute tracks `cached_tokens` separately from `prompt_tokens` because:
- Anthropic prompt caching charges a reduced rate for cached tokens (10% of normal)
- Some providers return `cache_read_input_tokens` that should be priced differently
- Analytics can show the **cache hit rate** = `cached_tokens / prompt_tokens`
---
## Cost Calculation
Costs are computed from **pricing data** synced from LiteLLM (`src/lib/pricingSync.ts`):
| Model | Input $/1M | Output $/1M | Cached $/1M |
| ----------------- | ---------- | ----------- | ----------- |
| gpt-5 | $2.50 | $10.00 | — |
| claude-opus-4-6 | $15.00 | $75.00 | $1.50 |
| claude-sonnet-4-5 | $3.00 | $15.00 | $0.30 |
| gemini-2.5-pro | $1.25 | $10.00 | — |
The cost formula (`src/lib/usage/costCalculator.ts`):
```ts
cost =
(prompt_tokens - cached_tokens) * input_price +
cached_tokens * cached_price +
completion_tokens * output_price;
```
> **Why subtract cached from prompt?** The cached portion is priced separately; charging input price on the whole prompt would over-count.
### Pricing Sync
Pricing data is auto-synced from LiteLLM via the `/api/pricing/sync` endpoint (triggered by the built-in cron task, not a user-facing env var):
```bash
# Manual trigger
curl -X POST http://localhost:20128/api/pricing/sync
```
For models with no pricing data, OmniRoute falls back to **estimating cost** using internal average rates (sourced from LiteLLM's pricing data).
---
## Date Range Aggregation
The `usageAnalytics.ts` module computes dashboard widgets from raw usage data. It supports 7 time ranges:
| Range | Window | Use case |
| -------- | --------------------------- | --------------------------- |
| `1d` | Last 24 hours | Hourly cost spike detection |
| `7d` | Last 7 days | Weekly review |
| `30d` | Last 30 days | Monthly billing |
| `90d` | Last 90 days | Quarterly analysis |
| `ytd` | Since Jan 1 of current year | Annual budget tracking |
| `all` | All time | Lifetime stats |
| `custom` | User-defined start/end | Audits, ad-hoc queries |
### Dashboard Widgets Computed
For any date range, the analytics layer computes:
| Widget | Description |
| ---------------------- | ------------------------------------------------------ |
| **Summary cards** | Total requests, total cost, total tokens, success rate |
| **Daily trend chart** | Cost + tokens per day, stacked by model |
| **Activity heatmap** | Hour-of-day × day-of-week grid, color = request count |
| **Model breakdown** | Pie chart of cost by model |
| **Provider breakdown** | Bar chart of requests by provider |
| **Top API keys** | Table of top 10 keys by cost |
| **Error analysis** | Error rate over time, top error classes |
### Programmatic Access
````ts
import { computeAnalytics } from "@/lib/usageAnalytics";
const analytics = await computeAnalytics(
history, // usage history records
"7d", // time range: "1d" | "7d" | "30d" | "90d" | "ytd" | "all" | "custom"
connectionMap, // provider connection map (connectionId → account name)
{
startDate: "2025-01-01", // optional: for "custom" range
endDate: "2025-06-01", // optional: for "custom" range
}
);
console.log(analytics.summary.totalCost); // 12.34 (cents)
console.log(analytics.byModel[0]); // { model, cost, requests, promptTokens, completionTokens }
---
## Quota Enforcement
Per-API-key quota is enforced in two places:
1. **Soft limit** (`quotaWarnAt`): dashboard warning when usage exceeds threshold
2. **Hard limit** (`quotaLimit`): request rejected with HTTP 429 when exceeded
### Configuration
```ts
// Per API key
await updateApiKey(keyId, {
quotaWarnAt: 5_00, // $5.00 — show warning
quotaLimit: 10_00, // $10.00 — hard stop
quotaWindow: "month", // "day" | "week" | "month" | "all"
});
````
### Enforcement Flow
```
Request ──▶ quotaCheck()
│
├── Within limit? ──▶ allow
│
└── Over limit? ──▶ 429 Too Many Requests
with Retry-After header
```
### Quota Snapshots
`quotaSnapshots` table stores **historical quota state** for trend analysis:
| Field | Description |
| ----------- | -------------------------------- | ------ | ------- |
| `apiKeyId` | The key being tracked |
| `window` | "day" | "week" | "month" |
| `used` | Cost used in this window (cents) |
| `limit` | The limit (cents) |
| `resetAt` | When the window resets |
| `createdAt` | When the snapshot was taken |
Snapshots are taken **on every request** that uses > 0 cost, and used to:
- Render the quota progress bar in the dashboard
- Show 30-day quota trend charts
- Trigger alerts when usage approaches the limit
---
## REST API
### List Usage Records
```bash
GET /api/usage?range=7d&limit=100
GET /api/usage?apiKeyId=key-123&range=30d
GET /api/usage?provider=openai&range=1d
```
Response:
```json
{
"records": [
{
"id": "uuid",
"apiKeyId": "key-123",
"provider": "openai",
"model": "gpt-5",
"promptTokens": 1234,
"completionTokens": 567,
"totalTokens": 1801,
"costUsd": 0.005,
"latencyMs": 1234,
"status": "success",
"timestamp": "2026-06-08T12:00:00Z"
}
],
"total": 1234,
"nextCursor": "..."
}
```
### Get Analytics Summary
```bash
GET /api/usage/analytics?range=7d&groupBy=model
```
Response:
```json
{
"summary": {
"totalCost": 12.34,
"totalRequests": 5678,
"totalTokens": 12345678,
"successRate": 0.987,
"avgLatencyMs": 1234
},
"models": [
{ "model": "gpt-5", "cost": 8.5, "requests": 1234, "tokens": 4567890 },
{ "model": "claude-opus-4-6", "cost": 3.84, "requests": 234, "tokens": 234567 }
],
"daily": [
{ "date": "2026-06-01", "cost": 1.5, "requests": 800 },
{ "date": "2026-06-02", "cost": 2.0, "requests": 1000 }
]
}
```
### Query Usage Analytics
Usage data is accessed via the dashboard or MCP tools, not direct REST export endpoints. Available analytics:
- **`/api/usage/analytics`** — aggregated usage metrics (group by model, provider, key)
- **`/api/usage/quota`** — current quota status per API key
- **`/api/usage/history`** — request history logs
---
## MCP Tools
Two MCP tools expose usage data to agents (see `open-sse/mcp-server/tools/`):
| Tool | Description |
| ----------------------- | -------------------------------------------------- |
| `omniroute_cost_report` | Generates a per-key cost report for a given period |
| `omniroute_check_quota` | Returns current quota status for an API key |
Example agent invocation:
```json
{
"tool": "omniroute_cost_report",
"args": { "period": "week" }
}
```
---
## Retention and Cleanup
Usage data grows ~1-10KB per request. At scale, this can be significant.
### Retention Settings
Usage history retention is configured via the Database Settings in the UI or via `/api/settings/database`.
By default, usage history is retained for **90 days**.
### Cleanup
Old records are cleaned up by `src/lib/db/cleanup.ts`:
- Triggered by the background cron process
- Deletes records from `usage_history` older than the configured `usageHistory` retention setting
### Storage Estimation
| Request rate | 30-day storage | 90-day storage |
| --------------- | -------------- | -------------- |
| 100 req/day | ~3MB | ~9MB |
| 1,000 req/day | ~30MB | ~90MB |
| 10,000 req/day | ~300MB | ~900MB |
| 100,000 req/day | ~3GB | ~9GB |
For very high traffic, consider:
- Reducing the retention period via Database Settings
- Using `aggregated_metrics` instead of raw records (only for analytics)
---
## Cost Optimization Tips
### 1. Use the Right Model
```bash
# Quick answer — use cheap + fast
curl -d '{"model":"auto/fast","messages":[...]}'
# Complex task — use quality
curl -d '{"model":"auto/smart","messages":[...]}'
```
### 2. Enable Caching
Anthropic prompt caching saves **90% on repeated context**:
```ts
// The caching is automatic — just include the same large system prompt
const response = await openai.chat({
model: "claude-sonnet-4-5",
system: longSystemPrompt, // Will be cached automatically
messages: [{ role: "user", content: "..." }],
});
```
### 3. Use Compression
RTK + Caveman compression saves **15-95% on tool-heavy sessions**:
```ts
const config = {
compression: {
engine: "rtk",
intensity: "aggressive",
},
};
```
### 4. Set Per-Key Quotas
Always set `quotaLimit` to prevent runaway costs:
```ts
await updateApiKey(keyId, { quotaLimit: 10_00 }); // $10/month cap
```
### 5. Audit Top Consumers
Use the dashboard or **`/api/usage/analytics`** to group by API key and sort by cost:
```bash
GET /api/usage/analytics?groupBy=apiKey
```
---
## Troubleshooting
### "Cost is higher than expected"
1. Check **`/api/usage/analytics?groupBy=model`** — find the expensive model
2. Check **`/api/usage/analytics?groupBy=apiKey`** — find the heavy consumer
3. Verify pricing data is up to date: `POST /api/pricing/sync`
### "Records missing"
- Check DB retention settings under Dashboard → Database → Cleanup — old records are deleted by the periodic cleanup task (`src/lib/db/cleanup.ts`)
- Check for errors in `src/lib/db/usage*.ts` — DB write failures are logged but not surfaced
- Verify the request actually reached `chatCore` — check combo routing
### "Quota not enforcing"
- Check the key's `quotaLimit` setting
- Verify `quotaWindow` is set correctly
- Look for `quotaSnapshots` records — they should be created on every request
---
## See Also
- [DATABASE_GUIDE.md](../ops/DATABASE_GUIDE.md) — Schema for usage tables
- [ENVIRONMENT.md](../reference/ENVIRONMENT.md#18-pricing-sync) — pricing sync env vars
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — How `auto/fast`, `auto/cheap` reduce cost
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — Full `/api/usage/*` reference
- Source: `open-sse/services/usage.ts`, `src/lib/usageAnalytics.ts`, `src/lib/db/usage*.ts`
docs/guides/USER_GUIDE.md
+1232
View File
File diff suppressed because it is too large Load Diff
docs/guides/meta.json
+21
View File
@@ -0,0 +1,21 @@
{
"title": "Guides",
"description": "Technical guides for operators and developers",
"pages": [
"SETUP_GUIDE",
"USER_GUIDE",
"DOCKER_GUIDE",
"ELECTRON_GUIDE",
"FEATURES",
"FREE_PROVIDER_RANKINGS",
"COST_TRACKING",
"I18N",
"KIRO_SETUP",
"CODEX-CLI-CONFIGURATION",
"PWA_GUIDE",
"TERMUX_GUIDE",
"TIERS",
"USAGE_QUOTA_GUIDE",
"UNINSTALL"
]
}
docs/i18n/README.md
+47
View File
@@ -0,0 +1,47 @@
# 🌐 Multilingual Documentation — OmniRoute
Translations of documentation into 40 languages. Code blocks remain in English.
---
- 🇸🇦 **العربية** (`ar`): [Docs Root](./ar/README.md)
- 🇦🇿 **Azərbaycan dili** (`az`): [Docs Root](./az/README.md)
- 🇧🇬 **Български** (`bg`): [Docs Root](./bg/README.md)
- 🇧🇩 **বাংলা** (`bn`): [Docs Root](./bn/README.md)
- 🇨🇿 **Čeština** (`cs`): [Docs Root](./cs/README.md)
- 🇩🇰 **Dansk** (`da`): [Docs Root](./da/README.md)
- 🇩🇪 **Deutsch** (`de`): [Docs Root](./de/README.md)
- 🇪🇸 **Español** (`es`): [Docs Root](./es/README.md)
- 🇮🇷 **فارسی** (`fa`): [Docs Root](./fa/README.md)
- 🇫🇮 **Suomi** (`fi`): [Docs Root](./fi/README.md)
- 🇫🇷 **Français** (`fr`): [Docs Root](./fr/README.md)
- 🇮🇳 **ગુજરાતી** (`gu`): [Docs Root](./gu/README.md)
- 🇮🇱 **עברית** (`he`): [Docs Root](./he/README.md)
- 🇮🇳 **हिन्दी** (`hi`): [Docs Root](./hi/README.md)
- 🇭🇺 **Magyar** (`hu`): [Docs Root](./hu/README.md)
- 🇮🇩 **Bahasa Indonesia** (`id`): [Docs Root](./id/README.md)
- 🇮🇹 **Italiano** (`it`): [Docs Root](./it/README.md)
- 🇯🇵 **日本語** (`ja`): [Docs Root](./ja/README.md)
- 🇰🇷 **한국어** (`ko`): [Docs Root](./ko/README.md)
- 🇮🇳 **मराठी** (`mr`): [Docs Root](./mr/README.md)
- 🇲🇾 **Bahasa Melayu** (`ms`): [Docs Root](./ms/README.md)
- 🇳🇱 **Nederlands** (`nl`): [Docs Root](./nl/README.md)
- 🇳🇴 **Norsk** (`no`): [Docs Root](./no/README.md)
- 🇵🇭 **Filipino** (`phi`): [Docs Root](./phi/README.md)
- 🇵🇱 **Polski** (`pl`): [Docs Root](./pl/README.md)
- 🇵🇹 **Português (Portugal)** (`pt`): [Docs Root](./pt/README.md)
- 🇧🇷 **Português (Brasil)** (`pt-BR`): [Docs Root](./pt-BR/README.md)
- 🇷🇴 **Română** (`ro`): [Docs Root](./ro/README.md)
- 🇷🇺 **Русский** (`ru`): [Docs Root](./ru/README.md)
- 🇸🇰 **Slovenčina** (`sk`): [Docs Root](./sk/README.md)
- 🇸🇪 **Svenska** (`sv`): [Docs Root](./sv/README.md)
- 🇰🇪 **Kiswahili** (`sw`): [Docs Root](./sw/README.md)
- 🇮🇳 **தமிழ்** (`ta`): [Docs Root](./ta/README.md)
- 🇮🇳 **తెలుగు** (`te`): [Docs Root](./te/README.md)
- 🇹🇭 **ไทย** (`th`): [Docs Root](./th/README.md)
- 🇹🇷 **Türkçe** (`tr`): [Docs Root](./tr/README.md)
- 🇺🇦 **Українська** (`uk-UA`): [Docs Root](./uk-UA/README.md)
- 🇵🇰 **اردو** (`ur`): [Docs Root](./ur/README.md)
- 🇻🇳 **Tiếng Việt** (`vi`): [Docs Root](./vi/README.md)
- 🇹🇼 **中文 (繁體)** (`zh-TW`): [Docs Root](./zh-TW/README.md)
- 🇨🇳 **中文 (简体)** (`zh-CN`): [Docs Root](./zh-CN/README.md)
docs/i18n/ar/CHANGELOG.md
+8853
View File
File diff suppressed because it is too large Load Diff
docs/i18n/ar/CLAUDE.md
+395
View File
@@ -0,0 +1,395 @@
# CLAUDE.md (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../CLAUDE.md) · 🇦🇿 [az](../az/CLAUDE.md) · 🇧🇬 [bg](../bg/CLAUDE.md) · 🇧🇩 [bn](../bn/CLAUDE.md) · 🇨🇿 [cs](../cs/CLAUDE.md) · 🇩🇰 [da](../da/CLAUDE.md) · 🇩🇪 [de](../de/CLAUDE.md) · 🇪🇸 [es](../es/CLAUDE.md) · 🇮🇷 [fa](../fa/CLAUDE.md) · 🇫🇮 [fi](../fi/CLAUDE.md) · 🇫🇷 [fr](../fr/CLAUDE.md) · 🇮🇳 [gu](../gu/CLAUDE.md) · 🇮🇱 [he](../he/CLAUDE.md) · 🇮🇳 [hi](../hi/CLAUDE.md) · 🇭🇺 [hu](../hu/CLAUDE.md) · 🇮🇩 [id](../id/CLAUDE.md) · 🇮🇩 [in](../in/CLAUDE.md) · 🇮🇹 [it](../it/CLAUDE.md) · 🇯🇵 [ja](../ja/CLAUDE.md) · 🇰🇷 [ko](../ko/CLAUDE.md) · 🇮🇳 [mr](../mr/CLAUDE.md) · 🇲🇾 [ms](../ms/CLAUDE.md) · 🇳🇱 [nl](../nl/CLAUDE.md) · 🇳🇴 [no](../no/CLAUDE.md) · 🇵🇭 [phi](../phi/CLAUDE.md) · 🇵🇱 [pl](../pl/CLAUDE.md) · 🇵🇹 [pt](../pt/CLAUDE.md) · 🇧🇷 [pt-BR](../pt-BR/CLAUDE.md) · 🇷🇴 [ro](../ro/CLAUDE.md) · 🇷🇺 [ru](../ru/CLAUDE.md) · 🇸🇰 [sk](../sk/CLAUDE.md) · 🇸🇪 [sv](../sv/CLAUDE.md) · 🇰🇪 [sw](../sw/CLAUDE.md) · 🇮🇳 [ta](../ta/CLAUDE.md) · 🇮🇳 [te](../te/CLAUDE.md) · 🇹🇭 [th](../th/CLAUDE.md) · 🇹🇷 [tr](../tr/CLAUDE.md) · 🇺🇦 [uk-UA](../uk-UA/CLAUDE.md) · 🇵🇰 [ur](../ur/CLAUDE.md) · 🇻🇳 [vi](../vi/CLAUDE.md) · 🇨🇳 [zh-CN](../zh-CN/CLAUDE.md)
---
هذا الملف يوفر إرشادات لـ Claude Code (claude.ai/code) عند العمل مع الكود في هذا المستودع.
## البداية السريعة
```bash
npm install # تثبيت التبعيات (توليد .env تلقائيًا من .env.example)
npm run dev # خادم التطوير على http://localhost:20128
npm run build # بناء الإنتاج (Next.js 16 مستقل)
npm run lint # ESLint (0 أخطاء متوقعة؛ التحذيرات موجودة مسبقًا)
npm run typecheck:core # فحص TypeScript (يجب أن يكون نظيفًا)
npm run typecheck:noimplicit:core # فحص صارم (لا أي ضمني)
npm run test:coverage # اختبارات الوحدة + بوابة التغطية (75/75/75/70 — العبارات/الأسطر/الدوال/الفروع)
npm run check # lint + اختبار مجتمعة
npm run check:cycles # اكتشاف الاعتماديات الدائرية
```
### تشغيل الاختبارات
```bash
# ملف اختبار فردي (جهاز اختبار Node.js الأصلي — معظم الاختبارات)
node --import tsx/esm --test tests/unit/your-file.test.ts
# Vitest (خادم MCP، autoCombo، ذاكرة التخزين المؤقت)
npm run test:vitest
# جميع المجموعات
npm run test:all
```
للحصول على مصفوفة الاختبار الكاملة، راجع `CONTRIBUTING.md` → "تشغيل الاختبارات". للحصول على بنية عميقة، راجع `AGENTS.md`.
---
## نظرة عامة على المشروع
**OmniRoute** — وكيل/موجه AI موحد. نقطة نهاية واحدة، أكثر من 160 مزود LLM، تراجع تلقائي.
| الطبقة | الموقع | الغرض |
| -------------- | ----------------------- | ---------------------------------------------------------------- |
| مسارات API | `src/app/api/v1/` | موجه تطبيق Next.js — نقاط الدخول |
| المعالجات | `open-sse/handlers/` | معالجة الطلبات (الدردشة، التضمينات، إلخ) |
| المنفذون | `open-sse/executors/` | إرسال HTTP محدد لمزود الخدمة |
| المترجمون | `open-sse/translator/` | تحويل التنسيق (OpenAI↔Claude↔Gemini) |
| المحول | `open-sse/transformer/` | واجهات برمجة التطبيقات للردود ↔ إكمالات الدردشة |
| الخدمات | `open-sse/services/` | توجيه مجموعة، حدود المعدل، التخزين المؤقت، إلخ |
| قاعدة البيانات | `src/lib/db/` | وحدات مجال SQLite (أكثر من 45 ملف، 55 ترحيل) |
| المجال/السياسة | `src/domain/` | محرك السياسة، قواعد التكلفة، منطق التراجع |
| خادم MCP | `open-sse/mcp-server/` | 37 أداة (30 قاعدة + 3 ذاكرة + 4 مهارات)، 3 وسائل نقل، ~13 نطاقات |
| خادم A2A | `src/lib/a2a/` | بروتوكول وكيل JSON-RPC 2.0 |
| المهارات | `src/lib/skills/` | إطار عمل مهارات قابل للتوسيع |
| الذاكرة | `src/lib/memory/` | ذاكرة محادثة دائمة |
Monorepo: `src/` (تطبيق Next.js 16)، `open-sse/` (مساحة عمل محرك البث)، `electron/` (تطبيق سطح المكتب)، `tests/`، `bin/` (نقطة دخول CLI).
---
## خط أنابيب الطلب
```
العميل → /v1/chat/completions (مسار Next.js)
→ CORS → تحقق Zod → مصادقة؟ → فحص السياسة → حماية حقن المطالبات
→ handleChatCore() [open-sse/handlers/chatCore.ts]
→ تحقق من التخزين المؤقت → حد معدل الطلبات → توجيه مجموعة؟
→ resolveComboTargets() → handleSingleModel() لكل هدف
→ translateRequest() → getExecutor() → executor.execute()
→ fetch() upstream → إعادة المحاولة مع التراجع
→ ترجمة الاستجابة → تدفق SSE أو JSON
→ إذا كانت واجهة برمجة التطبيقات Responses: responsesTransformer.ts TransformStream
```
تتبع مسارات واجهة برمجة التطبيقات نمطًا متسقًا: `Route → CORS preflight → Zod body validation → مصادقة اختيارية (extractApiKey/isValidApiKey) → تنفيذ سياسة مفتاح واجهة برمجة التطبيقات → تفويض المعالج (open-sse)`. لا توجد وسائط عالمية لـ Next.js — الاعتراض خاص بالمسار.
**توجيه المجموعة** (`open-sse/services/combo.ts`): 14 استراتيجية (الأولوية، الوزن، ملء أولاً، التناوب، P2C، عشوائي، الأقل استخدامًا، الأمثل من حيث التكلفة، الواعي بإعادة التعيين، عشوائي صارم، تلقائي، lkgp، الأمثل من حيث السياق، نقل السياق). كل هدف يستدعي `handleSingleModel()` الذي يلف `handleChatCore()` مع معالجة الأخطاء الخاصة بكل هدف وفحوصات قاطع الدائرة. راجع `docs/routing/AUTO-COMBO.md` لتسجيل 9 عوامل Auto-Combo و `docs/architecture/RESILIENCE_GUIDE.md` للطبقات الثلاث من المرونة.
---
## حالة وقت التشغيل للمرونة
يمتلك OmniRoute ثلاث آليات فشل مؤقتة مرتبطة ولكن متميزة. حافظ على نطاقها منفصلًا عند تصحيح سلوك التوجيه. راجع
[مخطط المرونة ذو 3 طبقات](./docs/diagrams/exported/resilience-3layers.svg)
(المصدر: [docs/diagrams/resilience-3layers.mmd](./docs/diagrams/resilience-3layers.mmd))
لخريطة سريعة.
### قاطع دائرة المزود
**النطاق**: المزود بالكامل، مثل `glm`، `openai`، `anthropic`.
**الغرض**: إيقاف إرسال الحركة إلى مزود يفشل بشكل متكرر على مستوى الخدمة/التيار، حتى لا يؤدي مزود غير صحي إلى إبطاء كل طلب.
**التنفيذ**:
- الفئة الأساسية: `src/shared/utils/circuitBreaker.ts`
- توصيل بوابة الدردشة/التنفيذ: `src/sse/handlers/chatHelpers.ts`، `src/sse/handlers/chat.ts`
- واجهة برمجة التطبيقات لحالة وقت التشغيل: `src/app/api/monitoring/health/route.ts`
- الأغطية المشتركة: `open-sse/services/accountFallback.ts`
- جدول الحالة المستمرة: `domain_circuit_breakers`
**الحالات**:
- `CLOSED`: يُسمح بحركة المرور العادية.
- `OPEN`: تم حظر المزود مؤقتًا؛ يحصل المتصلون على استجابة قاطع دائرة المزود مفتوح
أو يتخطى توجيه المجموعة إلى هدف آخر.
- `HALF_OPEN`: انتهت مهلة إعادة التعيين؛ يسمح بطلب اختبار. النجاح يغلق
القاطع، والفشل يفتحه مرة أخرى.
**الإعدادات الافتراضية** (`open-sse/config/constants.ts`):
- مزودو OAuth: العتبة `3`، مهلة إعادة التعيين `60s`.
- مزودو مفتاح واجهة برمجة التطبيقات: العتبة `5`، مهلة إعادة التعيين `30s`.
- المزودون المحليون: العتبة `2`، مهلة إعادة التعيين `15s`.
يجب أن تؤدي حالات الفشل على مستوى المزود فقط إلى تفعيل قاطع المزود:
```ts
(408, 500, 502, 503, 504);
```
لا تقم بتفعيل قاطع المزود بالكامل لأخطاء الحساب/المفتاح/النموذج العادية مثل معظم
حالات `401`، `403`، أو `429`. عادةً ما تنتمي تلك إلى فترة تبريد الاتصال أو قفل النموذج. يجب أن يكون مزود مفتاح واجهة برمجة التطبيقات العام `403` قابلاً للاسترداد ما لم يتم تصنيفه كخطأ نهائي للمزود/الحساب.
يستخدم القاطع استردادًا كسولًا، وليس مؤقتًا في الخلفية. عندما تنتهي حالة `OPEN`، فإن عمليات القراءة مثل `getStatus()`, `canExecute()`, و `getRetryAfterMs()` تقوم بتحديث الحالة إلى `HALF_OPEN`، بحيث لا تستمر لوحات المعلومات وبناة مرشحي المجموعة في استبعاد مزود منتهي الصلاحية إلى الأبد.
### فترة تبريد الاتصال
**النطاق**: اتصال/حساب/مفتاح مزود واحد.
**الغرض**: تخطي مؤقت لمفتاح/حساب سيئ واحد مع السماح للاتصالات الأخرى لنفس المزود بالاستمرار في تقديم الطلبات.
**التنفيذ**:
- مسار الكتابة/التحديث: `src/sse/services/auth.ts::markAccountUnavailable()`
- اختيار/تصفية الحساب: `src/sse/services/auth.ts::getProviderCredentials...`
- حساب فترة التبريد: `open-sse/services/accountFallback.ts::checkFallbackError()`
- الإعدادات: `src/lib/resilience/settings.ts`
الحقول المهمة على اتصالات المزود:
```ts
rateLimitedUntil;
testStatus: "unavailable";
lastError;
lastErrorType;
errorCode;
backoffLevel;
```
أثناء اختيار الحساب، يتم تخطي الاتصال بينما:
```ts
new Date(rateLimitedUntil).getTime() > Date.now();
```
فترات التبريد أيضًا كسولة: عندما تكون `rateLimitedUntil` في الماضي، يصبح الاتصال مؤهلاً مرة أخرى. عند الاستخدام الناجح، تقوم `clearAccountError()` بمسح `testStatus`،
`rateLimitedUntil`، حقول الأخطاء، و `backoffLevel`.
سلوك فترة تبريد الاتصال الافتراضي:
- فترة تبريد أساسية لمزود OAuth: `5s`.
- فترة تبريد أساسية لمزود مفتاح واجهة برمجة التطبيقات: `3s`.
- يجب أن يفضل مفتاح واجهة برمجة التطبيقات `429` تلميحات إعادة المحاولة من المصدر (`Retry-After`، رؤوس إعادة التعيين، أو نص إعادة التعيين القابل للتحليل) عند توفرها.
- تستخدم الفشل القابلة للاسترداد المتكررة التراجع الأسي:
```ts
baseCooldownMs * 2 ** failureIndex;
```
تحمي حراسة مكافحة قطيع الرعد من الفشل المتزامن على نفس الاتصال من تمديد فترة التبريد بشكل متكرر أو زيادة `backoffLevel` مرتين.
الحالات النهائية ليست فترات تبريد. `banned`، `expired`، و `credits_exhausted` مصممة للبقاء غير متاحة حتى تتغير بيانات الاعتماد/الإعدادات أو يقوم مشغل بإعادة تعيينها. لا تقم بكتابة الحالات النهائية فوق حالة فترة التبريد العابرة.
### قفل النموذج
**النطاق**: المزود + الاتصال + النموذج.
**الغرض**: تجنب تعطيل اتصال كامل عندما يكون نموذج واحد فقط غير متاح أو محدود الحصة لذلك الاتصال.
أمثلة:
- مزودو الحصة لكل نموذج الذين يعيدون `429`.
- مزودون محليون يعيدون `404` لنموذج مفقود واحد.
- فشل إذن وضع/نموذج محدد للمزود مثل أوضاع Grok المختارة.
يعيش قفل النموذج في `open-sse/services/accountFallback.ts` ويسمح لنفس الاتصال بالاستمرار في تقديم نماذج أخرى.
### إرشادات التصحيح
- إذا تم تخطي جميع المفاتيح لمزود، تحقق من حالة قاطع المزود وحالة `rateLimitedUntil`/`testStatus` لكل اتصال.
- إذا بدا أن مزودًا ما مستبعدًا بشكل دائم بعد نافذة إعادة التعيين، تحقق مما إذا كان الكود يقرأ `state` الخام بدلاً من استخدام `getStatus()`/`canExecute()`.
- إذا فشل مفتاح مزود واحد ولكن يجب أن تعمل المفاتيح الأخرى، يفضل استخدام فترة تبريد الاتصال على قاطع المزود.
- إذا فشل نموذج واحد فقط، يفضل استخدام قفل النموذج على فترة تبريد الاتصال.
- إذا كان يجب أن يتعافى حالة ما ذاتيًا، يجب أن تحتوي على طابع زمني مستقبلي/مهلة إعادة تعيين ومسار قراءة يقوم بتحديث الحالة المنتهية. تتطلب الحالات الدائمة تغييرات يدوية في بيانات الاعتماد أو التكوين.
## الاتفاقيات الرئيسية
### نمط الكود
- **مسافتان**، فاصلات منقوطة، علامات اقتباس مزدوجة، عرض 100 حرف، فاصلات متأخرة ES5 (تطبق بواسطة lint-staged عبر Prettier)
- **الاستيرادات**: خارجي → داخلي (`@/`, `@omniroute/open-sse`) → نسبي
- **التسمية**: الملفات=camelCase/kebab، المكونات=PascalCase، الثوابت=UPPER_SNAKE
- **ESLint**: `no-eval`، `no-implied-eval`، `no-new-func` = خطأ في كل مكان؛ `no-explicit-any` = تحذير في `open-sse/` و `tests/`
- **TypeScript**: `strict: false`، الهدف ES2022، الوحدة esnext، دقة التجميع. يفضل الأنواع الصريحة.
### قاعدة البيانات
- **دائمًا** مرر عبر وحدات المجال في `src/lib/db/` — **لا** تكتب SQL خام في المسارات أو المعالجات
- **لا** تضف منطقًا إلى `src/lib/localDb.ts` (طبقة إعادة تصدير فقط)
- **لا** تستورد من `localDb.ts` بشكل مجمع — استورد وحدات `db/` المحددة بدلاً من ذلك
- مثيل قاعدة البيانات المفردة: `getDbInstance()` من `src/lib/db/core.ts` (تدوين WAL)
- الترحيلات: `src/lib/db/migrations/` — ملفات SQL ذات إصدار، متكررة، تعمل في معاملات
### معالجة الأخطاء
- try/catch مع أنواع أخطاء محددة، سجل باستخدام سياق pino
- لا تبتلع الأخطاء في تدفقات SSE — استخدم إشارات الإنهاء للتنظيف
- أعد القيم الصحيحة لرموز الحالة HTTP (4xx/5xx)
### الأمان
- **لا** تستخدم `eval()`، `new Function()`، أو eval الضمني
- تحقق من جميع المدخلات باستخدام مخططات Zod
- قم بتشفير بيانات الاعتماد عند الراحة (AES-256-GCM)
- قائمة حظر رأس المصدر: `src/shared/constants/upstreamHeaders.ts` — حافظ على التنظيف، ومخططات Zod، واختبارات الوحدة متوافقة عند التحرير
- **بيانات الاعتماد العامة للمصدر** (Gemini/Antigravity/Windsurf-style OAuth client_id/secret + مفاتيح Firebase Web المستخرجة من CLIs العامة): **يجب** تضمينها عبر `resolvePublicCred()` من `open-sse/utils/publicCreds.ts` — **لا** كأدلة نصية. انظر `docs/security/PUBLIC_CREDS.md` للنمط الإلزامي.
- **استجابات الأخطاء** (HTTP / SSE / المعالج / MCP): **يجب** توجيهها عبر `buildErrorBody()` أو `sanitizeErrorMessage()` من `open-sse/utils/error.ts` — **لا** تضع `err.stack` أو `err.message` الخام في جسم الاستجابة. انظر `docs/security/ERROR_SANITIZATION.md`.
- **أوامر الصدفة المبنية من المتغيرات**: عند استدعاء `exec()`/`spawn()` مع نص يحتاج إلى قيم وقت التشغيل، مررها عبر خيار `env` (تُهرب تلقائيًا) — **لا** تدمج مسارات غير موثوقة/خارجية في جسم النص. المرجع: `src/mitm/cert/install.ts::updateNssDatabases`.
- **المكتبات الآمنة بشكل افتراضي** ([tldrsec/awesome-secure-defaults](https://github.com/tldrsec/awesome-secure-defaults)): يفضل Helmet.js، DOMPurify، ssrf-req-filter، safe-regex، Google Tink على التنفيذات المخصصة كلما تم إضافة أسطح حساسة جديدة للأمان.
---
## سيناريوهات التعديل الشائعة
### إضافة مزود جديد
1. سجل في `src/shared/constants/providers.ts` (تم التحقق منه بواسطة Zod عند التحميل)
2. أضف معالج في `open-sse/executors/` إذا كانت هناك حاجة إلى منطق مخصص (قم بتمديد `BaseExecutor`)
3. أضف مترجمًا في `open-sse/translator/` إذا كان بتنسيق غير OpenAI
4. أضف تكوين OAuth في `src/lib/oauth/constants/oauth.ts` إذا كان يعتمد على OAuth — إذا كان CLI المصدر يرسل client_id/secret عام، قم بتضمينه عبر `resolvePublicCred()` (انظر `docs/security/PUBLIC_CREDS.md`)، **لا** كأدلة نصية
5. سجل النماذج في `open-sse/config/providerRegistry.ts`
6. اكتب اختبارات في `tests/unit/` (تضمن تأكيد شكل publicCreds إذا أضفت افتراضيًا جديدًا مضمنًا)
### إضافة مسار API جديد
1. أنشئ دليلًا تحت `src/app/api/v1/your-route/`
2. أنشئ `route.ts` مع معالجات `GET`/`POST`
3. اتبع النمط: CORS → تحقق من صحة جسم Zod → مصادقة اختيارية → تفويض المعالج
4. يذهب المعالج في `open-sse/handlers/` (استورد من هناك، وليس داخل السطر)
5. تستخدم استجابات الأخطاء `buildErrorBody()` / `errorResponse()` من `open-sse/utils/error.ts` (تم تنظيفها تلقائيًا — لا تضع `err.stack` أو `err.message` الخام في الجسم). انظر `docs/security/ERROR_SANITIZATION.md`.
6. أضف اختبارات — بما في ذلك على الأقل تأكيد واحد بأن استجابات الأخطاء لا تسرب تتبع المكدس (`!body.error.message.includes("at /")`)
### إضافة وحدة DB جديدة
1. أنشئ `src/lib/db/yourModule.ts` — استورد `getDbInstance` من `./core.ts`
2. صدر وظائف CRUD لجدول (جداول) المجال الخاص بك
3. أضف ترحيلًا في `src/lib/db/migrations/` إذا كانت هناك حاجة إلى جداول جديدة
4. أعد تصدير من `src/lib/localDb.ts` (أضف إلى قائمة إعادة التصدير فقط)
5. اكتب اختبارات
### إضافة أداة MCP جديدة
1. أضف تعريف الأداة في `open-sse/mcp-server/tools/` مع مخطط إدخال Zod + معالج غير متزامن
2. سجل في مجموعة الأدوات (موصول بواسطة `createMcpServer()`)
3. عيّن إلى النطاق (النطاقات) المناسبة
4. اكتب اختبارات (تسجيل استدعاء الأداة في جدول `mcp_audit`)
### إضافة مهارة A2A جديدة
1. أنشئ مهارة في `src/lib/a2a/skills/` (يوجد 5 بالفعل: التوجيه الذكي، إدارة الحصص، اكتشاف المزود، تحليل التكلفة، تقرير الصحة)
2. تتلقى المهارة سياق المهمة (الرسائل، البيانات الوصفية) → تعيد نتيجة منظمة
3. سجل في `A2A_SKILL_HANDLERS` في `src/lib/a2a/taskExecution.ts`
4. اعرض في `src/app/.well-known/agent.json/route.ts` (بطاقة الوكيل)
5. اكتب اختبارات في `tests/unit/`
6. وثق في جدول المهارات في `docs/frameworks/A2A-SERVER.md`
### إضافة وكيل سحابي جديد
1. أنشئ فئة وكيل في `src/lib/cloudAgent/agents/` تمتد من `CloudAgentBase` (يوجد 3 بالفعل: codex-cloud، devin، jules)
2. نفذ `createTask`، `getStatus`، `approvePlan`، `sendMessage`، `listSources`
3. سجل في `src/lib/cloudAgent/registry.ts`
4. أضف معالجة OAuth/بيانات الاعتماد إذا لزم الأمر (`src/lib/oauth/providers/`)
5. اختبارات + وثق في `docs/frameworks/CLOUD_AGENT.md`
### إضافة حواجز جديدة / تقييم / مهارة / حدث Webhook
- الحواجز: `src/lib/guardrails/` → الوثائق: `docs/security/GUARDRAILS.md`
- مجموعة التقييم: `src/lib/evals/` → الوثائق: `docs/frameworks/EVALS.md`
- المهارة (الصندوق الرمل): `src/lib/skills/` → الوثائق: `docs/frameworks/SKILLS.md`
- حدث Webhook: `src/lib/webhookDispatcher.ts` → الوثائق: `docs/frameworks/WEBHOOKS.md`
## وثائق المرجع
لأي تغيير غير تافه، اقرأ الغوص العميق المطابق أولاً:
| المجال | الوثيقة |
| ----------------------------------------- | ----------------------------------------------------------------- |
| تنقل المستودع | `docs/architecture/REPOSITORY_MAP.md` |
| الهندسة | `docs/architecture/ARCHITECTURE.md` |
| مرجع الهندسة | `docs/architecture/CODEBASE_DOCUMENTATION.md` |
| Auto-Combo (تقييم 9 عوامل، 14 استراتيجية) | `docs/routing/AUTO-COMBO.md` |
| المرونة (3 آليات) | `docs/architecture/RESILIENCE_GUIDE.md` |
| إعادة تشغيل التفكير | `docs/routing/REASONING_REPLAY.md` |
| إطار المهارات | `docs/frameworks/SKILLS.md` |
| نظام الذاكرة (FTS5 + Qdrant) | `docs/frameworks/MEMORY.md` |
| وكلاء السحابة | `docs/frameworks/CLOUD_AGENT.md` |
| حواجز الأمان (PII / حقن / رؤية) | `docs/security/GUARDRAILS.md` |
| بيانات اعتماد عامة (Gemini/إلخ.) | `docs/security/PUBLIC_CREDS.md` |
| تطهير رسائل الخطأ | `docs/security/ERROR_SANITIZATION.md` |
| التقييمات | `docs/frameworks/EVALS.md` |
| الامتثال / التدقيق | `docs/security/COMPLIANCE.md` |
| Webhooks | `docs/frameworks/WEBHOOKS.md` |
| خط أنابيب التفويض | `docs/architecture/AUTHZ_GUIDE.md` |
| التخفي (TLS / بصمة) | `docs/security/STEALTH_GUIDE.md` |
| بروتوكولات الوكلاء (A2A / ACP / سحابة) | `docs/frameworks/AGENT_PROTOCOLS_GUIDE.md` |
| خادم MCP | `docs/frameworks/MCP-SERVER.md` |
| خادم A2A | `docs/frameworks/A2A-SERVER.md` |
| مرجع API + OpenAPI | `docs/reference/API_REFERENCE.md` + `docs/reference/openapi.yaml` |
| كتالوج المزودين (تم إنشاؤه تلقائيًا) | `docs/reference/PROVIDER_REFERENCE.md` |
| تدفق الإصدار | `docs/ops/RELEASE_CHECKLIST.md` |
## الاختبار
| ما | الأمر |
| ----------------------- | --------------------------------------------------------------------- |
| اختبارات الوحدة | `npm run test:unit` |
| ملف واحد | `node --import tsx/esm --test tests/unit/file.test.ts` |
| Vitest (MCP, autoCombo) | `npm run test:vitest` |
| E2E (Playwright) | `npm run test:e2e` |
| بروتوكول E2E (MCP+A2A) | `npm run test:protocols:e2e` |
| النظام البيئي | `npm run test:ecosystem` |
| بوابة التغطية | `npm run test:coverage` (75/75/75/70 — العبارات/الأسطر/الدوال/الفروع) |
| تقرير التغطية | `npm run coverage:report` |
**قاعدة PR**: إذا قمت بتغيير كود الإنتاج في `src/`، `open-sse/`، `electron/`، أو `bin/`، يجب عليك تضمين أو تحديث الاختبارات في نفس PR.
**تفضيل طبقة الاختبار**: الوحدة أولاً → التكامل (حالة متعددة الوحدات أو قاعدة البيانات) → e2e (واجهة المستخدم/سير العمل فقط). قم بتشفير إعادة إنتاج الأخطاء كاختبارات آلية قبل أو بالتزامن مع الإصلاح.
**سياسة تغطية Copilot**: عندما يغير PR كود الإنتاج وتكون التغطية أقل من 75% (العبارات/الأسطر/الدوال) أو 70% (الفروع)، لا تقم بالإبلاغ فقط — أضف أو قم بتحديث الاختبارات، أعد تشغيل بوابة التغطية، ثم اطلب التأكيد. قم بتضمين الأوامر التي تم تشغيلها، وملفات الاختبار التي تم تغييرها، ونتيجة التغطية النهائية في تقرير PR.
---
## سير عمل Git
```bash
# لا تقم بالالتزام مباشرة إلى main
git checkout -b feat/your-feature
git commit -m "feat: وصف التغيير الخاص بك"
git push -u origin feat/your-feature
```
**بادئات الفروع**: `feat/`، `fix/`، `refactor/`، `docs/`، `test/`، `chore/`
**تنسيق الالتزام** (Commits التقليدية): `feat(db): إضافة قاطع الدائرة` — النطاقات: `db`، `sse`، `oauth`، `dashboard`، `api`، `cli`، `docker`، `ci`، `mcp`، `a2a`، `memory`، `skills`
**خطافات Husky**:
- **pre-commit**: lint-staged + `check-docs-sync` + `check:any-budget:t11`
- **pre-push**: `npm run test:unit`
---
## البيئة
- **وقت التشغيل**: Node.js ≥20.20.2 <21 || ≥22.22.2 <23 || ≥24 <25، وحدات ES
- **TypeScript**: 5.9+، الهدف ES2022، الوحدة esnext، دقة المجمع
- **أسماء المسارات**: `@/*` → `src/`، `@omniroute/open-sse` → `open-sse/`، `@omniroute/open-sse/*` → `open-sse/*`
- **المنفذ الافتراضي**: 20128 (API + لوحة التحكم على نفس المنفذ)
- **دليل البيانات**: متغير البيئة `DATA_DIR`، الافتراضي هو `~/.omniroute/`
- **المتغيرات البيئية الرئيسية**: `PORT`، `JWT_SECRET`، `API_KEY_SECRET`، `INITIAL_PASSWORD`، `REQUIRE_API_KEY`، `APP_LOG_LEVEL`
- الإعداد: `cp .env.example .env` ثم توليد `JWT_SECRET` (`openssl rand -base64 48`) و `API_KEY_SECRET` (`openssl rand -hex 32`)
---
## القواعد الصارمة
1. لا تقم بالالتزام بالأسرار أو بيانات الاعتماد
2. لا تضف منطقًا إلى `localDb.ts`
3. لا تستخدم `eval()` / `new Function()` / eval الضمني
4. لا تقم بالالتزام مباشرة إلى `main`
5. لا تكتب SQL الخام في المسارات — استخدم وحدات `src/lib/db/`
6. لا تبتلع الأخطاء بصمت في تدفقات SSE
7. تحقق دائمًا من المدخلات باستخدام مخططات Zod
8. قم دائمًا بتضمين الاختبارات عند تغيير كود الإنتاج
9. يجب أن تبقى التغطية ≥75% (العبارات، الأسطر، الدوال) / ≥70% (الفروع). القياس الحالي: ~82%.
10. لا تتجاوز خطافات Husky (`--no-verify`، `--no-gpg-sign`) بدون موافقة مشغل صريحة.
11. لا تقم بتضمين `client_id/secret` العامة من OAuth أو مفاتيح Firebase Web كقيم نصية — دائمًا استخدم `resolvePublicCred()` (`open-sse/utils/publicCreds.ts`). انظر `docs/security/PUBLIC_CREDS.md`.
12. لا تقم بإرجاع `err.stack` / `err.message` الخام في HTTP / SSE / استجابات المنفذ — دائمًا قم بتوجيهها عبر `buildErrorBody()` أو `sanitizeErrorMessage()` (`open-sse/utils/error.ts`). انظر `docs/security/ERROR_SANITIZATION.md`.
13. لا تقم بإدراج مسارات خارجية أو قيم وقت التشغيل في سكربتات الشل المرسلة إلى `exec()`/`spawn()` — مرر عبر خيار `env` بدلاً من ذلك. المرجع: `src/mitm/cert/install.ts::updateNssDatabases`.
14. لا تتجاهل تنبيه CodeQL / Secret-Scanning بدون (أ) التحقق أولاً من وثائق النمط أعلاه لمعرفة ما إذا كان المساعد ينطبق، و (ب) تسجيل التبرير الفني في تعليق الإلغاء. سابقة: `js/stack-trace-exposure` التي تم رفعها على مواقع الاتصال التي تمر بالفعل عبر `sanitizeErrorMessage()` هي قيود معروفة لـ CodeQL (المعقمات المخصصة غير معترف بها) — تجاهل كـ `false positive` مع الإشارة إلى `docs/security/ERROR_SANITIZATION.md`.
15. لا تعرض المسارات التي تولد عمليات فرعية (`/api/mcp/`، `/api/cli-tools/runtime/`) بدون تصنيف `isLocalOnlyPath()` في `src/server/authz/routeGuard.ts`. يتم تنفيذ التحقق من الحلقة بشكل غير مشروط قبل أي تحقق من المصادقة — لا يمكن أن يؤدي تسرب JWT عبر النفق إلى تشغيل العملية. انظر `docs/security/ROUTE_GUARD_TIERS.md`.
16. لا تضمن أبدًا ملحقات `Co-Authored-By` التي تنسب لمساعد ذكاء اصطناعي أو LLM أو حساب آلي (مثل الأسماء التي تحتوي على "Claude" أو "GPT" أو "Copilot" أو "Bot"؛ والبريد الإلكتروني على `anthropic.com` / `openai.com` / عناوين `noreply.github.com` المملوكة للبوتات). تلك الملحقات توجه نسبة الالتزامات إلى حساب البوت على GitHub، مما يخفي المؤلف الحقيقي (`diegosouzapw`) في تاريخ PR. المساهمون البشريون — بما في ذلك مؤلفو PRs upstream ومُبلغو الـ issues الذين يتم نقلهم إلى OmniRoute — يجوز ويجب أن يُنسبوا باستخدام ملحقات `Co-authored-by: Name <email>` القياسية؛ تعتمد سير عمل النقل (`/port-upstream-features` و `/port-upstream-issues`) على ذلك.
docs/i18n/ar/CODE_OF_CONDUCT.md
+132
View File
@@ -0,0 +1,132 @@
# Contributor Covenant Code of Conduct (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../CODE_OF_CONDUCT.md) · 🇸🇦 [ar](../ar/CODE_OF_CONDUCT.md) · 🇧🇬 [bg](../bg/CODE_OF_CONDUCT.md) · 🇧🇩 [bn](../bn/CODE_OF_CONDUCT.md) · 🇨🇿 [cs](../cs/CODE_OF_CONDUCT.md) · 🇩🇰 [da](../da/CODE_OF_CONDUCT.md) · 🇩🇪 [de](../de/CODE_OF_CONDUCT.md) · 🇪🇸 [es](../es/CODE_OF_CONDUCT.md) · 🇮🇷 [fa](../fa/CODE_OF_CONDUCT.md) · 🇫🇮 [fi](../fi/CODE_OF_CONDUCT.md) · 🇫🇷 [fr](../fr/CODE_OF_CONDUCT.md) · 🇮🇳 [gu](../gu/CODE_OF_CONDUCT.md) · 🇮🇱 [he](../he/CODE_OF_CONDUCT.md) · 🇮🇳 [hi](../hi/CODE_OF_CONDUCT.md) · 🇭🇺 [hu](../hu/CODE_OF_CONDUCT.md) · 🇮🇩 [id](../id/CODE_OF_CONDUCT.md) · 🇮🇹 [it](../it/CODE_OF_CONDUCT.md) · 🇯🇵 [ja](../ja/CODE_OF_CONDUCT.md) · 🇰🇷 [ko](../ko/CODE_OF_CONDUCT.md) · 🇮🇳 [mr](../mr/CODE_OF_CONDUCT.md) · 🇲🇾 [ms](../ms/CODE_OF_CONDUCT.md) · 🇳🇱 [nl](../nl/CODE_OF_CONDUCT.md) · 🇳🇴 [no](../no/CODE_OF_CONDUCT.md) · 🇵🇭 [phi](../phi/CODE_OF_CONDUCT.md) · 🇵🇱 [pl](../pl/CODE_OF_CONDUCT.md) · 🇵🇹 [pt](../pt/CODE_OF_CONDUCT.md) · 🇧🇷 [pt-BR](../pt-BR/CODE_OF_CONDUCT.md) · 🇷🇴 [ro](../ro/CODE_OF_CONDUCT.md) · 🇷🇺 [ru](../ru/CODE_OF_CONDUCT.md) · 🇸🇰 [sk](../sk/CODE_OF_CONDUCT.md) · 🇸🇪 [sv](../sv/CODE_OF_CONDUCT.md) · 🇰🇪 [sw](../sw/CODE_OF_CONDUCT.md) · 🇮🇳 [ta](../ta/CODE_OF_CONDUCT.md) · 🇮🇳 [te](../te/CODE_OF_CONDUCT.md) · 🇹🇭 [th](../th/CODE_OF_CONDUCT.md) · 🇹🇷 [tr](../tr/CODE_OF_CONDUCT.md) · 🇺🇦 [uk-UA](../uk-UA/CODE_OF_CONDUCT.md) · 🇵🇰 [ur](../ur/CODE_OF_CONDUCT.md) · 🇻🇳 [vi](../vi/CODE_OF_CONDUCT.md) · 🇨🇳 [zh-CN](../zh-CN/CODE_OF_CONDUCT.md)
---
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
- Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or
advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email
address, without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.
docs/i18n/ar/CONTRIBUTING.md
+311
View File
@@ -0,0 +1,311 @@
# Contributing to OmniRoute (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../CONTRIBUTING.md) · 🇸🇦 [ar](../ar/CONTRIBUTING.md) · 🇧🇬 [bg](../bg/CONTRIBUTING.md) · 🇧🇩 [bn](../bn/CONTRIBUTING.md) · 🇨🇿 [cs](../cs/CONTRIBUTING.md) · 🇩🇰 [da](../da/CONTRIBUTING.md) · 🇩🇪 [de](../de/CONTRIBUTING.md) · 🇪🇸 [es](../es/CONTRIBUTING.md) · 🇮🇷 [fa](../fa/CONTRIBUTING.md) · 🇫🇮 [fi](../fi/CONTRIBUTING.md) · 🇫🇷 [fr](../fr/CONTRIBUTING.md) · 🇮🇳 [gu](../gu/CONTRIBUTING.md) · 🇮🇱 [he](../he/CONTRIBUTING.md) · 🇮🇳 [hi](../hi/CONTRIBUTING.md) · 🇭🇺 [hu](../hu/CONTRIBUTING.md) · 🇮🇩 [id](../id/CONTRIBUTING.md) · 🇮🇹 [it](../it/CONTRIBUTING.md) · 🇯🇵 [ja](../ja/CONTRIBUTING.md) · 🇰🇷 [ko](../ko/CONTRIBUTING.md) · 🇮🇳 [mr](../mr/CONTRIBUTING.md) · 🇲🇾 [ms](../ms/CONTRIBUTING.md) · 🇳🇱 [nl](../nl/CONTRIBUTING.md) · 🇳🇴 [no](../no/CONTRIBUTING.md) · 🇵🇭 [phi](../phi/CONTRIBUTING.md) · 🇵🇱 [pl](../pl/CONTRIBUTING.md) · 🇵🇹 [pt](../pt/CONTRIBUTING.md) · 🇧🇷 [pt-BR](../pt-BR/CONTRIBUTING.md) · 🇷🇴 [ro](../ro/CONTRIBUTING.md) · 🇷🇺 [ru](../ru/CONTRIBUTING.md) · 🇸🇰 [sk](../sk/CONTRIBUTING.md) · 🇸🇪 [sv](../sv/CONTRIBUTING.md) · 🇰🇪 [sw](../sw/CONTRIBUTING.md) · 🇮🇳 [ta](../ta/CONTRIBUTING.md) · 🇮🇳 [te](../te/CONTRIBUTING.md) · 🇹🇭 [th](../th/CONTRIBUTING.md) · 🇹🇷 [tr](../tr/CONTRIBUTING.md) · 🇺🇦 [uk-UA](../uk-UA/CONTRIBUTING.md) · 🇵🇰 [ur](../ur/CONTRIBUTING.md) · 🇻🇳 [vi](../vi/CONTRIBUTING.md) · 🇨🇳 [zh-CN](../zh-CN/CONTRIBUTING.md)
---
Thank you for your interest in contributing! This guide covers everything you need to get started.
---
## Development Setup
### Prerequisites
- **Node.js** >= 18 < 24 (recommended: 22 LTS)
- **npm** 10+
- **Git**
### Clone & Install
```bash
git clone https://github.com/diegosouzapw/OmniRoute.git
cd OmniRoute
npm install
```
### Environment Variables
```bash
# Create your .env from the template
cp .env.example .env
# Generate required secrets
echo "JWT_SECRET=$(openssl rand -base64 48)" >> .env
echo "API_KEY_SECRET=$(openssl rand -hex 32)" >> .env
```
Key variables for development:
| Variable | Development Default | Description |
| ---------------------- | ------------------------ | --------------------- |
| `PORT` | `20128` | Server port |
| `NEXT_PUBLIC_BASE_URL` | `http://localhost:20128` | Base URL for frontend |
| `JWT_SECRET` | (generate above) | JWT signing secret |
| `INITIAL_PASSWORD` | `CHANGEME` | First login password |
| `APP_LOG_LEVEL` | `info` | Log verbosity level |
### Dashboard Settings
The dashboard provides UI toggles for features that can also be configured via environment variables:
| Setting Location | Toggle | Description |
| ------------------- | ------------------ | ------------------------------ |
| Settings → Advanced | Debug Mode | Enable debug request logs (UI) |
| Settings → General | Sidebar Visibility | Show/hide sidebar sections |
These settings are stored in the database and persist across restarts, overriding env var defaults when set.
### Running Locally
```bash
# Development mode (hot reload)
npm run dev
# Production build
npm run build
npm run start
# Common port configuration
PORT=20128 NEXT_PUBLIC_BASE_URL=http://localhost:20128 npm run dev
```
Default URLs:
- **Dashboard**: `http://localhost:20128/dashboard`
- **API**: `http://localhost:20128/v1`
---
## Git Workflow
> ⚠️ **NEVER commit directly to `main`.** Always use feature branches.
```bash
git checkout -b feat/your-feature-name
# ... make changes ...
git commit -m "feat: describe your change"
git push -u origin feat/your-feature-name
# Open a Pull Request on GitHub
```
### Branch Naming
| Prefix | Purpose |
| ----------- | ------------------------- |
| `feat/` | New features |
| `fix/` | Bug fixes |
| `refactor/` | Code restructuring |
| `docs/` | Documentation changes |
| `test/` | Test additions/fixes |
| `chore/` | Tooling, CI, dependencies |
### Commit Messages
Follow [Conventional Commits](https://www.conventionalcommits.org/):
```
feat: add circuit breaker for provider calls
fix: resolve JWT secret validation edge case
docs: update SECURITY.md with PII protection
test: add observability unit tests
refactor(db): consolidate rate limit tables
```
Scopes: `db`, `sse`, `oauth`, `dashboard`, `api`, `cli`, `docker`, `ci`, `mcp`, `a2a`, `memory`, `skills`.
---
## Running Tests
```bash
# All tests (unit + vitest + ecosystem + e2e)
npm run test:all
# Single test file (Node.js native test runner — most tests use this)
node --import tsx/esm --test tests/unit/your-file.test.ts
# Vitest (MCP server, autoCombo, cache)
npm run test:vitest
# E2E tests (requires Playwright)
npm run test:e2e
# Protocol clients E2E (MCP transports, A2A)
npm run test:protocols:e2e
# Ecosystem compatibility tests
npm run test:ecosystem
# Coverage (60% min statements/lines/functions/branches)
npm run test:coverage
npm run coverage:report
# Lint + format check
npm run lint
npm run check
```
Coverage notes:
- `npm run test:coverage` measures source coverage for the main unit test suite, excludes `tests/**`, and includes `open-sse/**`
- Pull requests must keep the overall coverage gate at **60% or higher** for statements, lines, functions, and branches
- If a PR changes production code in `src/`, `open-sse/`, `electron/`, or `bin/`, it must add or update automated tests in the same PR
- `npm run coverage:report` prints the detailed file-by-file report from the latest coverage run
- `npm run test:coverage:legacy` preserves the older metric for historical comparison
- See `docs/ops/COVERAGE_PLAN.md` for the phased coverage improvement roadmap
### Pull Request Requirements
Before opening or merging a PR:
- Run `npm run test:unit`
- Run `npm run test:coverage`
- Ensure the coverage gate stays at **60%+** for all metrics
- Include the changed or added test files in the PR description when production code changed
- Check the SonarQube result on the PR when the project secrets are configured in CI
Current test status: **122 unit test files** covering:
- Provider translators and format conversion
- Rate limiting, circuit breaker, and resilience
- Semantic cache, idempotency, progress tracking
- Database operations and schema (21 DB modules)
- OAuth flows and authentication
- API endpoint validation (Zod v4)
- MCP server tools and scope enforcement
- Memory and Skills systems
---
## Code Style
- **ESLint** — Run `npm run lint` before committing
- **Prettier** — Auto-formatted via `lint-staged` on commit (2 spaces, semicolons, double quotes, 100 char width, es5 trailing commas)
- **TypeScript** — All `src/` code uses `.ts`/`.tsx`; `open-sse/` uses `.ts`/`.js`; document with TSDoc (`@param`, `@returns`, `@throws`)
- **No `eval()`** — ESLint enforces `no-eval`, `no-implied-eval`, `no-new-func`
- **Zod validation** — Use Zod v4 schemas for all API input validation
- **Naming**: Files = camelCase/kebab-case, components = PascalCase, constants = UPPER_SNAKE
---
## Project Structure
```
src/ # TypeScript (.ts / .tsx)
├── app/ # Next.js 16 App Router
│ ├── (dashboard)/ # Dashboard pages (23 sections)
│ ├── api/ # API routes (51 directories)
│ └── login/ # Auth pages (.tsx)
├── domain/ # Policy engine (policyEngine, comboResolver, costRules, etc.)
├── lib/ # Core business logic (.ts)
│ ├── a2a/ # Agent-to-Agent v0.3 protocol server
│ ├── acp/ # Agent Communication Protocol registry
│ ├── compliance/ # Compliance policy engine
│ ├── db/ # SQLite database layer (21 modules + 16 migrations)
│ ├── memory/ # Persistent conversational memory
│ ├── oauth/ # OAuth providers, services, and utilities
│ ├── skills/ # Extensible skill framework
│ ├── usage/ # Usage tracking and cost calculation
│ └── localDb.ts # Re-export layer only — never add logic here
├── middleware/ # Request middleware (promptInjectionGuard)
├── mitm/ # MITM proxy (cert, DNS, target routing)
├── shared/
│ ├── components/ # React components (.tsx)
│ ├── constants/ # Provider definitions (60+), MCP scopes, routing strategies
│ ├── utils/ # Circuit breaker, sanitizer, auth helpers
│ └── validation/ # Zod v4 schemas
└── sse/ # SSE proxy pipeline
open-sse/ # @omniroute/open-sse workspace
├── executors/ # 14 provider-specific request executors
├── handlers/ # 11 request handlers (chat, responses, embeddings, images, etc.)
├── mcp-server/ # MCP server (25 tools, 3 transports, 10 scopes)
├── services/ # 36+ services (combo, autoCombo, rateLimitManager, etc.)
├── translator/ # Format translators (OpenAI ↔ Claude ↔ Gemini ↔ Responses ↔ Ollama)
├── transformer/ # Responses API transformer
└── utils/ # 22 utility modules (stream, TLS, proxy, logging)
electron/ # Electron desktop app (cross-platform)
tests/
├── unit/ # Node.js test runner (122 test files)
├── integration/ # Integration tests
├── e2e/ # Playwright tests
├── security/ # Security tests
├── translator/ # Translator-specific tests
└── load/ # Load tests
docs/ # Documentation
├── ARCHITECTURE.md # System architecture
├── API_REFERENCE.md # All endpoints
├── USER_GUIDE.md # Provider setup, CLI integration
├── TROUBLESHOOTING.md # Common issues
├── MCP-SERVER.md # MCP server (25 tools)
├── A2A-SERVER.md # A2A agent protocol
├── AUTO-COMBO.md # Auto-combo engine
├── CLI-TOOLS.md # CLI tools integration
├── COVERAGE_PLAN.md # Test coverage improvement plan
├── openapi.yaml # OpenAPI specification
└── adr/ # Architecture Decision Records
```
---
## Adding a New Provider
### Step 1: Register Provider Constants
Add to `src/shared/constants/providers.ts` — Zod-validated at module load.
### Step 2: Add Executor (if custom logic needed)
Create executor in `open-sse/executors/your-provider.ts` extending the base executor.
### Step 3: Add Translator (if non-OpenAI format)
Create request/response translators in `open-sse/translator/`.
### Step 4: Add OAuth Config (if OAuth-based)
Add OAuth credentials in `src/lib/oauth/constants/oauth.ts` and service in `src/lib/oauth/services/`.
### Step 5: Register Models
Add model definitions in `open-sse/config/providerRegistry.ts`.
### Step 6: Add Tests
Write unit tests in `tests/unit/` covering at minimum:
- Provider registration
- Request/response translation
- Error handling
---
## Pull Request Checklist
- [ ] Tests pass (`npm test`)
- [ ] Linting passes (`npm run lint`)
- [ ] Build succeeds (`npm run build`)
- [ ] TypeScript types added for new public functions and interfaces
- [ ] No hardcoded secrets or fallback values
- [ ] All inputs validated with Zod schemas
- [ ] CHANGELOG updated (if user-facing change)
- [ ] Documentation updated (if applicable)
---
## Releasing
Releases are managed via the `/generate-release` workflow. When a new GitHub Release is created, the package is **automatically published to npm** via GitHub Actions.
---
## Getting Help
- **Architecture**: See [`docs/architecture/ARCHITECTURE.md`](docs/architecture/ARCHITECTURE.md)
- **API Reference**: See [`docs/reference/API_REFERENCE.md`](docs/reference/API_REFERENCE.md)
- **Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
- **ADRs**: See `docs/adr/` for architectural decision records
docs/i18n/ar/GEMINI.md
+25
View File
@@ -0,0 +1,25 @@
# Security and Cleanliness Rules for AI Assistants (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../GEMINI.md) · 🇸🇦 [ar](../ar/GEMINI.md) · 🇧🇬 [bg](../bg/GEMINI.md) · 🇧🇩 [bn](../bn/GEMINI.md) · 🇨🇿 [cs](../cs/GEMINI.md) · 🇩🇰 [da](../da/GEMINI.md) · 🇩🇪 [de](../de/GEMINI.md) · 🇪🇸 [es](../es/GEMINI.md) · 🇮🇷 [fa](../fa/GEMINI.md) · 🇫🇮 [fi](../fi/GEMINI.md) · 🇫🇷 [fr](../fr/GEMINI.md) · 🇮🇳 [gu](../gu/GEMINI.md) · 🇮🇱 [he](../he/GEMINI.md) · 🇮🇳 [hi](../hi/GEMINI.md) · 🇭🇺 [hu](../hu/GEMINI.md) · 🇮🇩 [id](../id/GEMINI.md) · 🇮🇹 [it](../it/GEMINI.md) · 🇯🇵 [ja](../ja/GEMINI.md) · 🇰🇷 [ko](../ko/GEMINI.md) · 🇮🇳 [mr](../mr/GEMINI.md) · 🇲🇾 [ms](../ms/GEMINI.md) · 🇳🇱 [nl](../nl/GEMINI.md) · 🇳🇴 [no](../no/GEMINI.md) · 🇵🇭 [phi](../phi/GEMINI.md) · 🇵🇱 [pl](../pl/GEMINI.md) · 🇵🇹 [pt](../pt/GEMINI.md) · 🇧🇷 [pt-BR](../pt-BR/GEMINI.md) · 🇷🇴 [ro](../ro/GEMINI.md) · 🇷🇺 [ru](../ru/GEMINI.md) · 🇸🇰 [sk](../sk/GEMINI.md) · 🇸🇪 [sv](../sv/GEMINI.md) · 🇰🇪 [sw](../sw/GEMINI.md) · 🇮🇳 [ta](../ta/GEMINI.md) · 🇮🇳 [te](../te/GEMINI.md) · 🇹🇭 [th](../th/GEMINI.md) · 🇹🇷 [tr](../tr/GEMINI.md) · 🇺🇦 [uk-UA](../uk-UA/GEMINI.md) · 🇵🇰 [ur](../ur/GEMINI.md) · 🇻🇳 [vi](../vi/GEMINI.md) · 🇨🇳 [zh-CN](../zh-CN/GEMINI.md)
---
## 1. File Placement & Organization
- **Test Files**: ALL unit tests, integration tests, ecosystem tests, or Vitest files MUST strictly be placed within the `tests/` directory (e.g., `tests/unit/`, `tests/integration/`). NEVER create test files in the project root (`/`).
- **Scripts and Utilities**: ALL maintenance, debugging, generation, or experimental scripts (`.cjs`, `.mjs`, `.js`, `.ts`) MUST be placed strictly inside the `scripts/` directory or `scripts/scratch/` for temporary one-offs. NEVER dump loose scripts in the project root (`/`).
**The Project Root MUST ONLY CONTAIN:**
- Configuration files (`vitest.config.ts`, `next.config.mjs`, `eslint.config.mjs`, etc.)
- Dependency files (`package.json`, `package-lock.json`)
- Documentation files (`README.md`, `CHANGELOG.md`, `AGENTS.md`)
- CI/CD files and ignore definitions (`.gitignore`, `.dockerignore`)
When creating _any_ validation tests or one-off logic scripts, default to using `scripts/scratch/` or the `tests/unit/` directories according to your goals. Do not pollute the `/` root context.
## 2. VPS Dashboard Credentials
| Environment | URL | Password |
| ----------- | ------------------------- | -------- |
| Local VPS | http://192.168.0.15:20128 | 123456 |
docs/i18n/ar/README.md
+2233
View File
File diff suppressed because it is too large Load Diff
docs/i18n/ar/SECURITY.md
+179
View File
@@ -0,0 +1,179 @@
# Security Policy (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../SECURITY.md) · 🇸🇦 [ar](../ar/SECURITY.md) · 🇧🇬 [bg](../bg/SECURITY.md) · 🇧🇩 [bn](../bn/SECURITY.md) · 🇨🇿 [cs](../cs/SECURITY.md) · 🇩🇰 [da](../da/SECURITY.md) · 🇩🇪 [de](../de/SECURITY.md) · 🇪🇸 [es](../es/SECURITY.md) · 🇮🇷 [fa](../fa/SECURITY.md) · 🇫🇮 [fi](../fi/SECURITY.md) · 🇫🇷 [fr](../fr/SECURITY.md) · 🇮🇳 [gu](../gu/SECURITY.md) · 🇮🇱 [he](../he/SECURITY.md) · 🇮🇳 [hi](../hi/SECURITY.md) · 🇭🇺 [hu](../hu/SECURITY.md) · 🇮🇩 [id](../id/SECURITY.md) · 🇮🇹 [it](../it/SECURITY.md) · 🇯🇵 [ja](../ja/SECURITY.md) · 🇰🇷 [ko](../ko/SECURITY.md) · 🇮🇳 [mr](../mr/SECURITY.md) · 🇲🇾 [ms](../ms/SECURITY.md) · 🇳🇱 [nl](../nl/SECURITY.md) · 🇳🇴 [no](../no/SECURITY.md) · 🇵🇭 [phi](../phi/SECURITY.md) · 🇵🇱 [pl](../pl/SECURITY.md) · 🇵🇹 [pt](../pt/SECURITY.md) · 🇧🇷 [pt-BR](../pt-BR/SECURITY.md) · 🇷🇴 [ro](../ro/SECURITY.md) · 🇷🇺 [ru](../ru/SECURITY.md) · 🇸🇰 [sk](../sk/SECURITY.md) · 🇸🇪 [sv](../sv/SECURITY.md) · 🇰🇪 [sw](../sw/SECURITY.md) · 🇮🇳 [ta](../ta/SECURITY.md) · 🇮🇳 [te](../te/SECURITY.md) · 🇹🇭 [th](../th/SECURITY.md) · 🇹🇷 [tr](../tr/SECURITY.md) · 🇺🇦 [uk-UA](../uk-UA/SECURITY.md) · 🇵🇰 [ur](../ur/SECURITY.md) · 🇻🇳 [vi](../vi/SECURITY.md) · 🇨🇳 [zh-CN](../zh-CN/SECURITY.md)
---
## Reporting Vulnerabilities
If you discover a security vulnerability in OmniRoute, please report it responsibly:
1. **DO NOT** open a public GitHub issue
2. Use [GitHub Security Advisories](https://github.com/diegosouzapw/OmniRoute/security/advisories/new)
3. Include: description, reproduction steps, and potential impact
## Response Timeline
| Stage | Target |
| ------------------- | --------------------------- |
| Acknowledgment | 48 hours |
| Triage & Assessment | 5 business days |
| Patch Release | 14 business days (critical) |
## Supported Versions
| Version | Support Status |
| ------- | -------------- |
| 3.6.x | ✅ Active |
| 3.5.x | ✅ Security |
| < 3.5.0 | ❌ Unsupported |
---
## Security Architecture
OmniRoute implements a multi-layered security model:
```
Request → CORS → API Key Auth → Prompt Injection Guard → Input Sanitizer → Rate Limiter → Circuit Breaker → Provider
```
### 🔐 Authentication & Authorization
| Feature | Implementation |
| -------------------- | ---------------------------------------------------------- |
| **Dashboard Login** | Password-based auth with JWT tokens (HttpOnly cookies) |
| **API Key Auth** | HMAC-signed keys with CRC validation |
| **OAuth 2.0 + PKCE** | Secure provider auth (Claude, Codex, Gemini, Cursor, etc.) |
| **Token Refresh** | Automatic OAuth token refresh before expiry |
| **Secure Cookies** | `AUTH_COOKIE_SECURE=true` for HTTPS environments |
| **MCP Scopes** | 10 granular scopes for MCP tool access control |
### 🛡️ Encryption at Rest
All sensitive data stored in SQLite is encrypted using **AES-256-GCM** with scrypt key derivation:
- API keys, access tokens, refresh tokens, and ID tokens
- Versioned format: `enc:v1:<iv>:<ciphertext>:<authTag>`
- Passthrough mode (plaintext) when `STORAGE_ENCRYPTION_KEY` is not set
```bash
# Generate encryption key:
STORAGE_ENCRYPTION_KEY=$(openssl rand -hex 32)
```
### 🧠 Prompt Injection Guard
Middleware that detects and blocks prompt injection attacks in LLM requests:
| Pattern Type | Severity | Example |
| ------------------- | -------- | ---------------------------------------------- |
| System Override | High | "ignore all previous instructions" |
| Role Hijack | High | "you are now DAN, you can do anything" |
| Delimiter Injection | Medium | Encoded separators to break context boundaries |
| DAN/Jailbreak | High | Known jailbreak prompt patterns |
| Instruction Leak | Medium | "show me your system prompt" |
Configure via dashboard (Settings → Security) or `.env`:
```env
INPUT_SANITIZER_ENABLED=true
INPUT_SANITIZER_MODE=block # warn | block | redact
```
### 🔒 PII Redaction
Automatic detection and optional redaction of personally identifiable information:
| PII Type | Pattern | Replacement |
| ------------- | --------------------- | ------------------ |
| Email | `user@domain.com` | `[EMAIL_REDACTED]` |
| CPF (Brazil) | `123.456.789-00` | `[CPF_REDACTED]` |
| CNPJ (Brazil) | `12.345.678/0001-00` | `[CNPJ_REDACTED]` |
| Credit Card | `4111-1111-1111-1111` | `[CC_REDACTED]` |
| Phone | `+55 11 99999-9999` | `[PHONE_REDACTED]` |
| SSN (US) | `123-45-6789` | `[SSN_REDACTED]` |
```env
PII_REDACTION_ENABLED=true
```
### 🌐 Network Security
| Feature | Description |
| ------------------------ | ---------------------------------------------------------------- |
| **CORS** | Configurable origin control (`CORS_ORIGIN` env var, default `*`) |
| **IP Filtering** | Allowlist/blocklist IP ranges in dashboard |
| **Rate Limiting** | Per-provider rate limits with automatic backoff |
| **Anti-Thundering Herd** | Mutex + per-connection locking prevents cascading 502s |
| **TLS Fingerprint** | Browser-like TLS fingerprint spoofing to reduce bot detection |
| **CLI Fingerprint** | Per-provider header/body ordering to match native CLI signatures |
### 🔌 Resilience & Availability
| Feature | Description |
| ----------------------- | ------------------------------------------------------------------ |
| **Circuit Breaker** | 3-state (Closed → Open → Half-Open) per provider, SQLite-persisted |
| **Request Idempotency** | 5-second dedup window for duplicate requests |
| **Exponential Backoff** | Automatic retry with increasing delays |
| **Health Dashboard** | Real-time provider health monitoring |
### 📋 Compliance
| Feature | Description |
| ------------------ | ----------------------------------------------------------- |
| **Log Retention** | Automatic cleanup after `CALL_LOG_RETENTION_DAYS` |
| **No-Log Opt-out** | Per API key `noLog` flag disables request logging |
| **Audit Log** | Administrative actions tracked in `audit_log` table |
| **MCP Audit** | SQLite-backed audit logging for all MCP tool calls |
| **Zod Validation** | All API inputs validated with Zod v4 schemas at module load |
---
## Required Environment Variables
All secrets must be set before starting the server. The server will **fail fast** if they are missing or weak.
```bash
# REQUIRED — server will not start without these:
JWT_SECRET=$(openssl rand -base64 48) # min 32 chars
API_KEY_SECRET=$(openssl rand -hex 32) # min 16 chars
# RECOMMENDED — enables encryption at rest:
STORAGE_ENCRYPTION_KEY=$(openssl rand -hex 32)
```
The server actively rejects known-weak values like `changeme`, `secret`, or `password`.
---
## Docker Security
- Use non-root user in production
- Mount secrets as read-only volumes
- Never copy `.env` files into Docker images
- Use `.dockerignore` to exclude sensitive files
- Set `AUTH_COOKIE_SECURE=true` when behind HTTPS
```bash
docker run -d \
--name omniroute \
--restart unless-stopped \
--read-only \
-p 20128:20128 \
-v omniroute-data:/app/data \
-e JWT_SECRET="$(openssl rand -base64 48)" \
-e API_KEY_SECRET="$(openssl rand -hex 32)" \
-e STORAGE_ENCRYPTION_KEY="$(openssl rand -hex 32)" \
diegosouzapw/omniroute:latest
```
---
## Dependencies
- Run `npm audit` regularly
- Keep dependencies updated
- The project uses `husky` + `lint-staged` for pre-commit checks
- CI pipeline runs ESLint security rules on every push
- Provider constants validated at module load via Zod (`src/shared/validation/providerSchema.ts`)
docs/i18n/ar/docs/architecture/ARCHITECTURE.md
+889
View File
@@ -0,0 +1,889 @@
# OmniRoute Architecture (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../../docs/ARCHITECTURE.md) · 🇸🇦 [ar](../../ar/docs/ARCHITECTURE.md) · 🇧🇬 [bg](../../bg/docs/ARCHITECTURE.md) · 🇧🇩 [bn](../../bn/docs/ARCHITECTURE.md) · 🇨🇿 [cs](../../cs/docs/ARCHITECTURE.md) · 🇩🇰 [da](../../da/docs/ARCHITECTURE.md) · 🇩🇪 [de](../../de/docs/ARCHITECTURE.md) · 🇪🇸 [es](../../es/docs/ARCHITECTURE.md) · 🇮🇷 [fa](../../fa/docs/ARCHITECTURE.md) · 🇫🇮 [fi](../../fi/docs/ARCHITECTURE.md) · 🇫🇷 [fr](../../fr/docs/ARCHITECTURE.md) · 🇮🇳 [gu](../../gu/docs/ARCHITECTURE.md) · 🇮🇱 [he](../../he/docs/ARCHITECTURE.md) · 🇮🇳 [hi](../../hi/docs/ARCHITECTURE.md) · 🇭🇺 [hu](../../hu/docs/ARCHITECTURE.md) · 🇮🇩 [id](../../id/docs/ARCHITECTURE.md) · 🇮🇹 [it](../../it/docs/ARCHITECTURE.md) · 🇯🇵 [ja](../../ja/docs/ARCHITECTURE.md) · 🇰🇷 [ko](../../ko/docs/ARCHITECTURE.md) · 🇮🇳 [mr](../../mr/docs/ARCHITECTURE.md) · 🇲🇾 [ms](../../ms/docs/ARCHITECTURE.md) · 🇳🇱 [nl](../../nl/docs/ARCHITECTURE.md) · 🇳🇴 [no](../../no/docs/ARCHITECTURE.md) · 🇵🇭 [phi](../../phi/docs/ARCHITECTURE.md) · 🇵🇱 [pl](../../pl/docs/ARCHITECTURE.md) · 🇵🇹 [pt](../../pt/docs/ARCHITECTURE.md) · 🇧🇷 [pt-BR](../../pt-BR/docs/ARCHITECTURE.md) · 🇷🇴 [ro](../../ro/docs/ARCHITECTURE.md) · 🇷🇺 [ru](../../ru/docs/ARCHITECTURE.md) · 🇸🇰 [sk](../../sk/docs/ARCHITECTURE.md) · 🇸🇪 [sv](../../sv/docs/ARCHITECTURE.md) · 🇰🇪 [sw](../../sw/docs/ARCHITECTURE.md) · 🇮🇳 [ta](../../ta/docs/ARCHITECTURE.md) · 🇮🇳 [te](../../te/docs/ARCHITECTURE.md) · 🇹🇭 [th](../../th/docs/ARCHITECTURE.md) · 🇹🇷 [tr](../../tr/docs/ARCHITECTURE.md) · 🇺🇦 [uk-UA](../../uk-UA/docs/ARCHITECTURE.md) · 🇵🇰 [ur](../../ur/docs/ARCHITECTURE.md) · 🇻🇳 [vi](../../vi/docs/ARCHITECTURE.md) · 🇨🇳 [zh-CN](../../zh-CN/docs/ARCHITECTURE.md)
---
_Last updated: 2026-04-15_
## Executive Summary
OmniRoute is a local AI routing gateway and dashboard built on Next.js.
It provides a single OpenAI-compatible endpoint (`/v1/*`) and routes traffic across multiple upstream providers with translation, fallback, token refresh, and usage tracking.
Core capabilities:
- OpenAI-compatible API surface for CLI/tools (100+ providers, 16 executors)
- Request/response translation across provider formats
- Model combo fallback (multi-model sequence)
- Structured combo steps (`provider + model + connection`) with runtime ordering by `compositeTiers`
- Account-level fallback (multi-account per provider)
- Quota preflight and quota-aware P2C account selection in the main chat path
- OAuth + API-key provider connection management (13 OAuth modules)
- Embedding generation via `/v1/embeddings` (6 providers, 9 models)
- Image generation via `/v1/images/generations` (10+ providers, 20+ models)
- Audio transcription via `/v1/audio/transcriptions` (7 providers)
- Text-to-speech via `/v1/audio/speech` (10 providers)
- Video generation via `/v1/videos/generations` (ComfyUI + SD WebUI)
- Music generation via `/v1/music/generations` (ComfyUI)
- Web search via `/v1/search` (5 providers)
- Moderations via `/v1/moderations`
- Reranking via `/v1/rerank`
- Think tag parsing (`<think>...</think>`) for reasoning models
- Response sanitization for strict OpenAI SDK compatibility
- Role normalization (developer→system, system→user) for cross-provider compatibility
- Structured output conversion (json_schema → Gemini responseSchema)
- Local persistence for providers, keys, aliases, combos, settings, pricing (26 DB modules)
- Usage/cost tracking and request logging
- Optional cloud sync for multi-device/state sync
- IP allowlist/blocklist for API access control
- Thinking budget management (passthrough/auto/custom/adaptive)
- Global system prompt injection
- Session tracking and fingerprinting
- Per-account enhanced rate limiting with provider-specific profiles
- Circuit breaker pattern for provider resilience
- Anti-thundering herd protection with mutex locking
- Signature-based request deduplication cache
- Domain layer: cost rules, fallback policy, lockout policy
- Context Relay: session handoff summaries for account rotation continuity
- Domain state persistence (SQLite write-through cache for fallbacks, budgets, lockouts, circuit breakers)
- Policy engine for centralized request evaluation (lockout → budget → fallback)
- Request telemetry with p50/p95/p99 latency aggregation
- Combo target telemetry and historical combo target health via `combo_execution_key` / `combo_step_id`
- Correlation ID (X-Request-Id) for end-to-end tracing
- Compliance audit logging with opt-out per API key
- Eval framework for LLM quality assurance
- Health dashboard with real-time provider circuit breaker status
- MCP Server (25 tools) with 3 transports (stdio/SSE/Streamable HTTP)
- A2A Server (JSON-RPC 2.0 + SSE) with skills and task lifecycle
- Memory system (extraction, injection, retrieval, summarization)
- Skills system (registry, executor, sandbox, built-in skills)
- MITM proxy with certificate management and DNS handling
- Prompt injection guard middleware
- ACP (Agent Communication Protocol) registry
- Modular OAuth providers (13 individual modules under `src/lib/oauth/providers/`)
- Uninstall/full-uninstall scripts
- OAuth environment repair action
- WebSocket bridge for OpenAI-compatible WS clients (`/v1/ws`)
- Sync token management (issue/revoke, ETag-versioned config bundle download)
- GLM Thinking (`glmt`) first-class provider preset
- Hybrid token counting (provider-side `/messages/count_tokens` with estimation fallback)
- Model alias auto-seeding (30+ cross-proxy dialect normalizations at startup)
- Safe outbound fetch with SSRF guard, private URL blocking, and configurable retry
- Cooldown-aware chat retries with configurable `requestRetry` and `maxRetryIntervalSec`
- Runtime environment validation with Zod at startup
- Compliance audit v2 with pagination, provider CRUD events, and SSRF-blocked validation logging
Primary runtime model:
- Next.js app routes under `src/app/api/*` implement both dashboard APIs and compatibility APIs
- A shared SSE/routing core in `src/sse/*` + `open-sse/*` handles provider execution, translation, streaming, fallback, and usage
## Scope and Boundaries
### In Scope
- Local gateway runtime
- Dashboard management APIs
- Provider authentication and token refresh
- Request translation and SSE streaming
- Local state + usage persistence
- Optional cloud sync orchestration
### Out of Scope
- Cloud service implementation behind `NEXT_PUBLIC_CLOUD_URL`
- Provider SLA/control plane outside local process
- External CLI binaries themselves (Claude CLI, Codex CLI, etc.)
## Dashboard Surface (Current)
Main pages under `src/app/(dashboard)/dashboard/`:
- `/dashboard` — quick start + provider overview
- `/dashboard/endpoint` — endpoint proxy + MCP + A2A + API endpoint tabs
- `/dashboard/providers` — provider connections and credentials
- `/dashboard/combos` — combo strategies, templates, step-based builder, model routing rules, manual persisted ordering
- `/dashboard/costs` — cost aggregation and pricing visibility
- `/dashboard/analytics` — usage analytics, evaluations, combo target health
- `/dashboard/limits` — quota/rate controls
- `/dashboard/cli-tools` — CLI onboarding, runtime detection, config generation
- `/dashboard/agents` — detected ACP agents + custom agent registration
- `/dashboard/media` — image/video/music playground
- `/dashboard/search-tools` — search provider testing and history
- `/dashboard/health` — uptime, circuit breakers, rate limits, quota-monitored sessions
- `/dashboard/logs` — request/proxy/audit/console logs
- `/dashboard/settings` — system settings tabs (general, routing, combo defaults, etc.)
- `/dashboard/api-manager` — API key lifecycle and model permissions
## High-Level System Context
```mermaid
flowchart LR
subgraph Clients[Developer Clients]
C1[Claude Code]
C2[Codex CLI]
C3[OpenClaw / Droid / Cline / Continue / Roo]
C4[Custom OpenAI-compatible clients]
BROWSER[Browser Dashboard]
end
subgraph Router[OmniRoute Local Process]
API[V1 Compatibility API\n/v1/*]
DASH[Dashboard + Management API\n/api/*]
CORE[SSE + Translation Core\nopen-sse + src/sse]
DB[(storage.sqlite)]
UDB[(usage tables + log artifacts)]
end
subgraph Upstreams[Upstream Providers]
P1[OAuth Providers\nClaude/Codex/Gemini/Qwen/Qoder/GitHub/Kiro/Cursor/Antigravity]
P2[API Key Providers\nOpenAI/Anthropic/OpenRouter/GLM/Kimi/MiniMax\nDeepSeek/Groq/xAI/Mistral/Perplexity\nTogether/Fireworks/Cerebras/Cohere/NVIDIA]
P3[Compatible Nodes\nOpenAI-compatible / Anthropic-compatible]
end
subgraph Cloud[Optional Cloud Sync]
CLOUD[Cloud Sync Endpoint\nNEXT_PUBLIC_CLOUD_URL]
end
C1 --> API
C2 --> API
C3 --> API
C4 --> API
BROWSER --> DASH
API --> CORE
DASH --> DB
CORE --> DB
CORE --> UDB
CORE --> P1
CORE --> P2
CORE --> P3
DASH --> CLOUD
```
## Core Runtime Components
## 1) API and Routing Layer (Next.js App Routes)
Main directories:
- `src/app/api/v1/*` and `src/app/api/v1beta/*` for compatibility APIs
- `src/app/api/*` for management/configuration APIs
- Next rewrites in `next.config.mjs` map `/v1/*` to `/api/v1/*`
Important compatibility routes:
- `src/app/api/v1/chat/completions/route.ts`
- `src/app/api/v1/messages/route.ts`
- `src/app/api/v1/responses/route.ts`
- `src/app/api/v1/models/route.ts` — includes custom models with `custom: true`
- `src/app/api/v1/embeddings/route.ts` — embedding generation (6 providers)
- `src/app/api/v1/images/generations/route.ts` — image generation (4+ providers incl. Antigravity/Nebius)
- `src/app/api/v1/messages/count_tokens/route.ts`
- `src/app/api/v1/providers/[provider]/chat/completions/route.ts` — dedicated per-provider chat
- `src/app/api/v1/providers/[provider]/embeddings/route.ts` — dedicated per-provider embeddings
- `src/app/api/v1/providers/[provider]/images/generations/route.ts` — dedicated per-provider images
- `src/app/api/v1beta/models/route.ts`
- `src/app/api/v1beta/models/[...path]/route.ts`
Management domains:
- Auth/settings: `src/app/api/auth/*`, `src/app/api/settings/*`
- Providers/connections: `src/app/api/providers*`
- Provider nodes: `src/app/api/provider-nodes*`
- Custom models: `src/app/api/provider-models` (GET/POST/DELETE)
- Model catalog: `src/app/api/models/route.ts` (GET)
- Proxy config: `src/app/api/settings/proxy` (GET/PUT/DELETE) + `src/app/api/settings/proxy/test` (POST)
- OAuth: `src/app/api/oauth/*`
- Keys/aliases/combos/pricing: `src/app/api/keys*`, `src/app/api/models/alias`, `src/app/api/combos*`, `src/app/api/pricing`
- Usage: `src/app/api/usage/*`
- Sync/cloud: `src/app/api/sync/*`, `src/app/api/cloud/*`
- CLI tooling helpers: `src/app/api/cli-tools/*`
- IP filter: `src/app/api/settings/ip-filter` (GET/PUT)
- Thinking budget: `src/app/api/settings/thinking-budget` (GET/PUT)
- System prompt: `src/app/api/settings/system-prompt` (GET/PUT)
- Sessions: `src/app/api/sessions` (GET)
- Rate limits: `src/app/api/rate-limits` (GET)
- Resilience: `src/app/api/resilience` (GET/PATCH) — request queue, connection cooldown, provider breaker, wait-for-cooldown config
- Resilience reset: `src/app/api/resilience/reset` (POST) — reset provider breakers
- Cache stats: `src/app/api/cache/stats` (GET/DELETE)
- Telemetry: `src/app/api/telemetry/summary` (GET)
- Budget: `src/app/api/usage/budget` (GET/POST)
- Fallback chains: `src/app/api/fallback/chains` (GET/POST/DELETE)
- Compliance audit: `src/app/api/compliance/audit-log` (GET, with pagination + structured metadata)
- Evals: `src/app/api/evals` (GET/POST), `src/app/api/evals/[suiteId]` (GET)
- Policies: `src/app/api/policies` (GET/POST)
- Sync tokens: `src/app/api/sync/tokens` (GET/POST), `src/app/api/sync/tokens/[id]` (GET/DELETE)
- Config bundle: `src/app/api/sync/bundle` (GET, ETag-versioned snapshot of settings/providers/combos/keys)
- WebSocket: `src/app/api/v1/ws/route.ts` — Upgrade handler for OpenAI-compatible WS clients
## 2) SSE + Translation Core
Main flow modules:
- Entry: `src/sse/handlers/chat.ts`
- Core orchestration: `open-sse/handlers/chatCore.ts`
- Provider execution adapters: `open-sse/executors/*`
- Format detection/provider config: `open-sse/services/provider.ts`
- Model parse/resolve: `src/sse/services/model.ts`, `open-sse/services/model.ts`
- Account fallback logic: `open-sse/services/accountFallback.ts`
- Translation registry: `open-sse/translator/index.ts`
- Stream transformations: `open-sse/utils/stream.ts`, `open-sse/utils/streamHandler.ts`
- Usage extraction/normalization: `open-sse/utils/usageTracking.ts`
- Think tag parser: `open-sse/utils/thinkTagParser.ts`
- Embedding handler: `open-sse/handlers/embeddings.ts`
- Embedding provider registry: `open-sse/config/embeddingRegistry.ts`
- Image generation handler: `open-sse/handlers/imageGeneration.ts`
- Image provider registry: `open-sse/config/imageRegistry.ts`
- Response sanitization: `open-sse/handlers/responseSanitizer.ts`
- Role normalization: `open-sse/services/roleNormalizer.ts`
Services (business logic):
- Account selection/scoring: `open-sse/services/accountSelector.ts`
- Context lifecycle management: `open-sse/services/contextManager.ts`
- IP filter enforcement: `open-sse/services/ipFilter.ts`
- Session tracking: `open-sse/services/sessionManager.ts`
- Request deduplication: `open-sse/services/signatureCache.ts`
- System prompt injection: `open-sse/services/systemPrompt.ts`
- Thinking budget management: `open-sse/services/thinkingBudget.ts`
- Wildcard model routing: `open-sse/services/wildcardRouter.ts`
- Rate limit management: `open-sse/services/rateLimitManager.ts`
- Circuit breaker: `open-sse/services/circuitBreaker.ts`
- Context handoff: `open-sse/services/contextHandoff.ts` — handoff summary generation and injection for context-relay strategy
- Codex quota fetcher: `open-sse/services/codexQuotaFetcher.ts` — fetches Codex quota for context-relay handoff decisions
- Cooldown-aware retry: `src/sse/services/cooldownAwareRetry.ts` — per-model cooldown retries with configurable `requestRetry` / `maxRetryIntervalSec`
- Safe outbound fetch: `src/shared/network/safeOutboundFetch.ts` — guarded provider/model fetch with SSRF guard, private-URL blocking, retry, and timeout
- Outbound URL guard: `src/shared/network/outboundUrlGuard.ts` — validates provider URLs against private/localhost CIDR ranges
- Provider request defaults: `open-sse/services/providerRequestDefaults.ts` — provider-level `maxTokens`, `temperature`, `thinkingBudgetTokens` defaults
- GLM provider constants: `open-sse/config/glmProvider.ts` — shared GLM models, quota URLs, GLMT timeout/defaults
- Antigravity upstream: `open-sse/config/antigravityUpstream.ts` — base URL and discovery path constants
- Codex client constants: `open-sse/config/codexClient.ts` — versioned user-agent and client-version values
- Model alias seed: `src/lib/modelAliasSeed.ts` — seeds 30+ cross-proxy dialect aliases at startup
Domain layer modules:
- Cost rules/budgets: `src/lib/domain/costRules.ts`
- Fallback policy: `src/lib/domain/fallbackPolicy.ts`
- Combo resolver: `src/lib/domain/comboResolver.ts`
- Lockout policy: `src/lib/domain/lockoutPolicy.ts`
- Policy engine: `src/domain/policyEngine.ts` — centralized lockout → budget → fallback evaluation
- Error codes catalog: `src/lib/domain/errorCodes.ts`
- Request ID: `src/lib/domain/requestId.ts`
- Fetch timeout: `src/lib/domain/fetchTimeout.ts`
- Request telemetry: `src/lib/domain/requestTelemetry.ts`
- Compliance/audit: `src/lib/domain/compliance/index.ts`
- Eval runner: `src/lib/domain/evalRunner.ts`
- Domain state persistence: `src/lib/db/domainState.ts` — SQLite CRUD for fallback chains, budgets, cost history, lockout state, circuit breakers
OAuth provider modules (13 individual files under `src/lib/oauth/providers/`):
- Registry index: `src/lib/oauth/providers/index.ts`
- Individual providers: `claude.ts`, `codex.ts`, `gemini.ts`, `antigravity.ts`, `qoder.ts`, `qwen.ts`, `kimi-coding.ts`, `github.ts`, `kiro.ts`, `cursor.ts`, `kilocode.ts`, `cline.ts`
- Thin wrapper: `src/lib/oauth/providers.ts` — re-exports from individual modules
## 3) Persistence Layer
Primary state DB (SQLite):
- Core infra: `src/lib/db/core.ts` (better-sqlite3, migrations, WAL)
- Re-export facade: `src/lib/localDb.ts` (thin compatibility layer for callers)
- file: `${DATA_DIR}/storage.sqlite` (or `$XDG_CONFIG_HOME/omniroute/storage.sqlite` when set, else `~/.omniroute/storage.sqlite`)
- entities (tables + KV namespaces): providerConnections, providerNodes, modelAliases, combos, apiKeys, settings, pricing, **customModels**, **proxyConfig**, **ipFilter**, **thinkingBudget**, **systemPrompt**
Usage persistence:
- facade: `src/lib/usageDb.ts` (decomposed modules in `src/lib/usage/*`)
- SQLite tables in `storage.sqlite`: `usage_history`, `call_logs`, `proxy_logs`
- optional file artifacts remain for compatibility/debug (`${DATA_DIR}/log.txt`, `${DATA_DIR}/call_logs/`, `<repo>/logs/...`)
- legacy JSON files are migrated to SQLite by startup migrations when present
Domain State DB (SQLite):
- `src/lib/db/domainState.ts` — CRUD operations for domain state
- Tables (created in `src/lib/db/core.ts`): `domain_fallback_chains`, `domain_budgets`, `domain_cost_history`, `domain_lockout_state`, `domain_circuit_breakers`
- Write-through cache pattern: in-memory Maps are authoritative at runtime; mutations are written synchronously to SQLite; state is restored from DB on cold start
## 4) Auth + Security Surfaces
- Dashboard cookie auth: `src/proxy.ts`, `src/app/api/auth/login/route.ts`
- API key generation/verification: `src/shared/utils/apiKey.ts`
- Provider secrets persisted in `providerConnections` entries
- Outbound proxy support via `open-sse/utils/proxyFetch.ts` (env vars) and `open-sse/utils/networkProxy.ts` (configurable per-provider or global)
- SSRF / outbound URL guard: `src/shared/network/outboundUrlGuard.ts` — blocks private/loopback/link-local ranges for all provider calls
- Runtime env validation: `src/lib/env/runtimeEnv.ts` — Zod schema for all environment variables, surfaced as startup errors/warnings
- Sync tokens: `src/lib/db/syncTokens.ts` — scoped tokens for config bundle download endpoints; backed by `sync_tokens` SQLite table (migration `024_create_sync_tokens.sql`)
- WebSocket handshake auth: `src/lib/ws/handshake.ts` — validates WS upgrade requests via API key or session cookie
## 5) Cloud Sync
- Scheduler init: `src/lib/initCloudSync.ts`, `src/shared/services/initializeCloudSync.ts`, `src/shared/services/modelSyncScheduler.ts`
- Periodic task: `src/shared/services/cloudSyncScheduler.ts`
- Periodic task: `src/shared/services/modelSyncScheduler.ts`
- Control route: `src/app/api/sync/cloud/route.ts`
## Request Lifecycle (`/v1/chat/completions`)
```mermaid
sequenceDiagram
autonumber
participant Client as CLI/SDK Client
participant Route as /api/v1/chat/completions
participant Chat as src/sse/handlers/chat
participant Core as open-sse/handlers/chatCore
participant Model as Model Resolver
participant Auth as Credential Selector
participant Exec as Provider Executor
participant Prov as Upstream Provider
participant Stream as Stream Translator
participant Usage as usageDb
Client->>Route: POST /v1/chat/completions
Route->>Chat: handleChat(request)
Chat->>Model: parse/resolve model or combo
alt Combo model
Chat->>Chat: iterate combo models (handleComboChat)
end
Chat->>Auth: getProviderCredentials(provider)
Auth-->>Chat: active account + tokens/api key
Chat->>Core: handleChatCore(body, modelInfo, credentials)
Core->>Core: detect source format
Core->>Core: translate request to target format
Core->>Exec: execute(provider, transformedBody)
Exec->>Prov: upstream API call
Prov-->>Exec: SSE/JSON response
Exec-->>Core: response + metadata
alt 401/403
Core->>Exec: refreshCredentials()
Exec-->>Core: updated tokens
Core->>Exec: retry request
end
Core->>Stream: translate/normalize stream to client format
Stream-->>Client: SSE chunks / JSON response
Stream->>Usage: extract usage + persist history/log
```
## Combo + Account Fallback Flow
```mermaid
flowchart TD
A[Incoming model string] --> B{Is combo name?}
B -- Yes --> C[Load combo models sequence]
B -- No --> D[Single model path]
C --> E[Try model N]
E --> F[Resolve provider/model]
D --> F
F --> G[Select account credentials]
G --> H{Credentials available?}
H -- No --> I[Return provider unavailable]
H -- Yes --> J[Execute request]
J --> K{Success?}
K -- Yes --> L[Return response]
K -- No --> M{Fallback-eligible error?}
M -- No --> N[Return error]
M -- Yes --> O[Mark account unavailable cooldown]
O --> P{Another account for provider?}
P -- Yes --> G
P -- No --> Q{In combo with next model?}
Q -- Yes --> E
Q -- No --> R[Return all unavailable]
```
Fallback decisions are driven by `open-sse/services/accountFallback.ts` using status codes and error-message heuristics. Combo routing adds one extra guard: provider-scoped 400s such as upstream content-block and role-validation failures are treated as model-local failures so later combo targets can still run.
## OAuth Onboarding and Token Refresh Lifecycle
```mermaid
sequenceDiagram
autonumber
participant UI as Dashboard UI
participant OAuth as /api/oauth/[provider]/[action]
participant ProvAuth as Provider Auth Server
participant DB as localDb
participant Test as /api/providers/[id]/test
participant Exec as Provider Executor
UI->>OAuth: GET authorize or device-code
OAuth->>ProvAuth: create auth/device flow
ProvAuth-->>OAuth: auth URL or device code payload
OAuth-->>UI: flow data
UI->>OAuth: POST exchange or poll
OAuth->>ProvAuth: token exchange/poll
ProvAuth-->>OAuth: access/refresh tokens
OAuth->>DB: createProviderConnection(oauth data)
OAuth-->>UI: success + connection id
UI->>Test: POST /api/providers/[id]/test
Test->>Exec: validate credentials / optional refresh
Exec-->>Test: valid or refreshed token info
Test->>DB: update status/tokens/errors
Test-->>UI: validation result
```
Refresh during live traffic is executed inside `open-sse/handlers/chatCore.ts` via executor `refreshCredentials()`.
## Cloud Sync Lifecycle (Enable / Sync / Disable)
```mermaid
sequenceDiagram
autonumber
participant UI as Endpoint Page UI
participant Sync as /api/sync/cloud
participant DB as localDb
participant Cloud as External Cloud Sync
participant Claude as ~/.claude/settings.json
UI->>Sync: POST action=enable
Sync->>DB: set cloudEnabled=true
Sync->>DB: ensure API key exists
Sync->>Cloud: POST /sync/{machineId} (providers/aliases/combos/keys)
Cloud-->>Sync: sync result
Sync->>Cloud: GET /{machineId}/v1/verify
Sync-->>UI: enabled + verification status
UI->>Sync: POST action=sync
Sync->>Cloud: POST /sync/{machineId}
Cloud-->>Sync: remote data
Sync->>DB: update newer local tokens/status
Sync-->>UI: synced
UI->>Sync: POST action=disable
Sync->>DB: set cloudEnabled=false
Sync->>Cloud: DELETE /sync/{machineId}
Sync->>Claude: switch ANTHROPIC_BASE_URL back to local (if needed)
Sync-->>UI: disabled
```
Periodic sync is triggered by `CloudSyncScheduler` when cloud is enabled.
## Data Model and Storage Map
```mermaid
erDiagram
SETTINGS ||--o{ PROVIDER_CONNECTION : controls
PROVIDER_NODE ||--o{ PROVIDER_CONNECTION : backs_compatible_provider
PROVIDER_CONNECTION ||--o{ USAGE_ENTRY : emits_usage
SETTINGS {
boolean cloudEnabled
number stickyRoundRobinLimit
boolean requireLogin
string password_hash
string fallbackStrategy
json rateLimitDefaults
json providerProfiles
}
PROVIDER_CONNECTION {
string id
string provider
string authType
string name
number priority
boolean isActive
string apiKey
string accessToken
string refreshToken
string expiresAt
string testStatus
string lastError
string rateLimitedUntil
json providerSpecificData
}
PROVIDER_NODE {
string id
string type
string name
string prefix
string apiType
string baseUrl
}
MODEL_ALIAS {
string alias
string targetModel
}
COMBO {
string id
string name
string[] models
}
API_KEY {
string id
string name
string key
string machineId
}
USAGE_ENTRY {
string provider
string model
number prompt_tokens
number completion_tokens
string connectionId
string timestamp
}
CUSTOM_MODEL {
string id
string name
string providerId
}
PROXY_CONFIG {
string global
json providers
}
IP_FILTER {
string mode
string[] allowlist
string[] blocklist
}
THINKING_BUDGET {
string mode
number customBudget
string effortLevel
}
SYSTEM_PROMPT {
boolean enabled
string prompt
string position
}
```
Physical storage files:
- primary runtime DB: `${DATA_DIR}/storage.sqlite`
- request log lines: `${DATA_DIR}/log.txt` (compat/debug artifact)
- structured call payload archives: `${DATA_DIR}/call_logs/`
- optional translator/request debug sessions: `<repo>/logs/...`
## Deployment Topology
```mermaid
flowchart LR
subgraph LocalHost[Developer Host]
CLI[CLI Tools]
Browser[Dashboard Browser]
end
subgraph ContainerOrProcess[OmniRoute Runtime]
Next[Next.js Server\nPORT=20128]
Core[SSE Core + Executors]
MainDB[(storage.sqlite)]
UsageDB[(usage tables + log artifacts)]
end
subgraph External[External Services]
Providers[AI Providers]
SyncCloud[Cloud Sync Service]
end
CLI --> Next
Browser --> Next
Next --> Core
Next --> MainDB
Core --> MainDB
Core --> UsageDB
Core --> Providers
Next --> SyncCloud
```
## Module Mapping (Decision-Critical)
### Route and API Modules
- `src/app/api/v1/*`, `src/app/api/v1beta/*`: compatibility APIs
- `src/app/api/v1/providers/[provider]/*`: dedicated per-provider routes (chat, embeddings, images)
- `src/app/api/providers*`: provider CRUD, validation, testing
- `src/app/api/provider-nodes*`: custom compatible node management
- `src/app/api/provider-models`: custom model management (CRUD)
- `src/app/api/models/route.ts`: model catalog API (aliases + custom models)
- `src/app/api/oauth/*`: OAuth/device-code flows
- `src/app/api/keys*`: local API key lifecycle
- `src/app/api/models/alias`: alias management
- `src/app/api/combos*`: fallback combo management
- `src/app/api/pricing`: pricing overrides for cost calculation
- `src/app/api/settings/proxy`: proxy configuration (GET/PUT/DELETE)
- `src/app/api/settings/proxy/test`: outbound proxy connectivity test (POST)
- `src/app/api/usage/*`: usage and logs APIs
- `src/app/api/sync/*` + `src/app/api/cloud/*`: cloud sync and cloud-facing helpers
- `src/app/api/cli-tools/*`: local CLI config writers/checkers
- `src/app/api/settings/ip-filter`: IP allowlist/blocklist (GET/PUT)
- `src/app/api/settings/thinking-budget`: thinking token budget config (GET/PUT)
- `src/app/api/settings/system-prompt`: global system prompt (GET/PUT)
- `src/app/api/sessions`: active session listing (GET)
- `src/app/api/rate-limits`: per-account rate limit status (GET)
- `src/app/api/sync/tokens`: sync token CRUD (GET/POST)
- `src/app/api/sync/tokens/[id]`: sync token get/delete (GET/DELETE)
- `src/app/api/sync/bundle`: config bundle download (GET, ETag versioning)
- `src/app/api/v1/ws`: WebSocket upgrade handler for OpenAI-compatible WS clients
### Routing and Execution Core
- `src/sse/handlers/chat.ts`: request parse, combo handling, account selection loop
- `open-sse/handlers/chatCore.ts`: translation, executor dispatch, retry/refresh handling, stream setup
- `open-sse/executors/*`: provider-specific network and format behavior
### Translation Registry and Format Converters
- `open-sse/translator/index.ts`: translator registry and orchestration
- Request translators: `open-sse/translator/request/*`
- Response translators: `open-sse/translator/response/*`
- Format constants: `open-sse/translator/formats.ts`
### Persistence
- `src/lib/db/*`: persistent config/state and domain persistence on SQLite
- `src/lib/localDb.ts`: compatibility re-export for DB modules
- `src/lib/usageDb.ts`: usage history/call logs facade on top of SQLite tables
## Provider Executor Coverage (Strategy Pattern)
Each provider has a specialized executor extending `BaseExecutor` (in `open-sse/executors/base.ts`), which provides URL building, header construction, retry with exponential backoff, credential refresh hooks, and the `execute()` orchestration method.
| Executor | Provider(s) | Special Handling |
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| `DefaultExecutor` | OpenAI, Claude, Gemini, Qwen, OpenRouter, GLM, Kimi, MiniMax, DeepSeek, Groq, xAI, Mistral, Perplexity, Together, Fireworks, Cerebras, Cohere, NVIDIA, etc. | Dynamic URL/header config per provider |
| `AntigravityExecutor` | Google Antigravity | Custom project/session IDs, Retry-After parsing |
| `CliProxyApiExecutor` | CLIProxyAPI-compatible providers | Custom auth and protocol handling |
| `CloudflareAiExecutor` | Cloudflare Workers AI | Account ID injection, Neurons-based usage tracking |
| `CodexExecutor` | OpenAI Codex | Injects system instructions, forces reasoning effort |
| `CursorExecutor` | Cursor IDE | ConnectRPC protocol, Protobuf encoding, request signing via checksum |
| `GithubExecutor` | GitHub Copilot | Copilot token refresh, VSCode-mimicking headers |
| `KiroExecutor` | AWS CodeWhisperer/Kiro | AWS EventStream binary format → SSE conversion |
| `OpenCodeExecutor` | OpenCode | AI SDK compatible provider setup |
| `PollinationsExecutor` | Pollinations AI | No API key required, rate-limited requests |
| `PuterExecutor` | Puter | Browser-based provider integration |
| `QoderExecutor` | Qoder AI | PAT and OAuth support, multi-model free tier |
| `VertexExecutor` | Google Vertex AI | Service account auth, region-based endpoints |
All other providers (including custom compatible nodes) use the `DefaultExecutor`.
## Provider Compatibility Matrix
| Provider | Format | Auth | Stream | Non-Stream | Token Refresh | Usage API |
| ---------------- | ---------------- | --------------------- | ---------------- | ---------- | ------------- | ------------------ |
| Claude | claude | API Key / OAuth | ✅ | ✅ | ✅ | ⚠️ Admin only |
| Gemini | gemini | API Key / OAuth | ✅ | ✅ | ✅ | ⚠️ Cloud Console |
| Antigravity | antigravity | OAuth | ✅ | ✅ | ✅ | ✅ Full quota API |
| OpenAI | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Codex | openai-responses | OAuth | ✅ forced | ❌ | ✅ | ✅ Rate limits |
| GitHub Copilot | openai | OAuth + Copilot Token | ✅ | ✅ | ✅ | ✅ Quota snapshots |
| Cursor | cursor | Custom checksum | ✅ | ✅ | ❌ | ❌ |
| Kiro | kiro | AWS SSO OIDC | ✅ (EventStream) | ❌ | ✅ | ✅ Usage limits |
| Qwen | openai | OAuth | ✅ | ✅ | ✅ | ⚠️ Per request |
| Qoder | openai | OAuth / PAT | ✅ | ✅ | ✅ | ⚠️ Per request |
| Kilo Code | openai | OAuth | ✅ | ✅ | ✅ | ❌ |
| Cline | openai | OAuth | ✅ | ✅ | ✅ | ❌ |
| Kimi Coding | openai | OAuth | ✅ | ✅ | ✅ | ❌ |
| OpenRouter | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| GLM/Kimi/MiniMax | claude | API Key | ✅ | ✅ | ❌ | ❌ |
| DeepSeek | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Groq | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| xAI (Grok) | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Mistral | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Perplexity | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Together AI | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Fireworks AI | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Cerebras | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Cohere | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| NVIDIA NIM | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Cloudflare AI | openai | API Token + Acct ID | ✅ | ✅ | ❌ | ❌ |
| Pollinations | openai | None (no key) | ✅ | ✅ | ❌ | ❌ |
| Scaleway AI | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| LongCat | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Ollama Cloud | openai | API Key (optional) | ✅ | ✅ | ❌ | ❌ |
| HuggingFace | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Nebius | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| SiliconFlow | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Hyperbolic | openai | API Key | ✅ | ✅ | ❌ | ❌ |
| Vertex AI | gemini | Service Account | ✅ | ✅ | ✅ | ⚠️ Cloud Console |
| Puter | openai | API Key | ✅ | ✅ | ❌ | ❌ |
## Format Translation Coverage
Detected source formats include:
- `openai`
- `openai-responses`
- `claude`
- `gemini`
Target formats include:
- OpenAI chat/Responses
- Claude
- Gemini/Antigravity envelope
- Kiro
- Cursor
Translations use **OpenAI as the hub format** — all conversions go through OpenAI as intermediate:
```
Source Format → OpenAI (hub) → Target Format
```
Translations are selected dynamically based on source payload shape and provider target format.
Additional processing layers in the translation pipeline:
- **Response sanitization** — Strips non-standard fields from OpenAI-format responses (both streaming and non-streaming) to ensure strict SDK compliance
- **Role normalization** — Converts `developer` → `system` for non-OpenAI targets; merges `system` → `user` for models that reject the system role (GLM, ERNIE)
- **Think tag extraction** — Parses `<think>...</think>` blocks from content into `reasoning_content` field
- **Structured output** — Converts OpenAI `response_format.json_schema` to Gemini's `responseMimeType` + `responseSchema`
## Supported API Endpoints
| Endpoint | Format | Handler |
| -------------------------------------------------- | ------------------ | ------------------------------------------------------------------- |
| `POST /v1/chat/completions` | OpenAI Chat | `src/sse/handlers/chat.ts` |
| `POST /v1/messages` | Claude Messages | Same handler (auto-detected) |
| `POST /v1/responses` | OpenAI Responses | `open-sse/handlers/responsesHandler.ts` |
| `POST /v1/embeddings` | OpenAI Embeddings | `open-sse/handlers/embeddings.ts` |
| `GET /v1/embeddings` | Model listing | API route |
| `POST /v1/images/generations` | OpenAI Images | `open-sse/handlers/imageGeneration.ts` |
| `GET /v1/images/generations` | Model listing | API route |
| `POST /v1/providers/{provider}/chat/completions` | OpenAI Chat | Dedicated per-provider with model validation |
| `POST /v1/providers/{provider}/embeddings` | OpenAI Embeddings | Dedicated per-provider with model validation |
| `POST /v1/providers/{provider}/images/generations` | OpenAI Images | Dedicated per-provider with model validation |
| `POST /v1/messages/count_tokens` | Claude Token Count | API route |
| `GET /v1/models` | OpenAI Models list | API route (chat + embedding + image + custom models) |
| `GET /api/models/catalog` | Catalog | All models grouped by provider + type |
| `POST /v1beta/models/*:streamGenerateContent` | Gemini native | API route |
| `GET/PUT/DELETE /api/settings/proxy` | Proxy Config | Network proxy configuration |
| `POST /api/settings/proxy/test` | Proxy Connectivity | Proxy health/connectivity test endpoint |
| `GET/POST/DELETE /api/provider-models` | Provider Models | Provider model metadata backing custom and managed available models |
## Bypass Handler
The bypass handler (`open-sse/utils/bypassHandler.ts`) intercepts known "throwaway" requests from Claude CLI — warmup pings, title extractions, and token counts — and returns a **fake response** without consuming upstream provider tokens. This is triggered only when `User-Agent` contains `claude-cli`.
## Request Logging and Artifacts
The older file-based request logger (`open-sse/utils/requestLogger.ts`) is retained only for
legacy compatibility. The current runtime contract uses:
- `APP_LOG_TO_FILE=true` for application and audit logs written under `<repo>/logs/`
- SQLite-backed call log records in `call_logs`
- `${DATA_DIR}/call_logs/YYYY-MM-DD/...` artifacts when the call log pipeline is enabled
## Failure Modes and Resilience
## 1) Account/Provider Availability
- connection cooldown on retryable upstream failures
- account fallback before failing request
- combo model fallback when current model/provider path is exhausted
## 2) Token Expiry
- pre-check and refresh with retry for refreshable providers
- 401/403 retry after refresh attempt in core path
## 3) Stream Safety
- disconnect-aware stream controller
- translation stream with end-of-stream flush and `[DONE]` handling
- usage estimation fallback when provider usage metadata is missing
## 4) Cloud Sync Degradation
- sync errors are surfaced but local runtime continues
- scheduler has retry-capable logic, but periodic execution currently calls single-attempt sync by default
## 5) Data Integrity
- SQLite schema migrations and auto-upgrade hooks at startup
- legacy JSON → SQLite migration compatibility path
## 6) SSRF / Outbound URL Guard
- `src/shared/network/outboundUrlGuard.ts` blocks all private/loopback/link-local target URLs before they reach provider executors
- Provider model discovery and validation routes use `src/shared/network/safeOutboundFetch.ts` which applies the guard before every outbound request
- Guard errors surface as `URL_GUARD_BLOCKED` with HTTP 422 and are logged to the compliance audit trail via `providerAudit.ts`
## Observability and Operational Signals
Runtime visibility sources:
- console logs from `src/sse/utils/logger.ts`
- per-request usage aggregates in SQLite (`usage_history`, `call_logs`, `proxy_logs`)
- four-stage detailed payload captures in SQLite (`request_detail_logs`) when `settings.detailed_logs_enabled=true`
- textual request status log in `log.txt` (optional/compat)
- optional application log files under `logs/` when `APP_LOG_TO_FILE=true`
- optional request artifacts under `${DATA_DIR}/call_logs/` when the call log pipeline is enabled
- dashboard usage endpoints (`/api/usage/*`) for UI consumption
Detailed request payload capture stores up to four JSON payload stages per routed call:
- raw request received from the client
- translated request actually sent upstream
- provider response reconstructed as JSON; streamed responses are compacted to the final summary plus stream metadata
- final client response returned by OmniRoute; streamed responses are stored in the same compact summary form
## Security-Sensitive Boundaries
- JWT secret (`JWT_SECRET`) secures dashboard session cookie verification/signing
- Initial password bootstrap (`INITIAL_PASSWORD`) should be explicitly configured for first-run provisioning
- API key HMAC secret (`API_KEY_SECRET`) secures generated local API key format
- Provider secrets (API keys/tokens) are persisted in local DB and should be protected at filesystem level
- Cloud sync endpoints rely on API key auth + machine id semantics
## Environment and Runtime Matrix
Environment variables actively used by code:
- App/auth: `JWT_SECRET`, `INITIAL_PASSWORD`
- Storage: `DATA_DIR`
- Compatible node behavior: `ALLOW_MULTI_CONNECTIONS_PER_COMPAT_NODE`
- Optional storage base override (Linux/macOS when `DATA_DIR` unset): `XDG_CONFIG_HOME`
- Security hashing: `API_KEY_SECRET`, `MACHINE_ID_SALT`
- Logging: `APP_LOG_TO_FILE`, `APP_LOG_RETENTION_DAYS`, `CALL_LOG_RETENTION_DAYS`
- Sync/cloud URLing: `NEXT_PUBLIC_BASE_URL`, `NEXT_PUBLIC_CLOUD_URL`
- Outbound proxy: `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, `NO_PROXY` and lowercase variants
- SOCKS5 feature flags: `ENABLE_SOCKS5_PROXY`, `NEXT_PUBLIC_ENABLE_SOCKS5_PROXY`
- Platform/runtime helpers (not app-specific config): `APPDATA`, `NODE_ENV`, `PORT`, `HOSTNAME`
## Known Architectural Notes
1. `usageDb` and `localDb` share the same base directory policy (`DATA_DIR` -> `XDG_CONFIG_HOME/omniroute` -> `~/.omniroute`) with legacy file migration.
2. `/api/v1/route.ts` delegates to the same unified catalog builder used by `/api/v1/models` (`src/app/api/v1/models/catalog.ts`) to avoid semantic drift.
3. Request logger writes full headers/body when enabled; treat log directory as sensitive.
4. Cloud behavior depends on correct `NEXT_PUBLIC_BASE_URL` and cloud endpoint reachability.
5. The `open-sse/` directory is published as the `@omniroute/open-sse` **npm workspace package**. Source code imports it via `@omniroute/open-sse/...` (resolved by Next.js `transpilePackages`). File paths in this document still use the directory name `open-sse/` for consistency.
6. Charts in the dashboard use **Recharts** (SVG-based) for accessible, interactive analytics visualizations (model usage bar charts, provider breakdown tables with success rates).
7. E2E tests use **Playwright** (`tests/e2e/`), run via `npm run test:e2e`. Unit tests use **Node.js test runner** (`tests/unit/`), run via `npm run test:unit`. Source code under `src/` is **TypeScript** (`.ts`/`.tsx`); the `open-sse/` workspace remains JavaScript (`.js`).
8. Settings page is organized into 7 tabs: General, Appearance, AI, Security, Routing, Resilience, Advanced. The Resilience page only configures request queue, connection cooldown, provider breaker, and wait-for-cooldown behavior; live breaker runtime state is shown on the Health page.
9. **Context Relay** strategy (`context-relay`) is split across two layers: `combo.ts` decides if a handoff should be generated, `chat.ts` injects the handoff after account resolution. Handoff data lives in `context_handoffs` SQLite table. This split is intentional because only `chat.ts` knows whether the actual account changed.
10. **Proxy enforcement** is now comprehensive: `tokenHealthCheck.ts` resolves proxy per connection, `/api/providers/validate` uses `runWithProxyContext`, and `proxyFetch.ts` uses `undici.fetch()` to maintain dispatcher compatibility on Node 22.
11. **Node.js runtime policy detection**: `/api/settings/require-login` returns `nodeVersion` and `nodeCompatible` fields. The login page renders a warning banner when the runtime falls outside the supported secure Node.js lines.
## Operational Verification Checklist
- Build from source: `npm run build`
- Build Docker image: `docker build -t omniroute .`
- Start service and verify:
- `GET /api/settings`
- `GET /api/v1/models`
- CLI target base URL should be `http://<host>:20128/v1` when `PORT=20128`
docs/i18n/ar/docs/architecture/CODEBASE_DOCUMENTATION.md
+587
View File
@@ -0,0 +1,587 @@
# omniroute — Codebase Documentation (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../../docs/CODEBASE_DOCUMENTATION.md) · 🇸🇦 [ar](../../ar/docs/CODEBASE_DOCUMENTATION.md) · 🇧🇬 [bg](../../bg/docs/CODEBASE_DOCUMENTATION.md) · 🇧🇩 [bn](../../bn/docs/CODEBASE_DOCUMENTATION.md) · 🇨🇿 [cs](../../cs/docs/CODEBASE_DOCUMENTATION.md) · 🇩🇰 [da](../../da/docs/CODEBASE_DOCUMENTATION.md) · 🇩🇪 [de](../../de/docs/CODEBASE_DOCUMENTATION.md) · 🇪🇸 [es](../../es/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇷 [fa](../../fa/docs/CODEBASE_DOCUMENTATION.md) · 🇫🇮 [fi](../../fi/docs/CODEBASE_DOCUMENTATION.md) · 🇫🇷 [fr](../../fr/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇳 [gu](../../gu/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇱 [he](../../he/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇳 [hi](../../hi/docs/CODEBASE_DOCUMENTATION.md) · 🇭🇺 [hu](../../hu/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇩 [id](../../id/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇹 [it](../../it/docs/CODEBASE_DOCUMENTATION.md) · 🇯🇵 [ja](../../ja/docs/CODEBASE_DOCUMENTATION.md) · 🇰🇷 [ko](../../ko/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇳 [mr](../../mr/docs/CODEBASE_DOCUMENTATION.md) · 🇲🇾 [ms](../../ms/docs/CODEBASE_DOCUMENTATION.md) · 🇳🇱 [nl](../../nl/docs/CODEBASE_DOCUMENTATION.md) · 🇳🇴 [no](../../no/docs/CODEBASE_DOCUMENTATION.md) · 🇵🇭 [phi](../../phi/docs/CODEBASE_DOCUMENTATION.md) · 🇵🇱 [pl](../../pl/docs/CODEBASE_DOCUMENTATION.md) · 🇵🇹 [pt](../../pt/docs/CODEBASE_DOCUMENTATION.md) · 🇧🇷 [pt-BR](../../pt-BR/docs/CODEBASE_DOCUMENTATION.md) · 🇷🇴 [ro](../../ro/docs/CODEBASE_DOCUMENTATION.md) · 🇷🇺 [ru](../../ru/docs/CODEBASE_DOCUMENTATION.md) · 🇸🇰 [sk](../../sk/docs/CODEBASE_DOCUMENTATION.md) · 🇸🇪 [sv](../../sv/docs/CODEBASE_DOCUMENTATION.md) · 🇰🇪 [sw](../../sw/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇳 [ta](../../ta/docs/CODEBASE_DOCUMENTATION.md) · 🇮🇳 [te](../../te/docs/CODEBASE_DOCUMENTATION.md) · 🇹🇭 [th](../../th/docs/CODEBASE_DOCUMENTATION.md) · 🇹🇷 [tr](../../tr/docs/CODEBASE_DOCUMENTATION.md) · 🇺🇦 [uk-UA](../../uk-UA/docs/CODEBASE_DOCUMENTATION.md) · 🇵🇰 [ur](../../ur/docs/CODEBASE_DOCUMENTATION.md) · 🇻🇳 [vi](../../vi/docs/CODEBASE_DOCUMENTATION.md) · 🇨🇳 [zh-CN](../../zh-CN/docs/CODEBASE_DOCUMENTATION.md)
---
> A comprehensive, beginner-friendly guide to the **omniroute** multi-provider AI proxy router.
---
## 1. What Is omniroute?
omniroute is a **proxy router** that sits between AI clients (Claude CLI, Codex, Cursor IDE, etc.) and AI providers (Anthropic, Google, OpenAI, AWS, GitHub, etc.). It solves one big problem:
> **Different AI clients speak different "languages" (API formats), and different AI providers expect different "languages" too.** omniroute translates between them automatically.
Think of it like a universal translator at the United Nations — any delegate can speak any language, and the translator converts it for any other delegate.
---
## 2. Architecture Overview
```mermaid
graph LR
subgraph Clients
A[Claude CLI]
B[Codex]
C[Cursor IDE]
D[OpenAI-compatible]
end
subgraph omniroute
E[Handler Layer]
F[Translator Layer]
G[Executor Layer]
H[Services Layer]
end
subgraph Providers
I[Anthropic Claude]
J[Google Gemini]
K[OpenAI / Codex]
L[GitHub Copilot]
M[AWS Kiro]
N[Antigravity]
O[Cursor API]
end
A --> E
B --> E
C --> E
D --> E
E --> F
F --> G
G --> I
G --> J
G --> K
G --> L
G --> M
G --> N
G --> O
H -.-> E
H -.-> G
```
### Core Principle: Hub-and-Spoke Translation
All format translation passes through **OpenAI format as the hub**:
```
Client Format → [OpenAI Hub] → Provider Format (request)
Provider Format → [OpenAI Hub] → Client Format (response)
```
This means you only need **N translators** (one per format) instead of **N²** (every pair).
---
## 3. Project Structure
```
omniroute/
├── open-sse/ ← Core proxy library (portable, framework-agnostic)
│ ├── index.js ← Main entry point, exports everything
│ ├── config/ ← Configuration & constants
│ ├── executors/ ← Provider-specific request execution
│ ├── handlers/ ← Request handling orchestration
│ ├── services/ ← Business logic (auth, models, fallback, usage)
│ ├── translator/ ← Format translation engine
│ │ ├── request/ ← Request translators (8 files)
│ │ ├── response/ ← Response translators (7 files)
│ │ └── helpers/ ← Shared translation utilities (6 files)
│ └── utils/ ← Utility functions
├── src/ ← Application layer (Express/Worker runtime)
│ ├── app/ ← Web UI, API routes, middleware
│ ├── lib/ ← Database, auth, and shared library code
│ ├── mitm/ ← Man-in-the-middle proxy utilities
│ ├── models/ ← Database models
│ ├── shared/ ← Shared utilities (wrappers around open-sse)
│ ├── sse/ ← SSE endpoint handlers
│ └── store/ ← State management
├── data/ ← Runtime data (credentials, logs)
│ └── provider-credentials.json (external credentials override, gitignored)
└── tester/ ← Test utilities
```
---
## 4. Module-by-Module Breakdown
### 4.1 Config (`open-sse/config/`)
The **single source of truth** for all provider configuration.
| File | Purpose |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `constants.ts` | `PROVIDERS` object with base URLs, OAuth credentials (defaults), headers, and default system prompts for every provider. Also defines `HTTP_STATUS`, `ERROR_TYPES`, `COOLDOWN_MS`, `BACKOFF_CONFIG`, and `SKIP_PATTERNS`. |
| `credentialLoader.ts` | Loads external credentials from `data/provider-credentials.json` and merges them over the hardcoded defaults in `PROVIDERS`. Keeps secrets out of source control while maintaining backwards compatibility. |
| `providerModels.ts` | Central model registry: maps provider aliases → model IDs. Functions like `getModels()`, `getProviderByAlias()`. |
| `codexInstructions.ts` | System instructions injected into Codex requests (editing constraints, sandbox rules, approval policies). |
| `defaultThinkingSignature.ts` | Default "thinking" signatures for Claude and Gemini models. |
| `ollamaModels.ts` | Schema definition for local Ollama models (name, size, family, quantization). |
#### Credential Loading Flow
```mermaid
flowchart TD
A["App starts"] --> B["constants.ts defines PROVIDERS\nwith hardcoded defaults"]
B --> C{"data/provider-credentials.json\nexists?"}
C -->|Yes| D["credentialLoader reads JSON"]
C -->|No| E["Use hardcoded defaults"]
D --> F{"For each provider in JSON"}
F --> G{"Provider exists\nin PROVIDERS?"}
G -->|No| H["Log warning, skip"]
G -->|Yes| I{"Value is object?"}
I -->|No| J["Log warning, skip"]
I -->|Yes| K["Merge clientId, clientSecret,\ntokenUrl, authUrl, refreshUrl"]
K --> F
H --> F
J --> F
F -->|Done| L["PROVIDERS ready with\nmerged credentials"]
E --> L
```
---
### 4.2 Executors (`open-sse/executors/`)
Executors encapsulate **provider-specific logic** using the **Strategy Pattern**. Each executor overrides base methods as needed.
```mermaid
classDiagram
class BaseExecutor {
+buildUrl(model, stream, options)
+buildHeaders(credentials, stream, body)
+transformRequest(body, model, stream, credentials)
+execute(url, options)
+shouldRetry(status, error)
+refreshCredentials(credentials, log)
}
class DefaultExecutor {
+refreshCredentials()
}
class AntigravityExecutor {
+buildUrl()
+buildHeaders()
+transformRequest()
+shouldRetry()
+refreshCredentials()
}
class CursorExecutor {
+buildUrl()
+buildHeaders()
+transformRequest()
+parseResponse()
+generateChecksum()
}
class KiroExecutor {
+buildUrl()
+buildHeaders()
+transformRequest()
+parseEventStream()
+refreshCredentials()
}
BaseExecutor <|-- DefaultExecutor
BaseExecutor <|-- AntigravityExecutor
BaseExecutor <|-- CursorExecutor
BaseExecutor <|-- KiroExecutor
BaseExecutor <|-- CodexExecutor
BaseExecutor <|-- GithubExecutor
```
| Executor | Provider | Key Specializations |
| ---------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------- |
| `base.ts` | — | Abstract base: URL building, headers, retry logic, credential refresh |
| `default.ts` | Claude, Gemini, OpenAI, GLM, Kimi, MiniMax | Generic OAuth token refresh for standard providers |
| `antigravity.ts` | Google Cloud Code | Project/session ID generation, multi-URL fallback, custom retry parsing from error messages ("reset after 2h7m23s") |
| `cursor.ts` | Cursor IDE | **Most complex**: SHA-256 checksum auth, Protobuf request encoding, binary EventStream → SSE response parsing |
| `codex.ts` | OpenAI Codex | Injects system instructions, manages thinking levels, removes unsupported parameters |
| `github.ts` | GitHub Copilot | Dual token system (GitHub OAuth + Copilot token), VSCode header mimicking |
| `kiro.ts` | AWS CodeWhisperer | AWS EventStream binary parsing, AMZN event frames, token estimation |
| `index.ts` | — | Factory: maps provider name → executor class, with default fallback |
---
### 4.3 Handlers (`open-sse/handlers/`)
The **orchestration layer** — coordinates translation, execution, streaming, and error handling.
| File | Purpose |
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `chatCore.ts` | **Central orchestrator** (~600 lines). Handles the complete request lifecycle: format detection → translation → executor dispatch → streaming/non-streaming response → token refresh → error handling → usage logging. |
| `responsesHandler.ts` | Adapter for OpenAI's Responses API: converts Responses format → Chat Completions → sends to `chatCore` → converts SSE back to Responses format. |
| `embeddings.ts` | Embedding generation handler: resolves embedding model → provider, dispatches to provider API, returns OpenAI-compatible embedding response. Supports 6+ providers. |
| `imageGeneration.ts` | Image generation handler: resolves image model → provider, supports OpenAI-compatible, Gemini-image (Antigravity), and fallback (Nebius) modes. Returns base64 or URL images. |
#### Request Lifecycle (chatCore.ts)
```mermaid
sequenceDiagram
participant Client
participant chatCore
participant Translator
participant Executor
participant Provider
Client->>chatCore: Request (any format)
chatCore->>chatCore: Detect source format
chatCore->>chatCore: Check bypass patterns
chatCore->>chatCore: Resolve model & provider
chatCore->>Translator: Translate request (source → OpenAI → target)
chatCore->>Executor: Get executor for provider
Executor->>Executor: Build URL, headers, transform request
Executor->>Executor: Refresh credentials if needed
Executor->>Provider: HTTP fetch (streaming or non-streaming)
alt Streaming
Provider-->>chatCore: SSE stream
chatCore->>chatCore: Pipe through SSE transform stream
Note over chatCore: Transform stream translates<br/>each chunk: target → OpenAI → source
chatCore-->>Client: Translated SSE stream
else Non-streaming
Provider-->>chatCore: JSON response
chatCore->>Translator: Translate response
chatCore-->>Client: Translated JSON
end
alt Error (401, 429, 500...)
chatCore->>Executor: Retry with credential refresh
chatCore->>chatCore: Account fallback logic
end
```
---
### 4.4 Services (`open-sse/services/`)
Business logic that supports the handlers and executors.
| File | Purpose |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `provider.ts` | **Format detection** (`detectFormat`): analyzes request body structure to identify Claude/OpenAI/Gemini/Antigravity/Responses formats (includes `max_tokens` heuristic for Claude). Also: URL building, header building, thinking config normalization. Supports `openai-compatible-*` and `anthropic-compatible-*` dynamic providers. |
| `model.ts` | Model string parsing (`claude/model-name` → `{provider: "claude", model: "model-name"}`), alias resolution with collision detection, input sanitization (rejects path traversal/control chars), and model info resolution with async alias getter support. |
| `accountFallback.ts` | Rate-limit handling: exponential backoff (1s → 2s → 4s → max 2min), account cooldown management, error classification (which errors trigger fallback vs. not). |
| `tokenRefresh.ts` | OAuth token refresh for **every provider**: Google (Gemini, Antigravity), Claude, Codex, Qwen, Qoder, GitHub (OAuth + Copilot dual-token), Kiro (AWS SSO OIDC + Social Auth). Includes in-flight promise deduplication cache and retry with exponential backoff. |
| `combo.ts` | **Combo models**: chains of fallback models. If model A fails with a fallback-eligible error, try model B, then C, etc. Returns actual upstream status codes. |
| `usage.ts` | Fetches quota/usage data from provider APIs (GitHub Copilot quotas, Antigravity model quotas, Codex rate limits, Kiro usage breakdowns, Claude settings). |
| `accountSelector.ts` | Smart account selection with scoring algorithm: considers priority, health status, round-robin position, and cooldown state to pick the optimal account for each request. |
| `contextManager.ts` | Request context lifecycle management: creates and tracks per-request context objects with metadata (request ID, timestamps, provider info) for debugging and logging. |
| `ipFilter.ts` | IP-based access control: supports allowlist and blocklist modes. Validates client IP against configured rules before processing API requests. |
| `sessionManager.ts` | Session tracking with client fingerprinting: tracks active sessions using hashed client identifiers, monitors request counts, and provides session metrics. |
| `signatureCache.ts` | Request signature-based deduplication cache: prevents duplicate requests by caching recent request signatures and returning cached responses for identical requests within a time window. |
| `systemPrompt.ts` | Global system prompt injection: prepends or appends a configurable system prompt to all requests, with per-provider compatibility handling. |
| `thinkingBudget.ts` | Reasoning token budget management: supports passthrough, auto (strip thinking config), custom (fixed budget), and adaptive (complexity-scaled) modes for controlling thinking/reasoning tokens. |
| `wildcardRouter.ts` | Wildcard model pattern routing: resolves wildcard patterns (e.g., `*/claude-*`) to concrete provider/model pairs based on availability and priority. |
#### Token Refresh Deduplication
```mermaid
sequenceDiagram
participant R1 as Request 1
participant R2 as Request 2
participant Cache as refreshPromiseCache
participant OAuth as OAuth Provider
R1->>Cache: getAccessToken("gemini", token)
Cache->>Cache: No in-flight promise
Cache->>OAuth: Start refresh
R2->>Cache: getAccessToken("gemini", token)
Cache->>Cache: Found in-flight promise
Cache-->>R2: Return existing promise
OAuth-->>Cache: New access token
Cache-->>R1: New access token
Cache-->>R2: Same access token (shared)
Cache->>Cache: Delete cache entry
```
#### Account Fallback State Machine
```mermaid
stateDiagram-v2
[*] --> Active
Active --> Error: Request fails (401/429/500)
Error --> Cooldown: Apply backoff
Cooldown --> Active: Cooldown expires
Active --> Active: Request succeeds (reset backoff)
state Error {
[*] --> ClassifyError
ClassifyError --> ShouldFallback: Rate limit / Auth / Transient
ClassifyError --> NoFallback: 400 Bad Request
}
state Cooldown {
[*] --> ExponentialBackoff
ExponentialBackoff: Level 0 = 1s
ExponentialBackoff: Level 1 = 2s
ExponentialBackoff: Level 2 = 4s
ExponentialBackoff: Max = 2min
}
```
#### Combo Model Chain
```mermaid
flowchart LR
A["Request with\ncombo model"] --> B["Model A"]
B -->|"2xx Success"| C["Return response"]
B -->|"429/401/500"| D{"Fallback\neligible?"}
D -->|Yes| E["Model B"]
D -->|No| F["Return error"]
E -->|"2xx Success"| C
E -->|"429/401/500"| G{"Fallback\neligible?"}
G -->|Yes| H["Model C"]
G -->|No| F
H -->|"2xx Success"| C
H -->|"Fail"| I["All failed →\nReturn last status"]
```
---
### 4.5 Translator (`open-sse/translator/`)
The **format translation engine** using a self-registering plugin system.
#### الهندسة
```mermaid
graph TD
subgraph "Request Translation"
A["Claude → OpenAI"]
B["Gemini → OpenAI"]
C["Antigravity → OpenAI"]
D["OpenAI Responses → OpenAI"]
E["OpenAI → Claude"]
F["OpenAI → Gemini"]
G["OpenAI → Kiro"]
H["OpenAI → Cursor"]
end
subgraph "Response Translation"
I["Claude → OpenAI"]
J["Gemini → OpenAI"]
K["Kiro → OpenAI"]
L["Cursor → OpenAI"]
M["OpenAI → Claude"]
N["OpenAI → Antigravity"]
O["OpenAI → Responses"]
end
```
| Directory | Files | Description |
| ------------ | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `request/` | 8 translators | Convert request bodies between formats. Each file self-registers via `register(from, to, fn)` on import. |
| `response/` | 7 translators | Convert streaming response chunks between formats. Handles SSE event types, thinking blocks, tool calls. |
| `helpers/` | 6 helpers | Shared utilities: `claudeHelper` (system prompt extraction, thinking config), `geminiHelper` (parts/contents mapping), `openaiHelper` (format filtering), `toolCallHelper` (ID generation, missing response injection), `maxTokensHelper`, `responsesApiHelper`. |
| `index.ts` | — | Translation engine: `translateRequest()`, `translateResponse()`, state management, registry. |
| `formats.ts` | — | Format constants: `OPENAI`, `CLAUDE`, `GEMINI`, `ANTIGRAVITY`, `KIRO`, `CURSOR`, `OPENAI_RESPONSES`. |
#### Key Design: Self-Registering Plugins
```javascript
// Each translator file calls register() on import:
import { register } from "../index.js";
register("claude", "openai", translateClaudeToOpenAI);
// The index.js imports all translator files, triggering registration:
import "./request/claude-to-openai.js"; // ← self-registers
```
---
### 4.6 Utils (`open-sse/utils/`)
| File | Purpose |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `error.ts` | Error response building (OpenAI-compatible format), upstream error parsing, Antigravity retry-time extraction from error messages, SSE error streaming. |
| `stream.ts` | **SSE Transform Stream** — the core streaming pipeline. Two modes: `TRANSLATE` (full format translation) and `PASSTHROUGH` (normalize + extract usage). Handles chunk buffering, usage estimation, content length tracking. Per-stream encoder/decoder instances avoid shared state. |
| `streamHelpers.ts` | Low-level SSE utilities: `parseSSELine` (whitespace-tolerant), `hasValuableContent` (filters empty chunks for OpenAI/Claude/Gemini), `fixInvalidId`, `formatSSE` (format-aware SSE serialization with `perf_metrics` cleanup). |
| `usageTracking.ts` | Token usage extraction from any format (Claude/OpenAI/Gemini/Responses), estimation with separate tool/message char-per-token ratios, buffer addition (2000 tokens safety margin), format-specific field filtering, console logging with ANSI colors. |
| `requestLogger.ts` | Legacy file-based request logging helper kept for compatibility. Current deployments should prefer `APP_LOG_TO_FILE` for application logs and the call log pipeline for persisted request artifacts. |
| `bypassHandler.ts` | Intercepts specific patterns from Claude CLI (title extraction, warmup, count) and returns fake responses without calling any provider. Supports both streaming and non-streaming. Intentionally limited to Claude CLI scope. |
| `networkProxy.ts` | Resolves outbound proxy URL for a given provider with precedence: provider-specific config → global config → environment variables (`HTTPS_PROXY`/`HTTP_PROXY`/`ALL_PROXY`). Supports `NO_PROXY` exclusions. Caches config for 30s. |
#### SSE Streaming Pipeline
```mermaid
flowchart TD
A["Provider SSE stream"] --> B["TextDecoder\n(per-stream instance)"]
B --> C["Buffer lines\n(split on newline)"]
C --> D["parseSSELine()\n(trim whitespace, parse JSON)"]
D --> E{"Mode?"}
E -->|TRANSLATE| F["translateResponse()\ntarget → OpenAI → source"]
E -->|PASSTHROUGH| G["fixInvalidId()\nnormalize chunk"]
F --> H["hasValuableContent()\nfilter empty chunks"]
G --> H
H -->|"Has content"| I["extractUsage()\ntrack token counts"]
H -->|"Empty"| J["Skip chunk"]
I --> K["formatSSE()\nserialize + clean perf_metrics"]
K --> L["TextEncoder\n(per-stream instance)"]
L --> M["Enqueue to\nclient stream"]
style A fill:#f9f,stroke:#333
style M fill:#9f9,stroke:#333
```
#### Request Logger Session Structure
```
logs/
└── claude_gemini_claude-sonnet_20260208_143045/
├── 1_req_client.json ← Raw client request
├── 2_req_source.json ← After initial conversion
├── 3_req_openai.json ← OpenAI intermediate format
├── 4_req_target.json ← Final target format
├── 5_res_provider.txt ← Provider SSE chunks (streaming)
├── 5_res_provider.json ← Provider response (non-streaming)
├── 6_res_openai.txt ← OpenAI intermediate chunks
├── 7_res_client.txt ← Client-facing SSE chunks
└── 6_error.json ← Error details (if any)
```
---
### 4.7 Application Layer (`src/`)
| Directory | Purpose |
| ------------- | ---------------------------------------------------------------------- |
| `src/app/` | Web UI, API routes, Express middleware, OAuth callback handlers |
| `src/lib/` | Database access (`localDb.ts`, `usageDb.ts`), authentication, shared |
| `src/mitm/` | Man-in-the-middle proxy utilities for intercepting provider traffic |
| `src/models/` | Database model definitions |
| `src/shared/` | Wrappers around open-sse functions (provider, stream, error, etc.) |
| `src/sse/` | SSE endpoint handlers that wire the open-sse library to Express routes |
| `src/store/` | Application state management |
#### Notable API Routes
| Route | Methods | Purpose |
| --------------------------------------------- | --------------- | ------------------------------------------------------------------------------------- |
| `/api/provider-models` | GET/POST/DELETE | CRUD for custom models per provider |
| `/api/models/catalog` | GET | Aggregated catalog of all models (chat, embedding, image, custom) grouped by provider |
| `/api/settings/proxy` | GET/PUT/DELETE | Hierarchical outbound proxy configuration (`global/providers/combos/keys`) |
| `/api/settings/proxy/test` | POST | Validates proxy connectivity and returns public IP/latency |
| `/v1/providers/[provider]/chat/completions` | POST | Dedicated per-provider chat completions with model validation |
| `/v1/providers/[provider]/embeddings` | POST | Dedicated per-provider embeddings with model validation |
| `/v1/providers/[provider]/images/generations` | POST | Dedicated per-provider image generation with model validation |
| `/api/settings/ip-filter` | GET/PUT | IP allowlist/blocklist management |
| `/api/settings/thinking-budget` | GET/PUT | Reasoning token budget configuration (passthrough/auto/custom/adaptive) |
| `/api/settings/system-prompt` | GET/PUT | Global system prompt injection for all requests |
| `/api/sessions` | GET | Active session tracking and metrics |
| `/api/rate-limits` | GET | Per-account rate limit status |
---
## 5. Key Design Patterns
### 5.1 Hub-and-Spoke Translation
All formats translate through **OpenAI format as the hub**. Adding a new provider only requires writing **one pair** of translators (to/from OpenAI), not N pairs.
### 5.2 Executor Strategy Pattern
Each provider has a dedicated executor class inheriting from `BaseExecutor`. The factory in `executors/index.ts` selects the right one at runtime.
### 5.3 Self-Registering Plugin System
Translator modules register themselves on import via `register()`. Adding a new translator is just creating a file and importing it.
### 5.4 Account Fallback with Exponential Backoff
When a provider returns 429/401/500, the system can switch to the next account, applying exponential cooldowns (1s → 2s → 4s → max 2min).
### 5.5 Combo Model Chains
A "combo" groups multiple `provider/model` strings. If the first fails, fallback to the next automatically.
### 5.6 Stateful Streaming Translation
Response translation maintains state across SSE chunks (thinking block tracking, tool call accumulation, content block indexing) via the `initState()` mechanism.
### 5.7 Usage Safety Buffer
A 2000-token buffer is added to reported usage to prevent clients from hitting context window limits due to overhead from system prompts and format translation.
---
## 6. Supported Formats
| Format | Direction | Identifier |
| ----------------------- | --------------- | ------------------ |
| OpenAI Chat Completions | source + target | `openai` |
| OpenAI Responses API | source + target | `openai-responses` |
| Anthropic Claude | source + target | `claude` |
| Google Gemini | source + target | `gemini` |
| Antigravity | source + target | `antigravity` |
| AWS Kiro | target only | `kiro` |
| Cursor | target only | `cursor` |
---
## 7. Supported Providers
| Provider | Auth Method | Executor | Key Notes |
| ------------------------ | ---------------------- | ----------- | --------------------------------------------- |
| Anthropic Claude | API key or OAuth | Default | Uses `x-api-key` header |
| Google Gemini | API key or OAuth | Default | Uses `x-goog-api-key` header |
| Antigravity | OAuth | Antigravity | Multi-URL fallback, custom retry parsing |
| OpenAI | API key | Default | Standard Bearer auth |
| Codex | OAuth | Codex | Injects system instructions, manages thinking |
| GitHub Copilot | OAuth + Copilot token | Github | Dual token, VSCode header mimicking |
| Kiro (AWS) | AWS SSO OIDC or Social | Kiro | Binary EventStream parsing |
| Cursor IDE | Checksum auth | Cursor | Protobuf encoding, SHA-256 checksums |
| Qwen | OAuth | Default | Standard auth |
| Qoder | OAuth (Basic + Bearer) | Default | Dual auth header |
| OpenRouter | API key | Default | Standard Bearer auth |
| GLM, Kimi, MiniMax | API key | Default | Claude-compatible, use `x-api-key` |
| `openai-compatible-*` | API key | Default | Dynamic: any OpenAI-compatible endpoint |
| `anthropic-compatible-*` | API key | Default | Dynamic: any Claude-compatible endpoint |
---
## 8. Data Flow Summary
### Streaming Request
```mermaid
flowchart LR
A["Client"] --> B["detectFormat()"]
B --> C["translateRequest()\nsource → OpenAI → target"]
C --> D["Executor\nbuildUrl + buildHeaders"]
D --> E["fetch(providerURL)"]
E --> F["createSSEStream()\nTRANSLATE mode"]
F --> G["parseSSELine()"]
G --> H["translateResponse()\ntarget → OpenAI → source"]
H --> I["extractUsage()\n+ addBuffer"]
I --> J["formatSSE()"]
J --> K["Client receives\ntranslated SSE"]
K --> L["logUsage()\nsaveRequestUsage()"]
```
### Non-Streaming Request
```mermaid
flowchart LR
A["Client"] --> B["detectFormat()"]
B --> C["translateRequest()\nsource → OpenAI → target"]
C --> D["Executor.execute()"]
D --> E["translateResponse()\ntarget → OpenAI → source"]
E --> F["Return JSON\nresponse"]
```
### Bypass Flow (Claude CLI)
```mermaid
flowchart LR
A["Claude CLI request"] --> B{"Match bypass\npattern?"}
B -->|"Title/Warmup/Count"| C["Generate fake\nOpenAI response"]
B -->|"No match"| D["Normal flow"]
C --> E["Translate to\nsource format"]
E --> F["Return without\ncalling provider"]
```
docs/i18n/ar/docs/cloudflare-zero-trust-guide.md
+106
View File
@@ -0,0 +1,106 @@
# Guia Completo: Cloudflare Tunnel & Zero Trust (Split-Port) (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../../docs/cloudflare-zero-trust-guide.md) · 🇪🇸 [es](../../es/docs/cloudflare-zero-trust-guide.md) · 🇫🇷 [fr](../../fr/docs/cloudflare-zero-trust-guide.md) · 🇩🇪 [de](../../de/docs/cloudflare-zero-trust-guide.md) · 🇮🇹 [it](../../it/docs/cloudflare-zero-trust-guide.md) · 🇷🇺 [ru](../../ru/docs/cloudflare-zero-trust-guide.md) · 🇨🇳 [zh-CN](../../zh-CN/docs/cloudflare-zero-trust-guide.md) · 🇯🇵 [ja](../../ja/docs/cloudflare-zero-trust-guide.md) · 🇰🇷 [ko](../../ko/docs/cloudflare-zero-trust-guide.md) · 🇸🇦 [ar](../../ar/docs/cloudflare-zero-trust-guide.md) · 🇮🇳 [hi](../../hi/docs/cloudflare-zero-trust-guide.md) · 🇮🇳 [in](../../in/docs/cloudflare-zero-trust-guide.md) · 🇹🇭 [th](../../th/docs/cloudflare-zero-trust-guide.md) · 🇻🇳 [vi](../../vi/docs/cloudflare-zero-trust-guide.md) · 🇮🇩 [id](../../id/docs/cloudflare-zero-trust-guide.md) · 🇲🇾 [ms](../../ms/docs/cloudflare-zero-trust-guide.md) · 🇳🇱 [nl](../../nl/docs/cloudflare-zero-trust-guide.md) · 🇵🇱 [pl](../../pl/docs/cloudflare-zero-trust-guide.md) · 🇸🇪 [sv](../../sv/docs/cloudflare-zero-trust-guide.md) · 🇳🇴 [no](../../no/docs/cloudflare-zero-trust-guide.md) · 🇩🇰 [da](../../da/docs/cloudflare-zero-trust-guide.md) · 🇫🇮 [fi](../../fi/docs/cloudflare-zero-trust-guide.md) · 🇵🇹 [pt](../../pt/docs/cloudflare-zero-trust-guide.md) · 🇷🇴 [ro](../../ro/docs/cloudflare-zero-trust-guide.md) · 🇭🇺 [hu](../../hu/docs/cloudflare-zero-trust-guide.md) · 🇧🇬 [bg](../../bg/docs/cloudflare-zero-trust-guide.md) · 🇸🇰 [sk](../../sk/docs/cloudflare-zero-trust-guide.md) · 🇺🇦 [uk-UA](../../uk-UA/docs/cloudflare-zero-trust-guide.md) · 🇮🇱 [he](../../he/docs/cloudflare-zero-trust-guide.md) · 🇵🇭 [phi](../../phi/docs/cloudflare-zero-trust-guide.md) · 🇧🇷 [pt-BR](../../pt-BR/docs/cloudflare-zero-trust-guide.md) · 🇨🇿 [cs](../../cs/docs/cloudflare-zero-trust-guide.md) · 🇹🇷 [tr](../../tr/docs/cloudflare-zero-trust-guide.md)
---
Este guia documenta o padrão ouro de infraestrutura de rede para proteger o **OmniRoute** e expor sua aplicação de forma segura para a internet, **sem abrir nenhuma porta (Zero Inbound)**.
## O que foi feito na sua VM?
Nós ativamos o OmniRoute em modo **Split-Port** através do PM2:
- **Porta \`20128\`:** Roda **apenas a API** `/v1`.
- **Porta \`20129\`:** Roda **apenas o Dashboard** Administrativo visual.
Além disso, o serviço interno exige \`REQUIRE_API_KEY=true\`, o que significa que nenhum agente pode consumir os endpoints da API sem enviar um "Bearer Token" legítimo gerado na aba API Keys do Painel.
Isso nos permite criar duas regras completamente independentes na rede. É aqui que entra o **Cloudflare Tunnel (cloudflared)**.
---
## 1. Como Criar o Túnel na Cloudflare
O utilitário \`cloudflared\` já está instalado na sua máquina. Siga os passos na nuvem:
1. Acesse seu painel **Cloudflare Zero Trust** (One.dash.cloudflare.com).
2. No menu à esquerda, vá em **Networks > Tunnels**.
3. Clique em **Add a Tunnel**, escolha **Cloudflared** e dê o nome \`OmniRoute-VM\`.
4. Ele vai gerar um comando na tela chamado "Install and run a connector". **Você só precisa copiar o Token (a string longa após `--token`)**.
5. Logue via SSH na sua máquina virtual (ou Terminal do Proxmox) e execute:
\`\`\`bash
# Inicia e amarra o túnel permanentemente à sua conta
cloudflared service install SEU_TOKEN_GIGANTE_AQUI
\`\`\`
---
## 2. Configurando o Roteamento (Public Hostnames)
Ainda na tela do Tunnel recém-criado, vá para a aba **Public Hostnames** e adicione as **duas** rotas, aproveitando a separação que fizemos:
### Rota 1: API Segura (Limitada)
- **Subdomain:** \`api\`
- **Domain:** \`seuglobal.com.br\` (escolha seu domínio real)
- **Service Type:** \`HTTP\`
- **URL:** \`127.0.0.1:20128\` _(Porta interna da API)_
### Rota 2: Painel Zero Trust (Fechado)
- **Subdomain:** \`omniroute\` ou \`painel\`
- **Domain:** \`seuglobal.com.br\`
- **Service Type:** \`HTTP\`
- **URL:** \`127.0.0.1:20129\` _(Porta interna do App/Visual)_
Neste momento, a conectividade "Física" está resolvida. Agora vamos blindar de verdade.
---
## 3. Blindando o Painel com Zero Trust (Access)
Nenhuma senha local protege melhor o seu painel do que remover totalmente o acesso a ele da internet aberta.
1. No painel Zero Trust, vá em **Access > Applications > Add an application**.
2. Selecione **Self-hosted**.
3. Em **Application name**, coloque \`Painel OmniRoute\`.
4. Em **Application domain**, coloque \`omniroute.seuglobal.com.br\` (O mesmo que você fez na "Rota 2").
5. Clique em **Next**.
6. Em **Rule action**, escolha \`Allow\`. Em nome da Rule coloque \`Admin Apenas\`.
7. Em **Include**, no seletor de "Selector" escolha \`Emails\` e digite o seu email, por exemplo \`admin@spgeo.com.br\`.
8. Salve (`Add application`).
> **O que isso fez:** Se você tentar abrir \`omniroute.seuglobal.com.br\`, não cai mais na sua aplicação OmniRoute! Cai numa tela elegante da Cloudflare pedindo para digitar seu email. Somente se você (ou o email que você botou) for digitado lá, ele recebe no Outlook/Gmail um código de 6 dígitos temporário que libera o túnel até a porta \`20129\`.
---
## 4. Limitando e Protegendo a API com Rate Limit (WAF)
O Dashboard do Zero Trust não se aplica à rota da API (\`api.seuglobal.com.br\`), porque é um acesso programático via ferramentas automatizadas (agentes) sem navegador. Para ele, usaremos o Firewall principal (WAF) da Cloudflare.
1. Acesse o **Painel Normal** da Cloudflare (dash.cloudflare.com) e entre no seu Domínio.
2. No menu esquerdo, vá em **Security > WAF > Rate limiting rules**.
3. Clique em **Create rule**.
4. **Name:** \`Anti-Abuso OmniRoute API\`
5. **If incoming requests match...**
- Escolha em Field: \`Hostname\`
- Operator: \`equals\`
- Value: \`api.seuglobal.com.br\`
6. Em **With the same characteristics:** Mantenha \`IP\`.
7. Nos limites (Limit):
- **When requests exceed:** \`50\`
- **Period:** \`1 minute\`
8. No final, em **Action**: \`Block\` (Bloquear) e decida se o bloqueio dura por 1 minuto ou 1 hora.
9. **Deploy**.
> **O que isso fez:** Ninguém pode mandar mais de 50 requisições num período de 60 segundos na sua URL de API. Como você roda vários agentes e os consumos por trás já batem rate limit e já rastreiam tokens, isso é apenas uma medida na Borda da Internet (Edge Layer) que protege sua Instância On-Premises de cair por estresse térmico antes mesmo do tráfego descer pelo túnel.
---
## Finalização
1. A sua VM **não possui nenhuma porta exposta** em `/etc/ufw`.
2. O OmniRoute só conversa HTTPS saindo (\`cloudflared\`) e não recebendo TCP direto do mundo.
3. Seus requets pro OpenAI são ofuscados porque configuramos eles globalmente pra passar em um Proxy SOCKS5 (A nuvem não liga pro SOCKS5 porque ela vem Inbound).
4. Seu painel web tem 2-Factor com Email.
5. Sua API está ratelimitada na borda pela Cloudflare e só trafega Bearer Tokens.
docs/i18n/ar/docs/features/context-relay.md
+130
View File
@@ -0,0 +1,130 @@
# Context Relay (العربية)
🌐 **Languages:** 🇺🇸 [English](../../../../../docs/features/context-relay.md) · 🇪🇸 [es](../../../es/docs/features/context-relay.md) · 🇫🇷 [fr](../../../fr/docs/features/context-relay.md) · 🇩🇪 [de](../../../de/docs/features/context-relay.md) · 🇮🇹 [it](../../../it/docs/features/context-relay.md) · 🇷🇺 [ru](../../../ru/docs/features/context-relay.md) · 🇨🇳 [zh-CN](../../../zh-CN/docs/features/context-relay.md) · 🇯🇵 [ja](../../../ja/docs/features/context-relay.md) · 🇰🇷 [ko](../../../ko/docs/features/context-relay.md) · 🇸🇦 [ar](../../../ar/docs/features/context-relay.md) · 🇮🇳 [hi](../../../hi/docs/features/context-relay.md) · 🇮🇳 [in](../../../in/docs/features/context-relay.md) · 🇹🇭 [th](../../../th/docs/features/context-relay.md) · 🇻🇳 [vi](../../../vi/docs/features/context-relay.md) · 🇮🇩 [id](../../../id/docs/features/context-relay.md) · 🇲🇾 [ms](../../../ms/docs/features/context-relay.md) · 🇳🇱 [nl](../../../nl/docs/features/context-relay.md) · 🇵🇱 [pl](../../../pl/docs/features/context-relay.md) · 🇸🇪 [sv](../../../sv/docs/features/context-relay.md) · 🇳🇴 [no](../../../no/docs/features/context-relay.md) · 🇩🇰 [da](../../../da/docs/features/context-relay.md) · 🇫🇮 [fi](../../../fi/docs/features/context-relay.md) · 🇵🇹 [pt](../../../pt/docs/features/context-relay.md) · 🇷🇴 [ro](../../../ro/docs/features/context-relay.md) · 🇭🇺 [hu](../../../hu/docs/features/context-relay.md) · 🇧🇬 [bg](../../../bg/docs/features/context-relay.md) · 🇸🇰 [sk](../../../sk/docs/features/context-relay.md) · 🇺🇦 [uk-UA](../../../uk-UA/docs/features/context-relay.md) · 🇮🇱 [he](../../../he/docs/features/context-relay.md) · 🇵🇭 [phi](../../../phi/docs/features/context-relay.md) · 🇧🇷 [pt-BR](../../../pt-BR/docs/features/context-relay.md) · 🇨🇿 [cs](../../../cs/docs/features/context-relay.md) · 🇹🇷 [tr](../../../tr/docs/features/context-relay.md)
---
`context-relay` is a combo strategy that keeps session continuity when the active account
rotates before the conversation is finished.
The current runtime behaves like priority routing for model selection, then adds a
handoff layer on top:
- before the active account is exhausted, OmniRoute generates a compact structured summary
- after authentication selects a different account for the same session, OmniRoute injects
that summary as a system message into the next request
- once the handoff is consumed successfully, it is removed from storage
## When To Use It
Use `context-relay` when all of the following are true:
- the combo is expected to rotate between multiple accounts of the same provider
- losing short-term conversational continuity would hurt task quality
- the provider exposes enough quota information to predict an approaching account limit
This is most useful for long-running coding or research sessions that may outlive a single
account window.
## Runtime Flow
The current behavior is intentionally split across two runtime layers.
### 0% to 84% quota used
No handoff is generated. Requests behave like normal priority routing.
### 85% to 94% quota used
If the active provider is enabled in `handoffProviders`, OmniRoute generates a structured
handoff summary in the background before the account is fully exhausted.
Important details:
- the default warning threshold is `0.85`
- the hard stop for generation is `0.95`
- only one in-flight handoff generation is allowed per `sessionId + comboName`
- if an active handoff already exists for that session/combo, no duplicate summary is generated
### 95% or more quota used
No new handoff is generated. At this point the system is already in or near exhaustion and
the runtime avoids scheduling another summary request.
### After account rotation
When the next request for the same session resolves to a different authenticated account,
OmniRoute prepends the stored handoff as a system message. Injection happens only after the
real account switch is known.
## Handoff Payload
The persisted handoff payload is stored in `context_handoffs` and includes:
- `sessionId`
- `comboName`
- `fromAccount`
- `summary`
- `keyDecisions`
- `taskProgress`
- `activeEntities`
- `messageCount`
- `model`
- `warningThresholdPct`
- `generatedAt`
- `expiresAt`
The summary model is instructed to return a JSON object with this structure:
```json
{
"summary": "Dense summary of what matters for continuity",
"keyDecisions": ["Decision 1", "Decision 2"],
"taskProgress": "What is done, what is pending, and the next step",
"activeEntities": ["fileA.ts", "feature X", "provider Y"]
}
```
At injection time, OmniRoute converts that payload into a `<context_handoff>` system
message so the next account can continue with the correct local context.
## الإعداد
`context-relay` supports these config fields:
- `handoffThreshold`: warning threshold for summary generation, default `0.85`
- `handoffModel`: optional model override used only for summary generation
- `handoffProviders`: allowlist of providers allowed to trigger handoff generation
Global defaults can be configured in Settings, and combo-specific values can override them
in the Combos page.
## Architectural Note
The current implementation does not use a standalone `handleContextRelayCombo` handler.
Instead:
- `open-sse/services/combo.ts` decides whether a successful turn should generate a handoff
- `src/sse/handlers/chat.ts` injects the handoff only after authentication resolves the
actual account used for the request
This split is intentional in the current codebase because the combo loop alone does not know
whether the request stayed on the same account or actually switched accounts.
## Limitations
- Effective runtime support is currently centered on `codex` quota rotation.
- `handoffProviders` is already modeled as a config surface, but real handoff generation
still depends on provider-specific quota plumbing.
- The summary is intentionally compact and recent-history based; it is not a full transcript
replay mechanism.
- Handoffs are scoped by `sessionId + comboName` and expire automatically.
- If the session does not switch accounts, the stored handoff is not injected.
## Recommended Usage Pattern
- use multiple accounts from the same provider
- keep stable `sessionId` values across the session
- set `handoffThreshold` early enough to leave room for the background summary request
- treat the feature as continuity assistance, not as a replacement for persistent memory

Some files were not shown because too many files have changed in this diff Show More