wehub/diegosouzapw--omniroute
1
0
Fork 0
Code Issues Pull Requests Actions 7 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:12 +08:00
commit d8dcd5f6d1
8604 changed files with 2479390 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/frameworks/A2A-SERVER.md
+295
View File
@@ -0,0 +1,295 @@
---
title: "OmniRoute A2A Server Documentation"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute A2A Server Documentation
> Agent-to-Agent Protocol v0.3 — OmniRoute as an intelligent routing agent
The A2A surface has two faces:
- **JSON-RPC 2.0** at `POST /a2a` (canonical entry point, defined in `src/app/a2a/route.ts`).
- **REST** under `/api/a2a/*` for dashboards and tooling (status, task list, cancel).
Tasks are tracked by `A2ATaskManager` (`src/lib/a2a/taskManager.ts`, default 5-minute TTL). Skills are dispatched via `A2A_SKILL_HANDLERS` in `src/lib/a2a/taskExecution.ts`.
## Agent Discovery
```bash
curl http://localhost:20128/.well-known/agent.json
```
Returns the Agent Card describing OmniRoute's capabilities, skills, and authentication requirements.
The Agent Card's `version` field is sourced from `process.env.npm_package_version` (see `src/app/.well-known/agent.json/route.ts:13`), so it stays auto-synced with `package.json` on every release.
---
## Authentication
All `/a2a` requests require an API key via the `Authorization` header:
```
Authorization: Bearer YOUR_OMNIROUTE_API_KEY
```
If no API key is configured on the server, authentication is bypassed.
## Enablement
A2A is controlled by the **Endpoints → A2A** toggle and is disabled by default. When disabled,
`GET /api/a2a/status` reports `status: "disabled"` and `online: false`; JSON-RPC calls to
`POST /a2a` return HTTP 503 with JSON-RPC error code `-32000`.
---
## JSON-RPC 2.0 Methods
### `message/send` — Synchronous Execution
Sends a message to a skill and waits for the complete response.
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{
"jsonrpc": "2.0",
"id": "1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Write a hello world in Python"}],
"metadata": {"model": "auto", "combo": "fast-coding"}
}
}'
```
**Response:**
```json
{
"jsonrpc": "2.0",
"id": "1",
"result": {
"task": { "id": "uuid", "state": "completed" },
"artifacts": [{ "type": "text", "content": "..." }],
"metadata": {
"routing_explanation": "Selected claude-sonnet via provider \"anthropic\" (latency: 1200ms, cost: $0.003)",
"cost_envelope": { "estimated": 0.005, "actual": 0.003, "currency": "USD" },
"resilience_trace": [
{ "event": "primary_selected", "provider": "anthropic", "timestamp": "..." }
],
"policy_verdict": { "allowed": true, "reason": "within budget and quota limits" }
}
}
}
```
### `message/stream` — SSE Streaming
Same as `message/send` but returns Server-Sent Events for real-time streaming.
```bash
curl -N -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{
"jsonrpc": "2.0",
"id": "1",
"method": "message/stream",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Explain quantum computing"}]
}
}'
```
**SSE Events:**
```
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"working"},"chunk":{"type":"text","content":"..."}}}
: heartbeat 2026-03-03T17:00:00Z
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"completed"},"metadata":{...}}}
```
### `tasks/get` — Query Task Status
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"jsonrpc":"2.0","id":"2","method":"tasks/get","params":{"taskId":"TASK_UUID"}}'
```
### `tasks/cancel` — Cancel a Task
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"jsonrpc":"2.0","id":"3","method":"tasks/cancel","params":{"taskId":"TASK_UUID"}}'
```
---
## Available Skills
OmniRoute exposes 6 A2A skills wired in `src/lib/a2a/taskExecution.ts::A2A_SKILL_HANDLERS`. Each skill module lives in `src/lib/a2a/skills/`.
| Skill | ID | Description | Tags | Examples |
| :----------------- | :------------------- | :-------------------------------------------------------------------------------------------------------------- | :------------------------- | :------------------------------------- |
| Smart Routing | `smart-routing` | Routes a prompt through the optimal provider/combo using OmniRoute's combo engine + scoring | routing, providers | "Route this prompt via the best model" |
| Quota Management | `quota-management` | Reports per-provider quota state, helps callers decide when to throttle/switch | quota, providers | "Check quota for anthropic" |
| Provider Discovery | `provider-discovery` | Lists installed providers with capabilities, free-tier flags, OAuth status | providers, discovery | "What providers are available?" |
| Cost Analysis | `cost-analysis` | Estimates cost of a request/conversation given the catalog + recent usage | cost, usage | "Estimate cost for this conversation" |
| Health Report | `health-report` | Aggregates circuit breaker, cooldown, lockout state per provider | health, resilience | "Show health status of all providers" |
| List Capabilities | `list-capabilities` | Returns the full 42-entry Agent Skills catalog as a markdown table with raw SKILL.md URLs for context injection | catalog, discovery, skills | "List all OmniRoute capabilities" |
> Note: the Agent Card description currently advertises "36+ providers" (`src/app/.well-known/agent.json/route.ts:26` and `:55`). The actual catalog has grown to 180+ providers — the string should be updated in a follow-up change (tracked as a separate doc/code TODO; not modified here).
### `list-capabilities` Skill Detail
The `list-capabilities` skill is particularly useful for external agents that need to discover what OmniRoute exposes before sending API calls. It returns a structured markdown table artifact:
```
| ID | Name | Category | Area | Endpoints/Commands | Raw URL |
| --- | --- | --- | --- | --- | --- |
| omni-auth | Auth & Sessions | api | auth | POST /api/auth/login, ... | https://raw.githubusercontent.com/... |
...
```
Each row includes the `rawUrl` column so agents can immediately fetch the full SKILL.md. The `metadata.totalSkills` field is always `42`. Implementation: `src/lib/a2a/skills/listCapabilities.ts`. See also [AGENT-SKILLS.md](./AGENT-SKILLS.md).
---
## REST API (auxiliary)
The JSON-RPC endpoint `/a2a` is the canonical A2A entry point. The REST endpoints below provide auxiliary access for dashboards and external tooling:
| Endpoint | Method | Description | Auth |
| :--------------------------- | :----- | :------------------------------- | :--------------------- |
| `/api/a2a/status` | GET | Server status, registered skills | (public) |
| `/api/a2a/tasks` | GET | List tasks with filters | management |
| `/api/a2a/tasks/[id]` | GET | Get task by ID | management |
| `/api/a2a/tasks/[id]/cancel` | POST | Cancel running task | management |
| `/.well-known/agent.json` | GET | Agent Card (A2A discovery) | (public, cached 3600s) |
---
## Adding a New Skill
1. **Create skill file:** `src/lib/a2a/skills/<your-skill>.ts`
Export an async function `(task: A2ATask) => Promise<{ artifacts, metadata }>`. Follow the shape of existing skills such as `smartRouting.ts`.
2. **Register handler:** in `src/lib/a2a/taskExecution.ts`, add an entry to `A2A_SKILL_HANDLERS`:
```typescript
export const A2A_SKILL_HANDLERS = {
// ...existing skills
"your-skill": async (task) => {
const skillModule = await import("./skills/yourSkill");
return skillModule.executeYourSkill(task);
},
};
```
3. **Expose in Agent Card:** in `src/app/.well-known/agent.json/route.ts`, append to the `skills` array:
```json
{
"id": "your-skill",
"name": "Your Skill",
"description": "Brief, intent-focused description",
"tags": ["routing", "quota"],
"examples": ["Sample natural-language invocation"]
}
```
4. **Write tests:** `tests/unit/a2a-<your-skill>.test.ts`. Cover happy path + error path.
5. **Document** the new skill in this file's `Available Skills` table.
---
## Task TTL
Tasks expire after `ttlMinutes` (default 5 min) — configured in the `A2ATaskManager` constructor at `src/lib/a2a/taskManager.ts:82`. To customize, fork the `A2ATaskManager` instantiation and pass a different value (e.g., `new A2ATaskManager(15)` for 15-minute TTL). A background interval sweeps expired tasks every 60 seconds.
---
## Task Lifecycle
```
submitted → working → completed
→ failed
→ cancelled
```
- Tasks expire after 5 minutes by default (see [Task TTL](#task-ttl))
- Terminal states: `completed`, `failed`, `cancelled`
- Event log tracks every state transition
---
## Error Codes
| Code | Meaning |
| :----- | :----------------------------- |
| -32700 | Parse error (invalid JSON) |
| -32600 | Invalid request / Unauthorized |
| -32601 | Method or skill not found |
| -32602 | Invalid params |
| -32603 | Internal error |
| -32000 | A2A endpoint is disabled |
---
## Integration Examples
### Python (requests)
```python
import requests
resp = requests.post("http://localhost:20128/a2a", json={
"jsonrpc": "2.0", "id": "1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Hello"}]
}
}, headers={"Authorization": "Bearer YOUR_KEY"})
result = resp.json()["result"]
print(result["artifacts"][0]["content"])
print(result["metadata"]["routing_explanation"])
```
### TypeScript (fetch)
```typescript
const resp = await fetch("http://localhost:20128/a2a", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: "Bearer YOUR_KEY",
},
body: JSON.stringify({
jsonrpc: "2.0",
id: "1",
method: "message/send",
params: {
skill: "smart-routing",
messages: [{ role: "user", content: "Hello" }],
},
}),
});
const { result } = await resp.json();
console.log(result.metadata.routing_explanation);
```
docs/frameworks/ACP.md
+559
View File
@@ -0,0 +1,559 @@
---
title: ACP (Agent Client Protocol)
---
# ACP (Agent Client Protocol)
> **TL;DR**: ACP lets OmniRoute spawn CLI agents (like Claude Code, Codex) as child processes instead of using HTTP APIs. This gives you "CLI-as-backend" transport.
---
## What Is ACP?
ACP (Agent Client Protocol) is a **"CLI-as-backend" transport** for OmniRoute. Instead of intercepting HTTP API calls to AI providers, ACP **spawns CLI agents as child processes** and feeds prompts through their native interface.
### Why Use ACP?
| Benefit | Description |
| ---------------------- | ------------------------------------------ |
| **No API keys needed** | Uses your existing CLI authentication |
| **Native protocol** | Uses each CLI's native input/output format |
| **Auto-discovery** | Detects installed CLIs on your system |
| **14 built-in agents** | Pre-configured for popular CLI tools |
| **Custom agents** | Add your own CLI tools via settings |
| **Process management** | Handles lifecycle (spawn, send, kill) |
---
## Supported CLI Agents
ACP supports **14 built-in CLI agents** out of the box:
| Agent ID | Display Name | Binary | Protocol |
| ------------- | ------------------ | ------------- | -------- |
| `codex` | OpenAI Codex CLI | `codex` | stdio |
| `claude` | Claude Code CLI | `claude` | stdio |
| `goose` | Goose CLI | `goose` | stdio |
| `openclaw` | OpenClaw | `openclaw` | stdio |
| `aider` | Aider | `aider` | stdio |
| `opencode` | OpenCode | `opencode` | stdio |
| `cline` | Cline | `cline` | stdio |
| `qwen-code` | Qwen Code | `qwen` | stdio |
| `forge` | ForgeCode | `forge` | stdio |
| `amazon-q` | Amazon Q Developer | `q` | stdio |
| `interpreter` | Open Interpreter | `interpreter` | stdio |
| `cursor-cli` | Cursor CLI | `cursor` | stdio |
| `warp` | Warp AI | `warp` | stdio |
### Custom Agents
You can add your own CLI agents via settings. Custom agents support the same features as built-in agents.
---
## Quick Start
### Step 1: Install a CLI Agent
```bash
# Example: Install Claude Code CLI
npm install -g @anthropic-ai/claude-code
# Verify installation
claude --version
```
### Step 2: ACP Auto-Detection
ACP automatically detects installed CLI agents on your system. No configuration needed!
### Step 3: Use ACP Transport
Once detected, ACP can be used as a transport for any supported provider. OmniRoute will automatically use ACP when the CLI is available.
---
## How ACP Works
### Architecture
```
┌─────────────────┐
│ OmniRoute │
│ (HTTP Proxy) │
└────────┬────────┘
│
│ spawn()
▼
┌─────────────────┐
│ Child Process │
│ (CLI Agent) │
│ │
│ stdin ◄──────┤ Send prompt
│ stdout ──────►│ Receive response
│ stderr ──────►│ Receive errors
└─────────────────┘
```
### Process Lifecycle
1. **Spawn** — ACP creates a child process for the CLI agent
2. **Send** — ACP writes prompts to the process's stdin
3. **Receive** — ACP reads responses from stdout/stderr
4. **Idle Detection** — ACP waits 2 seconds of inactivity before considering the response complete
5. **Kill** — ACP terminates the process (SIGTERM, then SIGKILL after 5s)
### Communication Protocol
ACP uses **stdio** (standard input/output) for communication with CLI agents. The protocol is:
1. **Send prompt** — Write to stdin with a newline
2. **Wait for response** — Read from stdout until idle (2s of no output)
3. **Timeout** — Default 120 seconds (configurable)
---
## API Reference
### Registry Functions
#### `detectInstalledAgents()`
Detects all installed CLI agents on the system. Results are cached for 60 seconds.
```typescript
import { detectInstalledAgents } from "@/lib/acp";
const agents = detectInstalledAgents();
// Returns: CliAgentInfo[]
interface CliAgentInfo {
id: string; // e.g., "codex", "claude"
name: string; // Display name
binary: string; // Binary name to spawn
versionCommand: string; // Version detection command
version: string | null; // Detected version (null if not installed)
installed: boolean; // Whether the agent is installed
providerAlias: string; // Provider ID in OmniRoute
spawnArgs: string[]; // Arguments to pass when spawning
protocol: "stdio" | "http"; // Communication protocol
isCustom?: boolean; // Whether this is a user-defined custom agent
}
```
#### `getAvailableAgents()`
Gets only the agents that are installed and available for ACP.
```typescript
import { getAvailableAgents } from "@/lib/acp";
const available = getAvailableAgents();
// Returns: CliAgentInfo[] (only installed agents)
```
#### `getAgentById(id)`
Gets a specific agent by ID.
```typescript
import { getAgentById } from "@/lib/acp";
const agent = getAgentById("claude");
// Returns: CliAgentInfo | undefined
```
#### `setCustomAgents(agents)`
Sets custom agent definitions from settings.
```typescript
import { setCustomAgents } from "@/lib/acp";
setCustomAgents([
{
id: "my-custom-cli",
name: "My Custom CLI",
binary: "mycli",
versionCommand: "mycli --version",
providerAlias: "my-provider",
spawnArgs: [],
protocol: "stdio",
},
]);
```
### Manager Functions
#### `acpManager.spawn(agentId, binary, args, env)`
Spawns a new CLI agent process.
```typescript
import { acpManager } from "@/lib/acp";
const session = acpManager.spawn("claude", "claude", ["--print", "--output-format", "json"], {
/* custom env vars */
});
// Returns: AcpSession
```
**Allowed agent IDs**: `["claude", "codex", "gemini", "qwen"]`
#### `acpManager.sendPrompt(sessionId, prompt, timeoutMs)`
Sends a prompt to a CLI agent and collects the response.
```typescript
import { acpManager } from "@/lib/acp";
const response = await acpManager.sendPrompt(
"acp-claude-1234567890-abc123",
"What is 2+2?",
120000 // 2 minutes timeout
);
// Returns: Promise<string>
```
#### `acpManager.kill(sessionId)`
Kills a session and cleans up.
```typescript
import { acpManager } from "@/lib/acp";
const killed = acpManager.kill("acp-claude-1234567890-abc123");
// Returns: boolean
```
#### `acpManager.getActiveSessions()`
Gets all active sessions.
```typescript
import { acpManager } from "@/lib/acp";
const sessions = acpManager.getActiveSessions();
// Returns: AcpSession[]
```
#### `acpManager.killAll()`
Kills all sessions.
```typescript
import { acpManager } from "@/lib/acp";
acpManager.killAll();
```
### Session Interface
```typescript
interface AcpSession {
id: string; // Unique session ID
agentId: string; // Agent ID (e.g., "claude")
process: ChildProcess; // Child process handle
alive: boolean; // Whether the process is alive
stdoutBuffer: string; // Accumulated stdout buffer
stderrBuffer: string; // Accumulated stderr buffer
createdAt: Date; // Created timestamp
}
```
### Events
The `AcpManager` extends `EventEmitter` and emits the following events:
#### `stdout`
Emitted when the CLI agent writes to stdout.
```typescript
acpManager.on("stdout", ({ sessionId, data }) => {
console.log(`[${sessionId}] stdout: ${data}`);
});
```
#### `stderr`
Emitted when the CLI agent writes to stderr.
```typescript
acpManager.on("stderr", ({ sessionId, data }) => {
console.error(`[${sessionId}] stderr: ${data}`);
});
```
#### `exit`
Emitted when the CLI agent process exits.
```typescript
acpManager.on("exit", ({ sessionId, code, signal }) => {
console.log(`[${sessionId}] exited with code ${code}, signal ${signal}`);
});
```
#### `error`
Emitted when the CLI agent process errors.
```typescript
acpManager.on("error", ({ sessionId, error }) => {
console.error(`[${sessionId}] error: ${error}`);
});
```
---
## Configuration
### Environment Variables
ACP inherits all environment variables from the parent process and can be extended with custom env vars:
```typescript
acpManager.spawn("claude", "claude", [], {
ANTHROPIC_API_KEY: "sk-...",
DEBUG: "true",
});
```
### Spawn Arguments
Each agent has default spawn arguments defined in the registry. You can override them:
```typescript
acpManager.spawn("claude", "claude", ["--print", "--verbose"], {});
```
### Timeouts
Default prompt timeout is **120 seconds** (2 minutes). You can override:
```typescript
await acpManager.sendPrompt(sessionId, prompt, 300000); // 5 minutes
```
### Detection Cache
Agent detection is cached for **60 seconds** to avoid expensive filesystem scans. Force refresh:
```typescript
import { refreshAgentCache } from "@/lib/acp";
refreshAgentCache();
```
---
## Security
### Command Injection Prevention
ACP validates version commands to prevent command injection attacks:
```typescript
const DISALLOWED_VERSION_COMMAND_CHARS = /[;&|<>`$\r\n]/;
```
Version commands containing these characters are rejected:
- `;` — Command separator
- `&` — Background process
- `|` — Pipe
- `<`, `>` — Redirection
- `` ` `` — Command substitution
- `$` — Variable expansion
- `\r`, `\n` — Line breaks
### Binary Name Validation
ACP validates that the version command binary matches the expected binary name (unless it's a custom agent).
### Process Isolation
Each ACP session runs in its own child process. The process is killed when the session ends or times out.
---
## Performance
### Detection Performance
- **First call**: ~50-200ms (runs `version` command for each agent)
- **Cached calls**: <1ms (returns from cache)
- **Cache TTL**: 60 seconds
### Prompt Performance
- **Spawn**: ~50-100ms
- **Send prompt**: ~10-50ms
- **Wait for response**: Depends on CLI agent (typically 1-30 seconds)
- **Kill**: ~5 seconds (SIGTERM) + immediate (SIGKILL)
### Resource Usage
- **Memory per session**: ~10-50MB (depends on CLI agent)
- **CPU**: Minimal (I/O bound)
- **Disk**: None
---
## Troubleshooting
### "Unknown agent" Error
**Problem**: `acpManager.spawn()` throws `Unknown agent: <id>`
**Solution**: Only 4 agents are allowed in `spawn()`:
- `claude`
- `codex`
- `gemini`
- `qwen`
Other agents must be spawned manually or via custom agent definitions.
### "Session not alive" Error
**Problem**: `acpManager.sendPrompt()` throws `Session ${sessionId} is not alive`
**Solution**: The session may have exited or been killed. Check session status:
```typescript
const session = acpManager.getSession(sessionId);
if (!session?.alive) {
// Re-spawn the session
acpManager.spawn("claude", "claude", [], {});
}
```
### "ACP timeout" Error
**Problem**: `acpManager.sendPrompt()` throws `ACP timeout after 120000ms`
**Solution**: Increase the timeout:
```typescript
await acpManager.sendPrompt(sessionId, prompt, 300000); // 5 minutes
```
### CLI Not Detected
**Problem**: `detectInstalledAgents()` doesn't find your CLI
**Solutions**:
1. **Check PATH**: Ensure the CLI is in your system PATH
2. **Check version command**: Run `claude --version` manually
3. **Check permissions**: Ensure the CLI is executable
4. **Custom agent**: Add a custom agent definition for non-standard CLIs
### Permission Denied
**Problem**: ACP can't execute the CLI
**Solutions**:
1. **Check file permissions**: `chmod +x /usr/local/bin/claude`
2. **Check ownership**: Ensure OmniRoute has read/execute permissions
3. **Check SELinux/AppArmor**: May block process spawning
---
## Examples
### Example 1: Spawn and Use Claude Code
```typescript
import { acpManager, detectInstalledAgents } from "@/lib/acp";
// Detect installed agents
const agents = detectInstalledAgents();
const claude = agents.find((a) => a.id === "claude");
if (claude?.installed) {
// Spawn a new session
const session = acpManager.spawn("claude", claude.binary, ["--print", "--output-format", "json"]);
// Send a prompt
const response = await acpManager.sendPrompt(
session.id,
"Explain quantum computing in 100 words"
);
console.log("Claude's response:", response);
// Clean up
acpManager.kill(session.id);
}
```
### Example 2: Auto-Discovery with Fallback
```typescript
import { acpManager, getAvailableAgents } from "@/lib/acp";
const available = getAvailableAgents();
// Try Claude first, fallback to Codex
let agentId = "claude";
if (!available.find((a) => a.id === "claude")) {
if (available.find((a) => a.id === "codex")) {
agentId = "codex";
} else {
throw new Error("No ACP-compatible CLI agent found");
}
}
const agent = available.find((a) => a.id === agentId)!;
const session = acpManager.spawn(agentId, agent.binary, agent.spawnArgs);
const response = await acpManager.sendPrompt(session.id, "Hello!");
acpManager.kill(session.id);
```
### Example 3: Custom Agent
```typescript
import { setCustomAgents, detectInstalledAgents } from "@/lib/acp";
// Register a custom CLI agent
setCustomAgents([
{
id: "my-llm-cli",
name: "My LLM CLI",
binary: "myllm",
versionCommand: "myllm --version",
providerAlias: "my-llm-provider",
spawnArgs: ["--format", "json"],
protocol: "stdio",
},
]);
// Now detectInstalledAgents() will include "my-llm-cli"
const agents = detectInstalledAgents();
```
---
## What's Next?
- **[API Reference](../reference/API_REFERENCE.md)** — REST API endpoints
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 226 providers
- **[MCP Server](./MCP-SERVER.md)** — Model Context Protocol integration
- **[A2A Server](./A2A-SERVER.md)** — Agent-to-Agent protocol
- **[Cloud Agent](./CLOUD_AGENT.md)** — Cloud-based agents
---
## Reference
- [AionUi Project](https://github.com/iOfficeAI/AionUi) — Inspiration for ACP auto-detection
- [ACP Source Code](../../src/lib/acp/) — Implementation details
- `manager.ts` — Process lifecycle management
- `registry.ts` — Agent discovery and registration
- `index.ts` — Public API exports
docs/frameworks/AGENT-SKILLS.md
+309
View File
@@ -0,0 +1,309 @@
---
title: "OmniRoute Agent Skills Catalog"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute Agent Skills Catalog
> **Source of truth:** `src/lib/agentSkills/` (catalog, generator, parsers) + `skills/` directory (SKILL.md files)
> **Last updated:** 2026-06-28 — v3.8.40
Agent Skills are structured SKILL.md files that teach external agents, MCP clients, and A2A orchestrators how to use OmniRoute's REST API and CLI. Unlike [Omni Skills](./SKILLS.md) (which are LLM tool definitions executed inside OmniRoute), Agent Skills are a _documentation catalog_ — static markdown that can be fed directly into agent context.
---
## Overview
The catalog contains **42 canonical Agent Skills** (22 REST API + 20 CLI). Each skill has:
- A **canonical ID** (`omni-auth`, `cli-serve`, etc.)
- A **SKILL.md** file in `skills/{id}/SKILL.md` with YAML frontmatter (`name`, `description`) + rich markdown body
- **REST endpoints** (API skills) or **CLI subcommands** (CLI skills) derived from the OpenAPI spec and CLI registry
- A **GitHub raw URL** for live fetch: `https://raw.githubusercontent.com/diegosouzapw/OmniRoute/refs/heads/main/skills/{id}/SKILL.md`
---
## Architecture
```
src/shared/constants/agentSkills.ts — 42-entry curated list (name/desc/category/area/icon)
src/lib/agentSkills/
catalog.ts — getCatalog(), getSkillById(), filterCatalog(), computeCoverage()
generator.ts — generateAgentSkills() writes SKILL.md to skills/{id}/
openapiParser.ts — extracts REST endpoints from docs/openapi.yaml
cliRegistryParser.ts — extracts CLI subcommands from bin/cli-registry.ts
schemas.ts — Zod schemas: AgentSkillSchema, SkillCoverageSchema, etc.
types.ts — TypeScript interfaces: AgentSkill, SkillCoverage, etc.
skills/{id}/SKILL.md — Generated + curated markdown files (42 total)
src/app/api/agent-skills/
route.ts — GET /api/agent-skills
[id]/route.ts — GET /api/agent-skills/{id}
[id]/raw/route.ts — GET /api/agent-skills/{id}/raw (text/markdown)
coverage/route.ts — GET /api/agent-skills/coverage
generate/route.ts — POST /api/agent-skills/generate (auth required)
open-sse/mcp-server/tools/agentSkillTools.ts — 3 MCP tools (list, get, coverage)
src/lib/a2a/skills/listCapabilities.ts — A2A skill: list-capabilities
```
---
## SKILL.md Format
```markdown
---
name: omni-providers
description: "Manage provider connections: add, test, rotate, and remove credentials."
---
<!-- generated by src/lib/agentSkills/generator.ts; manual edits will be overwritten -->
## Overview
...
## Authentication
...
## Endpoints
...
<!-- skill:custom-start -->
## Custom Section (preserved across regeneration)
...
<!-- skill:custom-end -->
```
The generator preserves content between `<!-- skill:custom-start -->` and `<!-- skill:custom-end -->` on regeneration. Ten skills have curated custom blocks:
`omni-mcp`, `omni-compression`, `cli-providers`, `cli-eval`, `omni-agents-a2a`, `omni-combos-routing`, `omni-auth`, `omni-resilience`, `omni-inference`, `cli-serve`.
---
## REST API Discovery
| Endpoint | Method | Description | Auth |
| :--------------------------- | :----- | :------------------------------------------------------- | :--------- |
| `/api/agent-skills` | GET | List catalog (optional `?category=api\|cli&area=<area>`) | none |
| `/api/agent-skills/{id}` | GET | Get single skill metadata | none |
| `/api/agent-skills/{id}/raw` | GET | Fetch SKILL.md as `text/markdown` | none |
| `/api/agent-skills/coverage` | GET | Coverage stats (how many SKILL.md files exist) | none |
| `/api/agent-skills/generate` | POST | Trigger generator (dryRun/prune/onlyIds) | management |
Example — list all API skills:
```bash
curl "http://localhost:20128/api/agent-skills?category=api"
```
Example — fetch a single SKILL.md:
```bash
curl -H "Accept: text/markdown" "http://localhost:20128/api/agent-skills/omni-providers/raw"
```
---
## MCP Discovery
Three MCP tools are registered under scope `read:catalog`:
| Tool | Description |
| :-------------------------------- | :------------------------------------------------- |
| `omniroute_agent_skills_list` | List skills (optional `category` / `area` filters) |
| `omniroute_agent_skills_get` | Get metadata + SKILL.md for one skill by `id` |
| `omniroute_agent_skills_coverage` | Coverage stats (API/CLI have/total) |
See [MCP-SERVER.md](./MCP-SERVER.md) for scope wiring and authentication.
---
## A2A Discovery
The A2A skill `list-capabilities` returns the full 42-skill catalog as a markdown table artifact. External orchestrators can invoke it via:
```json
{
"jsonrpc": "2.0",
"id": "1",
"method": "message/send",
"params": {
"skill": "list-capabilities",
"messages": [{ "role": "user", "content": "List all capabilities" }]
}
}
```
See [A2A-SERVER.md](./A2A-SERVER.md) for protocol details.
---
## Catalog — 42 Skill IDs
### API Skills (22)
| ID | Area | Entry Point |
| :--------------------- | :-------------- | :---------------------------------- |
| `omni-auth` | auth | Auth + session management |
| `omni-providers` | providers | Provider connection management |
| `omni-models` | models | Model catalog and capabilities |
| `omni-combos-routing` | combos-routing | Combo routing strategies |
| `omni-api-keys` | api-keys | API key management |
| `omni-usage-logs` | usage-logs | Usage and cost logs |
| `omni-budget` | budget | Budget guards |
| `omni-settings` | settings | Global settings |
| `omni-proxies` | proxies | Proxy pool management |
| `omni-cache` | cache | Semantic + prompt cache |
| `omni-compression` | compression | Context compression engines |
| `omni-context-rtk` | context-rtk | RTK compression |
| `omni-resilience` | resilience | Circuit breakers + cooldowns |
| `omni-cli-tools` | cli-tools | CLI tools REST proxy |
| `omni-tunnels` | tunnels | Tunnel management |
| `omni-sync-cloud` | sync-cloud | Cloud sync |
| `omni-db-backups` | db-backups | Database backups |
| `omni-webhooks` | webhooks | Webhook event dispatcher |
| `omni-mcp` | mcp | MCP server (87 tools, 3 transports) |
| `omni-agents-a2a` | agents-a2a | A2A agent protocol |
| `omni-version-manager` | version-manager | Version and update management |
| `omni-inference` | inference | Direct inference / completions |
### CLI Skills (20)
| ID | Area | CLI Command Root |
| :------------------- | :----------------- | :---------------------- |
| `cli-serve` | cli-serve | `omniroute serve` |
| `cli-health` | cli-health | `omniroute health` |
| `cli-providers` | cli-providers | `omniroute providers` |
| `cli-keys` | cli-keys | `omniroute keys` |
| `cli-models` | cli-models | `omniroute models` |
| `cli-chat` | cli-chat | `omniroute chat` |
| `cli-routing` | cli-routing | `omniroute routing` |
| `cli-resilience` | cli-resilience | `omniroute resilience` |
| `cli-compression` | cli-compression | `omniroute compression` |
| `cli-contexts` | cli-contexts | `omniroute contexts` |
| `cli-cost-usage` | cli-cost-usage | `omniroute cost` |
| `cli-mcp` | cli-mcp | `omniroute mcp` |
| `cli-a2a` | cli-a2a | `omniroute a2a` |
| `cli-tunnel` | cli-tunnel | `omniroute tunnel` |
| `cli-backup-sync` | cli-backup-sync | `omniroute backup` |
| `cli-policy-audit` | cli-policy-audit | `omniroute policy` |
| `cli-batches` | cli-batches | `omniroute batch` |
| `cli-eval` | cli-eval | `omniroute eval` |
| `cli-plugins-skills` | cli-plugins-skills | `omniroute plugins` |
| `cli-setup` | cli-setup | `omniroute setup` |
---
## How External Agents Consume Skills
### 1. Discovery via REST
```bash
# Get the full catalog
curl "http://your-omniroute/api/agent-skills" | jq '.skills[] | {id, name, category}'
# Get SKILL.md for context injection
curl "http://your-omniroute/api/agent-skills/omni-providers/raw" > omni-providers.md
```
### 2. Discovery via MCP
```typescript
// In a Claude Desktop / Cursor MCP client:
const result = await client.callTool("omniroute_agent_skills_list", { category: "api" });
// result.skills → array of AgentSkill with rawUrl for each
```
### 3. Discovery via A2A
```python
import requests
resp = requests.post("http://your-omniroute/a2a", json={
"jsonrpc": "2.0", "id": "1",
"method": "message/send",
"params": {"skill": "list-capabilities", "messages": [{"role": "user", "content": "list"}]}
})
table = resp.json()["result"]["artifacts"][0]["content"]
# table is a markdown table with all 42 skill IDs + rawUrl columns
```
### 4. Direct GitHub raw fetch (no server required)
```bash
BASE="https://raw.githubusercontent.com/diegosouzapw/OmniRoute/refs/heads/main/skills"
curl "${BASE}/omni-providers/SKILL.md"
```
---
## Generator
The generator reads the curated catalog + OpenAPI spec + CLI registry and writes `skills/{id}/SKILL.md` for each entry:
```bash
# Preview (dry run, no writes)
curl -X POST http://localhost:20128/api/agent-skills/generate \
-H "Authorization: Bearer <admin-key>" \
-H "Content-Type: application/json" \
-d '{"dryRun":true}'
# Full regeneration
curl -X POST http://localhost:20128/api/agent-skills/generate \
-H "Authorization: Bearer <admin-key>" \
-H "Content-Type: application/json" \
-d '{"dryRun":false,"prune":false}'
# Regenerate specific IDs
curl -X POST http://localhost:20128/api/agent-skills/generate \
-H "Authorization: Bearer <admin-key>" \
-H "Content-Type: application/json" \
-d '{"dryRun":false,"onlyIds":["omni-providers","cli-serve"]}'
```
The generator response is a `GeneratorReport`:
```json
{
"generated": ["omni-providers", "cli-serve"],
"unchanged": [],
"pruned": [],
"orphansDetected": [],
"errors": []
}
```
---
## Coverage API
```bash
curl "http://localhost:20128/api/agent-skills/coverage"
```
```json
{
"api": { "have": 22, "total": 22 },
"cli": { "have": 20, "total": 20 },
"totalSkills": 42,
"generatedAt": "2026-05-28T00:00:00.000Z"
}
```
---
## Related
- [SKILLS.md](./SKILLS.md) — Omni Skills framework (LLM tool injection + marketplace)
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP tool catalog (`omniroute_agent_skills_*` tools)
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A protocol (`list-capabilities` skill)
- `src/lib/agentSkills/` — catalog, generator, parsers
- `skills/` — generated SKILL.md files (42 entries)
docs/frameworks/AGENTBRIDGE.md
+537
View File
@@ -0,0 +1,537 @@
---
title: "AgentBridge"
version: 3.8.40
lastUpdated: 2026-06-28
---
# AgentBridge
AgentBridge is OmniRoute's MITM (Man-in-the-Middle) proxy that intercepts HTTPS traffic from IDE AI agents and reroutes it through OmniRoute's unified routing engine. It supports **9 IDE agents** — Antigravity, Kiro, GitHub Copilot, OpenAI Codex, Cursor, Zed, Claude Code, Open Code, and Trae (investigating) — making OmniRoute the broadest-coverage MITM proxy for AI coding assistants on the market.
**Dashboard location:** `/dashboard/tools/agent-bridge`
**Sidebar group:** Tools (after Cloud Agents)
**See also:** [`TRAFFIC_INSPECTOR.md`](./TRAFFIC_INSPECTOR.md) — monitor all intercepted traffic in real-time; [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md) — the Linux TPROXY transparent-decrypt capture mode driven by the `/api/tools/agent-bridge/tproxy` route.
---
## §1 Overview
### What is AgentBridge?
When an IDE agent (e.g., GitHub Copilot, Cursor, Claude Code) makes an API call, it connects directly to the upstream AI provider (OpenAI, Anthropic, etc.). AgentBridge intercepts that connection transparently at the TLS level — without requiring any agent configuration change — and rewrites the request through OmniRoute.
This means you can:
- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 226+ providers.
- **Apply model mappings**: `gemini-3-flash` → `claude-sonnet-4.7` transparently at the handler level.
- **Observe all agent traffic**: every intercepted request is published to the [Traffic Inspector](./TRAFFIC_INSPECTOR.md).
- **Apply OmniRoute resilience**: combo routing, circuit breakers, fallbacks, and cost tracking work for IDE agent traffic too.
### Positioning vs. the market
| Feature | 9router | anti-api | llm-interceptor | **OmniRoute AgentBridge** |
| ----------------- | :-----: | :------: | :-------------: | :-----------------------: |
| Antigravity | ✓ | ✓ | — | ✓ |
| GitHub Copilot | ✓ | ✓ | — | ✓ |
| Kiro (AWS) | ✓ | ✓ | — | ✓ |
| OpenAI Codex | — | ✓ | — | ✓ |
| Cursor IDE | ✓ | ✓ | — | ✓ |
| Zed Industries | — | ✓ | — | ✓ |
| Claude Code | — | — | ✓ | ✓ |
| Open Code | — | — | ✓ | ✓ |
| Trae | — | — | — | 🔍 Investigating |
| Dashboard UI | ✓ | ✗ | ✗ | ✓ |
| Traffic Inspector | ✗ | ✗ | ✓ | ✓ |
| OmniRoute routing | ✗ | ✗ | ✗ | ✓ |
| Model mapping UI | ✗ | ✗ | ✗ | ✓ |
| Bypass list | ✗ | ✗ | ✓ | ✓ |
| Upstream CA cert | ✗ | ✗ | ✓ | ✓ |
---
## §2 Architecture
### 2.1 Components overview
```
IDE Agent (VS Code / Cursor / etc.)
│ HTTPS (port 443)
▼
/etc/hosts — 127.0.0.1 api.githubcopilot.com ← DNS redirect
│
▼
src/mitm/server.cjs (port 443, CJS child process)
│ resolves target by Host header SNI
│ generates per-SNI TLS cert signed by AgentBridge CA
├── Bypass list match? → TCP passthrough (no decrypt)
├── Target match? → fetch → OmniRoute router (port 20128)
│ └── handler.intercept() — TypeScript
│ ├── maskSecrets() on request body/headers
│ ├── TrafficBuffer.push() — publishes to Traffic Inspector
│ └── fetchRouter() → /v1/chat/completions
└── No match? → TCP passthrough (no decrypt)
```
### 2.2 MITM server (`src/mitm/server.cjs`)
The core MITM server runs as a Node.js CJS child process (to avoid rewriting the existing CJS codebase). It:
- Listens on port 443 (requires privilege or `authbind`/`setcap`)
- Receives CONNECT tunnels from the OS (via `/etc/hosts` DNS redirect)
- Generates per-SNI TLS certificates signed by the AgentBridge CA (`DATA_DIR/mitm/ca.crt`)
- Resolves the target agent by Host header via `targets/index.ts` registry
- Dispatches to the TypeScript handler layer via HTTP to `http://127.0.0.1:20128`
`TARGET_HOSTS` is loaded from `DATA_DIR/mitm/targets.json` (written by `targets/index.ts` at boot), allowing dynamic updates without restarting the CJS server.
### 2.3 Handler base (`src/mitm/handlers/base.ts`)
All agent handlers extend `MitmHandlerBase`:
```ts
export abstract class MitmHandlerBase {
abstract readonly agentId: AgentId;
abstract intercept(
req: IncomingMessage,
res: ServerResponse,
body: Buffer,
mappedModel: string
): Promise<void>;
// Protected helpers: fetchRouter, pipeSSE, hookBufferStart, hookBufferUpdate
}
```
Each handler calls `hookBufferStart()` before proxying and `hookBufferUpdate()` when complete. These push `InterceptedRequest` entries into `globalTrafficBuffer` (see [Traffic Inspector](./TRAFFIC_INSPECTOR.md) §4).
### 2.4 Targets registry (`src/mitm/targets/`)
Each agent has a declarative target file:
```ts
// src/mitm/targets/copilot.ts
export const COPILOT_TARGET: MitmTarget = {
id: "copilot",
name: "GitHub Copilot",
hosts: ["api.githubcopilot.com", "copilot-proxy.githubusercontent.com"],
port: 443,
endpointPatterns: ["/chat/completions", "/v1/chat/completions"],
defaultModels: [{ id: "gpt-4o", name: "GPT-4o", alias: "gpt-4o" }],
handler: () => import("../handlers/copilot"),
riskNoticeKey: "providers.riskNotice.oauth",
};
```
The registry (`targets/index.ts`) exports `ALL_TARGETS` and emits `DATA_DIR/mitm/targets.json` on boot.
### 2.5 Passthrough and bypass list (`src/mitm/passthrough.ts`)
**Bypass list** (checked first, with precedence over target match):
- Default patterns: banking hosts, `.gov.`, OAuth/SSO providers (Okta, Auth0), etc.
- User patterns: stored in DB table `agent_bridge_bypass`
- Bypassed hosts receive a transparent TCP tunnel — TLS is **never decrypted**
**Passthrough default** (no target match and not in bypass):
- Also receives a TCP tunnel — connections are never broken
- Prevents the AgentBridge from disrupting general system HTTPS traffic
Routing precedence:
```
bypass list → target match → passthrough
```
### 2.6 Upstream CA cert (`src/mitm/upstreamTrust.ts`)
For corporate network environments with a custom CA:
```bash
AGENTBRIDGE_UPSTREAM_CA_CERT=/path/to/corporate-ca.pem
```
When set, configures `undici`'s global dispatcher with the extra CA cert, allowing AgentBridge to reach upstream providers through corporate TLS termination proxies.
### 2.7 Secret masking (`src/mitm/maskSecrets.ts`)
Applied to all request bodies and headers **before** they enter the Traffic Inspector buffer or any log:
- `sk-` / `ak-` / `pk-` prefixed tokens (OpenAI/Anthropic-style)
- `Authorization: Bearer <token>` headers
- Generic long tokens (≥40 chars)
---
## §3 Setup
### 3.1 Start/stop the MITM server
Use the AgentBridge Server Card at `/dashboard/tools/agent-bridge`:
| Action | Description |
| --------------- | ----------------------------------------------------------------------- |
| Start Server | Spawns `src/mitm/server.cjs` on port 443 |
| Stop Server | Gracefully shuts down the child process |
| Restart Server | Stop + start (picks up target changes) |
| Trust Cert | Installs `DATA_DIR/mitm/ca.crt` into OS trust store |
| Download Cert | Downloads `ca.crt` for manual installation |
| Regenerate Cert | Creates a new CA keypair (all existing per-agent certs are invalidated) |
### 3.2 Trust the certificate
The AgentBridge CA certificate must be trusted by the OS before IDEs will accept the MITM connection.
**Linux (NSS — Chrome/Firefox):**
```bash
certutil -A -d sql:$HOME/.pki/nssdb -n "OmniRoute AgentBridge" -t CT,, -i ~/.omniroute/mitm/ca.crt
```
**macOS (Keychain):**
```bash
sudo security add-trusted-cert -d -r trustRoot \
-k /Library/Keychains/System.keychain ~/.omniroute/mitm/ca.crt
```
**Windows (certmgr):**
```powershell
certutil -addstore -f Root $env:USERPROFILE\.omniroute\mitm\ca.crt
```
Or use the "Trust Cert" button in the dashboard (runs the appropriate command for your OS, with sudo prompt if needed).
#### Electron-based IDEs ignore the OS trust store (`NODE_EXTRA_CA_CERTS`)
Some IDEs — notably **Antigravity IDE**, and other Electron / VS Code-derived apps — bundle
their own Node.js runtime that **does not consult the OS trust store** for outbound
`fetch`/HTTPS. Trusting the CA at the OS/NSS level is enough for the IDE's native **backend**
(e.g. a Go language server, which uses the OS CA bundle), but the **Electron frontend** will
still fail TLS — it surfaces as the app being _logged out_ or showing a _"connection error"_
even though the MITM log shows the backend's bootstrap calls returning `200`. Two steps are
required, and both matter:
1. Point the runtime at the CA explicitly:
```bash
export NODE_EXTRA_CA_CERTS=/path/to/omniroute-agentbridge-ca.crt
```
2. **Launch the IDE from that shell.** Starting it from the desktop icon / Dock / Start menu
does **not** inherit shell exports, and `~/.config/environment.d/*.conf` only applies after
a fresh graphical login. Fully quit the IDE first — Electron's singleton lock means a second
launch just focuses the existing process and the new environment is ignored.
The OS-trust + NSS step above remains necessary (the Chromium network stack used by some auth
flows reads the per-user NSS store, and has its own static pins for `*.googleapis.com` that a
locally-trusted CA overrides). `NODE_EXTRA_CA_CERTS` covers the Node `fetch` path on top of it.
### 3.3 DNS routing
For each agent you want to intercept, its API host(s) must resolve to `127.0.0.1`. AgentBridge manages `/etc/hosts` entries automatically when you toggle DNS for an agent in the Setup Wizard.
Example `/etc/hosts` entries for GitHub Copilot:
```
127.0.0.1 api.githubcopilot.com
127.0.0.1 copilot-proxy.githubusercontent.com
```
### 3.4 Model mapping
Use the Model Mapping Table in each agent card to define source → target mappings:
| Source model (agent native) | Target model (OmniRoute) |
| --------------------------- | ------------------------ |
| `gpt-4o` | `claude-sonnet-4.7` |
| `*` (wildcard) | `claude-haiku-4.7` |
Wildcard `*` maps any unrecognized model to the specified target. Persisted in `agent_bridge_mappings` table.
> **Tip — discover the agent's real model IDs.** An IDE may send model names that differ from
> its UI labels and that change between major versions. For example **Antigravity 2** sends
> `gemini-3.1-pro-low`, `gemini-pro-agent`, and `gemini-3.1-flash-lite` over the wire — not the
> `gemini-2.5-pro` shown in older docs. Send one chat with no matching mapping in place: the MITM
> logs the exact incoming `model:` and passes the request through. Map that literal value, then
> the next request is intercepted and routed to your target.
### 3.5 Risk notice
AgentBridge intercepts credentials (OAuth tokens, API keys) that the IDE uses to authenticate with upstream providers. These are **masked before logging** (see §2.7) but are visible to OmniRoute's MITM layer. First activation of each agent shows a dismissible risk notice modal.
### 3.6 Maintenance & Diagnostics
The dashboard exposes a **Maintenance & Diagnostics** card (`AgentBridgeMaintenanceCard`, in `src/app/(dashboard)/dashboard/tools/agent-bridge/components/`) that surfaces operational MITM routes which previously had no UI. Its subtitle: _"Self-test the capture pipeline, undo leftover system state, and move your setup between machines."_ The card client helpers live in `src/lib/inspector/agentBridgeMaintenanceApi.ts`.
| Button | Route | What it does |
| ----------------- | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Diagnose** | `GET /api/tools/agent-bridge/diagnose` | Runs the capture-pipeline self-test and shows a per-check report (✓/✗ + remediation hint). |
| **Repair** | `POST /api/tools/agent-bridge/repair` | Undoes orphaned MITM system state (DNS spoof entries, root CA, system proxy) left behind by a crash or SIGKILL. Idempotent — reports "Nothing to repair" when state is clean. |
| **Remove CA** | `DELETE /api/tools/agent-bridge/cert` | Untrusts and removes the MITM root CA from the OS trust store (explicit, idempotent). Shown only when the CA is currently trusted; requires an inline "Remove CA?" confirmation. |
| **Export config** | `GET /api/tools/agent-bridge/config` | Downloads the portable config JSON (see §3.7). |
| **Import config** | `POST /api/tools/agent-bridge/config` | Uploads a previously-exported config JSON (see §3.7). |
**Diagnostics checks** (`summarizeDiagnostics()` in `src/mitm/inspector/diagnostics.ts`). The route runs the effectful probe for each and feeds the booleans into the pure summarizer; a single `healthy` verdict plus a per-failure hint is returned:
| Check name | What it verifies | Hint on failure |
| ------------------ | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `server-running` | The MITM server process is active | "The MITM server is not running. Start it from the AgentBridge tab." |
| `server-reachable` | The MITM server accepts connections on its port (TCP probe) | "The MITM server is not accepting connections on its port. Check that the port is free and that you have privileges to bind it." |
| `cert-exists` | The MITM certificate has been generated on disk | "No MITM certificate has been generated yet. Generate one from the AgentBridge tab." |
| `cert-trusted` | The MITM root CA is in the OS trust store | "The MITM root CA is not trusted by the OS store, so TLS interception will fail. Trust the certificate from the AgentBridge tab." |
| `dns-configured` | Target hostnames are spoofed in `/etc/hosts` | "Target hostnames are not spoofed in /etc/hosts, so traffic never reaches the proxy. Enable DNS for the agent(s) you want to capture." |
**Orphaned-state banner:** when the page detects state left behind by a crash (DNS spoof / CA / system proxy), the card shows an amber banner — _"A previous session left system state behind (DNS spoof, CA, or system proxy). Run Repair to clean it up."_ — and highlights the **Repair** button. `Repair` is the application-layer analogue of ProxyBridge's `--cleanup` flag (it delegates to `repairMitm()` in `src/mitm/manager.ts`).
> The MITM root CA is kept installed across stop/start to avoid repeated sudo
> prompts (the same behavior as mitmproxy/Charles), so removing it is an explicit
> **Remove CA** action rather than something that happens automatically on stop.
### 3.7 Portable config import/export
AgentBridge can serialize the **operator-tunable** state into a versioned JSON blob so a setup can be replicated across machines. The serializer is `src/lib/inspector/configPortability.ts` (`exportConfig()` / `importConfig()`), validated by `AgentBridgeConfigSchema`.
The export includes exactly three pieces (built-in defaults are intentionally **NOT** exported, so importing never duplicates or fights them):
| Field | Source | Notes |
| ---------------- | --------------------------------------------------------- | ----------------------------------------------------------------------- |
| `bypassPatterns` | user-defined bypass patterns (`agent_bridge_bypass`) | default bank/gov/okta patterns are excluded |
| `customHosts` | Traffic Inspector custom hosts (`inspector_custom_hosts`) | each: `{ host, kind: "llm"\|"app"\|"custom", label? }` |
| `agentMappings` | per-agent model mappings (`agent_bridge_mappings`) | `{ [agentId]: [{ source, target }] }` for every agent that has mappings |
```jsonc
// GET /api/tools/agent-bridge/config
{
"version": 1,
"bypassPatterns": ["*.internal.example.com"],
"customHosts": [{ "host": "api.example.com", "kind": "llm", "label": null }],
"agentMappings": { "copilot": [{ "source": "gpt-4o", "target": "claude-sonnet-4.7" }] },
}
```
**Import behavior** (`POST /api/tools/agent-bridge/config`): bypass patterns and per-agent mappings **replace wholesale**; custom hosts are added **idempotently** (`INSERT OR IGNORE`). The response reports how many of each were applied:
```jsonc
{ "ok": true, "bypassPatterns": 1, "customHosts": 1, "agents": 1 }
```
What is **NOT** in the config: server running state, cert paths, per-agent DNS state, upstream CA path, and TPROXY settings — those are host/runtime state, not portable preferences.
---
## §4 Per-agent reference
| # | Agent | Status | Hosts intercepted | Auth type |
| --- | ------------------ | ---------------- | ------------------------------------------------------------------ | -------------- |
| 1 | **Antigravity** | ✅ Supported | `daily-cloudcode-pa.googleapis.com`, `cloudcode-pa.googleapis.com` | Firebase OAuth |
| 2 | **Kiro (AWS)** | ✅ Supported | `prod.kiro.aws`, `dev.kiro.aws` | AWS SigV4 |
| 3 | **GitHub Copilot** | ✅ Supported | `api.githubcopilot.com`, `copilot-proxy.githubusercontent.com` | GitHub OAuth |
| 4 | **OpenAI Codex** | ✅ Supported | `api.openai.com` (Codex paths), `chatgpt.com` | OpenAI key |
| 5 | **Cursor IDE** | ✅ Supported | `api2.cursor.sh`, `api.cursor.sh` | Cursor OAuth |
| 6 | **Zed Industries** | ✅ Supported | `api.zed.dev`, `llm.zed.dev` | Zed OAuth |
| 7 | **Claude Code** | ✅ Supported | `api.anthropic.com` (opt-in) | Anthropic key |
| 8 | **Open Code** | ✅ Supported | `openrouter.ai`, `api.openai.com` (zen paths) | API key |
| 9 | **Trae** | 🔍 Investigating | TBD — see §8 | TBD |
### Setup wizard steps (per agent)
Each agent card has a 3-step setup wizard:
1. **Verify prerequisites** — Server running? Cert trusted? IDE installed (auto-detected)?
2. **Enable DNS** — Adds `/etc/hosts` entries (requires sudo). Shows exactly which lines will be added.
3. **Map models** — Optional model mapping table. Wildcards accepted.
### Agent detection
For agents 1–8, AgentBridge attempts to auto-detect IDE installation:
```ts
export async function detectAgent(agentId: AgentId): Promise<DetectionResult>;
// Returns: { installed: boolean, version?: string, path?: string }
```
Detection uses OS-specific paths and binary checks (e.g., `code --list-extensions | grep github.copilot` for Copilot, `~/.config/antigravity/` for Antigravity).
---
## §5 Security
### Hard Rules applied
| Rule | Application |
| --------------------------------- | ---------------------------------------------------------------------------------------- |
| **#12** `sanitizeErrorMessage` | All handler errors are sanitized before response or buffer entry |
| **#13** Shell env-passing | `/etc/hosts` edits use `env` option — no string interpolation of paths |
| **#15 + #17** `isLocalOnlyPath()` | `/api/tools/agent-bridge/` is LOCAL_ONLY + SPAWN_CAPABLE — loopback enforced before auth |
### Bypass list for sensitive hosts
The bypass list ensures that financial institutions, OAuth/SSO providers, and other sensitive hosts are **never decrypted**. Their TLS traffic passes through as a transparent TCP tunnel — OmniRoute never sees the plaintext.
Default bypass patterns include:
- `*.bank.*`, `*.gov.*` (financial/government)
- `*.okta.com`, `*.auth0.com`, `*.microsoft.com` (SSO/identity)
- `*.apple.com`, `*.icloud.com` (Apple system services)
User-added bypass patterns are stored in `agent_bridge_bypass` table and take precedence over everything.
### Secret masking
`maskSecrets()` from `src/mitm/maskSecrets.ts` is applied:
- On every request body before `TrafficBuffer.push()`
- On every header before logging or broadcasting
Patterns: `sk-`/`ak-`/`pk-` prefix tokens, `Bearer` tokens, and generic tokens ≥40 characters.
### Upstream CA cert
When `AGENTBRIDGE_UPSTREAM_CA_CERT` is set, the file is read at startup. If the path exists but the file is unreadable, AgentBridge logs a clear error and refuses to start (prevents silent TLS failures in corporate environments).
### Known limitations
- **Port 443 requires privilege**: On Linux, AgentBridge needs `setcap 'cap_net_bind_service=+ep'` on the Node binary, or run via `authbind`. The Setup Wizard displays OS-specific instructions.
- **IDE restart required**: After DNS redirect, the IDE must be restarted for the new host resolution to take effect.
- **Hardcoded OAuth tokens**: Some agents (Kiro, Antigravity) store OAuth refresh tokens locally. These are transparent to AgentBridge — it sees the Bearer token in each request, which is masked before logging.
- **Electron frontends need `NODE_EXTRA_CA_CERTS`**: IDEs whose frontend runs on a bundled Node/Electron runtime ignore the OS/NSS trust store and must be launched from a shell with `NODE_EXTRA_CA_CERTS` set (see §3.2). Symptom when missing: the IDE backend authenticates (MITM shows `200`s) but the UI stays logged out.
- **Multiple installs of the same IDE are independent**: a system install (e.g. `/usr/share/antigravity/antigravity`) and a user-local "Full" install (e.g. `~/AntigravityIDE_Full/antigravity-ide`) are separate processes with their own runtimes — each must be relaunched with the CA injected. Identify which one is running by its binary path before relaunching.
- **Identity is set by the agent's system prompt, not the routed model**: when you remap an agent's model to a different provider, the reply still claims the agent's native identity (e.g. Antigravity answers "I am powered by Gemini") because the IDE injects that into the system prompt. Confirm the real backend in `call_logs` / `proxy_logs` (`provider`, `model`, `target_format`), not by asking the model who it is.
---
## §6 Troubleshooting
### Port 443 conflict
If another process is already listening on port 443 (web server, VPN, etc.):
```bash
lsof -i :443 # find the process
sudo fuser -k 443/tcp # force-kill (use with care)
```
Alternatively, configure a non-privileged port in AgentBridge settings and set up `iptables` / `pf` redirect rules.
### Certificate not trusted
If the IDE shows TLS errors after starting AgentBridge:
1. Verify the cert was installed: `security find-certificate -c "OmniRoute AgentBridge"` (macOS) or `certutil -L -d sql:$HOME/.pki/nssdb` (Linux/NSS)
2. Some apps maintain their own trust store (Firefox, Chrome on Linux). Run "Trust Cert" again and check the NSS/Firefox-specific cert store.
3. Restart the IDE after trusting — in-flight TLS sessions use the old trust state.
### IDE logged out / "connection error" despite a trusted CA
Symptom: after redirecting DNS and trusting the CA, an Electron-based IDE (e.g. Antigravity)
opens **logged out** or shows an authentication/connection error, yet the MITM log shows the
bootstrap calls (`loadCodeAssist`, `fetchAvailableModels`, …) returning `200`.
Cause: the IDE's **bundled Node/Electron runtime ignores the OS trust store**. The native
backend (a Go language server) trusts the OS CA and authenticates, but the Electron frontend
does not — so the UI believes it is offline.
Fix (both steps): export `NODE_EXTRA_CA_CERTS=<ca.crt>` **and relaunch the IDE from that
shell**, not from the desktop icon. Fully quit the IDE first — Electron's singleton lock means
a second launch just focuses the existing process and the new environment is ignored. See §3.2.
This mirrors an open upstream report where a standalone agent works through a MITM but the IDE
variant fails under the same setup.
### DNS not propagated
Check that `/etc/hosts` was updated:
```bash
grep "omniroute\|127.0.0.1.*github\|127.0.0.1.*cursor" /etc/hosts
```
Flush DNS cache:
```bash
# macOS
sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder
# Linux (systemd-resolved)
sudo systemctl restart systemd-resolved
# Windows
ipconfig /flushdns
```
### IDE not detected
Auto-detection uses common installation paths. If detection fails but the IDE is installed:
- Check if the IDE binary is in a non-standard location
- The Setup Wizard still works — detection failure just means the badge won't show the install path
### Handler errors (upstream fetch fails)
If AgentBridge intercepts but all requests fail:
1. Verify at least one provider is connected at `/dashboard/providers`
2. Check OmniRoute server logs: `APP_LOG_LEVEL=debug` in `.env`
3. Verify `OMNIROUTE_BASE_URL` points to the correct router endpoint (default: `http://127.0.0.1:20128`)
---
## §7 API reference
All routes are `LOCAL_ONLY` (loopback-only, enforced before auth) and `SPAWN_CAPABLE`. See `src/server/authz/routeGuard.ts`.
Base path: `/api/tools/agent-bridge/`
| Method | Path | Description |
| ------------------- | ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| GET | `/api/tools/agent-bridge/state` | Global server state + per-agent detection/status |
| GET | `/api/tools/agent-bridge/agents` | List registered agents (id, name, hosts, viability, state) |
| GET | `/api/tools/agent-bridge/agents/{id}` | State of one agent (target config + detection + stored state) |
| PATCH | `/api/tools/agent-bridge/agents/{id}` | Update `setup_completed` for agent |
| GET | `/api/tools/agent-bridge/agents/{id}/detect` | Run detection probe for agent (`installed`, `version?`, `path?`) |
| POST | `/api/tools/agent-bridge/agents/{id}/dns` | Enable/disable DNS for agent (`{enabled: boolean}`) |
| GET | `/api/tools/agent-bridge/agents/{id}/mappings` | Model mappings for agent |
| PUT | `/api/tools/agent-bridge/agents/{id}/mappings` | Replace model mappings |
| POST | `/api/tools/agent-bridge/server` | Start/stop/restart server (`action: "start"\|"stop"\|"restart"\|"trust-cert"\|"regenerate-cert"`) |
| GET | `/api/tools/agent-bridge/cert` | Cert status (`exists`, `trusted`, `path`) |
| POST | `/api/tools/agent-bridge/cert` | Trust (install) the MITM root CA |
| DELETE | `/api/tools/agent-bridge/cert` | Untrust (remove) the MITM root CA — idempotent (see §3.6) |
| POST | `/api/tools/agent-bridge/cert/regenerate` | Regenerate the self-signed MITM cert |
| GET | `/api/tools/agent-bridge/cert/download` | Stream the PEM cert for download |
| GET | `/api/tools/agent-bridge/bypass` | List bypass patterns (`default` + `user`) |
| POST | `/api/tools/agent-bridge/bypass` | Replace user-defined bypass patterns wholesale |
| DELETE | `/api/tools/agent-bridge/bypass?pattern=...` | Remove a single user-defined bypass pattern |
| GET | `/api/tools/agent-bridge/diagnose` | Capture-pipeline self-test (see §3.6) |
| POST | `/api/tools/agent-bridge/repair` | Undo orphaned MITM system state (see §3.6) |
| GET | `/api/tools/agent-bridge/config` | Export portable config JSON (see §3.7) |
| POST | `/api/tools/agent-bridge/config` | Import portable config JSON (see §3.7) |
| GET | `/api/tools/agent-bridge/upstream-ca` | Get configured upstream CA path |
| POST | `/api/tools/agent-bridge/upstream-ca` | Validate + persist upstream CA path |
| POST | `/api/tools/agent-bridge/upstream-ca/test` | Validate-only (dry-run) an upstream CA path — does not persist |
| GET / POST / DELETE | `/api/tools/agent-bridge/tproxy` | TPROXY transparent-decrypt capture mode — see [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md) |
Full OpenAPI schemas: `docs/openapi.yaml` → tag `AgentBridge`.
---
## §8 Roadmap
### Trae investigation
Trae is a relatively new AI coding assistant. Before implementing a handler:
1. Identify the binary/extension in VS Code / JetBrains marketplaces or as a standalone app
2. Capture traffic with mitmproxy to discover API hosts and endpoint shapes
3. Determine authentication mechanism
4. Assess go/no-go based on TOS and API discoverability
Until investigation completes, the Trae card in the dashboard shows a "Investigating" badge with a "Report viability" link. The handler stub at `src/mitm/handlers/trae.ts` throws a structured `Not yet implemented` error.
### Backlog agents (MITM required — no custom base URL support)
The following tools do not support custom base URLs in their current versions, making MITM the only interception path. Viability assessment is pending:
- **Windsurf** (Codeium/Cognition)
- **Amp** (Sourcegraph)
- **Amazon Q / Kiro CLI** (AWS Bedrock — separate from Kiro IDE)
- **Cowork** (Anthropic desktop)
Note: GitHub Copilot CLI ≥v1.0.19 supports `COPILOT_PROVIDER_BASE_URL` — use direct config instead of MITM for that tool.
docs/frameworks/AGENT_PROTOCOLS_GUIDE.md
+283
View File
@@ -0,0 +1,283 @@
---
title: "Agent Protocols Guide"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Agent Protocols Guide
> **Source:** `src/lib/{a2a,acp,cloudAgent}/`, `src/app/api/{a2a,acp,cloud}/`, `src/app/api/v1/agents/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute exposes three different agent-related surfaces. They look similar at first glance but solve different problems. Use this page to pick the right one.
## TL;DR
| Surface | Best for | Transport | Standard |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- | -------------------- |
| **A2A — Agent-to-Agent** | Cross-agent collaboration with peer agents that speak the A2A protocol | JSON-RPC 2.0 over HTTP | A2A v0.3 (open spec) |
| **ACP — CLI Agents Registry** | Detecting / registering / launching CLI coding agents installed on the user's machine (Cursor, Cline, Codex CLI, Claude Code, Aider, etc.) | HTTP REST | OmniRoute-specific |
| **Cloud Agents** | Submitting long-running coding tasks to external cloud services (Codex Cloud, Devin, Jules, Cursor Cloud) | HTTP REST + DB-backed tasks | OmniRoute-specific |
The three are independent — pick any subset.
## Decision Tree
```
Do you need a cloud service to do work outside this machine (Codex Cloud / Devin / Jules)?
├─ YES → Cloud Agents (POST /api/v1/agents/tasks)
└─ NO → Continue
│
Do you have a peer agent that speaks A2A and wants to collaborate?
├─ YES → A2A (POST /a2a)
└─ NO → Continue
│
Do you need to list / configure CLI coding agents installed locally?
├─ YES → ACP (GET /api/acp/agents)
└─ NO → Use plain /v1/chat/completions
```
## 1. A2A — Agent-to-Agent
**Spec:** [A2A v0.3](https://a2a-protocol.org)
**OmniRoute endpoint:** `POST /a2a` (JSON-RPC 2.0)
**Agent Card:** `GET /.well-known/agent.json`
### When to use
- Building a multi-agent system where OmniRoute is one of the peers
- Exposing OmniRoute's routing intelligence (smart-routing, quota-management, etc.) to agents in frameworks like Google ADK or generic agent meshes
- Wrapping OmniRoute behind a standard discovery + invocation surface
### Methods
- `message/send` — submit a message, receive sync response
- `message/stream` — submit + receive SSE-streamed progress events
- `tasks/get` — read task by ID
- `tasks/cancel` — cancel a running task
### Built-in skills (6)
- `smart-routing` — route a prompt through the optimal combo
- `quota-management` — report per-provider quota state
- `provider-discovery` — list installed providers with capabilities
- `cost-analysis` — estimate cost of a request/conversation
- `health-report` — aggregate breaker/cooldown/lockout state per provider
- `list-capabilities` — enumerate the agent's available skills and metadata
### Deep dive
See [A2A-SERVER.md](./A2A-SERVER.md) for transport details, agent card structure, task TTL config, and the template for adding new skills.
## 2. ACP — CLI Agents Registry
**OmniRoute endpoint:** `GET /api/acp/agents`
**Source:** `src/lib/acp/{index,manager,registry}.ts`
### What it is
ACP is OmniRoute's **local CLI agent inventory**. It detects which coding CLIs are installed on the host (Cursor, Cline, Claude Code, Codex CLI, Continue, etc.), resolves their versions, and surfaces them to the dashboard so the user can wire each CLI to point at OmniRoute.
This is NOT an external protocol — it's an internal registry that powers the "CLI Tools" UI and the CLI fingerprint tracking (see [CLI-TOOLS.md](../reference/CLI-TOOLS.md)).
### What it does
- Probes the host for installed CLI binaries (uses `which` / `where` per OS)
- Reads each CLI's version (calls `<bin> --version`)
- Optionally accepts user-defined custom agents (binary path + version probe + spawn args)
- Persists custom agents in settings
- Returns the unified list to the dashboard
### REST API
| Endpoint | Method | Description | Auth |
| ----------------- | ------ | ------------------------------------------------------------- | ------- |
| `/api/acp/agents` | GET | List detected + custom agents (installed/total counts) | API key |
| `/api/acp/agents` | POST | Add/update/remove custom agent (action discriminator in body) | API key |
Body shape for POST (`customAgentBodySchema` in `src/app/api/acp/agents/route.ts`):
```json
{
"action": "add|update|remove",
"id": "cursor",
"name": "Cursor",
"binary": "/usr/local/bin/cursor",
"versionCommand": "--version",
"providerAlias": "cursor",
"spawnArgs": ["--api-base", "http://localhost:20128"],
"protocol": "stdio"
}
```
### Use cases
- Dashboard "CLI Tools" page lists what's installed and helps you point each at OmniRoute
- Custom agents let power users register internal/proprietary CLIs that OmniRoute doesn't know about by default
- Detection result fuels the `cli-tools` fingerprint matrix
### When NOT to use ACP
- ACP doesn't _run_ tasks. It only detects + configures CLIs. To actually invoke a CLI, you launch it yourself with the env vars OmniRoute provides (`OPENAI_BASE_URL`, `OPENAI_API_KEY`, etc.).
## 3. Cloud Agents
**OmniRoute endpoints:** `/api/v1/agents/tasks/*` (lifecycle) + `/api/cloud/*` (plumbing)
**Source:** `src/lib/cloudAgent/`
### What it is
A uniform interface over third-party cloud coding agents. You submit a prompt + repo URL, OmniRoute dispatches to the right cloud agent, polls status, returns results.
### Supported agents (3, all confirmed in `src/lib/cloudAgent/agents/`)
- `codex-cloud` — OpenAI Codex Cloud
- `devin` — Cognition Devin
- `jules` — Google Jules
### Lifecycle
```
POST /api/v1/agents/tasks
→ BaseAgent.createTask() per agent class
→ external service starts work
→ task row created in DB (cloud_agent_tasks)
↓
GET /api/v1/agents/tasks/[id]
→ lazy status sync from provider
→ returns current status + plan + activity log
↓
POST /api/v1/agents/tasks/[id] (action: "approve" | "message" | "cancel")
→ forwards to provider (or marks cancelled locally)
↓
DELETE /api/v1/agents/tasks/[id]
→ local cancel
```
### Auth
⚠️ **All `/api/v1/agents/tasks/*` endpoints require management auth** (commit `588a0333`). Bearer-only callers receive 401 since v3.8.0.
### Deep dive
See [CLOUD_AGENT.md](./CLOUD_AGENT.md) for the `CloudAgentBase` contract, per-agent specifics, schema details, and credential plumbing endpoints.
## Comparison: A2A vs Cloud Agents
Both have "long-running tasks" but at different layers:
| Aspect | A2A | Cloud Agents |
| ------------------ | --------------------------------------------------------------------------------- | ---------------------------------------- |
| Standard | Open A2A v0.3 | OmniRoute-specific |
| Where compute runs | Inside OmniRoute (uses configured combos) | External (Codex / Devin / Jules servers) |
| Task duration | Default TTL 5 min (configurable in `TaskManager`) | Minutes to hours |
| Repo-aware | No (passes prompts only) | Yes (repo URL + branch) |
| Use case | Cross-agent collab, smart routing as a service | Delegate "implement feature X in repo Y" |
| Auth | Optional `OMNIROUTE_API_KEY` for `/a2a`; management for `/api/a2a/*` REST helpers | Always management |
## Integration Examples
### Discover OmniRoute's A2A capabilities
```bash
curl http://localhost:20128/.well-known/agent.json
```
Returns the Agent Card with all 5 skills, transports, and version.
### Call OmniRoute as an A2A agent
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"method": "message/send",
"params": {
"messages": [{"role": "user", "content": "Route this prompt"}],
"skillId": "smart-routing"
},
"id": 1
}'
```
### List installed CLI agents via ACP
```bash
curl http://localhost:20128/api/acp/agents \
-H "Authorization: Bearer <api-key>"
```
### Add a custom CLI agent
```bash
curl -X POST http://localhost:20128/api/acp/agents \
-H "Authorization: Bearer <api-key>" \
-H "Content-Type: application/json" \
-d '{
"action": "add",
"id": "my-custom-cli",
"name": "My Custom CLI",
"binary": "/opt/mycli/bin/mycli",
"versionCommand": "--version",
"providerAlias": "openai"
}'
```
### Submit a Cloud Agent task
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"agentId": "devin",
"prompt": "Implement feature X in repo Y",
"repo": "https://github.com/user/repo",
"branch": "main"
}'
```
### Poll cloud task status
```bash
curl http://localhost:20128/api/v1/agents/tasks/<task-id> \
-H "Cookie: auth_token=..."
```
## When to Use What
- **Chatbot / copilot frontend** → `/v1/chat/completions` (OpenAI-compat — not an agent protocol)
- **Multi-agent collaboration** → A2A
- **Listing local CLIs in the dashboard** → ACP
- **Delegating long-running coding tasks to cloud services** → Cloud Agents
## Internal Architecture
```
┌─────────────────────┐
│ OmniRoute Core │
└─────────────────────┘
↑ ↑ ↑
┌─────────┘ │ └─────────┐
│ │ │
┌───────┐ ┌─────────┐ ┌────────────┐
│ A2A │ │ ACP │ │ Cloud │
│ (/a2a)│ │ (/acp) │ │ Agents │
└───────┘ └─────────┘ │ (/v1/agents│
│ │ │ /tasks) │
↓ ↓ └────────────┘
External peer Local CLI │
agents that binaries on ↓
speak A2A v0.3 the host Codex Cloud,
Devin, Jules
```
## See Also
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A deep dive
- [CLOUD_AGENT.md](./CLOUD_AGENT.md) — Cloud Agents deep dive
- [CLI-TOOLS.md](../reference/CLI-TOOLS.md) — External CLI integrations (uses ACP)
- [SKILLS.md](./SKILLS.md) — Skills framework (different from A2A skills — local execution sandbox)
- [API_REFERENCE.md](../reference/API_REFERENCE.md#agents-protocol) — endpoint reference
- Source: `src/lib/{a2a,acp,cloudAgent}/`
docs/frameworks/CLOUD_AGENT.md
+393
View File
@@ -0,0 +1,393 @@
---
title: "Cloud Agents"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Cloud Agents
> **Source of truth:** `src/lib/cloudAgent/` and `src/app/api/v1/agents/tasks/`
> **Last updated:** 2026-06-28 — v3.8.40 (frontmatter refresh; 4 agents incl. cursor-cloud)
OmniRoute orchestrates third-party cloud-hosted coding agents (Codex Cloud, Cursor,
Devin, Jules) as long-running tasks. Each agent is wrapped behind a uniform interface so
clients can submit a prompt + repo URL and receive results without dealing with
provider-specific APIs.
A Cloud Agent task is **not** a regular chat completion. It is a durable, multi-step
unit of work that may take minutes to hours, can produce a Pull Request as its
artifact, and supports follow-up messages and (in some providers) plan approval gates.
![Cloud Agent task lifecycle](../diagrams/exported/cloud-agent-flow.svg)
> Source: [diagrams/cloud-agent-flow.mmd](../diagrams/cloud-agent-flow.mmd)
## Supported Agents
| Provider ID | Class | Source | Upstream Base URL | Plan Approval |
| -------------- | ------------------ | ------------------------------------- | --------------------------------------- | ------------- |
| `jules` | `JulesAgent` | `src/lib/cloudAgent/agents/jules.ts` | `https://jules.googleapis.com/v1alpha` | Yes |
| `devin` | `DevinAgent` | `src/lib/cloudAgent/agents/devin.ts` | `https://api.devin.ai/v1` | Yes |
| `codex-cloud` | `CodexCloudAgent` | `src/lib/cloudAgent/agents/codex.ts` | `https://api.openai.com/v1/codex/cloud` | No (auto) |
| `cursor-cloud` | `CursorCloudAgent` | `src/lib/cloudAgent/agents/cursor.ts` | `https://api.cursor.com/v0` | No (auto) |
Registry: `src/lib/cloudAgent/registry.ts` — exports `getAgent(providerId)`,
`getAvailableAgents()`, and `isCloudAgentProvider(providerId)`. The registry is a
plain in-memory `Record<string, CloudAgentBase>` populated at module load.
## Architecture
```
Client (Dashboard / CLI / API)
→ POST /api/v1/agents/tasks (management auth required)
→ CreateCloudAgentTaskSchema validation (Zod)
→ registry.getAgent(providerId)
→ getCloudAgentCredentials(providerId)
└─ pulls from getProviderConnections({ provider, isActive: true })
(apiKey first, fallback to accessToken)
→ agent.createTask({ prompt, source, options }, credentials)
└─ HTTP POST to upstream provider API
└─ returns CloudAgentTask with internal id + externalId
→ insertCloudAgentTask(...) into cloud_agent_tasks (SQLite)
Polling (lazy sync on read):
GET /api/v1/agents/tasks/[id]
→ getCloudAgentTaskById(id)
→ agent.getStatus(externalId, credentials) // refreshes status + activities
→ updateCloudAgentTask(...) with new status, result, completed_at
→ return serialized task
Interactions:
POST /api/v1/agents/tasks/[id] body: { action: "approve" | "message" | "cancel" }
→ agent.approvePlan(externalId, credentials) for "approve"
→ agent.sendMessage(externalId, message, credentials) for "message"
→ status flips to "cancelled" for "cancel" (local-only)
```
Sync is **lazy**: status is refreshed from the upstream on every `GET /tasks/[id]`.
There is no background poller. Dashboards that need fresh state should poll the GET
endpoint at a sensible interval.
## `CloudAgentBase` Interface
Source: `src/lib/cloudAgent/baseAgent.ts`
```typescript
export interface AgentCredentials {
apiKey: string;
baseUrl?: string;
}
export interface CreateTaskParams {
prompt: string;
source: CloudAgentSource;
options: {
autoCreatePr?: boolean;
planApprovalRequired?: boolean;
environment?: Record<string, string>;
};
}
export interface GetStatusResult {
status: CloudAgentStatus;
externalId?: string;
result?: CloudAgentResult;
activities: CloudAgentActivity[];
error?: string;
}
export abstract class CloudAgentBase {
abstract readonly providerId: string;
abstract readonly baseUrl: string;
abstract createTask(p: CreateTaskParams, c: AgentCredentials): Promise<CloudAgentTask>;
abstract getStatus(externalId: string, c: AgentCredentials): Promise<GetStatusResult>;
abstract approvePlan(externalId: string, c: AgentCredentials): Promise<void>;
abstract sendMessage(
externalId: string,
message: string,
c: AgentCredentials
): Promise<CloudAgentActivity>;
abstract listSources(
c: AgentCredentials
): Promise<{ name: string; url: string; branch?: string }[]>;
protected mapStatus(raw: string): CloudAgentStatus; // heuristic upstream-string → enum
protected generateTaskId(): string; // `task_<ts>_<rand>`
protected generateActivityId(): string; // `act_<ts>_<rand>`
}
```
`CodexCloudAgent.approvePlan` intentionally throws — Codex Cloud auto-plans and has
no approval gate. `CodexCloudAgent.listSources` returns `[]`.
`CursorCloudAgent` drives Cursor's Background / Cloud Agents through its official REST
API (`api.cursor.com/v0`) with a **user or service-account API key** — the safer,
first-party alternative to re-using the Cursor IDE's OAuth session (provider `cursor`,
which carries a ban-risk warning). It is a plain REST adapter (no `@cursor/sdk` native
dependency). `approvePlan` throws (Cursor agents run autonomously); `listSources` lists
the repositories reachable by the key. Cursor returns UPPERCASE status enums
(`CREATING`/`RUNNING`/`FINISHED`/`ERROR`), mapped explicitly to the shared
`CloudAgentStatus`. `baseUrl` is overridable per-credential so the API version/path can
be corrected without a code change.
## Domain Types
Source: `src/lib/cloudAgent/types.ts`
```typescript
export const CLOUD_AGENT_STATUS = {
QUEUED: "queued",
RUNNING: "running",
AWAITING_APPROVAL: "awaiting_approval",
COMPLETED: "completed",
FAILED: "failed",
CANCELLED: "cancelled",
} as const;
export interface CloudAgentSource {
repoName: string;
repoUrl: string; // must be a valid URL
branch?: string;
}
export interface CloudAgentResult {
prUrl?: string;
prNumber?: number;
commitMessage?: string;
diffUrl?: string;
summary?: string;
duration?: number; // seconds, positive int
cost?: number; // positive float
}
export interface CloudAgentActivity {
id: string;
type: "plan" | "command" | "code_change" | "message" | "error" | "completion";
content: string;
timestamp: string; // ISO 8601
metadata?: Record<string, unknown>;
}
export interface CloudAgentTask {
id: string; // internal `task_...` id
providerId: "jules" | "devin" | "codex-cloud" | "cursor-cloud";
externalId?: string; // upstream provider's id
status: CloudAgentStatus;
prompt: string; // 1..10000 chars
source: CloudAgentSource;
options: {
autoCreatePr?: boolean;
planApprovalRequired?: boolean;
environment?: Record<string, string>;
};
result?: CloudAgentResult;
activities: CloudAgentActivity[];
error?: string;
createdAt: string;
updatedAt: string;
completedAt?: string;
}
```
Validation schemas (`CreateCloudAgentTaskSchema`, `UpdateCloudAgentTaskSchema`) are
exported alongside the types and are used by the route handlers.
## Database
Source: `src/lib/cloudAgent/db.ts` — table is created lazily via
`createCloudAgentTaskTable()` (also called from `src/lib/cloudAgent/index.ts` at
module import).
```sql
CREATE TABLE IF NOT EXISTS cloud_agent_tasks (
id TEXT PRIMARY KEY,
provider_id TEXT NOT NULL,
external_id TEXT,
status TEXT NOT NULL DEFAULT 'queued',
prompt TEXT NOT NULL,
source TEXT NOT NULL, -- JSON
options TEXT DEFAULT '{}', -- JSON
result TEXT, -- JSON
activities TEXT DEFAULT '[]', -- JSON
error TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
completed_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_provider ON cloud_agent_tasks(provider_id);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_status ON cloud_agent_tasks(status);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_created ON cloud_agent_tasks(created_at DESC);
```
`updateCloudAgentTask` enforces a **column whitelist** to prevent SQL injection:
`status`, `prompt`, `source`, `options`, `result`, `activities`, `error`,
`completed_at`. Any other key in the partial update is silently dropped.
## REST API — Task Lifecycle
**Auth:** All `/api/v1/agents/tasks*` endpoints require **management auth**
(`requireCloudAgentManagementAuth` wraps `requireManagementAuth` from
`src/lib/api/requireManagementAuth`). This is enforced after commit `588a0333`
(_"fix(auth): require management auth for agent and cooldown APIs"_).
| Method | Path | Purpose |
| ------- | ----------------------------- | ------------------------------------------------------ |
| OPTIONS | `/api/v1/agents/tasks` | CORS preflight |
| GET | `/api/v1/agents/tasks` | List tasks (filter: `provider`, `status`, `limit≤500`) |
| POST | `/api/v1/agents/tasks` | Create task (dispatches to upstream + persists) |
| DELETE | `/api/v1/agents/tasks?id=...` | Delete task by query id (does **not** cancel upstream) |
| OPTIONS | `/api/v1/agents/tasks/[id]` | CORS preflight |
| GET | `/api/v1/agents/tasks/[id]` | Read task + lazy-sync status from upstream |
| POST | `/api/v1/agents/tasks/[id]` | Action: `approve` / `message` / `cancel` |
| DELETE | `/api/v1/agents/tasks/[id]` | Delete task by path id |
### Create task
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"providerId": "devin",
"prompt": "Fix the bug in src/foo.ts where the parser returns null",
"source": {
"repoName": "user/repo",
"repoUrl": "https://github.com/user/repo",
"branch": "main"
},
"options": {
"autoCreatePr": true,
"planApprovalRequired": false
}
}'
```
Response `201`:
```json
{
"data": {
"id": "task_1731512345678_abc123def",
"providerId": "devin",
"externalId": "session_xyz",
"status": "queued",
"prompt": "...",
"source": { "repoName": "user/repo", "repoUrl": "...", "branch": "main" },
"options": { "autoCreatePr": true },
"createdAt": "2026-05-13T12:34:56.789Z"
}
}
```
### Approve a plan
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{"action":"approve"}'
```
### Send a follow-up message
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-d '{"action":"message","message":"Also add a unit test for the parser"}'
```
### Cancel (local status only)
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-d '{"action":"cancel"}'
```
`cancel` flips `status` to `"cancelled"` in the local DB but does **not** call the
upstream provider — there is no abort RPC in `CloudAgentBase`. To stop billing
upstream, terminate the task in the provider's own console.
## REST API — Cloud Provider Plumbing
These auxiliary endpoints under `src/app/api/cloud/` are used by remote clients
(the CLI, the Electron app, or sync workers) to read provider connection metadata
and resolve model aliases. They are authenticated with a **regular API key**
(via `validateApiKey`), not the management auth used by the task endpoints.
| Method | Path | Purpose |
| ------ | ------------------------------- | ------------------------------------------------------------------- |
| POST | `/api/cloud/auth` | Validate API key, return masked connection metadata + model aliases |
| PUT | `/api/cloud/credentials/update` | Refresh `accessToken` / `refreshToken` / `expiresAt` |
| POST | `/api/cloud/model/resolve` | Resolve a model alias to `{ provider, model }` |
| GET | `/api/cloud/models/alias` | List all model aliases |
| PUT | `/api/cloud/models/alias` | Set a model alias (and auto-sync to Cloud if enabled) |
`/api/cloud/auth` never returns raw `apiKey` / `accessToken` / `refreshToken`. It
returns `hasApiKey`, `hasAccessToken`, `hasRefreshToken`, and a masked preview
(`maskedApiKey`: first 4 + `****` + last 4).
## Credentials Resolution
`getCloudAgentCredentials(providerId)` in `src/lib/cloudAgent/api.ts`:
1. Loads active provider connections via `getProviderConnections({ provider: providerId, isActive: true })`.
2. For each connection, prefers `apiKey` (trimmed). Falls back to `accessToken`.
3. Returns the first non-empty token wrapped as `{ apiKey: token }`.
4. Returns `null` if no usable token is found — the API responds `400` with
`"No active credentials configured for cloud agent provider: <id>"`.
This means Cloud Agents reuse the same Provider Connection table as regular LLM
providers. To enable Jules, create an active connection with `provider: "jules"`
and a populated `apiKey`.
## Dashboard
Source: `src/app/(dashboard)/dashboard/cloud-agents/page.tsx`
A `"use client"` React page that:
- Lists tasks (polled via `GET /api/v1/agents/tasks`).
- Submits new tasks via a form that maps to `CreateCloudAgentTaskSchema`.
- Shows status badges (`queued`, `running`, `awaiting_approval`, `completed`,
`failed`, `cancelled`) and renders the `activities[]` timeline.
- Surfaces the `result.prUrl` / `commitMessage` / `summary` when `status === "completed"`.
## Integration with A2A
Cloud Agents can be exposed as A2A skills by registering an A2A skill that delegates
its `tasks/send` handler to `getAgent(...).createTask(...)` and translates A2A task
status events to the JSON-RPC 2.0 protocol. See [A2A-SERVER.md](./A2A-SERVER.md).
## Adding a New Cloud Agent
1. Create `src/lib/cloudAgent/agents/<name>.ts` extending `CloudAgentBase`.
2. Implement `createTask`, `getStatus`, `approvePlan` (or throw if N/A),
`sendMessage`, `listSources`. Use `this.mapStatus(...)` for status normalization.
3. Register in `src/lib/cloudAgent/registry.ts` under a stable `providerId`.
4. Extend the `providerId` literal union in `src/lib/cloudAgent/types.ts`
(`CloudAgentTask.providerId` and `CreateCloudAgentTaskSchema`).
5. Add the provider to `src/shared/constants/providers.ts` if it needs a connection
record. OAuth-based providers also need `src/lib/oauth/providers/`.
6. Add tests under `tests/unit/cloud-agent-*.test.ts`.
7. Update this doc and the dashboard's `CLOUD_AGENTS` constant.
## Configuration
| Env Var | Purpose |
| ---------------- | ----------------------------------------------------------- |
| `DATA_DIR` | Location of the SQLite database holding `cloud_agent_tasks` |
| `JWT_SECRET` | Required for management auth on task endpoints |
| `API_KEY_SECRET` | Required to encrypt provider connection credentials at rest |
No Cloud-Agent-specific env vars exist today — every secret lives in the
`provider_connections` table.
## See Also
- [A2A-SERVER.md](./A2A-SERVER.md)
- [API_REFERENCE.md](../reference/API_REFERENCE.md)
- [SKILLS.md](./SKILLS.md)
- [MEMORY.md](./MEMORY.md)
- Source: `src/lib/cloudAgent/`
- Routes: `src/app/api/v1/agents/tasks/`, `src/app/api/cloud/`
- Dashboard: `src/app/(dashboard)/dashboard/cloud-agents/page.tsx`
docs/frameworks/EMBEDDED-SERVICES.md
+856
View File
@@ -0,0 +1,856 @@
---
title: "Embedded Services"
description: "Reference for 9Router, CLIProxyAPI, Mux, and Bifrost"
---
# Embedded Services
> **Version:** v3.8.44
> **Last updated:** 2026-07-03
> **Audience:** Engineers adding, maintaining, or debugging embedded services (9Router, CLIProxyAPI, Mux, Bifrost).
Embedded services are locally-installed process sidecar tools that OmniRoute installs, supervises, and
exposes as first-class routing targets. Unlike external providers (which are reached over the internet
via API keys), embedded services run on the same machine as OmniRoute and communicate over loopback.
---
## Table of Contents
1. [Overview](#1-overview)
2. [Architecture — 4 layers](#2-architecture--4-layers)
3. [Lifecycle state machine](#3-lifecycle-state-machine)
4. [API reference](#4-api-reference)
5. [Security](#5-security)
6. [Adding a new embedded service](#6-adding-a-new-embedded-service)
7. [Troubleshooting](#7-troubleshooting)
8. [FAQ](#8-faq)
---
## 1. Overview
### Why embedded services?
Four services are embedded as of v3.8.44:
| Service | npm package | Default port | Purpose |
| --------------- | ----------------------------------------------- | :----------: | ------------------------------------------------------------------------------------------------------------------ |
| **9Router** | `9router` | 20130 | AI router that OmniRoute can use as a sub-provider. Models exposed as `9router/{sub}/{model}` |
| **CLIProxyAPI** | `@anthropic/cli-proxy` (via `cliproxy` binary) | auto | Local proxy adapter for Anthropic CLI auth flows. Provides fallback routing when OAuth tokens expire |
| **Mux** | `mux` (headless `mux server`) | 8322 | Local agent-orchestration daemon (coder/mux). Lifecycle-managed only — not a routing target (no LLM proxying). |
| **Bifrost** | `@maximhq/bifrost` | 8080 | Go AI-gateway relay backend. When running, auto-selected by the relay route (`/v1/relay/`) |
All four follow the same supervisory model:
- OmniRoute installs them under `DATA_DIR/services/{name}/` (isolated from OmniRoute's own `package.json`)
- OmniRoute spawns and monitors them as child processes
- OmniRoute injects an ephemeral API key into the child's environment and rotates it without downtime (where applicable)
- All management routes (`/api/services/*`) are **LOCAL_ONLY** — accessible only from loopback (hard rule #17)
### Key decisions (from design plan)
| Decision | Value |
| ------------------------------------- | ------------------------------------------------------------------------ |
| Dashboard access to 9Router native UI | Reverse proxy at `/dashboard/providers/services/9router/embed/*` |
| Installation mechanism | `npm install {package}` via `execFile` (no shell interpolation) |
| Consumption mode | Provider registered as `9router/{sub}/{model}` in routing engine |
| API key management | OmniRoute generates, encrypts at-rest (AES-256-GCM), and injects via env |
| Dashboard location | `/dashboard/providers/services` (three tabs) |
| Auto-start | Toggle per service, default OFF |
---
## 2. Architecture — 4 layers
```
┌────────────────────────────────────────────────────────────────────┐
│ Layer 1 — UI │
│ /dashboard/providers/services (tabs: CLIProxyAPI | 9Router | Mux)│
│ Logs live (SSE), Start/Stop/Restart/Update, Settings, Install │
│ │
│ src/app/(dashboard)/dashboard/providers/services/ │
│ ├── page.tsx Shell + tab routing by ?tab= │
│ ├── tabs/ CliproxyServiceTab, NinerouterServiceTab,│
│ │ MuxServiceTab │
│ └── components/ ServiceStatusCard, ServiceLifecycleButtons,│
│ ServiceLogsPanel, ApiKeyCard, ... │
└──────────────────────┬─────────────────────────────────────────────┘
│ HTTP (Next.js fetch)
┌──────────────────────▼─────────────────────────────────────────────┐
│ Layer 2 — API (LOCAL_ONLY — loopback only) │
│ │
│ /api/services/9router/{install|start|stop|restart|update| │
│ rotate-key|status|auto-start|logs} │
│ /api/services/cliproxy/{install|start|stop|restart|update| │
│ status|auto-start|logs} │
│ /api/services/mux/{install|start|stop|restart|update| │
│ status|auto-start|logs} │
│ /dashboard/providers/services/9router/embed/[...path] │
│ (reverse HTTP + WebSocket proxy → 9Router upstream) │
│ │
│ Gate: LOCAL_ONLY_API_PREFIXES includes "/api/services/" and │
│ "/dashboard/providers/services/*/embed/" │
└──────────────────────┬─────────────────────────────────────────────┘
│ in-process calls
┌──────────────────────▼─────────────────────────────────────────────┐
│ Layer 3 — ServiceSupervisor (src/lib/services/) │
│ │
│ ServiceSupervisor.ts Generic supervisor (child_process.spawn) │
│ ├── install: execFile('npm', ['install', pkg, '--prefix']) │
│ ├── start: spawn(node, [entrypoint], {env, cwd}) │
│ ├── api_key: crypto.randomBytes(32) → env NINEROUTER_API_KEY │
│ ├── port: 20130 for 9Router (configurable) │
│ ├── logs: stdio ring buffer 5 MB → SSE events │
│ ├── health: HTTP GET /health every 2–5 s, lazy recovery │
│ └── lifecycle: SIGTERM 15 s → SIGKILL │
│ │
│ registry.ts getSupervisor(name) / registerSupervisor() │
│ bootstrap.ts Bootstraps all SERVICES[] at process start │
│ apiKey.ts getOrCreateApiKey(), generateServiceApiKey() │
│ modelSync.ts Periodic GET /v1/models → service_models table │
│ ringBuffer.ts Circular log buffer (5 MB per service) │
│ healthCheck.ts Polling HTTP health probe │
│ installers/ ninerouter.ts, cliproxy.ts, mux.ts │
│ (installer adapters) │
└──────────────────────┬─────────────────────────────────────────────┘
│ OpenAI-compatible HTTP (loopback)
┌──────────────────────▼─────────────────────────────────────────────┐
│ Layer 4 — Provider / Routing │
│ │
│ open-sse/executors/ninerouter.ts │
│ Re-looks up port and API key per-request (no caching). │
│ Strips "9router/" prefix from model id before proxying. │
│ Returns 503 service_not_running if supervisor not in "running". │
│ │
│ src/shared/constants/providers.ts │
│ Entry for "9router": isEmbeddedService: true │
│ │
│ open-sse/config/providerRegistry.ts │
│ Models stored as "9router/{sub}/{model}" (prefixed). │
│ Synced every 5 min by modelSync.ts. │
│ │
│ Mux is lifecycle-managed ONLY (Layers 1-3) — it is an agent- │
│ orchestration daemon, not an LLM proxy, so it has no Layer 4 │
│ executor/provider entry and is never a routing target. │
└────────────────────────────────────────────────────────────────────┘
```
### Key source files
| File | Role |
| ------------------------------------------- | ------------------------------------------------ |
| `src/lib/services/ServiceSupervisor.ts` | Core class: lifecycle, lock, health, ring buffer |
| `src/lib/services/bootstrap.ts` | Process-level registration and auto-start |
| `src/lib/services/registry.ts` | Singleton map `tool → supervisor` |
| `src/lib/services/apiKey.ts` | Key generation, AES-256-GCM encryption at-rest |
| `src/lib/services/modelSync.ts` | Periodic model sync (5 min) + on-demand |
| `src/lib/services/ringBuffer.ts` | 5 MB circular log buffer with SSE subscribe |
| `src/lib/services/healthCheck.ts` | HTTP health probe (configurable interval) |
| `src/lib/services/installers/ninerouter.ts` | npm install/update/uninstall for 9Router |
| `src/lib/services/installers/cliproxy.ts` | npm install/update/uninstall for CLIProxyAPI |
| `src/lib/services/installers/mux.ts` | npm install/update/uninstall for Mux |
| `src/app/api/services/9router/_lib.ts` | `getOrInitSupervisor()` helper |
| `src/app/api/services/[name]/logs/route.ts` | Shared SSE logs endpoint |
| `open-sse/executors/ninerouter.ts` | Provider executor (Layer 4) |
---
## 3. Lifecycle state machine
```
install()
┌─────────────┐ ──────────► ┌─────────────┐
│ not_installed│ │ stopped │◄──────────────────┐
└─────────────┘ └──────┬──────┘ │
│ start() │
▼ │ stop()
┌──────────┐ │
│ starting │ │
└────┬─────┘ │
health probe ok │ crash / SIGTERM │
┌────▼─────┐ (exit within 5s) │
│ running │──── crash ──────────►┤
└────┬─────┘ ┌─▼────┐
stop() │ │error │
▼ └──────┘
┌──────────┐
│ stopping │
└──────────┘
```
States stored in the `version_manager` DB table (`status` column) and mirrored
in `ServiceSupervisor` in-memory state. The in-memory state is authoritative for
a running process; the DB state is the durable fallback at boot.
### State transitions
| From | Event | To |
| --------------- | ---------------------------------- | ---------------------- |
| `not_installed` | `install()` succeeds | `stopped` |
| `stopped` | `start()` called | `starting` |
| `starting` | health probe returns 200 | `running` |
| `starting` | process exits before healthy | `error` |
| `running` | `stop()` called | `stopping` → `stopped` |
| `running` | process exits unexpectedly (< 5 s) | `error` (fast crash) |
| `running` | process exits unexpectedly (> 5 s) | `error` |
| `error` | `start()` called | `starting` |
| any | `stop()` while `stopping` | no-op |
### Operation lock
`ServiceSupervisor` serializes lifecycle operations through an async operation lock
(`withLock()`). Concurrent `start()` calls on the same supervisor result in exactly
one spawn; the second caller waits and returns the existing status. This prevents
race conditions when, for example, auto-start and a UI button fire simultaneously.
---
## 4. API reference
All routes under `/api/services/` are **LOCAL_ONLY** (loopback only, hard rule #17).
Non-loopback requests receive `403 LOCAL_ONLY` regardless of auth token.
### 4.1 9Router endpoints (8 routes)
#### `POST /api/services/9router/install`
Install 9Router from npm. Creates `DATA_DIR/services/9router/` with its own
`package.json` and `node_modules/`. Does not conflict with OmniRoute's own deps.
**Request body** (all optional):
```json
{ "version": "latest" }
```
| Field | Type | Default | Description |
| --------- | -------- | ---------- | ------------------------------------ |
| `version` | `string` | `"latest"` | npm version tag or semver to install |
**Responses:**
| Status | Description |
| ------ | ------------------------------------------------------ |
| `200` | `{ ok: true, installedVersion: "x.y.z", path: "..." }` |
| `400` | Invalid request body (Zod validation failure) |
| `409` | Already installing (lock held) |
| `500` | npm install failed — see `message` for friendly error |
**Notes:** Uses `execFile('npm', [...])` — no shell, no interpolation (hard rule #13).
EACCES errors are surfaced as friendly messages.
---
#### `POST /api/services/9router/start`
Start 9Router. Registers a supervisor if not already registered, then calls
`supervisor.start()`. Idempotent when already running.
**Request body:** none
**Responses:**
| Status | Description |
| ------ | ---------------------------------------------------- |
| `200` | `ServiceStatus` object (see schema below) |
| `409` | 9Router is not installed (`status: "not_installed"`) |
| `503` | Start failed (process error — see `lastError`) |
**ServiceStatus schema:**
```json
{
"tool": "9router",
"state": "running",
"pid": 12345,
"port": 20130,
"health": "healthy",
"startedAt": "2026-05-25T10:00:00.000Z",
"lastError": null
}
```
---
#### `POST /api/services/9router/stop`
Gracefully stop 9Router. Sends SIGTERM, waits 15 s, then SIGKILL if still alive.
Idempotent when already stopped.
**Request body:** none
**Responses:**
| Status | Description |
| ------ | ---------------------------------- |
| `200` | `ServiceStatus` (state: "stopped") |
| `503` | Stop failed unexpectedly |
---
#### `POST /api/services/9router/restart`
Equivalent to `stop()` then `start()` under the operation lock.
**Request body:** none
**Responses:** same as `start` (returns final `ServiceStatus`).
---
#### `POST /api/services/9router/update`
Updates 9Router to a newer npm version. If the service is running, it is stopped
first, npm install is run (installing the newer version in-place), and then the
service is restarted.
**Request body** (all optional):
```json
{ "version": "latest" }
```
**Responses:**
| Status | Description |
| ------ | --------------------------------------------------------------- |
| `200` | `{ ok: true, previousVersion: "...", installedVersion: "..." }` |
| `400` | Invalid body |
| `500` | npm update failed |
---
#### `POST /api/services/9router/rotate-key`
Generates a new API key for 9Router, encrypts it at-rest, and restarts the service
(if running) so it picks up the new key from its environment. The old key is
invalidated immediately.
**Request body:** none
**Responses:**
| Status | Description |
| ------ | ------------------------------------------ |
| `200` | `{ keyRotated: true, restarted: boolean }` |
| `500` | Rotation failed |
**Security:** The new key is never returned in the response (no credential leak).
It is stored encrypted (AES-256-GCM) in the `version_manager` table.
---
#### `GET /api/services/9router/status`
Returns combined live + DB status including version metadata and API key preview.
**Responses:**
| Status | Description |
| ------ | ------------------ |
| `200` | See schema below |
| `500` | Status read failed |
**Response schema:**
```json
{
"tool": "9router",
"state": "running",
"pid": 12345,
"port": 20130,
"health": "healthy",
"startedAt": "2026-05-25T10:00:00.000Z",
"lastError": null,
"installedVersion": "1.2.3",
"latestVersion": "1.2.4",
"updateAvailable": true,
"apiKeyMasked": "nr_****abcd",
"autoStart": false,
"providerExpose": false
}
```
---
#### `POST /api/services/9router/auto-start`
Toggle the auto-start flag. When `enabled: true`, the service starts automatically
the next time OmniRoute boots (if the service is installed).
**Request body:**
```json
{ "enabled": true }
```
**Responses:**
| Status | Description |
| ------ | --------------------- |
| `200` | `{ autoStart: true }` |
| `400` | Invalid body |
---
#### `GET /api/services/9router/logs`
SSE stream of live logs from 9Router's stdout/stderr ring buffer.
**Query parameters:**
| Param | Type | Default | Description |
| -------- | --------- | ------- | --------------------------------------------------------- |
| `tail` | `integer` | 200 | How many historical lines to send first (max 1000) |
| `filter` | `string` | none | Case-insensitive substring filter (no regex — ReDoS-safe) |
**SSE events:**
| Event | Data | Description |
| ----------- | ----------- | ----------------------- |
| `snapshot` | `LogLine[]` | Initial historical tail |
| `log` | `LogLine` | Live log line |
| `heartbeat` | `{}` | Keep-alive every 15 s |
**LogLine schema:**
```json
{ "ts": 1716633600000, "stream": "stdout", "line": "[9router] Listening on :20130" }
```
**Responses:**
| Status | Description |
| ------ | --------------------------------------------- |
| `200` | `text/event-stream` |
| `400` | `filter` parameter too long (> 200 chars) |
| `404` | Service not found (supervisor not registered) |
---
### 4.2 CLIProxyAPI endpoints (7 routes)
CLIProxyAPI has the same endpoint shape as 9Router minus `rotate-key` (CLIProxyAPI
does not require an injected API key; it authenticates via the host's existing CLI
config) and `status` includes fewer fields.
| Method | Path | Description |
| ------ | ----------------------------------- | ------------------------------------ |
| `POST` | `/api/services/cliproxy/install` | Install CLIProxyAPI from npm |
| `POST` | `/api/services/cliproxy/start` | Start CLIProxyAPI |
| `POST` | `/api/services/cliproxy/stop` | Stop CLIProxyAPI |
| `POST` | `/api/services/cliproxy/restart` | Restart CLIProxyAPI |
| `POST` | `/api/services/cliproxy/update` | Update to newer version |
| `GET` | `/api/services/cliproxy/status` | Live + DB status (no `apiKeyMasked`) |
| `POST` | `/api/services/cliproxy/auto-start` | Toggle auto-start |
The shared `GET /api/services/{name}/logs` endpoint (see §4.1) works for all
four services using the `[name]` dynamic segment.
---
### 4.3 Mux endpoints (7 routes)
Mux has the same endpoint shape as CLIProxyAPI — no `rotate-key` route in the API
surface (the bearer token is generated the same way as 9Router's via
`getOrCreateApiKey("mux")` and injected via the `MUX_SERVER_AUTH_TOKEN` env var, but
there is no dedicated rotation endpoint yet). Mux is lifecycle-managed only: unlike
9Router, it has no Layer 4 executor and is never registered as a routing provider.
| Method | Path | Description |
| ------ | -------------------------------- | ------------------------------------- |
| `POST` | `/api/services/mux/install` | Install Mux from npm (`npm i mux`) |
| `POST` | `/api/services/mux/start` | Start Mux (`mux server`) |
| `POST` | `/api/services/mux/stop` | Stop Mux |
| `POST` | `/api/services/mux/restart` | Restart Mux |
| `POST` | `/api/services/mux/update` | Update to newer npm version |
| `GET` | `/api/services/mux/status` | Live + DB status |
| `POST` | `/api/services/mux/auto-start` | Toggle auto-start |
---
### 4.4 Bifrost endpoints (7 routes)
Bifrost is a Go AI-gateway relay backend (`@maximhq/bifrost`). It uses the same
endpoint shape as CLIProxyAPI (no `rotate-key` — Bifrost manages its own provider
keys in `config.json` under its `-app-dir`).
| Method | Path | Description |
| ------ | ---------------------------------- | ------------------------------------------------------ |
| `POST` | `/api/services/bifrost/install` | Install Bifrost from npm (`@maximhq/bifrost`) |
| `POST` | `/api/services/bifrost/start` | Start Bifrost on port 8080 (default) |
| `POST` | `/api/services/bifrost/stop` | Stop Bifrost |
| `POST` | `/api/services/bifrost/restart` | Restart Bifrost |
| `POST` | `/api/services/bifrost/update` | Update to newer version |
| `GET` | `/api/services/bifrost/status` | Live + DB status |
| `POST` | `/api/services/bifrost/auto-start` | Toggle auto-start |
| `GET` | `/api/services/bifrost/logs` | SSE log tail (via shared `[name]/logs` dynamic route) |
**Routing wiring:** When `BIFROST_BASE_URL` is unset and the supervised Bifrost
instance is running, `getBifrostRoutingConfig()` (in `routingBackend.ts`) automatically
uses `http://127.0.0.1:{port}` as the relay base URL. Explicit `BIFROST_BASE_URL` env
always takes precedence.
---
### 4.4 Reverse proxy (9Router dashboard embed)
The dashboard embeds the 9Router web UI inside an iframe via an internal reverse
proxy at:
```
GET|POST|... /dashboard/providers/services/9router/embed/[...path]
```
This proxy:
- Forwards the request to `http://127.0.0.1:{port}/{path}` (loopback only)
- Strips incoming `cookie` and `authorization` headers (no leakage of OmniRoute session)
- Injects `Authorization: Bearer {apiKey}` for 9Router authentication
- Strips `set-cookie`, `content-security-policy`, `x-frame-options`, `cross-origin-*` from the response
- Rewrites HTML responses to inject `<base href>` and normalize absolute paths (`/foo` → `/dashboard/.../embed/foo`)
WebSocket upgrades for the embedded dashboard are handled by a companion server on a
dedicated port (see `src/lib/services/embedWsProxy.ts`).
**Security:** The embed proxy routes are classified under `LOCAL_ONLY_API_PREFIXES`
and can only be reached from loopback. An attacker who obtains a JWT via a
Cloudflare/Ngrok tunnel cannot proxy into embedded services.
---
## 5. Security
### LOCAL_ONLY enforcement (hard rule #17)
All routes under `/api/services/` and `/dashboard/providers/services/*/embed/` are
classified as LOCAL_ONLY in `src/server/authz/routeGuard.ts`. The loopback check
runs unconditionally before any auth branch:
```
request arrives
→ isLocalOnlyPath(path)?
→ non-loopback → 403 LOCAL_ONLY (always, before auth check)
→ loopback → fall through to normal auth
```
This prevents a leaked JWT (e.g., via a tunnel) from triggering `npm install` or
process spawning. See `docs/security/ROUTE_GUARD_TIERS.md` for the full tier
matrix.
### API key injection
9Router and Mux require an API key/bearer token for their own HTTP endpoints.
OmniRoute:
1. Generates a key via `crypto.randomBytes(32).toString("base64url")` with a
service-specific prefix (`nr_` for 9Router, `mx_` for Mux).
2. Encrypts it at-rest using AES-256-GCM (same cipher used for provider credentials).
3. Decrypts and injects it as an environment variable at spawn time —
`NINEROUTER_API_KEY` for 9Router, `MUX_SERVER_AUTH_TOKEN` for Mux (never a CLI
flag, so the token never appears in `ps`/process listings).
4. Never returns the plaintext key in any HTTP response.
CLIProxyAPI does not require an injected key (it authenticates via the host's
existing CLI config).
### SSRF defense
The reverse HTTP proxy (`/dashboard/.../embed/[...path]`) is hardcoded to forward
only to `http://127.0.0.1:{port}`. It never follows redirects to non-loopback
destinations. The `ssrf-req-filter` library is used to reject any upstream URL that
resolves outside the loopback range.
### Shell safety (hard rule #13)
`npm install` is invoked via `execFile('npm', ['install', pkg, '--prefix', dir])` —
no template literals, no shell, no interpolation of external paths into the command
string. Runtime values (ports, API keys) are passed via the child's `env` object.
### Error sanitization (hard rule #12)
All error responses from `/api/services/*` go through `buildErrorBody()` or
`sanitizeErrorMessage()`. Raw `err.stack` and `err.message` are never returned
verbatim to the caller.
---
## 6. Adding a new embedded service
Follow these 8 steps. Read the existing implementations in `src/lib/services/installers/`
and `src/app/api/services/` as the canonical reference.
### Step 1 — Create the installer
Create `src/lib/services/installers/{name}.ts` modeled on `ninerouter.ts`:
```typescript
export const NAME_PACKAGE = "your-npm-package";
export const NAME_DEFAULT_PORT = 20132; // pick a free port
export async function install(version = "latest"): Promise<InstallResult> { ... }
export async function update(version = "latest"): Promise<InstallResult> { ... }
export async function uninstall(): Promise<void> { ... }
export function resolveSpawnArgs(apiKey: string, port: number): SpawnArgs { ... }
export async function getInstalledVersion(): Promise<string | null> { ... }
export async function getLatestVersion(): Promise<string | null> { ... }
```
Use `runNpm(['install', NAME_PACKAGE, '--prefix', dir])` from `installers/utils.ts`
— never `execSync` or shell interpolation.
### Step 2 — Register in bootstrap
Add a `ServiceEntry` to the `SERVICES` array in `src/lib/services/bootstrap.ts`:
```typescript
{
tool: "myservice",
port: NAME_DEFAULT_PORT,
healthPath: "/health",
healthIntervalMs: 5_000,
stopTimeoutMs: 15_000,
logsBufferBytes: 5_242_880,
needsApiKey: true, // false if no API key needed
}
```
Extend `buildSpawnArgsFactory()` to handle `cfg.tool === "myservice"`.
### Step 3 — Add migration and DB seed
Ensure the service has a row in `version_manager` via a migration in
`src/lib/db/migrations/`. The row should have:
```sql
INSERT OR IGNORE INTO version_manager (tool, status, auto_start, provider_expose)
VALUES ('myservice', 'not_installed', 0, 0);
```
### Step 4 — Create the 7 API endpoints
Under `src/app/api/services/{name}/`:
```
_lib.ts getOrInitSupervisor() helper
install/route.ts POST — calls installer.install()
start/route.ts POST — calls supervisor.start()
stop/route.ts POST — calls supervisor.stop()
restart/route.ts POST — calls supervisor.restart()
update/route.ts POST — calls installer.update()
status/route.ts GET — merges live + DB status
auto-start/route.ts POST — toggles auto_start flag
```
The shared `GET /api/services/[name]/logs` route is already wired — no changes
needed there.
Delegate all error responses through `createErrorResponse()` / `buildErrorBody()`.
### Step 5 — Add to LOCAL_ONLY_API_PREFIXES
In `src/server/authz/routeGuard.ts`, verify that `/api/services/` is already listed.
If you introduce a new prefix (e.g., `/api/tools/`), add it to both
`LOCAL_ONLY_API_PREFIXES` and, if it spawns processes, to `SPAWN_CAPABLE_PREFIXES`.
Add a test in `tests/unit/authz/routeGuard.test.ts`.
### Step 6 — Add the UI tab
Create `src/app/(dashboard)/dashboard/providers/services/tabs/{Name}ServiceTab.tsx`.
Reuse shared components:
- `ServiceStatusCard` — live state + health badge
- `ServiceLifecycleButtons` — Start / Stop / Restart / Update
- `ServiceLogsPanel` — SSE log tail (connects to `/api/services/{name}/logs`)
- `ApiKeyCard` — key reveal + rotate (if `needsApiKey: true`)
Register the tab in `ServicesPageShell.tsx`.
### Step 7 — Add the provider entry (if the service is a routing target)
If the embedded service exposes an OpenAI-compatible `/v1/chat/completions` endpoint:
1. Add a provider entry in `src/shared/constants/providers.ts` with `isEmbeddedService: true`.
2. Create `open-sse/executors/{name}.ts` extending `BaseExecutor`. Re-lookup port and
API key per-request (never cache in the constructor). Return a `503 service_not_running`
response when the supervisor state is not `"running"`.
3. Register models in `open-sse/config/providerRegistry.ts` with the service prefix
(e.g., `myservice/sub/model`). `modelSync.ts` will keep them updated.
### Step 8 — Document and test
1. Update `docs/frameworks/EMBEDDED-SERVICES.md` (this file) — add the service to the
table in §1 and any new endpoints to §4.
2. Add unit tests in `tests/unit/services/` (lifecycle, installer, API shape).
3. Add integration test in `tests/integration/services/` (behind `RUN_SERVICES_INT=1`).
4. Update `docs/openapi.yaml` with the new endpoints.
---
## 7. Troubleshooting
### Service does not start
**Symptoms:** Start button returns 503, state stays `"error"` or `"starting"`.
**Checklist:**
1. Check `GET /api/services/{name}/logs` (or the Logs panel in the dashboard). Look
for lines like `Error: ENOENT`, `address already in use`, or `Cannot find module`.
2. Verify `npm` is in PATH: `which npm` from the same user account that runs OmniRoute.
3. Verify the service is installed: check `GET /api/services/{name}/status` for
`installedVersion`. If `null`, run install first.
4. Check `DATA_DIR/services/{name}/node_modules/` exists and is not empty.
5. Check the `lastError` field in the status response for the sanitized exit reason.
---
### Cold start is slow (> 10 s to reach `running`)
**Symptoms:** State stays `"starting"` for a long time before going to `"running"` or `"error"`.
**Explanation:** 9Router's cold start includes importing large dependency trees (DNS,
tunnel, MITM modules). Default health interval is 2 s with 3 attempts before the
supervisor declares a timeout (but continues polling).
**Fix:** The `healthIntervalMs` and the `waitForHealthy` timeout
(`healthIntervalMs * 3`) are configurable in `bootstrap.ts`. For services with longer
startup times, increase `healthIntervalMs` to 5000 and `stopTimeoutMs` to 30 000.
---
### Port collision (`EADDRINUSE`)
**Symptoms:** Logs show `address already in use :::20130`.
**Causes:**
- Another process is already using port 20130.
- A previous 9Router process was not fully stopped (zombie PID).
**Fix:**
1. Change the default port via `NINEROUTER_PORT` environment variable in `.env`.
2. Find and kill the conflicting process: `lsof -ti :20130 | xargs kill -9`.
3. The port is configurable per service in `bootstrap.ts` via the `port` field.
**Note:** 9Router defaults to port 20130 specifically to avoid colliding with
OmniRoute's default port 20128.
---
### Permission denied (EACCES) on install
**Symptoms:** Install returns 500, logs show `EACCES` or `permission denied`.
**Causes:**
- `DATA_DIR` or its parent is not writable by the OmniRoute process.
- Running inside Docker rootless without write access to the mapped volume.
**Fix:**
1. Check `DATA_DIR` (default: `~/.omniroute/`): `ls -la ~/.omniroute/`
2. Ensure the OmniRoute process user owns the directory: `chown -R $USER ~/.omniroute/`
3. In Docker, ensure the volume mount has the correct permissions for the container user.
---
### Update fails (`npm install` timeout or network error)
**Symptoms:** Update returns 500 with `InstallError`, logs show network timeout.
**Checklist:**
1. Confirm npm registry is reachable: `npm ping`.
2. Check for corporate proxy: `npm config get proxy`, `npm config get https-proxy`.
3. Try the install manually: `npm install {package}@latest --prefix ~/.omniroute/services/{name}/`.
4. If behind an air-gap, pre-download the tarball and use `npm install /path/to/tarball.tgz`.
---
### Service shows `"error"` state immediately after start (fast crash)
**Symptoms:** State transitions from `"starting"` to `"error"` in under 5 seconds.
`lastError` shows `"Fast crash (exited with code 1)"`.
**Checklist:**
1. Read the full log tail: `GET /api/services/{name}/logs?tail=500`.
2. Common cause: missing environment variables expected by the service.
3. For 9Router: verify `NINEROUTER_DISABLE_MITM=true` and
`NINEROUTER_DISABLE_TUNNEL=true` are in the env passed at spawn (see
`installers/ninerouter.ts` `resolveSpawnArgs`).
---
## 8. FAQ
**Q: Can I expose the embedded services endpoints to non-loopback clients?**
No. The LOCAL_ONLY tier is intentional (hard rule #17). Routes that can invoke
`npm install` or spawn `node` processes must not be reachable from non-loopback
traffic, because a leaked JWT via a tunnel (Cloudflare, Ngrok, Tailscale) would
otherwise allow arbitrary process spawning. There is no opt-out carve-out for
`/api/services/` — unlike `/api/mcp/`, it is excluded from the manage-scope bypass
list. See `docs/security/ROUTE_GUARD_TIERS.md`.
---
**Q: Will 9Router and CLIProxyAPI be available in production/cloud deployments?**
Yes. Both services follow the same local-first model as OmniRoute itself. They run
on the same machine and communicate over loopback. "Production" here means the VPS
or local server where OmniRoute is deployed, not a remote cloud provider.
---
**Q: How do I debug the supervisor?**
1. Tail the SSE log stream: `curl -N http://localhost:20128/api/services/9router/logs`.
2. Check structured logs in OmniRoute's pino output filtered by
`service:supervisor` namespace.
3. Inspect the DB row: `sqlite3 ~/.omniroute/omniroute.db "SELECT * FROM version_manager WHERE tool='9router'"`.
4. Use `GET /api/services/9router/status` to see the current live state, PID, health,
and `lastError` in one call.
---
**Q: The supervisor shows `health: "degraded"` or `health: "unknown"` but state is `"running"`. Is that a problem?**
`"degraded"` means the health probe returned a non-200 response. `"unknown"` means no
probe has completed yet (race with first poll). Both are transient during startup.
If health stays `"degraded"` for more than `healthIntervalMs * 3` ms after
`"running"`, the embedded service is running but its HTTP API is not responding. Check
whether the port is correct in the status response and whether the service is actually
listening on that port.
---
**Q: Can I change the 9Router API key without a full restart?**
No. The API key is passed to 9Router via an environment variable at spawn time.
Environment variables cannot be changed in a running process. `POST .../rotate-key`
automatically stops and restarts the service to apply the new key. The key rotation
takes effect within the service's `stopTimeoutMs` (default 15 s) plus its startup
time.
---
**Q: What is the ring buffer limit and what happens when it fills?**
Each service has a dedicated 5 MB ring buffer. When the buffer is full, the oldest
log lines are evicted to make room for new ones. The SSE `snapshot` event returns
the most recent lines within the `tail` limit. Logs are not persisted to disk unless
`logsBufferPath` is set in the DB row.
---
## See also
- `docs/security/ROUTE_GUARD_TIERS.md` — LOCAL_ONLY tier details
- `docs/architecture/CODEBASE_DOCUMENTATION.md` — §3.2 Embedded Services module mapping
- `docs/architecture/ARCHITECTURE.md` — system-level context
- `docs/openapi.yaml` — machine-readable endpoint definitions
- `CLAUDE.md` §"Adding a New Embedded Service" — quick-reference checklist
docs/frameworks/EVALS.md
+250
View File
@@ -0,0 +1,250 @@
---
title: "Evaluations (Evals)"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Evaluations (Evals)
> **Source of truth:** `src/lib/evals/`, `src/lib/db/evals.ts`, `src/app/api/evals/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute ships a generic evaluation framework you can use to benchmark routing
configurations, single providers/models, or the bundled "golden set" suites.
Use it to verify routing changes, validate new providers, and gate releases
before promoting them to production traffic.
The framework is implemented as:
- A pure runner (`src/lib/evals/evalRunner.ts`) that registers in-memory
built-in suites, evaluates outputs against expected criteria, and aggregates
scorecards.
- A persistence layer (`src/lib/db/evals.ts`) for custom (user-defined) suites
and historical runs in SQLite.
- An orchestration layer (`src/lib/evals/runtime.ts`) that executes each case
by dispatching real calls to `POST /v1/chat/completions`, captures latency
and outputs, and persists the run.
- REST endpoints under `/api/evals/*` (management-auth only).
- A dashboard surface at `Dashboard → Usage → Evals` (`EvalsTab.tsx`).
## Concepts
### Suite
A suite is a named collection of test cases with a `description` and one or
more cases. Suites come from two sources:
| Source | Where defined | Mutable at runtime? |
| ---------- | --------------------------------------------- | ------------------- |
| `built-in` | Registered via `registerSuite()` at boot | No (code-defined) |
| `custom` | Stored in SQLite `eval_suites` + `eval_cases` | Yes (via API/UI) |
The current built-in suites (see `src/lib/evals/evalRunner.ts`):
- `golden-set` — 10 baseline cases across greeting/math/translation/safety
- `coding-proficiency` — Python/JS/SQL/TS/bug detection
- `reasoning-logic` — syllogisms, word problems, pattern recognition
- `multilingual` — translation and language detection
- `safety-guardrails` — PII, jailbreak, refusal, bias awareness
- `instruction-following` — JSON-only, numbered lists, language constraints
- `codex-comparison` — head-to-head coding tasks intended for compare mode
### Case
Each case carries:
| Field | Description |
| ---------- | ------------------------------------------------------------ |
| `id` | Stable identifier (used to key outputs and metrics) |
| `name` | Human-readable label |
| `model` | Default model when the run uses `suite-default` targeting |
| `input` | `{ messages, max_tokens? }` — sent to `/v1/chat/completions` |
| `expected` | `{ strategy, value }` — scoring rubric (see below) |
| `tags` | Optional labels (e.g. `safety`, `pii`, `jailbreak`) |
### Target
The same suite can be run against different targets. The target schema is
`evalTargetSchema` in `src/shared/validation/schemas.ts`:
| Target type | `id` | Behavior |
| --------------- | ---------- | --------------------------------------------------------------- |
| `suite-default` | `null` | Each case uses its built-in `model` field |
| `model` | model name | Force every case through one direct model (e.g. `gpt-4o`) |
| `combo` | combo name | Run every case through one combo (exercises the routing engine) |
For `model` and `combo`, the `id` field is required (enforced by Zod
`superRefine`). When `compareTarget` is provided, both targets must differ —
the runner persists both runs under the same `runGroupId` for A/B comparison.
## Scoring Rubrics
Implemented in `evaluateCase()` (evalRunner.ts):
| Strategy | Pass when… |
| ---------- | -------------------------------------------------------------------- |
| `exact` | `actualOutput === expected.value` |
| `contains` | `actualOutput.toLowerCase().includes(expected.value.toLowerCase())` |
| `regex` | `new RegExp(expected.value).test(actualOutput)` is truthy |
| `custom` | `expected.fn(actualOutput, evalCase)` returns truthy (built-in only) |
**Note:** Custom-function scoring is reserved for code-defined (built-in)
suites because functions cannot be serialized through the API. The
`evalCaseBuilderSchema` only accepts `contains | exact | regex` for
user-created suites.
There is no LLM-as-judge or embedding-based similarity scorer today — it would
be a clean extension point in `evaluateCase()`.
## Database Schema
Three tables (migrations `030_create_eval_runs.sql` and
`031_create_eval_suites.sql`):
| Table | Purpose |
| ------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `eval_suites` | Custom suite metadata (`id`, `name`, `description`) |
| `eval_cases` | Cases per suite — `input_json`, `expected_*`, `tags_json` |
| `eval_runs` | Historical runs — `pass_rate`, `total`, `passed`, `failed`, `avg_latency_ms`, `summary_json`, `results_json`, `outputs_json` |
Built-in suites are **not** stored in the DB. They live in memory and are
re-registered every time `evalRunner.ts` is imported.
## REST API
All endpoints require management auth (`requireManagementAuth`) — they are not
part of the public proxy surface.
| Endpoint | Method | Description |
| ----------------------------- | -------- | ------------------------------------------------------------- |
| `/api/evals` | `GET` | List suites + recent runs + scorecard + targets + keys |
| `/api/evals` | `POST` | Run a suite (single or compare) — schema `evalRunSuiteSchema` |
| `/api/evals/{suiteId}` | `GET` | Fetch one suite (built-in or custom) |
| `/api/evals/suites` | `POST` | Create a custom suite — schema `evalSuiteSaveSchema` |
| `/api/evals/suites/{suiteId}` | `GET` | Fetch a custom suite |
| `/api/evals/suites/{suiteId}` | `PUT` | Replace a custom suite (cases get re-inserted) |
| `/api/evals/suites/{suiteId}` | `DELETE` | Delete a custom suite and its cases |
### Running a suite
```bash
curl -X POST http://localhost:20128/api/evals \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"suiteId": "golden-set",
"target": { "type": "combo", "id": "my-combo" },
"apiKeyId": "optional-api-key-uuid"
}'
```
Optional fields:
- `outputs` — `Record<caseId, string>` of pre-computed outputs. When provided,
the runner **skips dispatch** and only scores the cached outputs (useful for
offline evaluation).
- `compareTarget` — second target to run in parallel; both runs share a
generated `runGroupId` for head-to-head viewing.
- `apiKeyId` — internal API key used to authenticate the dispatched
`/v1/chat/completions` calls. Required when `REQUIRE_API_KEY` is enabled.
### Creating a custom suite
```bash
curl -X POST http://localhost:20128/api/evals/suites \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"name": "Production smoke",
"description": "Quick sanity check before deploy",
"cases": [
{
"name": "JSON shape",
"model": "gpt-4o",
"input": { "messages": [{ "role": "user", "content": "Reply with {\"ok\": true}" }] },
"expected": { "strategy": "regex", "value": "\"ok\"\\s*:\\s*true" }
}
]
}'
```
## Dispatch Pipeline
`runEvalSuiteAgainstTarget()` (`src/lib/evals/runtime.ts`):
1. Resolves the suite (built-in or custom).
2. For each case, builds a `Request` to `/v1/chat/completions` with the case's
`messages`, the resolved `model`, `stream: false`, and `max_tokens: 512`
(or the case override).
3. Calls the chat handler directly (in-process — no extra HTTP hop).
4. Captures latency and extracts text from either `choices[0].message.content`
or the Responses-API `output[]` payload.
5. Scores all outputs via `runSuite()`, then persists via `saveEvalRun()`.
Cases run **sequentially**. There is no concurrency flag today.
## Dashboard
The UI lives at `Dashboard → Usage → Evals`
(`src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx`). From there you
can:
- Browse built-in and custom suites with case-by-case preview.
- Create/edit/delete custom suites with the case builder.
- Pick a target (suite defaults / model / combo), optionally a second
`compareTarget`, optionally an API key, then run on demand.
- Inspect run history, per-case pass/fail, latency, and captured outputs.
- See the rolling scorecard aggregated across the latest run per
`(suite, target)` scope.
## Relationship with the Auto-Assessment RFC
A separate, narrower assessment subsystem lives at `src/domain/assessment/`
(see also [AUTO-COMBO.md](../routing/AUTO-COMBO.md) for the live scoring engine).
That subsystem targets the Auto Combo engine — automatically scoring providers and
models so combos can self-heal when upstreams fail. It uses its own runner,
its own categorizer, and its own scoring logic.
The Evals framework documented here is the **broader, general-purpose
testing surface**. Prefer it for arbitrary regression suites, A/B comparisons,
and per-release smoke tests. Use the Auto-Assessment subsystem when you need
real-time provider health to influence routing decisions.
## CI Integration
There is no dedicated `eval:ci` npm script today. Two paths if you want to
gate releases on eval results:
- **HTTP path**: stand up the server, hit `POST /api/evals` with a known
`suiteId` + `target`, and assert `runs[].summary.passRate >= N` in the
response.
- **In-process path**: import `runEvalSuiteAgainstTarget()` from
`@/lib/evals/runtime` from a script, run against a test DB, and check the
returned `PersistedEvalRun.summary`.
Tests covering the route and history live at
`tests/unit/evals-route.test.ts` and `tests/unit/evals-history.test.ts`.
## Extension Points
Common changes and where to make them:
- **New scoring strategy** — extend the `switch (evalCase.expected.strategy)`
block in `evaluateCase()` (`evalRunner.ts`) and widen `EvalCaseStrategy` in
`src/lib/db/evals.ts` plus `evalCaseBuilderSchema` in `schemas.ts`.
- **New built-in suite** — define a suite object and call `registerSuite()` at
the bottom of `evalRunner.ts`. It will be auto-discovered by `listSuites()`.
- **Run with concurrency** — change the sequential `for` loop in
`runEvalSuiteAgainstTarget()` to a bounded `Promise.all` (no concurrency
control exists today).
- **Stream/tool-call cases** — currently the runner forces `stream: false`.
Streaming or tool-aware evaluation would require changes in `runtime.ts`
(capture and aggregate SSE chunks before scoring).
## See Also
- [USER_GUIDE.md](../guides/USER_GUIDE.md) — overall product walkthrough
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — request pipeline reference
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — Auto Combo scoring engine (live runtime)
- Source: `src/lib/evals/`, `src/lib/db/evals.ts`, `src/app/api/evals/`
- UI: `src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx`
docs/frameworks/GAMIFICATION.md
+1082
View File
File diff suppressed because it is too large Load Diff
docs/frameworks/MCP-SERVER.md
+421
View File
@@ -0,0 +1,421 @@
---
title: "OmniRoute MCP Server Documentation"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute MCP Server Documentation
> Model Context Protocol server with 94 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations.
>
> Source of truth: `open-sse/mcp-server/schemas/tools.ts` (34 base) + `memoryTools.ts` (3) + `skillTools.ts` (4) + `agentSkillTools.ts` (3) + `poolTools.ts` (6) + `gamificationTools.ts` (8) + `pluginTools.ts` (8) + `notionTools.ts` (6) + `obsidianTools.ts` (22) = **94** (`TOTAL_MCP_TOOL_COUNT`). Tool registration and scope wiring lives in `open-sse/mcp-server/server.ts`.
![MCP tool inventory (94 tools by category)](../diagrams/exported/mcp-tools-94.svg)
> Source: [diagrams/mcp-tools-94.mmd](../diagrams/mcp-tools-94.mmd) (regenerate via `npm run docs:render-diagrams`).
## Installation
OmniRoute MCP is built-in. Start it with:
```bash
omniroute --mcp
```
Or via the open-sse transport:
```bash
# HTTP streamable transport (port 20130)
omniroute --dev # MCP auto-starts on /mcp endpoint
```
## Transports
The MCP server exposes three transports, all backed by the same `createMcpServer()` factory:
| Transport | Where | When to use |
| :---------------- | :------------------------------------------ | :--------------------------------------------------- |
| `stdio` | `open-sse/mcp-server/server.ts` | IDE integrations (Claude Desktop, Cursor, etc.) |
| `sse` | `POST/GET /api/mcp/sse` via `httpTransport` | Browser/agent clients that need an event stream |
| `streamable-http` | `POST/GET/DELETE /api/mcp/stream` | Multi-session HTTP clients (`mcp-session-id` header) |
The active HTTP transport (`sse` or `streamable-http`) is selected by the `mcpTransport` setting. Switching transports closes existing sessions on the other transport.
### Remote access (manage-scope bypass)
`/api/mcp/*` is in the LOCAL_ONLY tier (`src/server/authz/routeGuard.ts`) — by default only loopback hosts (`localhost`, `127.0.0.1`, `::1`) can reach it. Since v3.8.2, non-loopback clients may connect if they present an `Authorization: Bearer <api-key>` whose key carries the `manage` scope. This is the only way to reach the remote MCP server through a tunnel, reverse proxy, or public hostname.
```bash
# Grant manage scope: open the dashboard API Keys page and toggle
# "Management Access" on the key, or POST scopes:["manage"] when creating.
# Then connect from a remote MCP client:
curl -i \
-H "Host: your-public-host.example" \
-H "Authorization: Bearer sk-…" \
-H "Content-Type: application/json" \
-H "Accept: application/json, text/event-stream" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"my-client","version":"0"}}}' \
https://your-public-host.example/api/mcp/stream
```
A non-manage key (or no Bearer) returns `403 LOCAL_ONLY`. The sibling prefix `/api/cli-tools/runtime/*` is intentionally NOT bypassable — see [Route Guard Tiers — Manage-scope carve-out](../security/ROUTE_GUARD_TIERS.md#manage-scope-carve-out).
## IDE Configuration
See [MCP Client Configuration](../guides/SETUP_GUIDE.md#mcp-client-configuration) for Claude Desktop,
Cursor, Cline, and compatible MCP client setup.
---
## Essential Tools (8) — Phase 1
| Tool | Scopes | Description |
| :------------------------------ | :-------------------- | :------------------------------------------------------------ |
| `omniroute_get_health` | `read:health` | Uptime, memory, circuit breakers, rate limits, cache stats |
| `omniroute_list_combos` | `read:combos` | All configured combos with strategies (optional metrics) |
| `omniroute_get_combo_metrics` | `read:combos` | Performance metrics for a specific combo |
| `omniroute_switch_combo` | `write:combos` | Activate or deactivate a combo |
| `omniroute_check_quota` | `read:quota` | Quota used/total, percent remaining, reset time, token health |
| `omniroute_route_request` | `execute:completions` | Send a chat completion through OmniRoute routing |
| `omniroute_cost_report` | `read:usage` | Cost report by period (session/day/week/month) |
| `omniroute_list_models_catalog` | `read:models` | Full model catalog with capabilities, status, pricing |
## Phase 1 — Search
| Tool | Scopes | Description |
| :--------------------- | :--------------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| `omniroute_web_search` | `execute:search` | Web search through OmniRoute search gateway (Serper/Brave/Perplexity/Exa/Tavily/Google PSE/Linkup/SearchAPI/SearXNG) with failover |
## Advanced Tools (11) — Phase 2
| Tool | Scopes | Description |
| :--------------------------------- | :----------------------------------- | :---------------------------------------------------------------------------------------- |
| `omniroute_simulate_route` | `read:health`, `read:combos` | Dry-run routing simulation with fallback tree |
| `omniroute_set_budget_guard` | `write:budget` | Session budget with degrade/block/alert action |
| `omniroute_set_routing_strategy` | `write:combos` | Update combo strategy at runtime (priority/weighted/auto/etc.) |
| `omniroute_set_resilience_profile` | `write:resilience` | Apply `aggressive` / `balanced` / `conservative` resilience preset |
| `omniroute_test_combo` | `execute:completions`, `read:combos` | Live test of every provider in a combo using a real upstream call |
| `omniroute_get_provider_metrics` | `read:health` | Per-provider metrics with p50/p95/p99 latency and circuit breaker state |
| `omniroute_best_combo_for_task` | `read:combos`, `read:health` | Recommend combo by task type with budget/latency constraints |
| `omniroute_explain_route` | `read:health`, `read:usage` | Explain why a request was routed to a provider (scoring factors + fallbacks) |
| `omniroute_get_session_snapshot` | `read:usage` | Full session snapshot: cost, tokens, top models/providers, errors, budget guard |
| `omniroute_db_health_check` | `read:health`, `write:resilience` | Diagnose (and optionally auto-repair) database drift like broken combo refs / orphan rows |
| `omniroute_sync_pricing` | `pricing:write` | Sync pricing data from external sources (LiteLLM); supports `dryRun` |
## Cache Tools (2)
| Tool | Scopes | Description |
| :---------------------- | :------------ | :-------------------------------------------------- |
| `omniroute_cache_stats` | `read:cache` | Semantic cache, prompt-cache, and idempotency stats |
| `omniroute_cache_flush` | `write:cache` | Flush cache globally or by signature/model |
## Compression Tools (5)
| Tool | Scopes | Description |
| :---------------------------------- | :------------------ | :----------------------------------------------------------------------------------------------------------------------- |
| `omniroute_compression_status` | `read:compression` | Compression settings, analytics summary, and cache-aware stats (includes `analytics.mcpDescriptionCompression` metadata) |
| `omniroute_compression_configure` | `write:compression` | Configure compression mode, threshold, target ratio, system-prompt preservation, MCP description compression toggle |
| `omniroute_set_compression_engine` | `write:compression` | Pick the active engine (off/caveman/rtk/stacked) and Caveman/RTK intensity |
| `omniroute_list_compression_combos` | `read:compression` | List named compression combos and their engine pipelines |
| `omniroute_compression_combo_stats` | `read:compression` | Analytics grouped by compression combo and engine |
`omniroute_compression_status` reports MCP description compression separately under
`analytics.mcpDescriptionCompression`. Those values are metadata-size estimates for MCP listable
descriptions (`tools`, `prompts`, `resources`, and `resourceTemplates`); they are not provider usage
receipts and are marked with `source: "mcp_metadata_estimate"`.
### MCP Accessibility Tree Filter (v3.8.0)
Separate from the 5 compression tools above, OmniRoute includes a post-execution filter that
compresses the **tool results** of MCP browser/accessibility tools before they are returned to the
agent. This filter is not itself a tool — it runs transparently on any tool result that contains
verbose accessibility-tree or browser-snapshot text (≥2000 chars).
Key behaviors:
- Collapses ≥30 consecutive repeated sibling lines into head + tail summary
- Preserves `[ref=eXX]` anchors required by Playwright/computer-use
- Hard-truncates oversized text (>50,000 chars) with a navigation hint
- Expected savings: **60–80%** on browser snapshot payloads
Configuration: `compression.mcpAccessibility` in global settings (migration 056).
Implementation: `open-sse/services/compression/engines/mcpAccessibility/`.
Full docs: [Compression Engines — MCP Accessibility Tree Filter](../compression/COMPRESSION_ENGINES.md#mcp-accessibility-tree-filter).
See [Compression Engines](../compression/COMPRESSION_ENGINES.md) and [RTK Compression](../compression/RTK_COMPRESSION.md) for
the runtime compression model behind these tools.
## 1Proxy Tools (3)
| Tool | Scopes | Description |
| :-------------------------- | :------------- | :-------------------------------------------------------------------------------------- |
| `omniroute_oneproxy_fetch` | `read:proxies` | Fetch free proxies from the 1proxy marketplace (protocol/country/quality/limit filters) |
| `omniroute_oneproxy_rotate` | `read:proxies` | Get the next available proxy by strategy (`random` / `quality` / `sequential`) |
| `omniroute_oneproxy_stats` | `read:proxies` | Pool stats, sync status, distribution by protocol and country |
## Memory Tools (3)
Defined in `open-sse/mcp-server/tools/memoryTools.ts`. Auth/scope is enforced through the standard MCP scope pipeline.
| Tool | Scopes | Description |
| :------------------------ | :------------- | :---------------------------------------------------------------------------------- |
| `omniroute_memory_search` | `read:memory` | Search memories by query / type / API key with token-budget enforcement |
| `omniroute_memory_add` | `write:memory` | Add a new memory entry (`factual` / `episodic` / `procedural` / `semantic`) |
| `omniroute_memory_clear` | `write:memory` | Clear memories for an API key, optionally filtered by type or `olderThan` timestamp |
## Skill Tools (4)
Defined in `open-sse/mcp-server/tools/skillTools.ts`. Backed by `src/lib/skills/registry` + `src/lib/skills/executor`.
| Tool | Scopes | Description |
| :---------------------------- | :--------------- | :-------------------------------------------------------------------------------- |
| `omniroute_skills_list` | `read:skills` | List registered skills with optional filtering by API key, name, or enabled state |
| `omniroute_skills_enable` | `write:skills` | Enable or disable a specific skill by ID |
| `omniroute_skills_execute` | `execute:skills` | Execute a skill with provided input and return the execution record |
| `omniroute_skills_executions` | `read:skills` | List recent skill execution history |
## Notion Context Source (6)
Defined in `open-sse/mcp-server/tools/notionTools.ts`. Token stored in `key_value` table via `src/lib/db/notion.ts`. REST client in `src/lib/notion/api.ts`. Settings API in `src/app/api/settings/notion/route.ts`. Dashboard UI in `src/app/(dashboard)/dashboard/endpoint/components/NotionSourceCard.tsx`.
Configure your Notion integration token from the **Context Sources** tab in the Endpoint dashboard, or via the REST API:
```bash
# Set token
curl -X POST http://localhost:20128/api/settings/notion \
-H "Content-Type: application/json" \
-d '{"token": "ntn_..."}'
# Check status
curl http://localhost:20128/api/settings/notion
# Disconnect
curl -X DELETE http://localhost:20128/api/settings/notion
```
| Tool | Scopes | Description |
| :--------------------------- | :------------- | :------------------------------------------------------------- |
| `notion_search` | `read:notion` | Full-text search across all pages and databases |
| `notion_get_page` | `read:notion` | Get a page by ID with its properties |
| `notion_list_block_children` | `read:notion` | List the child blocks of a page or block |
| `notion_query_database` | `read:notion` | Query a database with filters, sorts, and pagination |
| `notion_get_database` | `read:notion` | Get database schema by ID |
| `notion_append_blocks` | `write:notion` | Append children blocks to a parent block (max 100 per request) |
## Agent Skill Catalog Tools (3)
Defined in `open-sse/mcp-server/tools/agentSkillTools.ts`. Backed by `src/lib/agentSkills/catalog`. These tools expose the 42-entry Agent Skills documentation catalog to MCP clients and external agents. Scope: `read:catalog`.
| Tool | Scopes | Description |
| :-------------------------------- | :------------- | :--------------------------------------------------------------------------------------------------------------- |
| `omniroute_agent_skills_list` | `read:catalog` | List all 42 agent skills with optional `category` (api\|cli) and `area` filters; returns metadata + coverage |
| `omniroute_agent_skills_get` | `read:catalog` | Get full metadata + SKILL.md content for a single skill by canonical `id` |
| `omniroute_agent_skills_coverage` | `read:catalog` | Coverage stats: how many of the 22 API and 20 CLI skills have SKILL.md files on the filesystem vs catalog totals |
See [AGENT-SKILLS.md](./AGENT-SKILLS.md) for the full catalog and how external agents consume it.
## Related Frameworks (v3.8.0)
The MCP tool inventory above (94 tools = 34 core + 3 memory + 4 skills + 3 agent-skills + 6 pool + 8 gamification + 8 plugins + 6 notion + 22 obsidian) is intentionally
scoped to runtime routing/cache/compression/memory/skills/proxy/context-source operations. Two adjacent
frameworks ship alongside the MCP server in v3.8.0 and are documented separately:
### Cloud Agents
Cloud Agents are out-of-process AI coding agents (codex-cloud, devin, jules) wired into
OmniRoute through the same connection model used for LLM providers. They are exposed via
their own REST surface (`/api/v1/agents/*`) and are **not** part of the MCP tool catalog
— calling a Cloud Agent does not consume an MCP scope.
- Implementation: `src/lib/cloudAgent/` (`registry.ts`, `agents/codex-cloud.ts`, `agents/devin.ts`, `agents/jules.ts`).
- Lifecycle: `createTask`, `getStatus`, `approvePlan`, `sendMessage`, `listSources`.
- Documentation: [docs/frameworks/CLOUD_AGENT.md](./CLOUD_AGENT.md).
### Guardrails
Guardrails are pre/post-execution filters (vision-bridge, pii-masker, prompt-injection)
applied inside the chat pipeline. They run before the MCP tool/route layer is reached
and emit structured violations to the audit pipeline; they are not invoked as MCP tools.
- Implementation: `src/lib/guardrails/`.
- Documentation: [docs/security/GUARDRAILS.md](../security/GUARDRAILS.md).
When debugging an MCP call that appears blocked, check both the MCP audit log
(`scope_denied:*` entries) and the guardrails audit trail — a request may be rejected by
a guardrail **before** it ever reaches the MCP scope enforcement layer.
---
## REST API Endpoints
| Endpoint | Method | Description | Auth |
| :--------------------- | :-------------------- | :-------------------------------------------------------------------------------------------------- | :------------------------- |
| `/api/mcp/status` | `GET` | Server status: heartbeat, HTTP transport state, audit activity summary | Management (session/admin) |
| `/api/mcp/tools` | `GET` | Tool catalog (name, description, scopes, phase, source endpoints) | Management |
| `/api/mcp/sse` | `GET` / `POST` | SSE transport endpoint (gated by `mcpEnabled` + `mcpTransport === "sse"`) | API key + scopes |
| `/api/mcp/stream` | `POST`/`GET`/`DELETE` | Streamable HTTP transport (uses `mcp-session-id` header; `DELETE` ends the session) | API key + scopes |
| `/api/mcp/audit` | `GET` | Audit log entries from `mcp_tool_audit` (filters: `limit`, `offset`, `tool`, `success`, `apiKeyId`) | Management |
| `/api/mcp/audit/stats` | `GET` | Aggregated audit stats (`totalCalls`, `successRate`, `avgDurationMs`, top tools) | Management |
Source files: `src/app/api/mcp/{status,tools,sse,stream,audit,audit/stats}/route.ts`.
Both SSE and Streamable HTTP transports are blocked until the MCP server is enabled in Settings (`mcpEnabled`) and the appropriate `mcpTransport` is selected. If the wrong transport is configured the route returns HTTP 400 with a hint to switch settings.
---
## Authentication & Scopes
MCP tools are authenticated through API key scopes. Scope enforcement is centralized in
`open-sse/mcp-server/scopeEnforcement.ts`. Each tool requires specific scopes:
| Scope | Tools |
| :-------------------- | :---------------------------------------------------------------------------------------------------------------- |
| `read:health` | `get_health`, `get_provider_metrics`, `simulate_route`, `explain_route`, `best_combo_for_task`, `db_health_check` |
| `read:combos` | `list_combos`, `get_combo_metrics`, `simulate_route`, `best_combo_for_task`, `test_combo` |
| `write:combos` | `switch_combo`, `set_routing_strategy` |
| `read:quota` | `check_quota` |
| `read:usage` | `cost_report`, `get_session_snapshot`, `explain_route` |
| `read:models` | `list_models_catalog` |
| `execute:completions` | `route_request`, `test_combo` |
| `execute:search` | `web_search` |
| `write:budget` | `set_budget_guard` |
| `write:resilience` | `set_resilience_profile`, `db_health_check` |
| `pricing:write` | `sync_pricing` |
| `read:cache` | `cache_stats` |
| `write:cache` | `cache_flush` |
| `read:compression` | `compression_status`, `list_compression_combos`, `compression_combo_stats` |
| `write:compression` | `compression_configure`, `set_compression_engine` |
| `read:proxies` | `oneproxy_fetch`, `oneproxy_rotate`, `oneproxy_stats` |
| `read:notion` | `notion_search`, `notion_list_databases`, `notion_get_database`, `notion_query_database`, `notion_read` |
| `write:notion` | `notion_append_blocks` |
| `read:memory` | `memory_search` |
| `write:memory` | `memory_add`, `memory_clear` |
| `read:skills` | `skills_list`, `skills_executions` |
| `write:skills` | `skills_enable` |
| `execute:skills` | `skills_execute` |
| `read:catalog` | `agent_skills_list`, `agent_skills_get`, `agent_skills_coverage` |
Wildcard scopes are supported: `read:*` grants all read-scopes, `*` grants full access.
---
## Environment Variables
| Variable | Default | Purpose |
| :-------------------------------------- | :--------------------------------- | :----------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_BASE_URL` | `http://localhost:20128` | Base URL the MCP server uses when calling OmniRoute internal APIs |
| `OMNIROUTE_API_KEY` | (empty) | API key forwarded as `Authorization: Bearer` to internal API calls |
| `OMNIROUTE_MCP_ENFORCE_SCOPES` | `false` (only `"true"` enables it) | When enabled, missing scopes deny tool calls and log `scope_denied:<reason>` in audit log |
| `OMNIROUTE_MCP_SCOPES` | (empty) | Comma-separated allowlist of scopes considered "available" by default (used when caller does not provide its own scopes) |
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | (unset = on) | When set to `0/false/off/no`, disables MCP description compression at registration time |
| `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION` | (unset = on) | Alternate alias for the same toggle as above |
| `MCP_TOOL_DENY` | (unset = no filter) | Comma-separated tool names to drop from `tools/list` (tool-cardinality reduction — see below) |
| `MCP_TOOL_ALLOW` | (unset = no filter) | Comma-separated tool names to keep exclusively (allow-list mode — see below) |
| `DATA_DIR` | `~/.omniroute` | Heartbeat file is written to `${DATA_DIR}/runtime/mcp-heartbeat.json` |
---
## Description Compression
MCP tool, prompt, and resource registries can compress descriptions at registration/list time to reduce the metadata footprint exposed to clients (and therefore the prompt context cost). The implementation lives in `open-sse/mcp-server/descriptionCompressor.ts` and is wired into the MCP server via `compressMcpRegistryMetadata` inside `createMcpServer()`.
- Compression runs over the description text using the Caveman ruleset (`getRulesForContext("all", "full")`) with preserved-block extraction (code spans, fenced blocks, etc.) so structural content is not altered.
- Toggle per-deployment via the `compression.mcpDescriptionCompressionEnabled` value in the `key_value` settings table (default: enabled) — exposed in the UI as **Analytics → MCP description compression**.
- Toggle process-wide via either `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS=false` or `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION=false`.
- Realtime stats are surfaced via `omniroute_compression_status` under `analytics.mcpDescriptionCompression` and tagged `source: "mcp_metadata_estimate"` to disambiguate from real provider usage receipts.
---
## Tool Cardinality Reduction (F4.3)
Description compression shrinks each tool's metadata; **tool-cardinality reduction** goes one step further by reducing _how many_ tools are announced at all. Advertising fewer tools in the `tools/list` manifest cuts the per-request token cost the client's model pays for the tool catalog ("layer 5" compression). The implementation is a pure, stateless filter in `open-sse/mcp-server/toolCardinality.ts` (`reduceToolManifest`), wired into the registration loop in `createMcpServer()` (`open-sse/mcp-server/server.ts`).
**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 94 tools are announced unchanged.
| Variable | Mode |
| :--------------- | :-------------------------------------------------------------------------------------- |
| `MCP_TOOL_DENY` | Blacklist — comma-separated tool names that are always dropped from `tools/list` |
| `MCP_TOOL_ALLOW` | Allow-list — comma-separated tool names; only these survive, everything else is dropped |
`deny` takes priority over `allow`. Names are comma-separated, trimmed, and empty entries are ignored. Examples:
```bash
# Drop two tools from the catalog
MCP_TOOL_DENY="omniroute_get_health,omniroute_list_combos" omniroute --mcp
# Announce only the routing + quota tools (allow-list mode)
MCP_TOOL_ALLOW="omniroute_route_request,omniroute_check_quota" omniroute --mcp
```
**How filtered tools are removed:** registration always succeeds; a tool the profile rejects is then `.disable()`d on the MCP SDK handle, so it never appears in `tools/list` but the wiring stays intact (clean enable/disable, no re-registration). The profile parser is `readMcpToolProfileFromEnv(process.env)`, which returns `null` (no filtering) when both vars are empty.
The richer `ToolProfile` shape behind `reduceToolManifest` also supports scope-intersection filtering (`allowScopes`, with `read:*`-style wildcard matching) and a deterministic `maxTools` cap, but those two knobs need the full manifest at registration time and are **not** exposed through the environment variables today (a `tools/list`-level hook is a tracked follow-up). `estimateManifestTokens()` is available to compare the manifest token cost before and after reduction.
---
## Runtime Heartbeat
The stdio transport persists liveness to `${DATA_DIR}/runtime/mcp-heartbeat.json` every 5 seconds. The dashboard (`/api/mcp/status`) reads this file plus PID liveness to derive `online`. HTTP transports report state from in-process `getMcpHttpStatus()` instead (no file write).
The heartbeat snapshot contains:
```json
{
"pid": 12345,
"startedAt": "2026-05-13T12:34:56.000Z",
"lastHeartbeatAt": "2026-05-13T12:35:01.000Z",
"version": "1.8.1",
"transport": "stdio",
"scopesEnforced": false,
"allowedScopes": [],
"toolCount": 43
}
```
---
## Audit Logging
Every tool call is logged to the SQLite `mcp_tool_audit` table by `open-sse/mcp-server/audit.ts`:
- Tool name, arguments (hashed/truncated as per per-tool `auditLevel`), result
- Duration in ms, success/failure flag, error message (when applicable)
- API key hash, timestamp
- Scope denials are logged as `scope_denied:<reason>` with the missing scope list
Use the dashboard or the `/api/mcp/audit` and `/api/mcp/audit/stats` REST endpoints to inspect recent calls.
---
## Files
| File | Purpose |
| :----------------------------------------------------------------------- | :--------------------------------------------------------------- |
| `open-sse/mcp-server/server.ts` | MCP server factory, stdio entry point, scoped tool registrations |
| `open-sse/mcp-server/httpTransport.ts` | SSE + Streamable HTTP transport (session management) |
| `open-sse/mcp-server/scopeEnforcement.ts` | Tool scope evaluation and caller resolution |
| `open-sse/mcp-server/audit.ts` | Tool call audit logging (`mcp_tool_audit`) |
| `open-sse/mcp-server/runtimeHeartbeat.ts` | stdio heartbeat writer (`mcp-heartbeat.json`) |
| `open-sse/mcp-server/descriptionCompressor.ts` | Description compression for tool / prompt / resource registries |
| `open-sse/mcp-server/schemas/tools.ts` | Zod schemas + tool registry (`MCP_TOOLS`, 34 entries) |
| `open-sse/mcp-server/tools/advancedTools.ts` | Phase 2 + cache + 1proxy tool handlers |
| `open-sse/mcp-server/tools/compressionTools.ts` | Compression tool handlers |
| `open-sse/mcp-server/tools/memoryTools.ts` | Memory tool definitions (3 tools) |
| `open-sse/mcp-server/tools/skillTools.ts` | Skill tool definitions (4 tools) |
| `open-sse/mcp-server/tools/notionTools.ts` | Notion context source tool definitions (6 tools) |
| `open-sse/mcp-server/tools/gamificationTools.ts` | Gamification tool definitions (8 tools) |
| `open-sse/mcp-server/tools/pluginTools.ts` | Plugin registration and management tools (8 tools) |
| `src/app/api/mcp/status/route.ts` | `/api/mcp/status` endpoint |
| `src/app/api/mcp/tools/route.ts` | `/api/mcp/tools` endpoint |
| `src/app/api/mcp/sse/route.ts` | `/api/mcp/sse` SSE transport route |
| `src/app/api/mcp/stream/route.ts` | `/api/mcp/stream` Streamable HTTP transport route |
| `src/app/api/mcp/audit/route.ts` | `/api/mcp/audit` audit log query |
| `src/app/api/mcp/audit/stats/route.ts` | `/api/mcp/audit/stats` aggregated audit metrics |
| `src/lib/notion/api.ts` | Notion REST API client (retry, timeout, error classification) |
| `src/lib/db/notion.ts` | Notion token persistence (`key_value` table) |
| `src/app/api/settings/notion/route.ts` | Notion settings API (GET/POST/DELETE) |
| `src/app/(dashboard)/dashboard/endpoint/components/NotionSourceCard.tsx` | Notion token management UI |
| `tests/unit/notion-api.test.ts` | Notion API client tests (7) |
| `tests/unit/notion-tools.test.ts` | Notion tools scope enforcement tests (10) |
| `tests/unit/db/notion.test.mjs` | Notion DB module tests (3) |
docs/frameworks/MEMORY.md
+880
View File
@@ -0,0 +1,880 @@
---
title: "Memory System"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Memory System
> **Source of truth:** `src/lib/memory/` and `src/app/api/memory/`
> **Last updated:** 2026-06-28 — v3.8.40 (off-by-default + int8 quantization catch-up)
OmniRoute provides persistent conversational memory keyed by API key (and
optionally session id). Memories are extracted automatically from LLM responses
via lightweight regex pattern matching and injected back into subsequent
requests as a leading system message (or first user message for providers that
reject the system role).
> **Memory is OFF by default (v3.8.30+).** `DEFAULT_MEMORY_SETTINGS.enabled` is
> now `false` (`src/lib/memory/settings.ts`). Enabling memory injects up to
> `maxTokens` (~2k) of retrieved context into **every** chat request, which is
> billed — a surprising cost for new installs and for clients that manage their
> own context. Opt in explicitly under **Settings → Memory** (the
> `MemorySkillsTab` shows a token-cost warning callout when memory is enabled).
> A client can opt a single request out with the `x-omniroute-no-memory`
> request header (`true`/`1`/`yes`) — see the request-header table in
> [API_REFERENCE.md](../reference/API_REFERENCE.md). A no-memory request sets
> `memoryOwnerId = null`, which disables **both** memory and skill injection for
> that request (`open-sse/handlers/chatCore/headers.ts::isNoMemoryRequested`).
Memory is **scoped per API key**, not per user — every request authenticated
with the same API key shares the same memory pool, with optional further
scoping by `sessionId`.
## Architecture
```
Client → /v1/chat/completions (apiKeyInfo resolved upstream)
→ handleChatCore() [open-sse/handlers/chatCore.ts]
→ resolveMemoryOwnerId(apiKeyInfo) # extracts id
→ getMemorySettings() # cached settings
→ shouldInjectMemory(body, {enabled}) # gate
→ retrieveMemories(apiKeyId, config) # SQL + FTS5 + optional vector
→ injectMemory(body, memories, provider) # system or user message
→ upstream provider call
→ on response: extractFacts(text, apiKeyId, sessionId) # non-blocking
→ setImmediate → createMemory(fact) per match
→ embed(content) + upsertVector(id, vec)
```
The injection and extraction call-sites are wired in
`open-sse/handlers/chatCore.ts` (look for `retrieveMemories`, `injectMemory`,
and `extractFacts`).
## Engine architecture (3-tier resolution)
The Memory Engine resolves the retrieval path at runtime based on available
infrastructure and settings. Three tiers exist, applied in priority order:
```
┌─────────────────────────────────────────────────────────────┐
│ TIER 0 — Keyword (FTS5) │
│ Always available. SQLite FTS5 full-text search over │
│ content + key. Used when strategy = "exact" or as fallback. │
└──────────────────────────────────┬──────────────────────────┘
│ strategy = semantic|hybrid?
▼
┌─────────────────────────────────────────────────────────────┐
│ TIER 1 — Embedded Vector (sqlite-vec) │
│ sqlite-vec v0.1.9 loaded via db.loadExtension(). │
│ KNN brute-force over Float32 vectors. Active when: │
│ • sqlite-vec loadExtension succeeds │
│ • An embedding source is available (remote | static | │
│ transformers) that can produce a Float32Array │
│ • vec_memories table exists (created on first ready()) │
└──────────────────────────────────┬──────────────────────────┘
│ qdrant.enabled?
▼
┌─────────────────────────────────────────────────────────────┐
│ TIER 2 — Qdrant (opt-in external vector database) │
│ When enabled, replaces sqlite-vec for semantic/hybrid. │
│ Requires running Qdrant instance + configured host/port. │
└─────────────────────────────────────────────────────────────┘
```
Degradation is automatic and transparent:
- If sqlite-vec fails to load, tier 1 is unavailable → falls back to tier 0.
- If embedding source returns an error, tier 1 falls back to tier 0.
- If Qdrant is unhealthy, tier 2 falls back to tier 1 (or tier 0 if tier 1
is also unavailable).
## Embedding sources
The embedding layer (`src/lib/memory/embedding/`) resolves which source to use
based on `MemorySettingsExtended.embeddingSource`:
| Source | Description | Key required | Cold start |
| -------------- | ---------------------------------------------------------------------------- | ------------ | ---------------- |
| `remote` | Uses a configured provider's embedding API (OpenAI, Cohere, etc.) | Yes | None |
| `static` | Local lookup-table embedding via `potion-base-8M` (WordPiece + mean pooling) | No | ~200ms |
| `transformers` | Local ONNX inference via `@huggingface/transformers` v4, `all-MiniLM-L6-v2` | No | ~3s + ~400MB RAM |
| `auto` | Runtime resolution: remote (if key exists) → static → transformers → null | Depends | Depends |
**Resolution order for `auto`:**
1. Find first provider in `listEmbeddingProviders()` with `hasKey === true` → `remote`.
2. If `settings.staticEnabled === true` → `static`.
3. If `settings.transformersEnabled === true` → `transformers`.
4. Otherwise → `null` (degrades to FTS5 keyword search).
The embedding cache (`src/lib/memory/embedding/cache.ts`) uses an in-memory
LRU map keyed by `${source}:${model}:${dim}:${sha256(text)}`, capped at
`MEMORY_EMBEDDING_CACHE_MAX` entries (default 1000) with a TTL of
`MEMORY_EMBEDDING_CACHE_TTL_MS` (default 5 min). Shared across all callers
per process lifecycle.
## Hybrid RRF (k=60)
When `strategy = "hybrid"` and the vector store is available, retrieval uses
Reciprocal Rank Fusion to merge FTS5 and vector results:
```
RRF(d) = Σ 1 / (k + rank_i(d)) where k = 60 (configurable via MEMORY_RRF_K)
i
```
Concretely:
1. Run FTS5 search → ranked list `R_fts` (position 1..N).
2. Run KNN vector search → ranked list `R_vec` (position 1..M).
3. For each unique `memoryId`:
`rrf_score = 1/(60 + fts_rank)` + `1/(60 + vec_rank)` (0 if not in list).
4. Sort by `rrf_score` DESC, apply token budget walk.
RRF is well-known to be effective without needing score normalization across
heterogeneous retrieval systems. The default `k=60` is from the original
Cormack et al. paper and works well for small corpora (<10k memories).
## Backfill (lazy + reindex)
When the embedding model changes (detected via `embedding_signature`), the
vector store is rebuilt and all existing memories are marked
`needs_reindex = 1` in the `memories` table.
**Lazy backfill**: On the next retrieval, any memory missing a vector entry is
embedded and inserted into `vec_memories` before the search runs. This
amortizes the backfill cost across real requests without blocking startup.
**Explicit reindex**: The Engine tab in `/dashboard/memory` provides a
"Reindex Now" button that calls `POST /api/memory/reindex`. The handler calls
`runReindexBatch()` from `src/lib/memory/reindex.ts`, which processes up to
`limit` pending entries per request. Progress can be polled via
`GET /api/memory/engine-status` (`vectorStore.needsReindex`).
The `memory_vec_meta` table (migration `073_memory_vec.sql`) stores:
- `active_dim` — current vector dimension (null = not yet calibrated).
- `embedding_signature` — `${source}:${model}:${dim}` used to detect changes.
- `last_reset_at` — timestamp of last full reset.
- `vec_loaded` — 0/1 flag whether sqlite-vec loaded successfully.
## Settings extension
Seven new fields were added to `MemorySettingsExtended` (plan 21, D9) in
`src/shared/schemas/memory.ts`, persisted via `src/lib/db/settings.ts`:
| Field | Type | Default | Description |
| ------------------------ | -------------------------------------------------- | -------- | ------------------------------------------------ |
| `embeddingSource` | `"remote" \| "static" \| "transformers" \| "auto"` | `"auto"` | Which embedding source to use |
| `embeddingProviderModel` | `string \| null` | `null` | Provider/model in `provider/model` format |
| `transformersEnabled` | `boolean` | `false` | Opt-in for Transformers.js (MiniLM, ~400MB) |
| `staticEnabled` | `boolean` | `false` | Opt-in for static potion-base-8M local model |
| `rerankEnabled` | `boolean` | `false` | Enable reranking step (adds +200-500ms/req) |
| `rerankProviderModel` | `string \| null` | `null` | Rerank provider/model in `provider/model` format |
| `vectorStore` | `"sqlite-vec" \| "qdrant" \| "auto"` | `"auto"` | Which vector backend to use |
These are exposed via `GET /PUT /api/settings/memory` (schema `MemorySettingsExtendedSchema`).
> **TODO (D20):** Scope `global` (sharing memories across all API keys) is not
> implemented in this release. It requires schema changes and a global retrieval
> path. Track separately.
## Storage Layers
### Primary: SQLite (`memories` table)
Created by migration `015_create_memories.sql`:
| Column | Type | Notes |
| --------------------------- | ------------------ | -------------------------------------------------------------------- |
| `id` | `TEXT PRIMARY KEY` | UUID generated via `crypto.randomUUID()` |
| `api_key_id` | `TEXT NOT NULL` | Owning API key |
| `session_id` | `TEXT` | Optional per-conversation scope |
| `type` | `TEXT NOT NULL` | One of `factual`, `episodic`, `procedural`, `semantic` |
| `key` | `TEXT` | Stable upsert key, e.g. `preference:i_prefer_python` |
| `content` | `TEXT NOT NULL` | The actual fact text |
| `metadata` | `TEXT` | JSON blob (category, extractedAt, source, ...) |
| `created_at` / `updated_at` | `TEXT` | ISO 8601 strings |
| `expires_at` | `TEXT` | Optional expiry; `NULL` means permanent |
| `memory_id` | `INTEGER UNIQUE` | Added by `023_fix_memory_fts_uuid.sql` to bridge UUIDs ↔ FTS5 rowids |
Indexes: `api_key_id`, `session_id`, `type`, `expires_at`, plus the unique
`memory_id` index.
**Upsert semantics**: `createMemory()` looks for an existing row with the same
`(api_key_id, key)` and updates it in place when found (merging `metadata` via
shallow spread). This keeps the table from growing unbounded for repeated
preference statements.
### Full-text Search (`memory_fts` virtual table)
`022_add_memory_fts5.sql` creates an FTS5 virtual table over `content` and
`key`. `023_fix_memory_fts_uuid.sql` fixes a real-world bug where the UUID
primary key did not join to FTS5's integer rowid — the migration adds the
`memory_id` column, recreates the FTS table, and wires triggers
(`memory_fts_ai`, `memory_fts_ad`, `memory_fts_au`) that keep FTS in sync on
INSERT, DELETE, and UPDATE.
Used by `retrieval.ts` for the `semantic` and `hybrid` strategies (see below).
The retrieval code guards with `hasTable("memory_fts")` and falls back to
chronological order if the FTS table is missing or the FTS query throws.
### Optional: Qdrant (vector store tier 2)
`src/lib/memory/qdrant.ts` implements an optional Qdrant integration as tier 2
vector store. Retrieval only routes to Qdrant when the engine selector
`memoryVectorStore === "qdrant"` — the default `"auto"` (and `"sqlite-vec"`)
**never** select Qdrant. The Engine-tab toggle sets **both** `qdrantEnabled` and
`memoryVectorStore` together: enabling makes Qdrant the primary store, disabling
resets to `"auto"` (#5597 — before that fix, enabling was inert because nothing
wrote the engine selector). If Qdrant is unreachable or returns nothing, retrieval
falls back to sqlite-vec → FTS5.
- `upsertSemanticMemoryPoint()` — embed `key + content` with the configured
embedding model, ensure the collection exists (creates cosine-distance
vectors on first use), and upsert a point with payload `{memoryId,
apiKeyId, sessionId, key, content, metadata, createdAtUnix, expiresAtUnix}`.
- `searchSemanticMemory(query, topK, scope)` — embed the query, search the
collection filtered by `kind = "omniroute_memory"` and optionally by
`apiKeyId` / `sessionId`. Caps `topK` to `[1, 20]`.
- `deleteSemanticMemoryPoint(id)` — single point delete. Called by
`deleteMemory()` after the SQLite row is removed (D15).
- `cleanupSemanticMemoryPoints({retentionDays})` — bulk delete points whose
`expiresAtUnix` is in the past or whose `createdAtUnix` is older than the
retention cutoff. Counts first so the dashboard can show actual numbers.
- `checkQdrantHealth()` — `GET /readyz` health probe with latency.
The settings UI exposes Qdrant config, health check, semantic search test,
and cleanup in the **Engine tab** of `/dashboard/memory`. The corresponding
routes under `src/app/api/settings/qdrant/` are all wired as of v3.8.6:
| Route | Method | Description |
| --------------------------------------- | ------------- | ------------------------------- |
| `/api/settings/qdrant` | `GET` / `PUT` | Read / update Qdrant settings |
| `/api/settings/qdrant/health` | `GET` | Liveness probe + latency |
| `/api/settings/qdrant/search` | `POST` | Semantic search test |
| `/api/settings/qdrant/cleanup` | `POST` | Remove expired / old points |
| `/api/settings/qdrant/embedding-models` | `GET` | List available embedding models |
**Behavior notes (what to expect):**
- **Engine selection** — enabling Qdrant in the Engine tab makes it the primary
store (sets `memoryVectorStore="qdrant"`); disabling resets to `"auto"` (#5597).
- **No back-fill** — only memories created/updated **after** Qdrant is enabled are
written to it (fire-and-forget dual-write). Pre-existing SQLite memories are **not**
migrated; "Reindex Now" rebuilds the sqlite-vec index only, not Qdrant.
- **Vector dimension is auto-detected** from the actual embedding on first use — there
is no dimension field to fill in. Changing the embedding model after a collection
exists is **not** auto-handled: the existing collection is left untouched, dimension-
mismatched writes/searches fail and fall back to sqlite-vec. Recreate the collection
(new name, or delete it in Qdrant) to switch embedders.
- **Distance metric** — always **Cosine** (hardcoded on collection creation; not
configurable).
- **Auth** — API key only (sent as the `api-key` header; optional for unauthenticated
local Docker). JWT/RBAC are not used.
- **Config fields** — the UI exposes `host`, `port`, `collection`, `embeddingModel`,
`apiKey`. `vectorSize` / `hnswEfConstruct` are env/DB only and `vectorSize` is not
used for collection creation (dimension comes from the embedding).
### Vector quantization (int8 — opt-in, both backends)
Both vector backends support **opt-in int8 quantization** to cut the memory
footprint of stored vectors (~4× smaller than Float32) at a small recall cost.
Default is **off** on both — vectors stay full-precision unless explicitly
enabled.
| Backend | Setting | Type | Default | Where read |
| ---------- | ------------------------------- | ------------------------------ | -------- | ----------------------------------------------------------- |
| Qdrant | `qdrantQuantization` (DB key) | `"none" \| "int8" \| "binary"` | `"none"` | `src/lib/memory/qdrant.ts::normalizeQdrantConfig()` |
| sqlite-vec | `MEMORY_VEC_QUANTIZATION` (env) | `"none" \| "int8"` | `"none"` | `src/lib/memory/vectorStore.ts::requestedVecQuantization()` |
- **Qdrant** is configured per-instance via the `qdrantQuantization` setting
key (exposed as the `quantization` field on `PUT /api/settings/qdrant`). When
`"int8"`, `buildQuantizationConfig()` requests scalar quantization
(`always_ram`, quantile `0.99`) and searches enable `rescore: true` so the
full-precision vectors refine the int8 candidate set.
- **sqlite-vec** quantization is **environment-only** (not a DB setting): set
`MEMORY_VEC_QUANTIZATION=int8` to store the local vectors as an `int8[dim]`
column via `vec_quantize_int8(?, 'unit')`. The chosen mode is folded into the
`embedding_signature` (an `:int8` suffix), so switching modes triggers a full
reindex of the `vec_memories` table — the same lazy-backfill path used when
the embedding model changes.
## Memory Types
`MemoryType` (`src/lib/memory/types.ts`):
| Type | Used for |
| ------------ | ------------------------------------------------------------ |
| `factual` | Preferences, stable user facts, behavioral patterns |
| `episodic` | Decisions tied to a specific moment ("I chose Postgres") |
| `procedural` | Workflow / how-to memory (reserved; no auto-extractor today) |
| `semantic` | Reserved for vector-store entries |
`MemoryConfig` retrieval strategy is one of `exact`, `semantic`, or `hybrid`,
and scope is one of `session`, `apiKey`, or `global`. The default scope from
`getMemorySettings()` is `apiKey`.
## Fact Extraction (`extraction.ts`)
Extraction is **regex-based**, not LLM-based — it runs in-process with
`setImmediate()` so it never blocks the response stream:
- **Preference patterns** → `MemoryType.FACTUAL`
(e.g. `I prefer …`, `I really like …`, `my favorite is …`, `I hate …`)
- **Decision patterns** → `MemoryType.EPISODIC`
(e.g. `I'll use …`, `I chose …`, `I went with …`, `I'm going to adopt …`)
- **Pattern patterns** → `MemoryType.FACTUAL`
(e.g. `I usually …`, `I always …`, `I tend to …`)
Each match is sanitised (`trim`, whitespace-collapse, capped at 500 chars),
deduplicated within the batch via a stable `factKey(category, content)`, and
stored via `createMemory()` with metadata
`{category, extractedAt, source: "llm_response"}`. Input text is capped at
64 KiB (`MAX_EXTRACTION_TEXT_LENGTH`) — when longer, the **tail** of the text
is used so the most recent assistant content always participates.
`extractFactsFromText(text)` is exported for tests and returns the structured
facts without storing them.
## Retrieval (`retrieval.ts`)
`retrieveMemories(apiKeyId, config)` is the main entry point. It:
1. Normalises and validates the config through `MemoryConfigSchema`.
2. Returns `[]` immediately when `enabled` is false or `maxTokens <= 0`.
3. Clamps `maxTokens` to `[1, 8000]`.
4. Detects whether the modern `memories` table exists (vs the legacy `memory`
table) so older databases keep working.
5. Builds the base query with expiry guard
(`expires_at IS NULL OR datetime(expires_at) > datetime('now')`), optional
session scope, and optional `retentionDays` cutoff.
6. Branches on strategy:
- **`exact`** (default): chronological `ORDER BY created_at DESC LIMIT 100`.
- **`semantic`**: if `config.query` and `memory_fts` exists, JOIN
`memory_fts MATCH ?` and order by FTS rank; fall back to chronological
when FTS returns 0 rows.
- **`hybrid`**: union of FTS results (higher relevance) and the
chronological set, deduplicated by id.
7. Computes a keyword relevance score (`getRelevanceScore`) over
`content`, `key`, and `metadata` JSON when a query is provided. Rows with
zero score are filtered out.
8. Sorts by score desc, then `createdAt` desc.
9. Walks the ranked list and accepts entries while a running
`estimateTokens(content)` (≈ `length / 4`) stays under the budget. Always
returns at least one entry when any matched.
`estimateTokens` is exported and used by retrieval, summarisation, and the MCP
`omniroute_memory_search` tool.
## Injection (`injection.ts`)
`injectMemory(request, memories, provider)`:
1. Joins all memory contents into a single `Memory context: …` string.
2. Picks a strategy by provider name:
- **System message** (default for OpenAI, Anthropic, Gemini, …) — prepends
a `{role: "system", content: memoryText}` ahead of any existing system
messages so user system prompts still take precedence.
- **User message** (fallback) — for providers in
`PROVIDERS_WITHOUT_SYSTEM_MESSAGE`: `o1`, `o1-mini`, `o1-preview`,
`glm`, `glmt`, `glm-cn`, `zai`, `qianfan`. These reject the system role
and would 400 otherwise (cf. issue #1701 for GLM/Zhipu).
3. Logs the count, strategy, and model under `memory.injection.injected`.
`providerSupportsSystemMessage(provider)` is exported for callers that need to
make routing decisions of their own. Unknown providers default to `true`
(system role allowed) for safety.
## Settings (`settings.ts`)
Memory configuration is **stored in the DB settings table**, not in env vars.
`getMemorySettings()` reads from `getSettings()` and caches the result
in-process; `invalidateMemorySettingsCache()` is called by the settings PUT
route after writes.
### Legacy fields (all versions)
| DB key | Type | Default | UI control |
| --------------------- | ------- | -------------------------------------------------- | ----------------------------------------------- |
| `memoryEnabled` | boolean | `false` (off by default since v3.8.30) | Memory on/off |
| `memoryMaxTokens` | integer | `2000` (range `0–16000`) | Token budget for injection |
| `memoryRetentionDays` | integer | `30` (range `1–365`) | Retention window |
| `memoryStrategy` | enum | `"hybrid"` (one of `recent`, `semantic`, `hybrid`) | Retrieval strategy |
| `skillsEnabled` | boolean | `false` | Toggles per-key skill injection (see SKILLS.md) |
Note: the UI strategy `"recent"` maps to the internal `"exact"` retrieval
strategy via `toMemoryRetrievalConfig()` (chronological order).
### New fields (v3.8.6, plan 21 D9)
See also the "Settings extension" section above for field descriptions.
| DB key | API field | Default |
| --------------------------- | ------------------------ | -------- |
| `memoryEmbeddingSource` | `embeddingSource` | `"auto"` |
| `memoryEmbeddingModel` | `embeddingProviderModel` | `null` |
| `memoryTransformersEnabled` | `transformersEnabled` | `false` |
| `memoryStaticEnabled` | `staticEnabled` | `false` |
| `memoryRerankEnabled` | `rerankEnabled` | `false` |
| `memoryRerankModel` | `rerankProviderModel` | `null` |
| `memoryVectorStore` | `vectorStore` | `"auto"` |
Qdrant-related DB keys (`qdrantEnabled`, `qdrantHost`, `qdrantPort`,
`qdrantApiKey`, `qdrantCollection` default `"omniroute_memory"`,
`qdrantEmbeddingModel` default `"openai/text-embedding-3-small"`) are read by
`normalizeQdrantConfig()` in `qdrant.ts`.
### Environment variables (v3.8.6)
Six optional env vars tune the engine's runtime behaviour (documented in `.env.example`):
| Variable | Default | Description |
| ------------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `MEMORY_EMBEDDING_CACHE_TTL_MS` | `300000` | Embedding cache TTL (5 min) |
| `MEMORY_EMBEDDING_CACHE_MAX` | `1000` | Max entries in embedding LRU cache |
| `MEMORY_TRANSFORMERS_MODEL` | `Xenova/all-MiniLM-L6-v2` | HF repo for Transformers.js model |
| `MEMORY_STATIC_MODEL` | `minishlab/potion-base-8M` | HF repo for static potion model |
| `MEMORY_STATIC_CACHE_DIR` | `<DATA_DIR>/embeddings` | Where to store downloaded models |
| `MEMORY_VEC_TOP_K` | `20` | Default top-K for vector search |
| `MEMORY_RRF_K` | `60` | RRF k constant for hybrid search |
| `MEMORY_VEC_QUANTIZATION` | `none` | Set to `int8` to store local sqlite-vec vectors quantized (~4× smaller; opt-in). Mode change forces a reindex. |
## Summarisation (`summarization.ts`)
`summarizeMemories(apiKeyId, sessionId?, maxTokens = 4000)` compacts older
content when the running token total over a key's memories exceeds the
budget. It iterates rows DESC by `created_at`, keeps rows that fit, and for
the rest replaces `content` in place with the first three sentences of the
original. `tokensSaved` is the difference in `estimateTokens` between old and
new content.
This routine is **available but not called automatically** in the current
chat pipeline — call it from a cron, an admin action, or
`MemoryConfig.autoSummarize` glue if you need ongoing compaction. The data
loss is one-way: original text is overwritten.
## REST API
All endpoints require management auth (`requireManagementAuth`).
### Core memory endpoints (existing + updated)
| Method | Path | Description |
| -------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET` | `/api/memory` | Paginated list with filters: `apiKeyId`, `type`, `sessionId`, `q`, `limit`, `page`, `offset`. Response includes `stats.total`, `stats.tokensUsed`, `stats.hitRate`, `cacheStats` |
| `POST` | `/api/memory` | Create entry (Zod-validated: `content`, `key`, optional `type`, `sessionId`, `apiKeyId`, `metadata`, `expiresAt`). Calls `createMemory()` which upserts on `(apiKeyId, key)` |
| `GET` | `/api/memory/[id]` | Fetch a single entry by UUID |
| `PUT` | `/api/memory/[id]` | Update entry fields (`type`, `key`, `content`, `metadata`). Body: `MemoryUpdatePutSchema`. Also syncs vector if embedding source available. |
| `DELETE` | `/api/memory/[id]` | Delete an entry; also deletes from `vec_memories` (D15) and Qdrant best-effort. Returns 404 when missing. |
| `GET` | `/api/memory/health` | Runs `verifyExtractionPipeline("health-check")` — round-trip create→list→delete. Returns `{working, latencyMs, error?}` |
### New memory engine endpoints (plan 21)
| Method | Path | Description |
| ------ | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `POST` | `/api/memory/retrieve-preview` | Dry-run of `retrieveMemories` — returns ranked results with score, tier, tokens. Body: `RetrievePreviewSchema`. Does NOT inject or modify memories. |
| `GET` | `/api/memory/embedding-providers` | Lists providers with embedding models, indicating which have a configured API key. |
| `GET` | `/api/memory/engine-status` | Returns full engine status: keyword tier, embedding resolution, vector store stats, Qdrant health, rerank config. Shape: `MemoryEngineStatusSchema`. |
| `POST` | `/api/memory/summarize` | Manually trigger memory compaction. Body: `MemorySummarizeSchema` (`olderThanDays`, `apiKeyId?`, `dryRun`). Returns `{candidates, tokensSaved}`. |
| `POST` | `/api/memory/reindex` | Trigger vector reindex for memories with `needs_reindex=1`. Body: `MemoryReindexSchema` (`force`). Returns `{started, pending}`. |
### Settings endpoints
| Method | Path | Description |
| ------ | --------------------------------------- | ------------------------------------------------------------------------------------------------ |
| `GET` | `/api/settings/memory` | Current normalised `MemorySettingsExtended` (7 new fields + legacy) |
| `PUT` | `/api/settings/memory` | Update any field from `MemorySettingsExtendedSchema` (12 total fields) |
| `GET` | `/api/settings/qdrant` | Current Qdrant settings (`QdrantSettingsSchema`) |
| `PUT` | `/api/settings/qdrant` | Update Qdrant settings. Body: `QdrantSettingsUpdateSchema`. `apiKey` = empty string removes key. |
| `GET` | `/api/settings/qdrant/health` | Liveness probe against configured Qdrant instance. Returns `QdrantHealthResultSchema`. |
| `POST` | `/api/settings/qdrant/search` | Semantic search test against Qdrant. Body: `QdrantSearchSchema` (`query`, `topK`). |
| `POST` | `/api/settings/qdrant/cleanup` | Remove Qdrant points for expired / old memories. |
| `GET` | `/api/settings/qdrant/embedding-models` | List embedding models available for Qdrant. |
The `/api/memory` list query supports either `page`-based pagination
(`parsePaginationParams`) **or** raw `offset` — when `offset` is present it
takes precedence and a derived `page` is computed for the response shape.
## MCP Tools (`open-sse/mcp-server/tools/memoryTools.ts`)
When the MCP server is enabled, three memory tools are registered:
- `omniroute_memory_search` — `{apiKeyId, query?, type?, maxTokens?, limit?}`
→ wraps `retrieveMemories()`. As of v3.8.6 (D16), the `strategy` is read
from `getMemorySettings()` instead of being hardcoded to `"exact"`. If
`query` is provided and `strategy` is `semantic` or `hybrid`, the vector
store is used when available.
- `omniroute_memory_add` — `{apiKeyId, sessionId?, type, key, content,
metadata?}` → wraps `createMemory()`. Accepts only the 4 canonical types:
`factual`, `episodic`, `procedural`, `semantic` (D17).
- `omniroute_memory_clear` — `{apiKeyId, type?, olderThan?}` → lists matching
entries, optionally filters by created-before timestamp, then deletes each
via `deleteMemory()` (which also removes vectors from sqlite-vec + Qdrant).
See [MCP-SERVER.md](./MCP-SERVER.md) for transport and scope details.
## Dashboard (Memory Studio)
`src/app/(dashboard)/dashboard/memory/page.tsx` is now a **3-tab Studio**:
### Tab: Memories
- Concept card (collapsible "How it works" explainer).
- Real-time list, search, and pagination (debounced 300 ms).
- Type filter (`factual` / `episodic` / `procedural` / `semantic` / all).
- Add-memory modal (key, content, type).
- Inline edit (pencil button → `PUT /api/memory/[id]`).
- Delete per row (with confirmation dialog).
- JSON export of the current page; JSON import via file picker.
- Stat cards: `totalEntries`, `tokensUsed`, `hitRate`.
- "Compact old" button → `POST /api/memory/summarize` (dry-run first shows
candidate count, then confirms).
- A green/red health dot driven by `GET /api/memory/health`.
### Tab: Playground
- Query input + strategy selector (Exact / Semantic / Hybrid) + token budget.
- "Simulate" → `POST /api/memory/retrieve-preview` — shows ranked results with
`score`, `tier`, `tokens`, `vecScore`, `ftsScore`.
- Resolution panel showing which embedding source / vector store was used and
whether a fallback occurred.
### Tab: Engine
- Engine status panel (keyword FTS5 chip, embedding chip, vector store chip,
Qdrant health chip, rerank chip).
- "Reindex Now" button → `POST /api/memory/reindex`.
- Embedding source selector (auto / remote / static / transformers + toggles).
- Qdrant config card (enable toggle, host/port/collection/key, test connection,
semantic search test, cleanup).
- Rerank config card (enable toggle, provider/model selector).
Memory and Qdrant settings also live under
`/dashboard/settings → Memory & Skills` (`MemorySkillsTab.tsx`) for
the legacy/global settings surface.
## Caching
`src/lib/memory/store.ts` keeps an in-process LRU-ish cache
(`MEMORY_CACHE_TTL = 5 min`, `MEMORY_MAX_CACHE_SIZE = 10 000`, with 20 %
oldest eviction) for `getMemory(id)` reads, plus a generic key/value
`memoryCache` layer (`src/lib/memory/cache.ts`) with `get`/`set`/`invalidate`
methods used by callers that want their own scoped cache (1 000-entry LRU,
default TTL 5 min).
## Privacy & Lifecycle
- Memory ownership is the API key id (`resolveMemoryOwnerId` in
`chatCore.ts`). Without an `apiKeyInfo.id` neither retrieval nor injection
nor extraction runs.
- Entries with a future `expires_at` are filtered out of retrieval; old
entries beyond `retentionDays` are excluded by the
`created_at >= cutoff` clause in `retrieveMemories`.
- For hard deletion, use `DELETE /api/memory/[id]` or `omniroute_memory_clear`.
- Extraction is fire-and-forget via `setImmediate`; failures are logged under
`memory.extraction.background.failed` and never surface to the caller.
- Verification round-trips (`verifyExtractionPipeline`) clean up their own
test entries in a `finally` block.
## See Also
- [SKILLS.md](./SKILLS.md) — the `skillsEnabled` setting injects tool
definitions alongside memory.
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP transport / scopes.
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — broader API surface.
- Source modules:
- `src/lib/memory/types.ts`, `schemas.ts`
- `src/lib/memory/store.ts`, `retrieval.ts`, `injection.ts`, `reindex.ts`
- `src/lib/memory/extraction.ts`, `summarization.ts`, `verify.ts`
- `src/lib/memory/settings.ts`, `qdrant.ts`, `cache.ts`
- `src/lib/memory/vectorStore.ts` — sqlite-vec + hybrid RRF
- `src/lib/memory/embedding/index.ts` — multi-source embedding layer
- `src/lib/memory/embedding/types.ts`, `remote.ts`, `staticPotion.ts`,
`transformersLocal.ts`, `cache.ts`
- `src/shared/schemas/memory.ts` — Zod schemas for all memory API bodies
- `src/shared/schemas/qdrant.ts` — Zod schemas for Qdrant settings/ops
- `src/lib/db/memoryVec.ts` — CRUD for `memory_vec_meta`
- `src/lib/db/migrations/015_create_memories.sql`,
`022_add_memory_fts5.sql`, `023_fix_memory_fts_uuid.sql`,
`073_memory_vec.sql`
- `src/app/api/memory/route.ts`, `[id]/route.ts`, `health/route.ts`
- `src/app/api/memory/retrieve-preview/route.ts`
- `src/app/api/memory/engine-status/route.ts`
- `src/app/api/memory/embedding-providers/route.ts`
- `src/app/api/memory/summarize/route.ts`
- `src/app/api/memory/reindex/route.ts`
- `src/app/api/settings/memory/route.ts`
- `src/app/api/settings/qdrant/route.ts` + sub-routes
- `src/app/(dashboard)/dashboard/memory/` — Studio UI (page + components +
tabs + hooks)
- `open-sse/handlers/chatCore.ts` (injection / extraction wiring)
- `open-sse/mcp-server/tools/memoryTools.ts`
---
## Choosing an Embedding Provider (v3.8.16+)
OmniRoute's memory engine supports **four embedding sources** (`src/lib/memory/embedding/`). Each has different trade-offs in **latency, cost, model quality, and setup complexity**.
### The Four Providers
| Provider | Source | Latency | Cost | Quality | Setup |
| -------------- | ------------------------------------------ | ------------------------------- | -------------------- | -------------------------- | ------------------ |
| `transformers` | Local ONNX model (Xenova/all-MiniLM-L6-v2) | ~50-150ms (CPU) | Free | Good | `npm install` only |
| `static` | Pre-computed vectors (cached) | <1ms | Free | N/A (depends on cache hit) | None |
| `remote` | OpenAI / Cohere / Voyage API | ~100-300ms | $0.02-0.10/1M tokens | Excellent | API key |
| `cache` | In-memory LRU layer over any source | <1ms (hit), full latency (miss) | Free | Same as underlying | None |
### Decision Tree
```
What's your deployment context?
│
┌───────────┼───────────┬──────────────┐
│ │ │ │
DEV/TEST SMALL PROD LARGE PROD EDGE / OFFLINE
│ │ │ │
▼ ▼ ▼ ▼
transformers transformers remote (Qdrant) transformers
(free, no API) (best quality) (no internet)
│ │ │ │
└────────┬──┴───────────┴──────────────┘
│
▼
ALWAYS add `cache` layer on top
(LruCache wraps any provider)
```
### Database & API Configuration
Memory embedding options are configured via the Settings API/UI, not environment variables. The relevant settings database keys under Settings (`normalizeMemorySettings` in `src/lib/memory/settings.ts`) are:
- `memoryEmbeddingSource`: `"transformers"` (local), `"remote"` (API-based, e.g. OpenAI), `"static"` (external store), or `"auto"`
- `memoryEmbeddingProviderModel`: Model identifier for remote/static sources (e.g., `"text-embedding-3-small"`)
- `memoryTransformersEnabled`: `true` | `false`
- `memoryStaticEnabled`: `true` | `false`
- `memoryVectorStore`: `"sqlite-vec"`, `"qdrant"`, or `"auto"`
#### Local Model (`transformers`)
Uses transformers.js internally to run local models:
```bash
# Env vars read in code (src/lib/memory/embedding/index.ts):
MEMORY_TRANSFORMERS_MODEL=Xenova/all-MiniLM-L6-v2 # HF model repo
MEMORY_STATIC_MODEL=minishlab/potion-base-8M # HF static potion model
MEMORY_STATIC_CACHE_DIR=<DATA_DIR>/embeddings # Cache directory
```
#### LRU Embedding Cache
The cache is always on by default and configured via env vars:
```bash
MEMORY_EMBEDDING_CACHE_MAX=1000 # Max cached items
MEMORY_EMBEDDING_CACHE_TTL_MS=300000 # TTL (5 min)
```
### Performance Numbers
Benchmark on a typical 4-core x86 server (texts ~100 tokens each):
| Provider | p50 | p95 | p99 | Cost / 1M embeddings |
| -------------------- | ----- | ----- | ----- | ---------------------------------- |
| `transformers` (CPU) | 80ms | 180ms | 350ms | Free |
| `remote` (OpenAI) | 120ms | 220ms | 400ms | ~$0.02 (ada-002) / $0.13 (3-large) |
| `static` (Qdrant) | 15ms | 30ms | 60ms | Depends on Qdrant hosting |
| `cache` (hit) | <1ms | <1ms | 2ms | Free |
---
## Fact Extraction Patterns (v3.8.16+)
The `extraction.ts` module (`src/lib/memory/extraction.ts`) uses **regex pattern matching** to extract structured facts from conversation messages. Understanding these patterns helps you tune extraction quality for your use case.
### Default Pattern Categories
| Category | Example pattern | Captures |
| ------------------- | ----------------------------------------------------------- | ------------------------------ |
| PREFERENCE_PATTERNS | `"I prefer <X>"`, `"I like <X>"`, `"I hate <X>"` | User preferences |
| DECISION_PATTERNS | `"I'll use <X>"`, `"I decided to <X>"`, `"I went with <X>"` | User decisions (episodic) |
| PATTERN_PATTERNS | `"I usually <X>"`, `"I always <X>"`, `"I never <X>"` | Persistent behavioral patterns |
### Example Patterns (Simplified)
```ts
// From src/lib/memory/extraction.ts
const PREFERENCE_PATTERNS = [
/\bI\s+(?:really\s+)?prefer\s+([^.,\n]+)/gi,
/\bI\s+(?:really\s+)?like\s+([^.,\n]+)/gi,
/\bI\s+(?:hate|dislike|avoid)\s+([^.,\n]+)/gi,
];
const DECISION_PATTERNS = [
/\bI'?(?:ll|will)\s+use\s+([^.,\n]+)/gi,
/\bI\s+(?:have\s+)?decided\s+(?:to\s+)?([^.,\n]+)/gi,
];
const PATTERN_PATTERNS = [/\bI\s+usually\s+([^.,\n]+)/gi, /\bI\s+always\s+([^.,\n]+)/gi];
```
### What Gets Extracted
When a user says:
> "I prefer TypeScript. I'll use Postgres for this project. I always commit before pushing. I don't like Python."
> Extraction produces 4 memories:
>
> | Key | Category | Type | Content |
> | ------------------------------------ | ---------- | -------- | --------------------------- |
> | `preference:typescript` | preference | factual | "TypeScript" |
> | `decision:postgres_for_this_project` | decision | episodic | "Postgres for this project" |
> | `pattern:commit_before_pushing` | pattern | factual | "commit before pushing" |
> | `preference:python` | preference | factual | "Python" |
### Extraction Limits
To prevent runaway extraction, the following limits apply:
| Min content length | 3 chars |
| Max content length | 500 chars |
### When to Disable Extraction
Extraction runs automatically whenever memory is enabled; there is no separate
extraction-only toggle. To turn it off, disable memory entirely (`enabled: false`
via `PUT /api/settings/memory`). Consider doing so when:
- You have high message volume and the extraction cost is non-trivial
- Your conversations are mostly transient (chat, debugging) with no long-term value
- You're already capturing context via custom plugins
---
## Hybrid RRF Tuning (v3.8.16+)
The **Reciprocal Rank Fusion (RRF)** algorithm combines FTS5 (keyword) and vector (semantic) results. The `k` parameter controls how much weight is given to lower-ranked results.
### The Formula
For each candidate memory, the RRF score is:
```
RRF(d) = Σ 1 / (k + rank_i(d))
```
Where:
- `k` is the constant (default 60)
- `rank_i(d)` is the rank of document `d` in the i-th retrieval system (FTS, vector)
- The sum runs over all retrieval systems
### How `k` Affects Results
| `k` value | Effect | Best for |
| -------------------- | ------------------------------------------------------------------------------- | -------------------------------------- |
| `k=0` | Pure rank fusion (no smoothing) | Theoretical baseline |
| `k=10-30` | Heavily weights top results, low-rank barely contributes | When top-3 results are usually correct |
| **`k=60`** (default) | Balanced — top-10 results all contribute meaningfully | General-purpose retrieval |
| `k=100+` | Flatter — even low-rank results can dominate if they appear in multiple systems | When recall > precision is critical |
### Tuning `k` in Practice
```bash
# Default
MEMORY_RRF_K=60
# Aggressive precision (small memory, few docs)
MEMORY_RRF_K=20
# Maximum recall (large memory, varied queries)
MEMORY_RRF_K=120
```
**Example with `k=20`:**
- FTS rank 1 → contribution `1/21 = 0.048`
- FTS rank 10 → contribution `1/30 = 0.033`
- Vector rank 1 → contribution `0.048`
- Combined max: `0.096`
**Example with `k=60`:**
- FTS rank 1 → contribution `1/61 = 0.016`
- FTS rank 10 → contribution `1/70 = 0.014`
- Vector rank 1 → contribution `0.016`
- Combined max: `0.033`
With higher `k`, the **relative difference** between top-1 and rank-10 is smaller, so the algorithm relies more on **consensus across retrieval systems** than on top-rank confidence.
### When to Change `k`
| Symptom | Try |
| -------------------------------------- | ------------------------------------------------------------ |
| Top result always wins, but it's wrong | **Lower** k (e.g., 20) — top-rank confidence matters more |
| Right answer is in top-5 but not top-1 | **Higher** k (e.g., 100) — flatter scoring rewards consensus |
| Recall is high but precision is low | **Lower** k — sharpen the ranking |
| Recall is low (missing relevant docs) | **Higher** k — give lower-ranked docs a chance |
### RRF Weighting
The reciprocal rank fusion uses equal weights for semantic vector rank and full-text search rank:
```
RRF(d) = 1/(k + rank_vector) + 1/(k + rank_fts)
```
There are no environment variables to adjust individual weights (`MEMORY_RRF_VECTOR_WEIGHT`/`MEMORY_RRF_FTS_WEIGHT` do not exist).
---
## Summarization Strategy (v3.8.16+)
The `summarization.ts` module (`src/lib/memory/summarization.ts`) compresses older memories to keep the active set small while preserving recall.
### When Summarization Triggers
| Trigger | Threshold (default) |
| ---------------------- | ------------------- |
| Manual trigger via API | n/a |
### What Gets Summarized
Two entry points are exported from `summarization.ts`:
- **`summarizeMemories(apiKeyId, sessionId?, maxTokens = 4000)`** — condenses the
memories for a session into a single summary text bounded by a token budget.
- **`summarizeMemoriesOlderThan(apiKeyId, days, dryRun)`** — the age-based
compaction used by the API: it selects every memory older than `days`, builds
one condensed summary memory from them, and (when `dryRun` is `false`) deletes
the originals. Pass `dryRun: true` to preview the candidate set and token total
without modifying anything.
There is no tag/key clustering pass or per-memory "core vs summarizable" scoring —
selection is purely the age cutoff, and the summary text is a condensed,
type-prefixed line per candidate.
### Triggering Summarization
Summarization is **manual / opt-in** — the `autoSummarize` setting is `false` by
default, so nothing is compacted automatically. Trigger it via the API:
```bash
curl -X POST http://localhost:20128/api/memory/summarize \
-H "Authorization: Bearer $OMNIROUTE_KEY"
```
To leave it off, simply keep `autoSummarize` at its default (`false`).
### Summarization Quality Tips
- **Preview first with `dryRun`** — `summarizeMemoriesOlderThan(..., true)` returns
the candidate list and total token count so you can confirm what would be merged
before deleting the originals.
- **Run summarization during low-traffic hours** if you have a large memory corpus — the LLM call is the slow part
```bash
# Cron-style: summarize at 3am daily
0 3 * * * curl -X POST http://localhost:20128/api/memory/summarize \
-H "Authorization: Bearer $OMNIROUTE_KEY"
```
docs/frameworks/NOTION_CONTEXT.md
+131
View File
@@ -0,0 +1,131 @@
---
title: "Notion Context Source"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Notion Context Source
> **Source of truth:** `src/lib/notion/api.ts` (REST client), `src/lib/db/notion.ts`
> (token persistence), `open-sse/mcp-server/tools/notionTools.ts` (6 MCP tools),
> `src/app/api/settings/notion/route.ts` (settings API). Tool registration and scope
> wiring lives in `open-sse/mcp-server/server.ts`.
## What it is
OmniRoute can connect to a **Notion** workspace as a **context source** — a read/write
knowledge base that agents reach through the built-in MCP server. Once a Notion
integration token is configured, the MCP tools let an LLM search pages and databases,
read page content and block trees, query databases with filters/sorts, and append new
blocks — all proxied through OmniRoute (with retry, timeout, and error classification)
so the model never touches the Notion API directly.
The integration is a thin, hardened wrapper over the official Notion REST API
(`https://api.notion.com/v1`, `Notion-Version: 2026-03-11`). The client
(`src/lib/notion/api.ts`) adds:
- **Retry with exponential backoff** (up to 3 attempts) for `429` and `5xx`.
- **55-second request timeout** via `AbortController`.
- **Typed error classification** — `NotionAuthError` (401/403),
`NotionNotFoundError` (404), `NotionRateLimitError` (429, honors `retry after`
hints), `NotionValidationError` (400/409), `NotionServerError` (5xx),
`NotionTimeoutError`.
- **Message sanitization** that strips stack-trace-like fragments before surfacing.
## Setup
There is **no environment variable** for the Notion token — it is stored in the
SQLite `key_value` table (namespace `notion`, key `integration_token`) via
`src/lib/db/notion.ts`. Configure it from the **Context Sources** tab of the Endpoint
dashboard (`ObsidianSourceCard`'s sibling `NotionSourceCard`), or via the settings REST API.
> [!NOTE]
> The token is a **Notion internal integration token**. Create an integration at
> <https://www.notion.com/my-integrations>, then share the pages/databases you want
> OmniRoute to access with that integration (Notion's permission model is share-based,
> not workspace-wide).
### Configure via REST
```bash
# Save + validate the integration token (POST validates by issuing a test search)
curl -X POST http://localhost:20128/api/settings/notion \
-H "Content-Type: application/json" \
-d '{"token":"ntn_xxx"}'
# Check connection status
curl http://localhost:20128/api/settings/notion
# Disconnect (clears the stored token)
curl -X DELETE http://localhost:20128/api/settings/notion
```
All three methods require dashboard authentication (`isAuthenticated`). On `POST`,
OmniRoute saves the token and immediately runs a 1-result test search; if Notion
returns an error object the token is cleared and the call fails with `400`.
## MCP tools (6)
Defined in `open-sse/mcp-server/tools/notionTools.ts`. The token is resolved at call
time via `getNotionToken()`; if none is configured the tool throws
`"Notion integration token not configured. Set it in Settings > Context Sources."`
| Tool | Scope | Description |
| ---------------------------- | -------------- | --------------------------------------------------------------------------------- |
| `notion_search` | `read:notion` | Search pages and databases by text query (returns titles, IDs, URLs). Paginated. |
| `notion_get_page` | `read:notion` | Get content and metadata of a page by its ID. |
| `notion_list_block_children` | `read:notion` | List all block children of a block or page (the block tree). Paginated. |
| `notion_query_database` | `read:notion` | Query a database with optional `filter` + `sorts` (Notion API format). Paginated. |
| `notion_get_database` | `read:notion` | Get the schema/metadata of a database by ID. |
| `notion_append_blocks` | `write:notion` | Append block children to an existing block or page (max 100 blocks per request). |
### Input parameters
- `notion_search` — `query` (1–500 chars), `pageSize` (1–100, default 20),
`startCursor` (optional).
- `notion_get_page` — `pageId` (32-char hex or UUID).
- `notion_list_block_children` — `blockId`, `pageSize` (1–100, default 50),
`startCursor` (optional).
- `notion_query_database` — `databaseId`, `filter` (optional, Notion filter format),
`sorts` (optional array), `pageSize` (1–100, default 50), `startCursor` (optional).
- `notion_get_database` — `databaseId`.
- `notion_append_blocks` — `blockId`, `children` (array of block objects),
`after` (optional position).
### Scopes
The read tools require `read:notion` and the write tool requires `write:notion`.
Scopes are enforced by `withScopeEnforcement()` in
`open-sse/mcp-server/server.ts` only when `OMNIROUTE_MCP_ENFORCE_SCOPES=true`; the
caller's allowed scopes come from `OMNIROUTE_MCP_SCOPES` (comma-separated) or the
authenticated API key's scope context. See [MCP-SERVER.md](./MCP-SERVER.md) for the
full scope model.
## Endpoints
| Method | Path | Purpose |
| -------- | ---------------------- | -------------------------------------- |
| `GET` | `/api/settings/notion` | Return `{ connected, hasToken }`. |
| `POST` | `/api/settings/notion` | Save + validate the integration token. |
| `DELETE` | `/api/settings/notion` | Disconnect (clear the stored token). |
> These are dashboard settings routes. There is **no public `/v1` Notion proxy
> endpoint** — Notion is reached exclusively through the MCP tools above.
## Use cases
- **Knowledge-grounded answers** — let an agent `notion_search` the workspace and
`notion_get_page` the top hit before answering, so responses cite real internal docs.
- **Database-backed workflows** — `notion_query_database` a tasks/CRM database with
filters + sorts, then summarize or triage the rows.
- **Write-back / logging** — `notion_append_blocks` to append meeting notes, run
summaries, or agent output into an existing page (append-only; no destructive edits).
- **Structure exploration** — `notion_list_block_children` to walk a page's block tree,
or `notion_get_database` to discover a database's property schema before querying it.
## Related
- [MCP Server](./MCP-SERVER.md) — transports, scope enforcement, full tool inventory.
- [Obsidian Context Source](./OBSIDIAN_CONTEXT.md) — the other built-in context source.
- [Memory System](./MEMORY.md) — persistent conversational memory (complementary
context layer, injected automatically rather than tool-fetched).
docs/frameworks/OBSIDIAN_CONTEXT.md
+191
View File
@@ -0,0 +1,191 @@
---
title: "Obsidian Context Source"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Obsidian Context Source
> **Source of truth:** `src/lib/obsidian/api.ts` (REST + sync client),
> `src/lib/db/obsidian.ts` (token / base-URL / WebDAV persistence),
> `src/lib/obsidianSync.ts` (WebDAV vault sync), `open-sse/mcp-server/tools/obsidianTools.ts`
> (22 MCP tools), `src/app/api/settings/obsidian/route.ts` +
> `src/app/api/settings/obsidian/webdav/route.ts` (settings APIs). Tool registration
> and scope wiring lives in `open-sse/mcp-server/server.ts`.
## What it is
OmniRoute connects to an **Obsidian** vault as a **context source** — a local Markdown
knowledge base that agents read and write through the built-in MCP server. The
integration talks to the **Obsidian Local REST API** community plugin running inside the
desktop app, so agents can search notes, read/write/patch files, list the vault, work
with daily/weekly periodic notes, manage tags, run Obsidian commands, and (optionally)
coordinate a bidirectional desktop↔mobile vault sync.
The client (`src/lib/obsidian/api.ts`) wraps the Local REST API with:
- **Retry with backoff** for transient `5xx`, **30-second timeout** via `AbortController`.
- **Typed error classification** — `ObsidianAuthError` (401/403),
`ObsidianNotFoundError` (404), `ObsidianServerError` (5xx), `ObsidianTimeoutError`.
- A **friendly "cannot reach Obsidian" hint** that calls out the common port mistake
(HTTP on `27123`, **not** the MCP endpoint on `27124`) and the Tailscale form.
- Vault-relative **path encoding** so note paths with spaces/slashes are safe.
## Setup
There is **no environment variable** for the Obsidian token or base URL — both are
stored in the SQLite `key_value` table (namespace `obsidian`) via
`src/lib/db/obsidian.ts`. The token is **encrypted at rest** (AES-256-GCM, with
plaintext backward-compat fallback). Configure from the **Context Sources** tab of the
Endpoint dashboard (`ObsidianSourceCard`), or via the settings REST API.
> [!IMPORTANT]
> The **Obsidian Local REST API** plugin must be installed and running. Its REST
> interface listens on **HTTP `127.0.0.1:27123`** (the default base URL). Port `27124`
> is a _separate_ MCP/HTTPS endpoint and is explicitly rejected by the settings route.
> If connecting from another device, use `http://<tailscale-ip>:27123`.
### Configuration keys (SQLite `key_value`, namespace `obsidian`)
| Key | Purpose | Encrypted |
| ----------------- | ------------------------------------------------ | --------- |
| `api_key` | Local REST API bearer token | yes |
| `base_url` | REST base URL (default `http://127.0.0.1:27123`) | no |
| `vault_path` | Absolute path to the vault directory (for sync) | no |
| `webdav_username` | Generated WebDAV username (vault sync) | no |
| `webdav_password` | Generated WebDAV password (vault sync) | yes |
| `webdav_enabled` | Whether WebDAV vault sync is enabled | no |
### Configure via REST
```bash
# Save + validate the Local REST API token (POST validates via a status check)
curl -X POST http://localhost:20128/api/settings/obsidian \
-H "Content-Type: application/json" \
-d '{"token":"<obsidian-rest-api-key>","baseUrl":"http://127.0.0.1:27123"}'
# Check connection status (returns connected, hasToken, baseUrl, vaultPath)
curl http://localhost:20128/api/settings/obsidian
# Disconnect (clears the stored token)
curl -X DELETE http://localhost:20128/api/settings/obsidian
```
All methods require dashboard authentication. `POST` rejects any URL on port `27124`
and validates the token by calling the Local REST API status endpoint before persisting.
### WebDAV vault sync
`src/app/api/settings/obsidian/webdav/route.ts` manages an optional WebDAV-backed
vault sync (driven by `src/lib/obsidianSync.ts`). Enabling it points OmniRoute at a
local vault directory and mints a random WebDAV username/password pair:
```bash
# Enable WebDAV sync for a vault directory (mints username/password)
curl -X POST http://localhost:20128/api/settings/obsidian/webdav \
-H "Content-Type: application/json" \
-d '{"vaultPath":"/home/me/MyVault"}'
# Get WebDAV sync status (credentials returned only while enabled)
curl http://localhost:20128/api/settings/obsidian/webdav
# Disable WebDAV sync (clears credentials + managed .stignore)
curl -X DELETE http://localhost:20128/api/settings/obsidian/webdav
```
### Per-API-key context source (optional)
Obsidian config can be scoped **per API key** via the `api_key_context_sources` table
(`src/lib/db/apiKeyContextSources.ts`). When an MCP call carries an authenticated API
key id, `getObsidianConfigForApiKey()` prefers that key's own token/base-URL/vault-path
(`source: "api_key"`) and otherwise falls back to the global config (`source: "global"`).
## MCP tools (22)
Defined in `open-sse/mcp-server/tools/obsidianTools.ts`. The token/base-URL are resolved
per call (per-API-key first, then global). Tools that hit the OmniRoute **sync server**
(the four `obsidian_sync_*` tools) additionally require the sync auth token configured
in OmniRoute settings.
### Read tools (`read:obsidian`)
| Tool | Description |
| ---------------------------- | ------------------------------------------------------------------------------------ |
| `obsidian_check_status` | Check whether the Local REST API is reachable and authenticated. |
| `obsidian_search_simple` | Full-text search of note content; returns snippets with file paths. |
| `obsidian_search_structured` | Search using a JSON Logic expression (and/or/regex/path filters). |
| `obsidian_read_note` | Read a note by vault-relative path; optionally a specific heading/block/frontmatter. |
| `obsidian_list_vault` | List files and directories in the vault (tree of entries). |
| `obsidian_get_document_map` | Get the note's heading structure as a map of headings → line numbers. |
| `obsidian_get_note_metadata` | Get frontmatter, tags, links, char/word count without the full content. |
| `obsidian_get_active_file` | Get the path + content of the currently active file in Obsidian. |
| `obsidian_get_periodic_note` | Get the daily/weekly/monthly periodic note for a date (today if omitted). |
| `obsidian_get_tags` | List all vault tags with their frequencies. |
| `obsidian_list_commands` | List available Obsidian command IDs (use with `obsidian_execute_command`). |
| `obsidian_sync_status` | OmniRoute sync server status: running, vault name, port, uptime, last sync. |
| `obsidian_sync_conflicts` | List unresolved sync conflicts (path, conflict path, detected-at). |
### Write tools (`write:obsidian`)
| Tool | Description |
| -------------------------------- | ----------------------------------------------------------------------------------- |
| `obsidian_write_note` | Create or overwrite a note with given Markdown content. |
| `obsidian_append_note` | Append content to a note; optionally to a specific heading/block. |
| `obsidian_patch_note` | Surgically append/prepend/replace at a heading, block, or frontmatter field. |
| `obsidian_delete_note` | Permanently delete a note from the vault. |
| `obsidian_move_note` | Move or rename a note within the vault. |
| `obsidian_execute_command` | Execute an Obsidian command by its command ID. |
| `obsidian_open_file` | Open a file in Obsidian (creates it if it does not exist). |
| `obsidian_sync_trigger` | Trigger an immediate bidirectional desktop↔mobile vault sync. |
| `obsidian_sync_resolve_conflict` | Resolve a sync conflict: keep `local` (mobile), `remote` (desktop), or `keep-both`. |
> [!NOTE]
> `obsidian_patch_note` targets accept `targetType` of `heading | block | frontmatter`
> and `operation` of `append | prepend | replace`, with an optional
> `createTargetIfMissing`. The four `obsidian_sync_*` tools talk to the local sync
> server (`http://127.0.0.1:27781` by default) and require the sync token.
### Scopes
Read tools require `read:obsidian`; write tools require `write:obsidian`. Enforcement
is identical to Notion — handled by `withScopeEnforcement()` in
`open-sse/mcp-server/server.ts`, gated on `OMNIROUTE_MCP_ENFORCE_SCOPES=true`, with
allowed scopes sourced from `OMNIROUTE_MCP_SCOPES` or the API key's scope context. See
[MCP-SERVER.md](./MCP-SERVER.md).
## Endpoints
| Method | Path | Purpose |
| -------- | ------------------------------- | ----------------------------------------------------- |
| `GET` | `/api/settings/obsidian` | Return `{ connected, hasToken, baseUrl, vaultPath }`. |
| `POST` | `/api/settings/obsidian` | Save + validate token (rejects port `27124`). |
| `DELETE` | `/api/settings/obsidian` | Disconnect (clear stored token). |
| `GET` | `/api/settings/obsidian/webdav` | WebDAV sync status + credentials (while enabled). |
| `POST` | `/api/settings/obsidian/webdav` | Enable WebDAV sync for a vault directory. |
| `DELETE` | `/api/settings/obsidian/webdav` | Disable WebDAV sync. |
> These are dashboard settings routes. The vault itself is reached through the Obsidian
> Local REST API (the configured `base_url`) and through the MCP tools above — there is
> no public `/v1` Obsidian proxy endpoint.
## Use cases
- **Vault-grounded answers** — `obsidian_search_simple` / `obsidian_search_structured`
then `obsidian_read_note` so an agent answers from your real notes.
- **Note authoring / journaling** — `obsidian_write_note`, `obsidian_append_note`, or
the surgical `obsidian_patch_note` to log agent output, summaries, or daily notes
(`obsidian_get_periodic_note`) into the vault.
- **Vault navigation** — `obsidian_list_vault`, `obsidian_get_document_map`, and
`obsidian_get_tags` to explore structure before reading/writing.
- **Obsidian automation** — `obsidian_list_commands` + `obsidian_execute_command` to
drive plugins/commands from an agent; `obsidian_open_file` to surface a note in the UI.
- **Mobile sync** — enable WebDAV sync, then `obsidian_sync_trigger` /
`obsidian_sync_status` / `obsidian_sync_conflicts` / `obsidian_sync_resolve_conflict`
to coordinate desktop↔mobile and resolve conflicts.
## Related
- [MCP Server](./MCP-SERVER.md) — transports, scope enforcement, full tool inventory.
- [Notion Context Source](./NOTION_CONTEXT.md) — the other built-in context source.
- [Memory System](./MEMORY.md) — persistent conversational memory (complementary
context layer, injected automatically rather than tool-fetched).
docs/frameworks/OPENCODE.md
+165
View File
@@ -0,0 +1,165 @@
---
title: "OpenCode Integration"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OpenCode Integration
> **Status:** Generally available.
> **Audience:** Operators wiring OpenCode to an OmniRoute deployment.
> **Source of truth (config schema):** `src/shared/services/opencodeConfig.ts`
> **Source of truth (npm package):** `@omniroute/opencode-provider/` (publishable workspace)
[OpenCode](https://opencode.ai) is an agentic CLI/desktop AI client. It reads its provider catalog from `~/.config/opencode/opencode.json` (or `opencode.jsonc`) and follows the schema at `https://opencode.ai/config.json`. OmniRoute exposes itself to OpenCode as one of those providers — every request flows through OmniRoute's standard OpenAI-compatible `/v1` surface, so OpenCode automatically benefits from Auto-Combo routing, circuit breakers, key policies, observability, etc.
There are **two supported integration paths**. Pick one — they generate the same config.
---
## Path 1 — CLI generator (no npm install)
Recommended for end users. Ships with OmniRoute. Writes `opencode.json` in place.
```bash
# After installing OmniRoute (npm i -g @omniroute/cli or local clone)
omniroute config opencode \
--baseUrl http://localhost:20128 \
--apiKey "$OMNIROUTE_API_KEY"
```
Behind the scenes the CLI calls `mergeOpenCodeConfigText()` (`src/shared/services/opencodeConfig.ts:104`), so an existing `opencode.json` keeps its other providers and comments. The OmniRoute entry is added/replaced atomically.
Resulting file (default model catalog):
```jsonc
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"omniroute": {
"npm": "@ai-sdk/openai-compatible",
"name": "OmniRoute",
"options": {
"baseURL": "http://localhost:20128/v1",
"apiKey": "<your-key>",
},
"models": {
"claude-opus-4-5-thinking": { "name": "claude-opus-4-5-thinking" },
"claude-sonnet-4-5-thinking": { "name": "claude-sonnet-4-5-thinking" },
"gemini-3.1-pro-high": { "name": "gemini-3.1-pro-high" },
"gemini-3-flash": { "name": "gemini-3-flash" },
},
},
},
}
```
---
## Path 2 — npm package `@omniroute/opencode-provider`
Recommended when you're scripting the config from Node/TS (CI pipelines, monorepos, custom installer flows).
```bash
npm install --save-dev @omniroute/opencode-provider
```
```ts
import { writeFileSync } from "node:fs";
import { buildOmniRouteOpenCodeConfig } from "@omniroute/opencode-provider";
const config = buildOmniRouteOpenCodeConfig({
baseURL: "http://localhost:20128",
apiKey: process.env.OMNIROUTE_API_KEY ?? "sk_omniroute",
// Optional: override the model catalog exposed to OpenCode
models: ["auto", "claude-opus-4-7", "gpt-5.5"],
modelLabels: { auto: "Auto-Combo" },
});
writeFileSync("opencode.json", JSON.stringify(config, null, 2));
```
For a non-destructive merge against an existing file, replicate `mergeOpenCodeConfigText()` from `opencodeConfig.ts` or call the CLI generator.
See the [package README](../../@omniroute/opencode-provider/README.md) for the full API.
---
## What the runtime actually does
Both paths produce the same `provider.omniroute.npm: "@ai-sdk/openai-compatible"`. At runtime, OpenCode loads `@ai-sdk/openai-compatible` (already a transitive dependency of OpenCode) and configures it with `baseURL` + `apiKey`. From there:
```
OpenCode UI/agent
→ @ai-sdk/openai-compatible
→ HTTP POST {baseURL}/chat/completions (OmniRoute OpenAI surface)
→ OmniRoute /v1/chat/completions handler (open-sse/handlers/chatCore.ts)
→ combo routing / Auto-Combo / executor
→ upstream provider
```
The plugin never touches HTTP. It only emits configuration.
---
## Model catalog defaults
```ts
export const OMNIROUTE_DEFAULT_OPENCODE_MODELS = [
"claude-opus-4-5-thinking",
"claude-sonnet-4-5-thinking",
"gemini-3.1-pro-high",
"gemini-3-flash",
] as const;
```
You can override via `models: [...]`. Recommended additions:
- `"auto"` — surfaces OmniRoute's [Auto-Combo](../routing/AUTO-COMBO.md) zero-config router. Lets OpenCode pick "the best available model" without you hard-coding the catalog.
- `"<combo-name>"` — any combo you've defined in the dashboard; OmniRoute resolves it transparently.
---
## URL normalisation
The helper accepts both forms and emits exactly one `/v1`:
| Input | Output (`options.baseURL`) |
| ------------------------------ | --------------------------- |
| `http://localhost:20128` | `http://localhost:20128/v1` |
| `http://localhost:20128/` | `http://localhost:20128/v1` |
| `http://localhost:20128/v1` | `http://localhost:20128/v1` |
| `http://localhost:20128/v1///` | `http://localhost:20128/v1` |
This deduplication is **the most common breakage** seen in older configs. If you have an `opencode.json` from before v3.8.0 that points at `/v1/v1/...`, re-run the generator or call `createOmniRouteProvider` again.
---
## Authentication modes
| OmniRoute setting | Recommended `apiKey` value |
| ------------------------------------------- | -------------------------------------------------- |
| `REQUIRE_API_KEY=false` (default for local) | `sk_omniroute` (literal placeholder) |
| `REQUIRE_API_KEY=true` | A real per-user API key from Dashboard → API Keys. |
For Anthropic-style clients that send `x-api-key` + `anthropic-version`, OmniRoute's `extractApiKey` also honours the key from `x-api-key`. OpenCode uses the OpenAI surface, so it'll always send `Authorization: Bearer ${apiKey}` — no Anthropic special-case applies here.
---
## Troubleshooting
| Symptom | Cause | Fix |
| ---------------------------------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `404` on every request with URL containing `/v1/v1/` | Stale config from pre-v3.8 plugin that double-suffixed `/v1`. | Regenerate via Path 1 or 2. |
| `401 Invalid API key` | OmniRoute has `REQUIRE_API_KEY=true` and the key is unknown. | Create the key in the dashboard, or set `REQUIRE_API_KEY=false` (local only) and use `sk_omniroute`. |
| Model list empty in OpenCode UI | All 4 default models are hidden in OmniRoute's provider visibility. | Pass `models: ["auto", ...]` to surface ones you've enabled. |
| OpenCode 500 with `cannot read property 'models'` | Older OpenCode (< 0.1.x) didn't accept inline `models`. | Upgrade OpenCode to a version that follows the v1 schema (`opencode.ai/config.json`). |
---
## See also
- [API reference](../reference/API_REFERENCE.md) — full OmniRoute REST surface
- [Auto-Combo](../routing/AUTO-COMBO.md) — what `model: "auto"` means
- [`@omniroute/opencode-provider` README](../../@omniroute/opencode-provider/README.md)
- Source: `src/shared/services/opencodeConfig.ts`, `src/lib/cli-helper/config-generator/opencode.ts`, `@omniroute/opencode-provider/src/index.ts`
docs/frameworks/OPEN_SSE_ARCHITECTURE.md
+576
View File
@@ -0,0 +1,576 @@
---
title: "open-sse Architecture"
version: 3.8.40
lastUpdated: 2026-06-28
---
# open-sse Architecture
> **TL;DR**: `open-sse/` is the core streaming engine that powers every LLM request in OmniRoute. It contains ~900 files implementing the request pipeline, executors, services, MCP server, and translation layer. This guide explains how the pieces fit together.
**Source:** `open-sse/` (workspace package, ~900 files; 811 `.ts`)
---
## Why a Separate Workspace Package?
`open-sse/` is a **standalone workspace** in the OmniRoute monorepo for several reasons:
1. **Reusability** — `open-sse` is published as `@omniroute/open-sse` on npm, so other projects can use it independently
2. **Clean boundaries** — the streaming engine is decoupled from the OmniRoute-specific UI/DB layer
3. **Performance** — the engine has no Next.js dependencies, enabling faster cold starts in CLI/serverless contexts
4. **Versioning** — `open-sse` can release on its own cadence
```json
// package.json
"workspaces": ["open-sse"]
```
---
## Top-Level Structure
```
open-sse/
├── index.ts # Public entry point
├── types.d.ts # Public type exports
├── package.json # @omniroute/open-sse
├── config/ # Provider configs, constants, registries
├── executors/ # Per-provider HTTP executors (67 + base.ts/index.ts)
├── handlers/ # Request handlers (chatCore, responses, etc.)
├── lib/ # Internal utilities
├── mcp-server/ # Model Context Protocol server
├── services/ # ~298 service modules
├── transformer/ # Responses API format transformer
├── translator/ # Format translation (OpenAI ↔ Claude ↔ Gemini)
└── utils/ # Shared utilities (logging, error, stream, etc.)
```
### Module Counts
| Directory | Files | Purpose |
| `executors/` | 68 | Per-provider HTTP executors (unified via DefaultExecutor factory) |
| `handlers/` | 16 | Request entry points (chatCore, responses, embeddings) |
| `services/` | ~298 | Routing, caching, rate limiting, refresh, etc. |
| `translator/` | ~27 | Format conversion (OpenAI ↔ Claude ↔ Gemini) |
| `mcp-server/` | 32 | MCP tools and transports |
| `utils/` | ~65 | Cross-cutting utilities (logging, error, stream) |
| `config/` | ~10 | Provider configs, constants, registries |
---
## The Request Pipeline
Every LLM request flows through a **5-stage pipeline**:
```
┌──────────────┐
HTTP request │ 1. ROUTE │ combo resolution, model selection
(Next.js route) └──────┬───────┘
│
▼
┌──────────────┐
│ 2. TRANSLATE│ format conversion (OpenAI ↔ Claude ↔ Gemini)
└──────┬───────┘
│
▼
┌──────────────┐
│ 3. EXECUTE │ provider executor, HTTP, retry, breaker
└──────┬───────┘
│
▼
┌──────────────┐
│ 4. STREAM │ SSE transformation, backpressure
└──────┬───────┘
│
▼
┌──────────────┐
│ 5. RECORD │ usage tracking, call log, error classification
└──────┬───────┘
│
▼
HTTP response (SSE or JSON)
```
### Stage 1: Route (services/combo.ts)
**Entry point**: `handleComboChat()` in `services/combo.ts`
Resolves the request to a concrete `(provider, model, account, credentials)` tuple:
- Look up the combo by ID (or build a virtual combo for `auto/*` models)
- Apply routing strategy (priority, weighted, round-robin, etc.)
- Filter out unhealthy providers (circuit breaker)
- Pick the next viable target
For `auto/*` models, this stage also:
- Runs the **9-factor scoring** algorithm (`services/autoCombo/`)
- Selects a `provider+model` pair based on health, cost, latency, etc.
### Stage 2: Translate (translator/)
If the source format (e.g., OpenAI) differs from the target format (e.g., Claude), the request is **translated**:
- System prompt → system message
- Tool definitions → provider-specific tool format
- Reasoning/thinking parameters → provider-specific equivalents
- Message role normalization (`developer` → `system` for non-OpenAI)
The `translator/index.ts` exposes:
```ts
translateRequest(body, sourceFormat, targetFormat): TranslatedRequest
needsTranslation(source, target): boolean
```
### Stage 3: Execute (executors/)
**Entry point**: `getExecutor(providerId).execute(request, options)`
All providers use `DefaultExecutor` (`executors/default.ts`) via the `getExecutor()` factory fallback. The executor:
- Builds the upstream URL (`buildUrl()`)
- Adds provider-specific headers (`buildHeaders()`)
- Transforms the request body (`transformRequest()`)
- Sends the HTTP request with retry + exponential backoff
- Handles auth refresh if needed (OAuth providers)
All executors extend `BaseExecutor` (`executors/base.ts`, 1170 LOC) which provides:
- Common retry logic
- Proxy integration
- Circuit breaker integration
- Usage recording hooks
### Stage 4: Stream (utils/stream.ts)
For streaming responses, the executor returns a **ReadableStream**. The handler:
- Pipes through an SSE transform (`createSSETransformStreamWithLogger`)
- Applies heartbeat pings to detect dead connections
- Handles client disconnect gracefully (`pipeWithDisconnect`)
- Transforms SSE → JSON for non-streaming clients
For non-streaming responses, the executor returns a parsed JSON object that is passed through unchanged.
### Stage 5: Record (services/usage.ts)
After the response (success or failure), usage is recorded:
- `prompt_tokens`, `completion_tokens`, `cached_tokens` from the response
- `cost_usd` computed from pricing data
- `latency_ms`, `status`, `error_class` if failed
- Persisted to `usage_history` table
Call log artifacts (if enabled) are written to `${DATA_DIR}/call_logs/`.
---
## Key Files Deep-Dive
### chatCore.ts (5977 lines)
The **main request handler**. Despite its size, it has a clear structure:
```ts
// Pseudo-structure of chatCore.ts
export async function handleChat(request: NextRequest) {
// 1. Auth + CORS
await authenticateRequest(request);
applyCorsHeaders(response);
// 2. Body validation
const body = await parseRequestBody(request);
// 3. Format detection + translation
const sourceFormat = detectFormat(request);
const targetFormat = getTargetFormat(providerId);
if (needsTranslation(sourceFormat, targetFormat)) {
body = translateRequest(body, sourceFormat, targetFormat);
}
// 4. Combo routing
const targets = await resolveComboTargets(comboId, body);
for (const target of targets) {
try {
const result = await executeOnTarget(target, body);
await recordUsage(result);
return result;
} catch (err) {
// Continue to next target
}
}
// 5. Emergency fallback
return await emergencyFallback(body);
}
```
Despite being one giant function, it's organized into **commented sections** that map to the 5-stage pipeline.
### combo.ts (4456 LOC)
The **routing engine** that resolves a combo to ordered targets.
```ts
// services/combo.ts
export async function handleComboChat(body, comboId): Promise<ChatResult> {
const targets = await resolveComboTargets(comboId, body);
for (const target of targets) {
try {
return await handleSingleModel(target, body);
} catch (err) {
log.warn("target failed, trying next", { target, err });
}
}
throw new ComboExhaustedError("All targets failed");
}
```
Supports **17 routing strategies** (see `src/shared/constants/routingStrategies.ts`):
| Strategy | Behavior |
| ------------------- | ------------------------------------------------------------------------- |
| `priority` | First-target ordered list |
| `weighted` | Probabilistic by per-target weight |
| `round-robin` | Cycle through targets in order |
| `context-relay` | Hand off context across targets |
| `fill-first` | Fill quota before moving to next |
| `p2c` | Power of two choices |
| `random` | Uniform random |
| `least-used` | Pick the one with fewest recent uses |
| `cost-optimized` | Cheapest healthy target first |
| `reset-aware` | Aware of provider reset windows |
| `reset-window` | Reset window-based routing |
| `headroom` | Most remaining quota headroom first |
| `strict-random` | Truly uniform (no quality weighting) |
| `auto` | Use 9-factor scoring (`autoCombo/`) |
| `lkgp` | Last known good provider first |
| `context-optimized` | Best for long-context requests |
| `fusion` | Fan out to a panel in parallel, then synthesize via a judge (`fusion.ts`) |
### base.ts (1170 LOC)
The **abstract executor** that all 67 executors extend. It contains:
- `buildUrl()` — default URL construction (subclasses override for custom)
- `buildHeaders()` — default headers (auth, content-type)
- `transformRequest()` — pass-through by default
- `execute()` — the main HTTP loop with retry/backoff/breaker
```ts
// open-sse/executors/default.ts
export class DefaultExecutor extends BaseExecutor {
// Handles all OpenAI/Anthropic-compatible providers
// Providers register configurations (URL, auth, headers) but share executor logic
}
```
Provider-specific behavior (auth headers, base URL, version headers) is configured via the provider registry, not separate executor classes.
````
---
## Services (117 modules)
Services are **focused, single-purpose modules** that handlers compose. The big categories:
### Routing & Combo
- `combo.ts` — entry point for combo-routed requests
- `services/autoCombo/` — 9-factor scoring, 8 auto routing strategies
- `wildcardRouter.ts` — matches wildcard routes (`gpt-*`)
- `modelFamilyFallback.ts` — T5 intra-family fallback
### Rate Limiting & Quota
- `rateLimitManager.ts` — token bucket per key+provider
- `usage.ts` — usage recording
- `quotaCache.ts` — in-memory quota snapshots
### Account & Token
- `tokenRefresh.ts` — OAuth refresh on 401
- `accountFallback.ts` — switch to alternate account
- `sessionManager.ts` — multi-turn session state
### Intelligence
- `intentClassifier.ts` — classify request intent
- `taskAwareRouter.ts` — route by task type
- `thinkingBudget.ts` — allocate thinking tokens
- `contextManager.ts` — inject routing context
### Resilience
- `resilience.ts` — retry, backoff, breaker orchestration
- `emergencyFallback.ts` — last-resort fallback
- `modelDeprecation.ts` — auto-route to successor models
### State
- `signatureCache.ts` — dedup by request signature
- `volumeDetector.ts` — load shedding
- `contextHandoff.ts` — session serialization
### Compression
- `compression/` (subdirectory) — full compression pipeline
- 39 files covering engines, rule packs, adapters
### Skills
- (covered in [SKILLS.md](./SKILLS.md))
### Memory
- (covered in [MEMORY.md](./MEMORY.md))
---
## Executors (75+ files)
One file per provider. They all extend `BaseExecutor` and override what differs.
### Common Patterns
Providers are resolved via `getExecutor(providerId)`, which returns the configured executor. OpenAI/Anthropic-compatible providers use `DefaultExecutor` (`executors/default.ts`). Provider-specific behavior (base URL, auth headers, API version) is configured in `open-sse/config/providers/`, while request body transformations are handled in `open-sse/translator/`.
**Custom URL** is set via provider configuration:
```ts
// Provider config in open-sse/config/providers/
export default {
id: "together",
baseURL: "https://api.together.xyz/v1/chat/completions",
}
````
**Custom auth** is handled through the provider registry's auth configuration (API key, OAuth, header profiles).
**Custom request body** transformations (e.g., Anthropic separating `system` from `messages`) are registered per-provider in `open-sse/translator/`.
````
### The Executor Factory
`executors/index.ts` exports `getExecutor(providerId)`:
```ts
import { getExecutor } from "@omniroute/open-sse/executors";
const executor = getExecutor("anthropic");
const result = await executor.execute({
model: "claude-sonnet-4-5",
messages: [...],
});
````
The factory is generated from `config/providerRegistry.ts` which lists all 212+ providers and their executor class.
---
## Translators
Translate between **3 formats**: OpenAI, Anthropic, Gemini, plus the new Responses API.
### When Translation Happens
```ts
import { needsTranslation, translateRequest } from "@omniroute/open-sse/translator";
if (needsTranslation(sourceFormat, targetFormat)) {
body = translateRequest(body, sourceFormat, targetFormat);
}
```
Common translations:
- `OpenAI → Anthropic`: separate `system` field, `x-api-key` header
- `OpenAI → Gemini`: `contents` instead of `messages`, `systemInstruction`
- `OpenAI → Responses API`: `input` array, `previous_response_id` state
### Edge Cases Handled
- `developer` role → `system` for non-OpenAI
- `system` role → merged into first user message for GLM/ERNIE
- `json_schema` → Gemini's `responseMimeType` + `responseSchema`
- `tools` → provider-specific tool format
- Thinking parameters (o1, Claude) → provider-specific equivalents
---
## MCP Server
`open-sse/mcp-server/` implements the **Model Context Protocol** server:
- **30+ tools** (provider management, combos, memory, cache, compression, 1proxy, skills)
- **3 transports**: stdio, SSE, Streamable HTTP
- **13 scopes** for fine-grained authorization
### Tool Registration
Tools are registered as standalone files in `open-sse/mcp-server/tools/`, each exporting a name, schema, handler, and scope:
```ts
// open-sse/mcp-server/tools/getHealth.ts
import { z } from "zod";
export default {
name: "omniroute_get_health",
description: "Get system health snapshot",
scope: "read:health",
inputSchema: z.object({}),
handler: async (_args, ctx) => {
return await getSystemHealth();
},
};
```
### Transports
```ts
// stdio (CLI usage)
startMcpStdio(server);
// SSE (HTTP-based streaming)
startMcpSse(server, port);
// Streamable HTTP (modern MCP)
startMcpStreamable(server, port);
```
### Authorization
Every tool call goes through scope checks (`open-sse/mcp-server/auth/`):
```ts
if (!hasScope(apiKey, "providers:read")) {
throw new Error("Insufficient scope");
}
```
---
## Transformers
`open-sse/transformer/` converts between **Chat Completions** and **Responses API** formats.
### Why a Separate Transformer?
The Responses API is OpenAI's new format with **stateful conversations** (`previous_response_id`). When a client sends a Responses request, OmniRoute:
1. Converts Responses → Chat Completions internally
2. Sends to provider (any provider that supports Chat Completions)
3. Converts the response back to Responses format
4. Streams the converted response to the client
The transformer (`transformer/responsesTransformer.ts`) provides:
```ts
createResponsesApiTransformStream(): TransformStream
```
This handles:
- `response.output_item.added` events
- `response.output_text.delta` events
- `response.completed` event
- Tool call mapping (`function_call` ↔ `tool_calls`)
---
## Configuration
`open-sse/config/` holds the configuration layer:
| File | Purpose |
| ----------------------------- | --------------------------------- |
| `providerRegistry.ts` | 212+ provider definitions |
| `providerModels.ts` | Model aliases, format mapping |
| `constants.ts` | Timeouts, limits, status codes |
| `defaultThinkingSignature.ts` | Default Claude thinking signature |
| `modelStrip.ts` (in services) | Per-provider field stripping |
### Provider Registry Schema
```ts
interface ProviderConfig {
id: string;
name: string;
baseUrl: string;
authType: "bearer" | "api-key" | "oauth" | "cookie";
executorClass: string;
defaultModel: string;
capabilities: ProviderCapabilities;
models: ModelDefinition[];
}
```
Zod validation at module load ensures all provider configs are valid.
---
## Performance Constraints
The routing engine has strict performance budgets:
| Operation | Target | Measurement |
| --------------------------------------- | ------ | ------------------------- |
| Combo resolution | <10ms | For 50 targets |
| Rate limit check | <1ms | In-memory token bucket |
| Model family fallback | <5ms | Cached family definitions |
| Request routing dispatch | <2ms | Hot path |
| **No blocking I/O in routing hot path** | — | All async |
---
## Anti-Patterns
❌ **Synchronous DB calls in `combo.ts`** — pre-compute and cache
❌ **Retry logic in handlers** — use `retry()` from resilience service
❌ **Direct provider config access** — use `providerRegistry` getters
❌ **Hardcoded fallback chains** — define in `modelFamilyFallback.ts`
❌ **State mutations across concurrent requests** — use request-scoped context only
---
## Adding a New Component
### Adding a New Service
1. Create `open-sse/services/[serviceName].ts` with focused responsibility
2. Export main handler function and any constants
3. Add unit tests in `tests/unit/services/[serviceName].test.mjs`
4. Integrate into request pipeline in `handlers/chatCore.ts` (if routing-related)
5. Update routing logic in `combo.ts` if service affects target selection
6. Document in this file
### Adding a New Executor
1. Create `open-sse/executors/[provider].ts` extending `BaseExecutor`
2. Register in `config/providerRegistry.ts`
3. Add to `executors/index.ts` factory
4. Add unit tests for the executor
5. Document in `docs/architecture/ARCHITECTURE.md`
### Adding a New MCP Tool
1. Create or update `open-sse/mcp-server/tools/[category]Tools.ts`
2. Define Zod schema for inputs
3. Register tool in `mcp-server/index.ts`
4. Add to scope matrix in `mcp-server/auth/`
5. Add unit tests
---
## See Also
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — high-level architecture
- [CODEBASE_DOCUMENTATION.md](../architecture/CODEBASE_DOCUMENTATION.md) — engineering reference
- [REPOSITORY_MAP.md](../architecture/REPOSITORY_MAP.md) — directory-by-directory
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — 9-factor scoring
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP server
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A server
- Source: `open-sse/` (400+ files, ~143K LOC)
docs/frameworks/PLAYGROUND_STUDIO.md
+212
View File
@@ -0,0 +1,212 @@
---
title: "Playground Studio"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Playground Studio
> **Feature:** Playground Studio — unified AI testing workspace for `/dashboard/playground`.
> **Plans:** `17-playground-studio-redesign.plan.md` + `_orchestration/master-plan-group-C.md`
> **Status:** Released in v3.8.6
---
## Overview
Playground Studio transforms `/dashboard/playground` from a simple Monaco-based editor into
a full-featured testing workspace. It replaces the legacy `page.tsx` with a `PlaygroundStudio`
shell that renders four tabs and a shared config pane.
```
┌ Playground ──────────────────────────────────────────────────────────┐
│ [💬 Chat] [⚖ Compare] [{} API] [🔧 Build] 142↑ 38↓ · $0.002 </>│
├──────────────────────────────────────────┬───────────────────────────┤
│ {active tab content} │ ─ Config │
│ │ Endpoint [chat ∨] │
│ │ Model [gpt-5.4 ∨] │
│ │ System [textarea] │
│ │ Temp ▕▕▔▔ 0.7 │
│ │ Presets [▾ load][save] │
│ │ [✨ Improve prompt] │
└──────────────────────────────────────────┴───────────────────────────┘
```
---
## Tabs
### Chat Tab
Evolves `ChatPlayground.tsx` into a multi-turn streaming workbench:
- Full markdown rendering via `MarkdownMessage.tsx` (code blocks, tables, lists, links).
- System prompt sourced from the shared Config pane.
- Token/cost per message (prompt + completion tokens).
- Regenerate last response.
- Sends to `POST /v1/chat/completions` with SSE streaming.
### Compare Tab
The key differentiator for a proxy: run 1 prompt across up to **4 models in parallel**.
- Up to 4 columns, each independently streaming from `/v1/chat/completions`.
- `+ Add model` button (Cmd+K shortcut) to add columns.
- `Run all ▶` triggers all streams simultaneously via `Promise.all` + per-column `AbortController`.
- Global **Cancel all** aborts every in-flight stream.
- Per-column `ProviderMetrics` shows TTFT, TPS, tokens, and estimated cost in real time.
- Metrics labelled **"client-side estimate"** (D12) — measured from first SSE chunk.
### API Tab
Preserves 100% of the original Monaco editor for power users (D14):
- 10 endpoints: chat completions, completions, embeddings, images, audio, speech, transcriptions, moderations, rerank, search.
- Multimodal file upload.
- SSE streaming with real-time output.
- Wrapped as `ApiTab.tsx` (lazy-loaded, `ssr: false`).
### Build Tab
Tools/function calling and structured output UI:
- `ToolsBuilder.tsx` — add/edit/remove `tools[]` with JSON schema editor per tool.
Validates parameters via `ToolDefinitionSchema` (Zod).
- `StructuredOutputEditor.tsx` — toggle JSON mode + JSON schema editor.
Validates response against schema via `StructuredOutputSchema` (Zod).
- Sends request to `/v1/chat/completions` with `tools[]` and/or `response_format`.
---
## Config Pane (Shared)
`StudioConfigPane.tsx` — always visible, collapsible.
| Field | Component | Notes |
| -------------- | --------------------- | ---------------------------------------------------------------------- |
| Endpoint | `<select>` | 10 options matching `PlaygroundEndpoint` |
| Model | `<input>` | free text, e.g. `openai/gpt-4o` |
| System prompt | `<textarea>` | fed into all tabs |
| Parameters | `ParamSliders` | temperature, max_tokens, top_p, presence/frequency penalty, seed, stop |
| Presets | `PresetPicker` | load/save named config snapshots (persisted in DB) |
| Improve prompt | `ImprovePromptButton` | opens quota-warning modal, calls `/api/playground/improve-prompt` |
State is lifted to `PlaygroundStudio.tsx` and passed down to all tabs. Switching tabs
preserves config state.
---
## Top Bar
`StudioTopBar.tsx`:
- Tab switcher (role="tablist").
- `TokenCostCounter` — live token (↑/↓) and estimated cost display.
- Export code button (`</>`) — opens `ExportCodeModal`.
---
## Export Code Modal
`ExportCodeModal.tsx` uses `codeExport.ts` to generate curl / Python / TypeScript snippets
from the current `PlaygroundState`. API key placeholder is always `$OMNIROUTE_API_KEY` (D11).
---
## Prompt Improver
`ImprovePromptButton.tsx` → `useImprovePrompt.ts` → `POST /api/playground/improve-prompt`:
1. Modal warns "will consume quota".
2. On confirm, sends `{ system, prompt, model, tone }` to the route.
3. Route calls `/v1/chat/completions` internally with `promptImprover.META_SYSTEM_PROMPT`.
4. Returns `{ improvedSystem?, improvedPrompt?, tokensIn, tokensOut }`.
5. UI patches the Config pane system prompt and the Chat tab user prompt.
---
## Presets
`PresetPicker.tsx` → `usePresets.ts` → `/api/playground/presets/*`:
- Stored in `playground_presets` SQLite table (migration `084_playground_presets.sql`).
- Each preset stores: `name`, `endpoint`, `model`, `system`, `params_json`, `created_at`.
- CRUD: `GET` list, `POST` create, `GET /:id`, `PUT /:id`, `DELETE /:id`.
---
## Stream Metrics
`useStreamMetrics.ts` + `streamMetrics.ts` (pure function):
- `start()` — records request start time.
- `onFirstChunk()` — records TTFT.
- `onChunk(n)` — accumulates completion token count.
- `finish(usage?)` — computes final metrics: `ttftMs`, `totalMs`, `tps`, `tokensIn`, `tokensOut`, `costUsd`.
- Pricing from static table in `src/lib/playground/types.ts` (labelled "estimated" — D13).
---
## Backend Routes
| Method | Path | Handler |
| -------- | -------------------------------- | ----------------------------------------------------------------------------------------- |
| `POST` | `/api/playground/improve-prompt` | Zod-validates `ImprovePromptRequestSchema`; calls `/v1/chat/completions` with meta-prompt |
| `GET` | `/api/playground/presets` | Returns `{ presets: PlaygroundPresetListItem[] }` |
| `POST` | `/api/playground/presets` | Creates preset; validates `PlaygroundPresetCreateSchema` |
| `GET` | `/api/playground/presets/:id` | Returns one preset or 404 |
| `PUT` | `/api/playground/presets/:id` | Partial update |
| `DELETE` | `/api/playground/presets/:id` | 204 |
Auth: optional (`REQUIRE_API_KEY`). Errors via `buildErrorBody()` (Hard Rule #12).
---
## Key Files
| Path | Purpose |
| -------------------------------------------------------------------------- | --------------------------------------------------- |
| `src/app/(dashboard)/dashboard/playground/PlaygroundStudio.tsx` | Shell component, tab orchestrator |
| `src/app/(dashboard)/dashboard/playground/components/StudioTopBar.tsx` | Tabs + counter + export button |
| `src/app/(dashboard)/dashboard/playground/components/StudioConfigPane.tsx` | Shared config panel |
| `src/app/(dashboard)/dashboard/playground/components/tabs/ChatTab.tsx` | Chat workbench |
| `src/app/(dashboard)/dashboard/playground/components/tabs/CompareTab.tsx` | Multi-model compare |
| `src/app/(dashboard)/dashboard/playground/components/tabs/ApiTab.tsx` | Monaco editor (preserved) |
| `src/app/(dashboard)/dashboard/playground/components/tabs/BuildTab.tsx` | Tools + structured output |
| `src/app/(dashboard)/dashboard/playground/components/ExportCodeModal.tsx` | Code export modal |
| `src/app/(dashboard)/dashboard/playground/components/CompareColumn.tsx` | Single compare column |
| `src/app/(dashboard)/dashboard/playground/components/ProviderMetrics.tsx` | TTFT/TPS display |
| `src/app/(dashboard)/dashboard/playground/hooks/useStreamMetrics.ts` | Client-side metric hook |
| `src/app/(dashboard)/dashboard/playground/hooks/usePresets.ts` | Presets CRUD hook |
| `src/app/(dashboard)/dashboard/playground/hooks/useImprovePrompt.ts` | Improve-prompt hook |
| `src/lib/playground/codeExport.ts` | curl/Python/TS generator (shared with Search Tools) |
| `src/lib/playground/promptImprover.ts` | Meta-prompt builder |
| `src/lib/playground/streamMetrics.ts` | Pure metrics computation |
| `src/lib/db/playgroundPresets.ts` | DB module (CRUD) |
| `src/app/api/playground/improve-prompt/route.ts` | Improve-prompt REST route |
| `src/app/api/playground/presets/route.ts` | Presets list + create |
| `src/app/api/playground/presets/[id]/route.ts` | Presets get/update/delete |
| `src/lib/db/migrations/084_playground_presets.sql` | DB migration |
---
## Troubleshooting
| Symptom | Cause | Fix |
| -------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------- |
| Monaco editor not rendering in API tab | SSR loaded Monaco | Verify `ApiTab` uses `dynamic(..., { ssr: false })` |
| Compare streams fire sequentially | Wrong `Promise.all` usage | All stream starts must be dispatched in one `Promise.all` call |
| Metrics show `null` TTFT | First chunk handler not wired | Check `useStreamMetrics.onFirstChunk()` is called in the SSE reader loop |
| Preset not persisting | DB migration not run | Run `npm run db:migrate` or restart the server (migration auto-runs on startup) |
| Improve prompt returns 502 | Model not set in Config | User must enter a model name in the Config pane before improving |
| Export code shows `MISSING_API_KEY` | Placeholder not inserted | `codeExport.ts` always uses `API_KEY_PLACEHOLDER = "$OMNIROUTE_API_KEY"` |
---
## References
- Master plan: `_tasks/features-v3.8.6/refactorpages/_orchestration/master-plan-group-C.md`
- Feature plan: `_tasks/features-v3.8.6/refactorpages/17-playground-studio-redesign.plan.md`
- Code export: `src/lib/playground/codeExport.ts`
- Prompt improver: `src/lib/playground/promptImprover.ts`
- Search Tools Studio: `docs/frameworks/SEARCH_TOOLS_STUDIO.md`
docs/frameworks/PLUGINS.md
+114
View File
@@ -0,0 +1,114 @@
---
title: "OmniRoute CLI Plugin System"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute CLI Plugin System
Extend the `omniroute` CLI without modifying its core. Plugins follow the `omniroute-cmd-*` naming convention, similar to `gh extension` or `kubectl plugin`.
## Quick start
```bash
# Install a plugin from npm
omniroute plugin install stripe
# Install a local plugin in development
omniroute plugin install ./my-plugin
# List installed plugins
omniroute plugin list
# Scaffold a new plugin
omniroute plugin scaffold myplugin
cd omniroute-cmd-myplugin
omniroute plugin install .
```
## Plugin anatomy
A plugin is an npm package named `omniroute-cmd-<name>` (or `@scope/omniroute-cmd-<name>`).
```
omniroute-cmd-myplugin/
├── package.json # must have "type": "module" and "main": "index.mjs"
├── index.mjs # exports register(program, ctx) + optional meta
└── README.md
```
### `package.json`
```json
{
"name": "omniroute-cmd-myplugin",
"version": "0.1.0",
"type": "module",
"main": "index.mjs",
"engines": { "omniroute": ">=4.0.0" },
"keywords": ["omniroute-plugin", "omniroute-cmd"]
}
```
### `index.mjs`
```js
export const meta = {
name: "myplugin",
version: "0.1.0",
description: "My plugin for OmniRoute",
omnirouteApi: ">=4.0.0",
};
export function register(program, ctx) {
program
.command("myplugin")
.description(meta.description)
.option("-n, --name <name>")
.action(async (opts, cmd) => {
const gOpts = cmd.optsWithGlobals();
const res = await ctx.apiFetch("/api/combos", {
baseUrl: gOpts.baseUrl,
apiKey: gOpts.apiKey,
});
const data = await res.json();
ctx.emit(data, gOpts);
});
}
```
## Plugin context API
The `ctx` object passed to `register(program, ctx)`:
| Property | Type | Description |
| ---------------------------- | ---------------- | -------------------------------------------------- |
| `ctx.apiFetch(path, opts)` | `async function` | Authenticated fetch to the OmniRoute server |
| `ctx.emit(data, opts)` | `function` | Output in table/json/jsonl/csv per `--output` flag |
| `ctx.t(key)` | `async function` | i18n translation lookup |
| `ctx.withSpinner(label, fn)` | `async function` | Wraps async fn with ora spinner |
| `ctx.baseUrl` | `string` | Resolved base URL |
| `ctx.apiKey` | `string \| null` | API key if provided |
## Discovery
Plugins are discovered from:
1. `~/.omniroute/plugins/<name>/` — user-local installs
2. `OMNIROUTE_PLUGIN_PATH` env var — custom directory
Loading errors are caught and printed as warnings — a broken plugin never crashes the CLI.
## Security
Plugins run with the same Node.js process privileges as `omniroute`. Only install plugins from sources you trust. `omniroute plugin install` shows an explicit warning and requires `--yes` or interactive confirmation.
## Publishing
1. Ensure `package.json` has `"keywords": ["omniroute-plugin"]`
2. `npm publish` as normal
3. Users discover via `omniroute plugin search <query>` (searches npm registry)
## Example plugin
See [`examples/omniroute-cmd-hello/`](../../examples/omniroute-cmd-hello/index.mjs) for a minimal working example with `meta` + `register()`.
docs/frameworks/PLUGIN_MARKETPLACE.md
+348
View File
@@ -0,0 +1,348 @@
---
title: "Plugin Marketplace"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Plugin Marketplace
> **Source of truth:** `src/lib/plugins/` (`marketplace.ts`, `manager.ts`, `manifest.ts`,
> `scanner.ts`, `loader.ts`), `src/app/api/plugins/`, and
> `src/app/(dashboard)/dashboard/plugins/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute ships a WordPress-style plugin system. Plugins are self-contained
directories — each with a `plugin.json` manifest and an entry file — that hook
into the request pipeline (`onRequest` / `onResponse` / `onError`) and into
lifecycle events (`onInstall` / `onActivate` / `onDeactivate` / `onUninstall`).
The **Plugin Marketplace** is the discovery layer on top of that system. It
exposes a browsable catalog of installable plugins. By default the catalog is a
small built-in seed registry; an operator can point it at a custom remote
registry URL, in which case the fetch is hardened by a DNS-resolving SSRF guard
(see [Security](#security)).
Every plugin route is **loopback-only** (Tier 1 — `LOCAL_ONLY`): plugins load
and execute code in child processes, so the routes are unreachable from a
non-loopback origin regardless of auth. See
[`docs/security/ROUTE_GUARD_TIERS.md`](../security/ROUTE_GUARD_TIERS.md).
## How It Fits Together
```
Dashboard (/dashboard/plugins)
├─ "Installed" tab → GET /api/plugins (listPlugins)
│ POST /api/plugins/scan (pluginManager.scan)
│ POST /api/plugins/{name}/activate|deactivate
│ DELETE /api/plugins/{name} (uninstall)
└─ "Marketplace" tab → GET /api/plugins/marketplace
→ listMarketplacePlugins()
├─ no custom URL → built-in SEED_REGISTRY
└─ custom URL → isSafeMarketplaceUrl() SSRF guard
→ safeOutboundFetch(guard:"public-only")
```
- **Registry layer** — `src/lib/plugins/marketplace.ts`: lists / searches the
catalog, falling back to the seed registry on any failure.
- **Lifecycle layer** — `src/lib/plugins/manager.ts` (`pluginManager` singleton):
install, upgrade, activate, deactivate, uninstall, scan, startup load.
- **Manifest layer** — `src/lib/plugins/manifest.ts`: Zod schema + defaults for
`plugin.json`.
- **Scanner** — `src/lib/plugins/scanner.ts`: discovers plugins on disk under
the plugin directory.
- **Loader** — `src/lib/plugins/loader.ts`: spawns each plugin in an isolated
child process and brokers hook calls over IPC.
## Marketplace Catalog
`listMarketplacePlugins()` (`src/lib/plugins/marketplace.ts`) returns a list of
`MarketplaceEntry` objects:
| Field | Type | Notes |
| ------------- | -------- | ------------------------------------ |
| `name` | string | kebab-case plugin name |
| `version` | string | semver |
| `description` | string | Short summary |
| `author` | string | Author / org |
| `license` | string | SPDX-style license id |
| `downloadUrl` | string | Source download URL (may be empty) |
| `repository` | string? | Optional repository URL |
| `tags` | string[] | Search/filter tags |
| `downloads` | number | Download count |
| `rating` | number | 0–5 |
| `verified` | boolean | Whether the entry is marked verified |
| `lastUpdated` | string | ISO-ish date string |
When no custom registry URL is configured, the catalog is the built-in
`SEED_REGISTRY` (currently `request-logger`, `rate-limiter`, `cost-tracker`, and
`theme-manager`). The seed registry is always available — if a configured remote
registry is unreachable, returns a non-`200` status, or returns an unrecognized
body, `listMarketplacePlugins()` logs a warning and falls back to the seed list.
> Note: the marketplace **catalog** (browse/search) is wired end to end, but
> one-click marketplace **install** from the catalog is not yet implemented — the
> dashboard's "Install" button on a marketplace entry currently shows a
> "coming soon" notice. Installation today goes through the local-path install
> flow (`POST /api/plugins`) and on-disk discovery (`POST /api/plugins/scan`).
## REST API
All endpoints require management auth (`requireManagementAuth`) **and** are
loopback-only — `/api/plugins` and `/api/plugins/` are listed in
`LOCAL_ONLY_API_PREFIXES` (`src/server/authz/routeGuard.ts`).
| Endpoint | Method | Description |
| -------------------------------- | ------ | --------------------------------------------------- |
| `/api/plugins` | GET | List installed plugins (optional `?status=` filter) |
| `/api/plugins` | POST | Install a plugin from an absolute local path |
| `/api/plugins/scan` | POST | Scan the plugin directory and register new plugins |
| `/api/plugins/marketplace` | GET | List marketplace catalog entries |
| `/api/plugins/[name]` | GET | Get installed plugin details |
| `/api/plugins/[name]` | DELETE | Uninstall a plugin |
| `/api/plugins/[name]/activate` | POST | Activate (load + register hooks) |
| `/api/plugins/[name]/deactivate` | POST | Deactivate (fire `onDeactivate`, unregister hooks) |
| `/api/plugins/[name]/config` | GET | Get plugin config + config schema |
| `/api/plugins/[name]/config` | PUT | Update plugin config (validated against schema) |
The `GET /api/plugins` `status` filter accepts one of
`installed` / `active` / `inactive` / `error`. An invalid value returns `400`.
### List installed plugins
```bash
curl http://localhost:20128/api/plugins \
-H "Cookie: auth_token=..."
```
### Install from a local path
```bash
curl -X POST http://localhost:20128/api/plugins \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{ "path": "/absolute/path/to/my-plugin" }'
```
The `path` must be **absolute** and may not contain `..` traversal segments or
null bytes (enforced by Zod). The source directory must contain a valid
`plugin.json` (or be a parent of one). On success the response is `201` with the
installed plugin row.
### Browse the marketplace
```bash
curl http://localhost:20128/api/plugins/marketplace \
-H "Cookie: auth_token=..."
```
### Update plugin config
```bash
curl -X PUT http://localhost:20128/api/plugins/my-plugin/config \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{ "config": { "level": "debug", "maxItems": 100 } }'
```
`PUT .../config` validates each provided value against the plugin's
`configSchema` (declared in the manifest): `number` fields honor `min`/`max`,
`select` fields must match the declared `enum`. Keys not present in the schema
are allowed through.
## Configuration
### Plugin directory
Plugins live under the OmniRoute data directory:
```
~/.omniroute/plugins/<plugin-name>/
├─ plugin.json
└─ index.js # (or whatever manifest.main points to)
```
`getDefaultPluginDir()` (`src/lib/plugins/scanner.ts`) resolves this to
`<home>/.omniroute/plugins`, where `<home>` is taken from the `HOME` /
`USERPROFILE` environment variables. `POST /api/plugins/scan` discovers any
subdirectory there that holds a valid `plugin.json` and registers it.
### Custom marketplace registry URL
The marketplace catalog source is read from the `pluginMarketplaceUrl` setting
(`src/lib/plugins/marketplace.ts` reads `settings.pluginMarketplaceUrl`). When
set to an `http(s)` URL, `listMarketplacePlugins()` fetches that URL and accepts
either a top-level JSON array of entries or an object with a `plugins` array;
entries without a string `name` are filtered out. When unset (or when the fetch
fails the SSRF guard / returns a bad response), the built-in seed registry is
used.
The dashboard "Marketplace" tab exposes a field for this URL (read back from
`GET /api/settings`).
> Implementation note: the dashboard "Save" action sends
> `pluginMarketplaceUrl` to `PATCH /api/settings`. At the time of writing this
> key is not declared in `updateSettingsSchema`
> (`src/shared/validation/settingsSchemas.ts`), so verify persistence in your
> release before relying on it — the **read** path (`getSettings()` →
> `listMarketplacePlugins()`) honors the key once it is present in the settings
> store.
## Security
### Route tier — loopback only
Plugins execute code in spawned child processes, so the entire `/api/plugins`
surface is classified `LOCAL_ONLY` (Tier 1). Loopback enforcement runs
unconditionally **before** any auth check, so a leaked management token reaching
the box over a tunnel still cannot install, activate, or uninstall a plugin.
See [`docs/security/ROUTE_GUARD_TIERS.md`](../security/ROUTE_GUARD_TIERS.md) and
Hard Rules #15 / #17.
### Marketplace registry SSRF guard
A custom registry URL is attacker-influenceable configuration, so before
fetching it `listMarketplacePlugins()` runs it through two layers:
1. **`isSafeMarketplaceUrl(url)`** (`src/lib/plugins/marketplace.ts`):
- Rejects anything that is not `http:` / `https:`.
- Rejects literal private/loopback/link-local/ULA hosts (IPv4 **and** IPv6,
including IPv4-mapped) via the canonical `isPrivateHost`
(`src/shared/network/outboundUrlGuard.ts`).
- Resolves **both** `A` and `AAAA` records and rejects if **any** resolved
address is private — closing the public-hostname → private-IP bypass.
- **Fails closed**: a DNS resolution failure rejects the URL.
2. **`safeOutboundFetch(url, { guard: "public-only", timeoutMs: 5000 })`**
(`src/shared/network/safeOutboundFetch.ts`): re-applies the public-only URL
guard at fetch time and **blocks redirects** (no public → private `30x`
pivot).
A URL that fails either layer does not abort the request — the marketplace
silently falls back to the built-in seed registry and logs a warning.
> This guard was hardened in PR #3774 specifically to resolve A + AAAA and use
> the canonical `isPrivateHost` instead of an IPv4-only check.
### Plugin execution isolation
- **Process isolation** — `loadPlugin()` (`src/lib/plugins/loader.ts`) spawns
each plugin in a separate Node.js child process and communicates over IPC.
Hook calls have a timeout with `SIGTERM` → `SIGKILL` escalation.
- **Env allowlist** — the child receives only an allowlisted set of environment
variables; the broader set is only granted when the manifest requests the
`env` permission.
- **Path containment** — install/upgrade/uninstall assert that the plugin
directory and `manifest.main` resolve **within** the managed plugin root
before any copy or recursive delete (guards against tampered DB paths and
`../` traversal in `manifest.main`). Activation resolves symlinks via
`realpath` and refuses to load an entry point that escapes the plugin
directory.
- **Optional integrity pin** — a manifest may declare an `integrity`
(`sha256-<base64>`, SRI format) field. When present, the loader verifies the
entry file hash at load time and refuses to activate on mismatch. It is
opt-in tamper-detection, **not** a security boundary — loopback-only routing
and the permission model are the real boundaries.
## Manifest (`plugin.json`)
Validated by `PluginManifestSchema` (`src/lib/plugins/manifest.ts`):
| Field | Type | Notes |
| ------------------ | --------- | ----------------------------------------------------------- |
| `name` | string | Required; kebab-case (`^[a-z0-9-]+$`), 1–100 chars |
| `version` | string | Required; semver (`MAJOR.MINOR.PATCH`) |
| `description` | string? | ≤ 500 chars |
| `author` | string? | ≤ 200 chars |
| `license` | string? | Defaults to `MIT` |
| `main` | string? | Entry file; defaults to `index.js` |
| `source` | enum? | `local` \| `marketplace` (defaults to `local`) |
| `tags` | string[]? | Search tags |
| `requires` | object? | `{ omniroute?, permissions[] }` |
| `hooks` | object? | Booleans declaring which hooks the plugin implements |
| `skills` | object[]? | Optional skill definitions |
| `enabledByDefault` | boolean? | Auto-activate on install |
| `configSchema` | object? | Map of config fields (`string`/`number`/`boolean`/`select`) |
| `integrity` | string? | Optional `sha256-<base64>` entry-file pin |
Permissions are drawn from the enum
`network` / `file-read` / `file-write` / `env` / `exec`.
## Lifecycle Flow
```
install (POST /api/plugins, path)
→ scan/validate manifest → copy to staging → assert main within dir
→ atomic rename into ~/.omniroute/plugins/<name> → insert DB row
→ fire onInstall → if enabledByDefault: activate
activate (POST /api/plugins/{name}/activate)
→ realpath containment check → loadPlugin() (spawn child process)
→ register declared hooks → status = "active" → fire onActivate
deactivate (POST /api/plugins/{name}/deactivate)
→ fire onDeactivate (BEFORE unregister) → unregister hooks
→ kill child process → status = "inactive"
uninstall (DELETE /api/plugins/{name})
→ deactivate if active → fire onUninstall
→ containment-checked recursive delete of plugin dir → delete DB row
```
Re-running `install` against a directory whose manifest version is **strictly
newer** than the installed version auto-upgrades (clean reinstall; config resets
to defaults). A same-or-older version is rejected.
## Database
Table `plugins` (migration `076_create_plugins.sql`):
| Column | Type | Notes |
| --------------- | ------- | ------------------------------------------------ |
| `id` | TEXT PK | UUID |
| `name` | TEXT | Unique |
| `version` | TEXT | semver; default `1.0.0` |
| `description` | TEXT | Optional |
| `author` | TEXT | Optional |
| `license` | TEXT | Default `MIT` |
| `main` | TEXT | Entry file; default `index.js` |
| `source` | TEXT | Default `local` |
| `tags` | TEXT | JSON array; default `[]` |
| `status` | TEXT | `installed` \| `active` \| `inactive` \| `error` |
| `enabled` | INT | 0/1; default 0 |
| `manifest` | TEXT | Full manifest JSON |
| `config` | TEXT | JSON; default `{}` |
| `config_schema` | TEXT | JSON; default `{}` |
| `hooks` | TEXT | JSON array of declared hook names; default `[]` |
| `permissions` | TEXT | JSON array; default `[]` |
| `plugin_dir` | TEXT | Absolute install directory |
| `error_message` | TEXT | Set when `status = "error"` |
| `installed_at` | TEXT | `datetime('now')` |
| `updated_at` | TEXT | `datetime('now')` |
| `activated_at` | TEXT | Set on activation |
Plugin metrics/analytics are tracked in additional tables
(`090_plugin_metrics.sql`, `091_plugin_analytics.sql`).
## Dashboard
The dashboard page at `/dashboard/plugins`
(`src/app/(dashboard)/dashboard/plugins/page.tsx`) provides two tabs:
- **Installed** — lists installed plugins with their declared hooks, an
activate/deactivate toggle, an uninstall button, and a "Scan for plugins"
action (`POST /api/plugins/scan`).
- **Marketplace** — shows the catalog from `GET /api/plugins/marketplace` with a
field to set the custom registry URL.
A per-plugin config page lives at `/dashboard/plugins/[name]/config`
(`src/app/(dashboard)/dashboard/plugins/[name]/config/page.tsx`).
## See Also
- [`docs/security/ROUTE_GUARD_TIERS.md`](../security/ROUTE_GUARD_TIERS.md) —
why `/api/plugins` is loopback-only (Tier 1)
- [`docs/frameworks/SKILLS.md`](./SKILLS.md) — the related skills framework
(`src/lib/skills/`); plugins may declare skills in their manifest
- [`docs/frameworks/WEBHOOKS.md`](./WEBHOOKS.md) — event-driven outbound
integrations
- [`docs/security/ERROR_SANITIZATION.md`](../security/ERROR_SANITIZATION.md) —
the `buildErrorBody()` pattern every plugin route uses for error responses
docs/frameworks/PLUGIN_SDK.md
+253
View File
@@ -0,0 +1,253 @@
---
title: "OmniRoute Plugin SDK"
version: 3.8.40
lastUpdated: 2026-06-28
---
# OmniRoute Plugin SDK
## Quick Start
```ts
import { definePlugin } from "omniroute/plugins/sdk";
export default definePlugin({
name: "my-plugin",
priority: 50,
onRequest: async (ctx) => {
console.log(`Request ${ctx.requestId} for ${ctx.model}`);
},
onResponse: async (ctx, response) => {
console.log(`Response for ${ctx.requestId}`);
return response;
},
onError: async (ctx, error) => {
console.error(`Error: ${error.message}`);
},
});
```
## API Reference
### `definePlugin(def: PluginDefinition): Plugin`
Factory function that creates a Plugin object with defaults.
**Parameters:**
- `name` (string, required) — Plugin name in kebab-case
- `priority` (number, optional, default: 100) — Lower runs first
- `enabled` (boolean, optional, default: true) — Start enabled?
- `onRequest` (function, optional) — Runs before chat handler
- `onResponse` (function, optional) — Runs after chat handler
- `onError` (function, optional) — Runs on handler error
### `blockRequest(response?): BlockingHookResult`
Block the request and optionally return a custom response.
```ts
onRequest: (ctx) => {
if (!ctx.headers["authorization"]) {
return blockRequest({ error: "Unauthorized", status: 401 });
}
};
```
### `modifyBody(body): PluginResult`
Modify the request body before it reaches the provider.
```ts
onRequest: (ctx) => {
return modifyBody({ ...ctx.body, temperature: 0.7 });
};
```
### `addMetadata(metadata): PluginResult`
Attach metadata to the request context.
```ts
onRequest: (ctx) => {
return addMetadata({ source: "my-plugin", version: "1.0.0" });
};
```
## Plugin Context (`PluginContext`)
| Field | Type | Description |
| ----------- | ------------------------- | ------------------------- |
| `requestId` | `string` | Unique request identifier |
| `model` | `string` | Requested model name |
| `provider` | `string` | Target provider ID |
| `body` | `Record<string, unknown>` | Request body |
| `headers` | `Record<string, string>` | Request headers |
| `metadata` | `Record<string, unknown>` | Mutable metadata |
| `timestamp` | `number` | Request timestamp |
## Manifest (`plugin.json`)
```json
{
"name": "my-plugin",
"version": "1.0.0",
"description": "A sample plugin",
"author": "your-name",
"main": "index.js",
"hooks": {
"onRequest": { "enabled": true, "priority": 50 },
"onResponse": true,
"onError": false
},
"requires": {
"permissions": ["network", "file-read"]
},
"enabledByDefault": false,
"configSchema": {
"apiKey": { "type": "string", "description": "API key for external service" },
"maxRetries": { "type": "number", "min": 1, "max": 10, "default": 3 },
"debug": { "type": "boolean", "default": false },
"mode": { "type": "string", "enum": ["fast", "slow"], "default": "fast" }
}
}
```
### Hook Priority
Hooks can be configured with priority (lower = runs first):
```json
{
"hooks": {
"onRequest": { "enabled": true, "priority": 10 },
"onResponse": { "enabled": true, "priority": 100 }
}
}
```
Or as simple booleans (default priority 100):
```json
{
"hooks": {
"onRequest": true,
"onResponse": true
}
}
```
## Permission System
Plugins run in a sandboxed VM context. Access to external resources requires explicit permissions:
| Permission | Grants |
| ------------ | ------------------------------------------------------------ |
| `network` | `fetch`, `AbortController`, `Headers`, `Request`, `Response` |
| `file-read` | `fs.readFile`, `fs.readdir`, `fs.stat` |
| `file-write` | `fs.writeFile`, `fs.mkdir`, `fs.rm` |
| `env` | Read-only `process.env` proxy |
| `exec` | `child_process.exec`, `child_process.execSync` |
Without a permission, the corresponding globals are simply not available in the sandbox.
## Config Schema
Define configurable settings in `configSchema`:
```json
{
"configSchema": {
"apiKey": { "type": "string", "description": "External API key" },
"maxRetries": { "type": "number", "min": 1, "max": 10, "default": 3 },
"debug": { "type": "boolean", "default": false },
"mode": { "type": "string", "enum": ["fast", "slow"], "default": "fast" }
}
}
```
Field types: `string`, `number`, `boolean`, `select`
Field options: `default`, `min`, `max`, `enum`, `description`
Config values are persisted in the database and accessible via the dashboard config page.
## Built-in Events
| Event | When | Payload |
| ----------------- | ----------------------------------------- | ----------------------------- |
| `onRequest` | Before chat handler | Request context |
| `onResponse` | After chat handler | Response data |
| `onError` | On handler error | Error object |
| `onModelSelect` | Model selected for routing | Model info |
| `onComboResolve` | Combo routing resolved | Combo targets |
| `onRateLimit` | Rate limit hit | Limit info |
| `onQuotaExhaust` | Quota exhausted | Quota info |
| `onProviderError` | Provider returned error | Error details |
| `onStreamStart` | SSE stream started | Stream info |
| `onStreamEnd` | SSE stream ended | Stream stats |
| `onInstall` | Plugin installed | `{ name, version, manifest }` |
| `onActivate` | Plugin activated | `{ name, version, manifest }` |
| `onDeactivate` | Plugin deactivated | `{ name, version, manifest }` |
| `onUninstall` | Plugin uninstalled (before files deleted) | `{ name, version, manifest }` |
## Examples
### Request Logger
```ts
import { definePlugin } from "omniroute/plugins/sdk";
export default definePlugin({
name: "request-logger",
onRequest: async (ctx) => {
console.log(`[${new Date().toISOString()}] ${ctx.method} ${ctx.model} -> ${ctx.provider}`);
},
});
```
### Rate Limiter
```ts
import { definePlugin, blockRequest } from "omniroute/plugins/sdk";
const requests = new Map<string, number[]>();
export default definePlugin({
name: "rate-limiter",
priority: 10,
onRequest: async (ctx) => {
const key = ctx.headers["x-api-key"] || "anonymous";
const now = Date.now();
const window = 60000; // 1 minute
const maxRequests = 100;
const timestamps = (requests.get(key) || []).filter((t) => t > now - window);
timestamps.push(now);
requests.set(key, timestamps);
if (timestamps.length > maxRequests) {
return blockRequest({ error: "Rate limit exceeded", status: 429 });
}
},
});
```
### Response Transformer
```ts
import { definePlugin } from "omniroute/plugins/sdk";
export default definePlugin({
name: "response-transformer",
onResponse: async (ctx, response) => {
if (response.choices) {
response.choices = response.choices.map((c: any) => ({
...c,
message: { ...c.message, content: c.message.content.trim() },
}));
}
return response;
},
});
```
docs/frameworks/SEARCH_TOOLS_STUDIO.md
+183
View File
@@ -0,0 +1,183 @@
---
title: "Search Tools Studio"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Search Tools Studio
> **Feature:** Search Tools Studio — unified web tools workspace for `/dashboard/search-tools`.
> **Plans:** `18-search-tools-studio-redesign.plan.md` + `_orchestration/master-plan-group-C.md`
> **Status:** Released in v3.8.6
---
## Overview
Search Tools Studio transforms `/dashboard/search-tools` from a basic search playground into a
three-tab Studio unifying web search, web scraping, and side-by-side provider comparison.
```
┌ Search Tools ──────────────────────────────────────────────────────────┐
│ [🔍 Search] [📄 Scrape] [⚖ Compare] 142ms · $0.001 </> │
│ ⓘ [Modalities guide] │
├──────────────────────────────────────────┬─────────────────────────────┤
│ {active tab content} │ ─ Config │
│ │ Provider [auto ∨] │
│ │ 🟢 Serper $0.001 │
│ │ 🟢 Tavily $0.008 │
│ │ 🔥 Firecrawl (fetch) │
│ │ Type [web | news] │
│ │ Full page [ ] (scrape) │
│ │ Format [md|text|html] │
│ │ Rerank model [∨] │
└──────────────────────────────────────────┴─────────────────────────────┘
```
---
## Tabs
### Search Tab
Evolves the existing `SearchForm` + `ResultsPanel` + `RerankPanel` into a tab:
- Query → results (title, URL, snippet, relevance score).
- Provider metadata in Config pane (cost, quota, status).
- Rerank section: pick a rerank model, reorder results, show `positionDelta`.
- Empty state with CTA when no search providers are configured.
- Search history via `SearchHistory.tsx`.
- Calls `POST /v1/search` (existing endpoint, no changes).
### Scrape Tab
New tab for extracting content from a URL via `POST /v1/web/fetch` (created in plan 05):
- Input: URL + full-page toggle + format selector (markdown / text / HTML).
- Submit → fetch → render `ScrapeResult.tsx`.
- `ScrapeResult` renders markdown preview + raw toggle.
- Cap: if response body > **256 KB**, UI shows `(truncated, view raw)` and opens raw in a Monaco modal (D21).
- Metadata panel: provider (firecrawl/jina-reader/tavily-search/tinyfish), latency, cost, response size, links count.
- Uses `useScrapeFetch.ts` hook.
### Compare Tab
Runs the same query/URL across up to **4 providers in parallel** (D22):
- Side-by-side columns per provider.
- Metrics: latency, cost, result count, response size.
- URL overlap calculation for search (number of shared URLs vs initial result).
- Calls `POST /v1/search` (search) or `POST /v1/web/fetch` (scrape) per provider.
---
## Config Pane (Shared)
`SearchToolsConfigPane.tsx` — always visible, collapsible.
| Field | Notes |
| ------------ | ---------------------------------------------------------------- |
| Provider | Dropdown with status badge (configured / missing / rate limited) |
| Type | `web` or `news` (search only) |
| Full page | Toggle for scrape — fetches entire page vs first visible content |
| Format | `markdown`, `text`, or `html` (scrape only) |
| Rerank model | Optional model for post-search reranking |
| History | Collapsible search history section |
---
## SearchConceptCard
`SearchConceptCard.tsx` — always visible, collapsible accordion. Explains:
| Concept | One-liner |
| ------------------- | -------------------------------------------------------------------- |
| **Search** | Fetches a list of web results (title, URL, snippet, relevance score) |
| **Scrape** | Extracts the full content of a URL (markdown, text or HTML) |
| **Compare** | Runs the same query in N providers side-by-side |
| **Rerank** | Reorders results via LLM to improve query relevance |
| **Auto (cheapest)** | Picks the cheapest available provider automatically |
---
## Provider Catalog
`ProviderCatalog.tsx` exposes the full provider list from `GET /api/search/providers`
(extended in F4 to include fetch providers):
| Field | Source |
| ------------------------------ | ------------------------------------------------------------------------------------------ |
| `id`, `name` | `searchRegistry.ts` |
| `kind` | `"search"` (12 providers) or `"fetch"` (firecrawl, jina-reader, tavily-search, tinyfish) |
| `costPerQuery` | Registry data |
| `freeMonthlyQuota` | Registry data |
| `searchTypes` / `fetchFormats` | Registry data |
| `status` | `"configured"` / `"missing"` / `"rate_limited"` — derived at runtime from credential store |
| `configureHref` | `/dashboard/providers` |
The status is **derived at request time** by checking whether credentials exist and whether
all keys are currently in cooldown.
---
## Export Code
`ExportCodeModal` (imported from Playground Studio) + `codeExport.ts` generate
curl / Python / TypeScript snippets for both `/v1/search` and `/v1/web/fetch` calls.
API key placeholder is always `$OMNIROUTE_API_KEY` (D11, shared with Playground Studio).
---
## Backend Changes
Only one backend change was needed for this feature:
### Extended `GET /api/search/providers`
`src/app/api/search/providers/route.ts` was extended to:
- Include all 4 fetch providers (`firecrawl`, `jina-reader`, `tavily-search`, `tinyfish`) in the array.
- Add `kind: "search" | "fetch"` to every item.
- Add `status: "configured" | "missing" | "rate_limited"` derived from live credential state.
- Maintain backward compatibility — existing fields (`id`, `name`, etc.) unchanged.
---
## Key Files
| Path | Purpose |
| --------------------------------------------------------------------------------- | ------------------------------------------------- |
| `src/app/(dashboard)/dashboard/search-tools/SearchToolsClient.tsx` | Studio shell, tab orchestrator |
| `src/app/(dashboard)/dashboard/search-tools/components/SearchToolsTopBar.tsx` | Tabs + metrics + export button |
| `src/app/(dashboard)/dashboard/search-tools/components/SearchToolsConfigPane.tsx` | Shared config panel |
| `src/app/(dashboard)/dashboard/search-tools/components/SearchConceptCard.tsx` | Explainer cards (always visible) |
| `src/app/(dashboard)/dashboard/search-tools/components/ProviderCatalog.tsx` | Provider list with metadata |
| `src/app/(dashboard)/dashboard/search-tools/components/ScrapeResult.tsx` | Markdown preview + raw toggle |
| `src/app/(dashboard)/dashboard/search-tools/components/tabs/SearchTab.tsx` | Search + rerank tab |
| `src/app/(dashboard)/dashboard/search-tools/components/tabs/ScrapeTab.tsx` | Scrape tab |
| `src/app/(dashboard)/dashboard/search-tools/components/tabs/CompareTab.tsx` | Multi-provider compare tab |
| `src/app/(dashboard)/dashboard/search-tools/hooks/useScrapeFetch.ts` | Scrape fetch hook |
| `src/app/api/search/providers/route.ts` | Extended with `kind` + `status` + fetch providers |
| `open-sse/config/searchRegistry.ts` | Source of truth for search provider metadata |
---
## Troubleshooting
| Symptom | Cause | Fix |
| ----------------------------------------- | -------------------------- | ------------------------------------------------------------------------------- |
| Scrape tab shows "endpoint not available" | `/v1/web/fetch` not wired | Verify plan 05 is merged; check `src/app/api/v1/web/fetch/route.ts` exists |
| Provider catalog shows all as "missing" | Credentials not configured | Add credentials in `/dashboard/providers` |
| Scrape content is truncated | Response > 256 KB cap | Expected behavior (D21). Use "view raw" button for full content |
| Compare tab only shows 2 providers | Rate limit active | Two or more providers may be in cooldown — check provider status in Config pane |
| "Size" shown as raw key in table | Missing i18n key | Verify `search.size` exists in the locale file; rebuild i18n |
---
## References
- Master plan: `_tasks/features-v3.8.6/refactorpages/_orchestration/master-plan-group-C.md`
- Feature plan: `_tasks/features-v3.8.6/refactorpages/18-search-tools-studio-redesign.plan.md`
- Search provider registry: `open-sse/config/searchRegistry.ts`
- Playground Studio (shared `ExportCodeModal` + `codeExport.ts`): `docs/frameworks/PLAYGROUND_STUDIO.md`
- Web fetch backend: `src/app/api/v1/web/fetch/route.ts`
docs/frameworks/SKILLS.md
+487
View File
@@ -0,0 +1,487 @@
---
title: "Skills Framework"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Skills Framework
> **Source of truth:** `src/lib/skills/` and `src/app/api/skills/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute exposes an extensible Skills framework that lets language models (and operators) compose reusable capabilities — from filesystem reads and HTTP requests to sandboxed code execution and curated marketplace skills.
A skill is a versioned, schema-defined unit of work. OmniRoute can inject skills as tool definitions into outbound requests, intercept tool calls coming back from the model, run the matching handler, and feed the result back to the model so the conversation can continue. The model never sees the implementation — only the tool interface.
---
## Agent Skills vs Omni Skills
OmniRoute has two distinct but complementary skill systems:
| Dimension | **Omni Skills** (this doc) | **Agent Skills** |
| :-------------- | :------------------------------------------------------------ | :------------------------------------------------------------------------------------------ |
| Purpose | LLM tool injection + sandboxed execution | SKILL.md catalog for external agents to discover and consume |
| Source of truth | `src/lib/skills/` + marketplace | `src/lib/agentSkills/` + `skills/` directory |
| Runtime mode | Injected into outbound requests, executed on tool-call events | Static markdown catalog + REST/MCP/A2A discovery endpoints |
| Who uses it | OmniRoute itself (combo routing, inbound LLM calls) | External agents, MCP clients, A2A orchestrators |
| Count | Variable (marketplace-driven) | 42 canonical entries (22 API + 20 CLI) |
| Format | `SkillDefinition` with tool schema + handler | `SKILL.md` frontmatter + markdown body |
| Discovery | `/api/skills/*` REST + `omniroute_skills_*` MCP tools | `/api/agent-skills/*` REST + `omniroute_agent_skills_*` MCP tools + A2A `list-capabilities` |
**Omni Skills** are the execution engine — they define what OmniRoute _can do_ when an LLM invokes a tool.
**Agent Skills** are the documentation catalog — they explain to external agents _how to use_ OmniRoute's REST API and CLI, with structured SKILL.md files that can be fed directly into agent prompts.
For the Agent Skills catalog, generator, MCP tools, and A2A skill, see [docs/frameworks/AGENT-SKILLS.md](./AGENT-SKILLS.md).
---
## Concepts
### Skill Sources
Three sources of skills coexist in the same registry:
1. **Built-in skills** (`src/lib/skills/builtins.ts`) — shipped with OmniRoute. Cover the common cases:
- `file_read`, `file_write` — per-API-key sandbox workspace under `<DATA_DIR>/skills/workspaces/<hashed-key>/`
- `http_request` — outbound HTTP through `safeOutboundFetch` with `guard: "public-only"`
- `web_search` — pluggable search provider with caching (`executeWebSearch`)
- `eval_code` — Docker-sandboxed `node` or `python` execution
- `execute_command` — Docker-sandboxed shell command
- `browser` — Playwright-backed scaffolding, disabled by default (`builtin/browser.ts`)
2. **SkillsMP** (the OmniRoute Marketplace) — fetched from `https://skillsmp.com/api/v1/skills/search`. Requires `skillsmpApiKey` in Settings.
3. **SkillsSH** (`skills.sh` community catalog) — fetched from `https://skills.sh/api/search`. No auth needed; SKILL.md content pulled from GitHub raw.
A single "active provider" controls which catalog the dashboard installs from (`src/lib/skills/providerSettings.ts`). Switch it under **Settings → Memory & Skills**. Default: `skillsmp`.
### Skill Identity
Skills are keyed by `name@version` in the in-memory registry (`src/lib/skills/registry.ts`). Version must be semver (`^\d+\.\d+\.\d+$`). `resolveVersion()` understands `^`, `~`, `>`, `>=`, `<`, `<=`, `==`, and exact-match constraints.
### Skill Mode
Each skill has a runtime mode that controls when it is injected:
| Mode | Behavior |
| ------ | ------------------------------------------------------------------------------------------ |
| `on` | Always injected as a tool definition |
| `off` | Never injected, never executable |
| `auto` | Scored against the incoming request; injected only if score ≥ `AUTO_MIN_SCORE` (default 3) |
`auto` is the default for marketplace-installed skills. `enabled=true` and `mode="off"` together mean "registered but inactive" — toggling `enabled` via the legacy column also bumps `mode` so older codepaths stay consistent (`src/app/api/skills/[id]/route.ts`).
### Status (executions)
Skill executions are tracked in the `skill_executions` table with the following statuses (`src/lib/skills/types.ts`):
```ts
enum SkillStatus {
PENDING = "pending",
RUNNING = "running",
SUCCESS = "success",
ERROR = "error",
TIMEOUT = "timeout",
}
```
### Registry Cache
`SkillRegistry` is a singleton with a 60-second TTL cache (`registry.ts:14`). `loadFromDatabase()` is idempotent and dedupes concurrent calls via `pendingLoad`. Any write (`register`/`unregister`/`unregisterById`) invalidates the cache. Look up versions via `getSkillVersions(name)` and `resolveVersion(name, constraint)`.
### Provider-Aware Injection
`injectSkills()` in `src/lib/skills/injection.ts` is the entry point that turns registered skills into provider-specific tool definitions:
- **OpenAI** — `{ type: "function", function: { name, description, parameters } }`
- **Anthropic** — `{ name, description, input_schema }`
- **Google (Gemini)** — `{ name, description, parameters }`
The tool name is encoded as `name@version` so the handler can pick the right version when the model calls it back.
### AUTO Scoring
When `mode="auto"`, each candidate skill is scored against the request context (`scoreAutoSkill()` in `injection.ts`):
| Signal | Points |
| ---------------------------------------------- | ------------ |
| Skill name appears verbatim in context | +6 |
| Each name token matches a context token | +2 |
| Each tag substring matches context | +3 |
| Each description token matches context | +1 |
| Background reason matches a name token | +2 per token |
| Background reason matches a tag | +2 per token |
| Provider hint in tags matches request provider | +2 / −2 |
Top `AUTO_MAX_SKILLS = 5` skills with `score >= AUTO_MIN_SCORE = 3` are injected. Ties are broken by `installCount` (desc), then alphabetical name (`injection.ts:225-235`).
### Tool Call Interception
`handleToolCallExecution()` in `src/lib/skills/interception.ts` is invoked by the chat handler after the upstream returns a tool-calling response:
1. `extractToolCalls()` reads provider-specific shapes (OpenAI `tool_calls` / Responses `function_call`, Anthropic `tool_use`, Gemini `functionCalls`).
2. Built-in tool aliases (e.g. `omniroute_web_search` → `web_search`) are resolved first. Built-in handlers run inline.
3. Anything else routes through `skillExecutor.execute(name@version, args, { apiKeyId, sessionId })`.
4. Results are spliced back into the response — `tool_results`, `function_call_output` items, or Anthropic `tool_result` blocks as appropriate.
`customSkillExecutionEnabled` in the execution context can be set to `false` to allow only built-in interception (used by request paths that explicitly disable user-defined handlers).
---
## Docker Sandbox
Non-builtin code paths (`eval_code`, `execute_command`) run inside Docker via `SandboxRunner` (`src/lib/skills/sandbox.ts`). Every container is launched with:
```
--rm --network none|bridge --cap-drop ALL
--security-opt no-new-privileges --pids-limit 100
--cpus <cpuLimit/1000> --memory <memoryLimit>m
--tmpfs /tmp:rw,noexec,nosuid,size=64m
--tmpfs /workspace:rw,noexec,nosuid,size=64m
--read-only (when readOnly=true)
```
Defaults (`SandboxRunner.DEFAULT_CONFIG`):
| Field | Default | Notes |
| ---------------- | --------------- | ---------------------------------------------------- |
| `cpuLimit` | 100 (= 0.1 CPU) | Divided by 1000 before passing to `--cpus` |
| `memoryLimit` | 256 MB | Hard limit |
| `timeout` | 30000 ms | Soft kill via `SIGTERM` + `docker kill` |
| `networkEnabled` | `false` | Becomes `--network none` |
| `readOnly` | `true` | Root FS read-only; `/tmp` and `/workspace` are tmpfs |
`SandboxRunner.kill(id)` and `killAll()` are exposed for shutdown; running containers are tracked in `runningContainers: Map<string, ChildProcess>`.
### Sandbox Env Vars
Configured via `process.env` in `src/lib/skills/builtins.ts`:
| Env Var | Default | Purpose |
| --------------------------------- | ---------------- | ------------------------------------------------------------------ |
| `SKILLS_MAX_FILE_BYTES` | `1048576` (1 MB) | Cap for `file_read` and `file_write` |
| `SKILLS_MAX_HTTP_RESPONSE_BYTES` | `256000` | Cap for `http_request` response body |
| `SKILLS_MAX_SANDBOX_OUTPUT_CHARS` | `100000` | Cap for stdout/stderr returned to the caller |
| `SKILLS_SANDBOX_TIMEOUT_MS` | `10000` | Default timeout for sandboxed commands; capped at 60 s |
| `SKILLS_SANDBOX_NETWORK_ENABLED` | `false` | Master gate for egress. Set `1` or `true` to allow per-call opt-in |
| `SKILLS_ALLOWED_SANDBOX_IMAGES` | (see below) | Comma-separated allowlist of Docker images |
Default allowed images: `alpine:3.20`, `node:22-alpine`, `python:3.12-alpine`. Any additions via `SKILLS_ALLOWED_SANDBOX_IMAGES` are merged with the defaults; unknown images are rejected by `normalizeImage()`.
> Note: there is no separate `SKILLS_EXECUTION_TIMEOUT_MS` env var. The non-sandbox handler timeout is hard-coded to 30 s in `SkillExecutor` (`executor.ts:13`) but can be overridden at runtime via `skillExecutor.setTimeout(ms)`.
### Workspace Isolation
`file_read` and `file_write` resolve every path relative to a per-API-key workspace at `<DATA_DIR>/skills/workspaces/<sha256(apiKeyId).slice(0,24)>/`. Path traversal (`..`) and forbidden segments (`.env`, `.git`, `.ssh`, `.omniroute`, `.codex`, `secrets`) are rejected before any disk I/O.
### HTTP Hardening
`http_request` (`builtins.ts:257`):
- Method allowlist: `GET, HEAD, POST, PUT, PATCH, DELETE`
- Blocked outbound headers: `host, connection, content-length, cookie, set-cookie, authorization, proxy-authorization`
- Redirects disabled (`allowRedirect: false`)
- Routed through `safeOutboundFetch` with `guard: "public-only"` (private/loopback ranges blocked)
- Response truncated at `SKILLS_MAX_HTTP_RESPONSE_BYTES`; client sees `truncated: true`
---
## Hybrid Executor (preview)
`src/lib/skills/hybrid.ts` defines a `HybridExecutor` that decides between `direct` (in-process) and `sandbox` execution per call, with an `autoUpgrade` retry path on timeout/memory errors. The wired-in `directExecutor` / `sandboxRunner` implementations are stubs (`executeDirect`, `executeInSandbox` return placeholder objects) — treat this module as a contract under construction. Real execution still goes through `skillExecutor` + `SandboxRunner`.
---
## Storage
Schema lives in two migrations:
- `src/lib/db/migrations/016_create_skills.sql` — base `skills` and `skill_executions` tables, with indexes on `(api_key_id, name)` and `(skill_id, status, created_at)`.
- `src/lib/db/migrations/027_skill_mode_and_metadata.sql` — adds `mode`, `source_provider`, `tags` (JSON), `install_count` to `skills`.
`skill_executions.status` is constrained at the database level: `CHECK(status IN ('pending', 'running', 'success', 'error', 'timeout'))`.
---
## REST API
All endpoints live under `src/app/api/skills/`. Management endpoints (`/api/skills`, `/api/skills/[id]`, `/api/skills/install`) require **management auth** via `requireManagementAuth()`. The marketplace/install flows use the lighter `isAuthenticated()` (session or API key).
| Endpoint | Method | Purpose |
| --------------------------------- | ------ | ------------------------------------------------------------------------ | --- | ------------------------ | -------- | ------------------ |
| `/api/skills` | GET | List registered skills. Supports `?q=`, `?mode=on | off | auto`, `?source=skillsmp | skillssh | local`, pagination |
| `/api/skills/[id]` | PUT | Update `enabled` or `mode` |
| `/api/skills/[id]` | DELETE | Unregister by id |
| `/api/skills/install` | POST | Install a custom skill (handler code + schema) |
| `/api/skills/marketplace` | GET | Search the SkillsMP catalog (returns popular defaults when `q` is empty) |
| `/api/skills/marketplace/install` | POST | Install a SkillsMP skill (requires active provider = `skillsmp`) |
| `/api/skills/skillssh` | GET | Search the skills.sh catalog (`?q=&limit=`, capped at 100) |
| `/api/skills/skillssh/install` | POST | Install a skills.sh skill (requires active provider = `skillssh`) |
| `/api/skills/executions` | GET | Paginated execution history (`?apiKeyId=`) |
| `/api/skills/executions` | POST | Execute a registered skill ad-hoc |
The `POST /api/skills/executions` endpoint returns HTTP `503` with `{ error: "Skills execution is disabled..." }` when `settings.skillsEnabled === false` (`executor.ts:42-45`). Operators can flip the master switch from **Settings → AI**.
### Example: install a custom skill
```bash
curl -X POST http://localhost:20128/api/skills/install \
-H "Authorization: Bearer $OMNIROUTE_MGMT_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "reverse-text",
"version": "1.0.0",
"description": "Reverses a string",
"schema": {
"input": { "type": "object", "properties": { "text": { "type": "string" } }, "required": ["text"] },
"output": { "type": "object", "properties": { "reversed": { "type": "string" } } }
},
"handlerCode": "echo-handler",
"apiKeyId": "your-api-key-id"
}'
```
The `handlerCode` string is a **handler name lookup** — not executable code. The executor maps it via `skillExecutor.registerHandler(name, fn)` (`executor.ts:25`). Marketplace installs store the SKILL.md text in this field as documentation and route execution through model-generated tool calls. Arbitrary user-supplied source is not eval'd.
---
## MCP Tools
Four MCP tools wrap the skills surface (`open-sse/mcp-server/tools/skillTools.ts`). They are auto-registered when the MCP server boots.
| Tool | Description |
| ----------------------------- | ------------------------------------------------------------ |
| `omniroute_skills_list` | List skills, optional filters: `apiKeyId`, `name`, `enabled` |
| `omniroute_skills_enable` | Enable/disable a skill by `skillId` |
| `omniroute_skills_execute` | Execute a skill with an input payload |
| `omniroute_skills_executions` | Recent execution history (default 50, max 100) |
See [MCP-SERVER.md](./MCP-SERVER.md) for transport setup and scope assignments.
---
## A2A Integration
`src/lib/skills/a2a.ts` exports the `memory_aware_routing` A2A skill descriptor and a `registerA2ASkill(registry)` helper. Custom A2A skills live in `src/lib/a2a/skills/` and are dispatched via `A2A_SKILL_HANDLERS` (`src/lib/a2a/taskExecution.ts`). See [A2A-SERVER.md](./A2A-SERVER.md) for the full task lifecycle.
---
## Adding a New Built-in Skill
1. **Define the handler** in `src/lib/skills/builtins.ts` (or a sibling file under `src/lib/skills/builtin/`). Signature: `(input, { apiKeyId, sessionId }) => Promise<output>`.
2. **Sandboxed code path?** Call `sandboxRunner.run(image, command, env, sandboxConfig({...}))`. Use `normalizeImage()` against the allowlist.
3. **Filesystem path?** Always pass through `resolveWorkspacePath(input, context)` before touching disk.
4. **Network call?** Use `safeOutboundFetch` with `guard: "public-only"`; sanitize headers via `sanitizeHeaders()`.
5. **Register** by adding the entry to `builtinSkills` (or calling `registerBrowserSkill(executor)`-style at boot).
6. **Wire built-in tool aliases** (optional) in `BUILTIN_TOOL_ALIASES` (`interception.ts:23`) if the upstream model emits a different name.
7. **Tests** in `src/lib/skills/__tests__/` (Vitest).
---
## Adding a Custom (Non-Builtin) Skill
1. Register the handler at process startup:
```ts
skillExecutor.registerHandler("my-handler", async (input, ctx) => { ... });
```
2. Insert the skill via `POST /api/skills/install` (the `handlerCode` field must match the registered handler name).
3. Toggle `mode` to `on` or `auto` via `PUT /api/skills/[id]`.
---
## Operational Tips
- **Master switch:** `settings.skillsEnabled = false` blocks all execution and returns HTTP `503` on `/api/skills/executions`. The registry continues to load.
- **Lock down egress:** keep `SKILLS_SANDBOX_NETWORK_ENABLED` unset (default) for fully air-gapped sandboxing. Per-call `networkEnabled: true` still requires the master gate.
- **Allow specific images:** set `SKILLS_ALLOWED_SANDBOX_IMAGES="myorg/sandbox:1.0,node:22-alpine"` to extend the allowlist.
- **Audit executions:** `/dashboard/skills/executions` and `omniroute_skills_executions` both query `skill_executions`. Successful runs include `durationMs`; failures include `errorMessage`.
- **Cache invalidation:** call `skillRegistry.invalidateCache()` after manual DB edits; otherwise wait 60 s.
- **Anonymous workspace:** when `apiKeyId` is empty, all calls hash to the same `"anonymous"` workspace — share-aware code should always pass a real key.
---
## Execution Lifecycle (v3.8.16+)
The `SkillExecutor` (`src/lib/skills/executor.ts`) is a **singleton** that manages every skill invocation. Understanding its lifecycle is critical for debugging timeouts, retries, and execution state.
### The 5-Stage Lifecycle
```
execute() called
│
▼
┌─────────────┐
│ PENDING │ ← queued, not yet started (DB row created)
└──────┬──────┘
│ start handler
▼
┌─────────────┐
│ RUNNING │ ← handler invoked with timeout
└──────┬──────┘
│
┌────┴────┬──────────┬──────────┐
│ │ │ │
▼ ▼ ▼ ▼
SUCCESS ERROR TIMEOUT (no other path — killed by parent)
│ │ │
└────┬────┴──────────┘
│
▼
DB row updated with status, output, durationMs
```
### Default Configuration
| Setting | Default | Configurable via |
| ------------ | ------------- | ------------------------------------ |
| `timeout` | `30000` (30s) | `skillExecutor.setTimeout(ms)` |
| `maxRetries` | `3` | `skillExecutor.setMaxRetries(count)` |
> **Important**: The executor is a singleton — calling `setTimeout()` affects all subsequent invocations globally. Per-skill timeouts are not currently supported; if you need different timeouts per skill, submit separate processes or fork the executor.
### Status Values
From `src/lib/skills/types.ts`:
```ts
enum SkillStatus {
PENDING = "pending", // Queued, not yet started
RUNNING = "running", // Handler invoked
SUCCESS = "success", // Handler returned valid output
ERROR = "error", // Handler threw an exception
TIMEOUT = "timeout", // Exceeded the executor's timeout
}
```
> **Note**: The `TIMEOUT` status is defined in the enum but is **not actually written to the DB** by the current executor implementation — timeouts surface as `ERROR` with the message `"Skill execution timed out"`. The status enum is reserved for future use.
### Inspecting Executions
```ts
import { skillExecutor } from "omniroute/skills/executor";
// Get a specific execution by ID
const exec = skillExecutor.getExecution("exec-uuid-123");
if (exec) {
console.log(`${exec.skillName}: ${exec.status} in ${exec.durationMs}ms`);
}
// List recent executions for an API key
const recent = skillExecutor.listExecutions("api-key-id", 50, 0);
for (const e of recent) {
console.log(`${e.skillName} → ${e.status} (${e.durationMs}ms)`);
}
// Count total executions
const total = skillExecutor.countExecutions("api-key-id");
```
### Retry Behavior
The `maxRetries` setting is stored but **not currently used** by the executor's `execute()` method — it only performs a single attempt. The `maxRetries` value is exposed for future implementation and for hooks that want to read it.
For now, retries must be implemented inside the skill handler itself. Built-in
skills are registered against the executor (e.g. `registerBuiltinSkills(executor)`
/ `registerBrowserSkill(executor)` in `src/lib/skills/builtin/`); whichever handler
you register can wrap its own retry loop:
```ts
// inside a skill handler
async function handler(input, ctx) {
const maxRetries = 3;
let lastError: Error | null = null;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await fetchSomething(input);
} catch (err) {
lastError = err as Error;
if (attempt < maxRetries) {
await new Promise((r) => setTimeout(r, 1000 * attempt));
}
}
}
throw lastError;
}
```
---
## SkillMode in Detail
The `SkillMode` enum (`src/lib/skills/types.ts`) controls **when and how** skills are invoked:
```ts
enum SkillMode {
AUTO = "auto", // LLM decides when to call the skill
MANUAL = "manual", // Only invoked by explicit user request
HYBRID = "hybrid", // AUTO scoring + manual override
}
```
> **Note**: The codebase defines `SkillMode` (AUTO/MANUAL/HYBRID), while the `Skill.mode` field uses a different shape (`"on" | "off" | "auto"`). They are related but not identical — `SkillMode` is for executor policy, `Skill.mode` is for per-skill enablement.
### When to Use Each Mode
| Mode | LLM behavior | Use case |
| -------- | ------------------------------------------------------------------------------ | -------------------------------------------------- |
| `AUTO` | LLM can call the skill when it deems necessary | General-purpose skills (file reads, HTTP requests) |
| `MANUAL` | LLM cannot call the skill; only an explicit `executeSkill` API call invokes it | Sensitive operations (database writes, payments) |
| `HYBRID` | LLM can suggest the skill; user must confirm | Skills that have side effects but aren't dangerous |
### AUTO Scoring
When `AUTO` mode is active, each candidate skill is scored against the request
context by `scoreAutoSkill()` in `src/lib/skills/injection.ts` — an additive,
integer point system (skill-name match, name/tag/description token overlap,
background-reason hints, provider-hint bonus/penalty). The top
`AUTO_MAX_SKILLS = 5` skills with `score >= AUTO_MIN_SCORE = 3` are injected as
callable tools, ties broken by `installCount` then name. See the full point table
in [**Tool Schema Generation → AUTO Scoring**](#auto-scoring) earlier in this
document; there is no float `0.6`-style threshold and no `registry.ts` scoring.
---
## Built-in Skills Catalog
OmniRoute ships with a curated set of built-in skills in `src/lib/skills/builtin/`. The most common ones:
### Browser Automation Skill
The browser skill (`src/lib/skills/builtin/browser.ts`) provides headless browser automation via Playwright/Puppeteer. **It is implemented but not in the default skills catalog** — to use it, install the browser extension plugin separately.
```ts
// Enable in your config
const config: SkillConfig = {
enabled: true,
mode: SkillMode.MANUAL, // Always require explicit invocation
allowedSkills: ["browser"],
timeout: 60000, // 60s for page loads
maxRetries: 1,
};
```
### Other Built-in Categories
| Category | Skills | Mode |
| --------- | ------------------------------------------- | ------ |
| File I/O | `file_read`, `file_write` | AUTO |
| HTTP | `http_request` | AUTO |
| Search | `web_search` | AUTO |
| Code Exec | `eval_code` (sandboxed JavaScript/Python) | HYBRID |
| System | `execute_command` (sandboxed CLI execution) | MANUAL |
### Adding a Custom Skill
See the [Plugin SDK & Skills Integration](./PLUGIN_SDK.md) for how to add a custom skill via the plugin system.
---
## See Also
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP tool registration and transports
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A task lifecycle and skill dispatch
- [USER_GUIDE.md](../guides/USER_GUIDE.md#-skills-system) — user-facing introduction
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — request pipeline and component map
- Source: `src/lib/skills/`, `src/app/api/skills/`, `open-sse/mcp-server/tools/skillTools.ts`
- Tests: `src/lib/skills/__tests__/integration.test.ts`
docs/frameworks/TRAFFIC_INSPECTOR.md
+491
View File
@@ -0,0 +1,491 @@
---
title: "Traffic Inspector"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Traffic Inspector
Traffic Inspector is OmniRoute's built-in HTTPS traffic debugger — a Charles Proxy / mitmweb / HTTP Toolkit-like tool that is **LLM-aware** and **agent-aware**. It lives at `/dashboard/tools/traffic-inspector` and receives live traffic from up to 5 simultaneous capture sources.
**Dashboard location:** `/dashboard/tools/traffic-inspector`
**Sidebar group:** Tools (after AgentBridge)
**See also:** [`AGENTBRIDGE.md`](./AGENTBRIDGE.md) — AgentBridge is capture mode 1.
---
## §1 Overview
### What makes Traffic Inspector unique
| Feature | mitmweb | Charles | Fiddler | **OmniRoute Traffic Inspector** |
| ------------------------------------------------------------------- | :-----: | :-----: | :-----: | :-----------------------------: |
| Web-based | ✓ | ✗ | ✗ | ✓ |
| Open-source | ✓ | ✗ | partial | ✓ |
| **Agent-aware** (knows if request is from Antigravity/Copilot/etc.) | ✗ | ✗ | ✗ | ✓ |
| **LLM-aware** (parses OpenAI/Anthropic/Gemini shape, tokens, model) | ✗ | ✗ | ✗ | ✓ |
| **Model mapping visible** (gemini-3-flash → claude-sonnet-4.7) | ✗ | ✗ | ✗ | ✓ |
| **Proxy/upstream latency split** | partial | ✗ | ✗ | ✓ |
| **Integrated with OmniRoute** routing, fallback, cost | ✗ | ✗ | ✗ | ✓ |
| **System-wide proxy debug** (any app on the machine) | ✓ | ✓ | ✓ | ✓ |
| **Custom host capture** (per-host DNS redirect) | ✓ | ✓ | ✓ | ✓ |
| **HTTP_PROXY env mode** | ✓ | ✓ | ✓ | ✓ |
| **Conversation view** (multi-turn bubbles, tool_use/tool_result) | ✗ | ✗ | ✗ | ✓ |
| **SSE stream merger** (reconstruct from delta events) | ✗ | ✗ | ✗ | ✓ |
| **Session recording** (named, exportable .har/.jsonl) | ✗ | ✓ | ✓ | ✓ |
### Architecture in one paragraph
The `TrafficBuffer` (`src/mitm/inspector/buffer.ts`) is a shared in-memory ring buffer (default 1000 entries, configurable via `INSPECTOR_BUFFER_SIZE`). All capture sources write to it via `push()`. The buffer classifies each entry using `kindDetector.ts` (determines if it's an LLM request), computes a `contextKey` (SHA-256 fingerprint of the system prompt), and broadcasts to all WebSocket subscribers via `globalTrafficBuffer.subscribe()`. The dashboard connects via `GET /api/tools/traffic-inspector/ws` and receives a snapshot on connect, followed by `new`/`update`/`clear` events.
---
## §2 Capture modes
Traffic Inspector supports **5 simultaneous capture sources**. Each is independently toggleable. The `source` field on every `InterceptedRequest` (`src/mitm/inspector/types.ts`) is one of `"agent-bridge"`, `"custom-host"`, `"http-proxy"`, `"system-proxy"`, or `"tproxy"`.
### Mode 1 — AgentBridge (default, always on)
**Source:** AgentBridge handlers (`src/mitm/handlers/base.ts`)
**Mechanism:** Every `intercept()` call in `MitmHandlerBase` calls `hookBufferStart()` before forwarding and `hookBufferUpdate()` on completion. Zero extra config — works as soon as AgentBridge is running.
**Reach:** The 9 IDE agents configured in AgentBridge
**Note:** `source` field in `InterceptedRequest` = `"agent-bridge"`
### Mode 2 — Custom Hosts (DNS redirect)
**Source:** User-defined host list (`inspector_custom_hosts` table)
**Mechanism:** Adding a host via the UI adds `127.0.0.1 <host>` to `/etc/hosts` (requires sudo). The existing AgentBridge MITM server (port 443) generates a SNI cert dynamically for the new host.
**Reach:** Any application using the added host — no app config change needed
**Note:** `source` = `"custom-host"`
Example use cases:
- Monitor `api.openai.com` from Python scripts
- Debug `my-internal-llm.company.com`
- Capture traffic from mobile devices on the same network (via ARP spoofing — advanced)
### Mode 3 — HTTP_PROXY listener (port 8080)
**Source:** Applications using `HTTP_PROXY`/`HTTPS_PROXY` environment variables
**Mechanism:** Secondary listener at port 8080 (`src/mitm/inspector/httpProxyServer.ts`) that acts as a standard explicit HTTP/HTTPS proxy. Accepts `CONNECT` tunnels (HTTPS) and direct HTTP requests.
**Reach:** Any application that respects `HTTP_PROXY` env — no DNS change, no sudo
**Note:** `source` = `"http-proxy"`
```bash
# Quick capture for a single command:
HTTPS_PROXY=http://127.0.0.1:8080 curl https://api.openai.com/v1/models
# Persistent capture in a shell session:
export HTTP_PROXY=http://127.0.0.1:8080
export HTTPS_PROXY=http://127.0.0.1:8080
```
**TLS limitation:** HTTPS `CONNECT` tunnels are captured as metadata only (host, port, timing) — TLS body is not decrypted by default. Enable "Decrypt HTTPS in proxy mode" toggle (opt-in, requires AgentBridge cert to be trusted) for full body inspection.
**Port conflict:** If port 8080 is in use, AgentBridge returns a 409 with a structured error. Change the port via `INSPECTOR_HTTP_PROXY_PORT` env var.
### Mode 4 — System-wide proxy (advanced, opt-in)
**Source:** OS-level proxy settings (applies to all apps on the machine)
**Mechanism:** Uses OS APIs to redirect all HTTP/HTTPS traffic through the HTTP_PROXY listener:
- **macOS:** `networksetup -setwebproxy / -setsecurewebproxy`
- **Linux:** `gsettings set org.gnome.system.proxy` + `/etc/environment`
- **Windows:** `netsh winhttp set proxy 127.0.0.1:8080`
**Reach:** Every application on the machine that respects system proxy settings
**Note:** `source` = `"system-proxy"`
**Safety mechanisms:**
- Auto-disable timer (default 30 min, configurable via `INSPECTOR_SYSTEM_PROXY_GUARD_MINUTES`)
- Previous system proxy state is saved in DB and restored on revert
- Dashboard shows "Reverting system proxy" prompt if user navigates away while active
- UI shows `⚠ Advanced` badge + explicit confirmation checkbox
### Mode 5 — TPROXY transparent decrypt (Linux, root, opt-in)
**Source:** Kernel TPROXY + policy routing (`src/mitm/tproxy/`)
**Mechanism:** Marks new local outbound TCP connections to a target port (default `443`) in `mangle OUTPUT`, an `ip rule` reroutes the marked packets to local delivery, and `mangle PREROUTING`'s `TPROXY` target hands them to a transparent (**IP_TRANSPARENT**) listener (default port `8443`). The listener terminates TLS with a leaf certificate issued **per SNI hostname on demand** by a dynamic CA, captures the decrypted exchange, and forwards the request re-encrypted to the original destination.
**Reach:** **Arbitrary** destination hosts on the target port — no `/etc/hosts` spoof, no `HTTP_PROXY` env, no system-wide proxy mutation. The intercepted process needs no config change, but must trust the dynamic CA.
**Note:** `source` = `"tproxy"`
**Requirements:** Linux only (**IP_TRANSPARENT** is Linux-only), the **CAP_NET_ADMIN** capability (root), and a native N-API addon that must be built with a C toolchain (`npm run build:native:tproxy`). When unavailable, the dashboard toggle is disabled with the tooltip "TPROXY decrypt requires Linux + root + the native addon". The firewall rules apply/revert transactionally (a crash never leaves a `mangle` rule behind) and flush on reboot. An SO_MARK-based anti-loop keeps the proxy's own re-encrypted forward from being re-intercepted.
This is a substantial subsystem with its own dedicated operator guide — see **[`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md)** for the full firewall recipe, the per-SNI dynamic CA + trust-store installer, the local-only route, anti-loop details, and the configuration schema. The toggle is driven by `GET / POST / DELETE /api/tools/agent-bridge/tproxy` (note: the route lives under the AgentBridge prefix, not the Traffic Inspector prefix).
### Capture mode comparison
| Mode | Setup | Sudo? | Reach | Notes |
| ----------------- | ----------------------------- | :---------------------: | --------------------------- | ----------------------------------------------------------------------------------------------------------- |
| 1. AgentBridge | Automatic | Once (cert+hosts) | 9 IDE agents | Default on |
| 2. Custom Hosts | Per-host input | Yes (hosts file) | Any app using that host | Persisted in DB |
| 3. HTTP_PROXY | `export HTTPS_PROXY=...` | No | Apps respecting env | Port 8080, no TLS decrypt by default |
| 4. System-wide | Toggle + confirm | Yes | All apps on machine | Auto-disable in 30 min |
| 5. TPROXY decrypt | Toggle (Linux + native addon) | Yes (root + CA install) | Any host on the target port | Decrypts arbitrary hosts; off by default — see [MITM-TPROXY-DECRYPT.md](../security/MITM-TPROXY-DECRYPT.md) |
---
## §3 UI
### 3.1 Layout
```
┌─ Traffic Inspector ─────────────────────────────────────────────────────┐
│ ┌─ Capture sources toolbar ─────────────────────────────────────────┐ │
│ │ [✓ AgentBridge] [✓ Custom hosts (3)] [○ HTTP_PROXY] [○ System]│ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ ┌─ Filter/control bar ──────────────────────────────────────────────┐ │
│ │ Profile: (●) LLM only (○) Custom (○) All │ │
│ │ [⎉ Pause] [🗑 Clear] [⬇ .har] [● REC session] ● live 482/1k │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
├══◀▶══════════════════════════════╬══════════════════════════════════════╤╡
│ REQUEST LIST (resizable) ║ DETAIL PANE ▲ │
│ ────────────────────────────── │ ║ [Conversation][Headers][Request] │ │
│ ▎ 14:32 POST 200 12k AG openai ║ [Response][Timing][LLM][Stats] │ │
│ ▎ 14:31 POST 200 8k CP openai ║ ▼ │
│ ▎ 14:31 POST 503 ⚠ KR ... ║ │
│ ▎ 14:30 GET 200 3k 🌐 custom ║ │
└══════════════════════════════════╝══════════════════════════════════════╝
```
### 3.2 Request list (left panel)
- **Virtualized** (`useVirtualList` + `ResizeObserver`): handles 1000 items without freezing
- **Auto-scroll** with toggle to pause while inspecting
- **Color-coded status**: green (2xx), yellow (3xx), red (4xx/5xx), gray (in-flight)
- **Agent emoji**: 🔵 Antigravity, 🟢 Copilot, 🟠 Kiro, 🟣 Codex, 🔷 Cursor, 🟤 Zed, 🟡 Claude Code, ⚫ Open Code, 🌐 custom host
- **Context color bar**: 1px left border colored by `contextKey` (SHA-256 of system prompt) — visually groups related conversations
- **Lazy body**: only the selected request's body is materialized in the detail tabs (avoids rendering 1000 × 1MB bodies)
### 3.3 Detail pane — 7 tabs
| Tab | Content | Notes |
| ---------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| **Conversation** | Multi-turn chat bubbles (system/user/assistant + tool_use/tool_result) | Normalized from any provider format; only shown for `detectedKind === "llm"` |
| **Headers** | Request + response header tables | Sensitive headers (Authorization, Cookie, api-key) masked by default; "Show secrets" toggle |
| **Request** | Raw body, JSON tree view, model field badge | Pretty-printed JSON or raw text |
| **Response** | Raw body or SSE event list; toggle "Raw ↔ Merged" | SSE merger reconstructs final message from delta events |
| **Timing** | Waterfall: proxy overhead vs upstream latency | Total, TTFB, and size |
| **LLM Details** | Provider, model, messages count, tokens in/out, cost estimate, mapped target | Only shown for LLM requests |
| **Stats** | Recharts: latency timeline, token bar chart, tool call scatter | Only shown when a recorded session is loaded |
### 3.4 Toolbar controls
| Control | Action |
| ---------------- | --------------------------------------------------------------------- |
| ⎉ Pause | Stops rendering new requests; "X new" badge accumulates |
| 🗑 Clear | Clears the UI list (server buffer is not affected) |
| ⬇ Export .har | Downloads current filtered list as HAR file |
| ● Record session | Starts a named recording session |
| Profile selector | LLM only / Custom hosts / All |
| Host filter | Substring match on `host` field |
| Agent filter | Dropdown: All / per-agent |
| Status filter | All / 2xx / 3xx / 4xx / 5xx / error |
| Source filter | All / agent-bridge / custom-host / http-proxy / system-proxy / tproxy |
| **Live** filter | Show only in-flight (open) requests — `liveOnly` toggle (see §4.6) |
### 3.5 Resizable panels
- List and detail pane separated by a drag handle
- List width: min 280px, max 720px, persisted in `localStorage` (`inspector.listWidth`)
- Collapsible to a 48px rail (icon-only); click a row in the rail to expand
---
## §4 LLM-aware features
### 4.1 Kind detector (`src/mitm/inspector/kindDetector.ts`)
Classifies each request as `"llm"`, `"app"`, or `"unknown"` using 4 signals:
1. **Host registry** — ~18 known LLM API hostnames (OpenAI, Anthropic, Gemini, Groq, Mistral, Together, Fireworks, Cohere, Perplexity, Hugging Face, OpenRouter, xAI, Moonshot, etc.)
2. **Path patterns** — `/v1/chat/completions`, `/v1/messages`, `/generateContent`, `/v1/responses`, etc.
3. **Body shape** — detects `messages[]` (OpenAI/Claude), `contents[]` (Gemini), `prompt`, `input` fields
4. **User-agent hints** — `codex`, `claude`, `gemini`, `antigravity`, `kiro`, `copilot`, `cursor` in UA string
Custom hosts added via Mode 2 inherit their `kind` from the form input (defaults to `"custom"`).
### 4.2 SSE merger (`src/mitm/inspector/sseMerger.ts`)
**MIT port from [chouzz/llm-interceptor](https://github.com/chouzz/llm-interceptor)**
Reconstructs the final assistant message from raw SSE delta events:
- **Anthropic**: accumulates `content_block_delta` by index; handles `text_delta`, `input_json_delta` (tool calls), `thinking_delta`
- **OpenAI**: accumulates `choices[i].delta.content` and `tool_calls` by index
- **Gemini**: accumulates `candidates[i].content.parts`
- **Unknown**: returns raw events as-is
The Response tab shows a toggle: **"Raw events ↔ Merged"**.
### 4.3 Conversation normalizer (`src/mitm/inspector/conversationNormalizer.ts`)
**MIT port from [chouzz/llm-interceptor](https://github.com/chouzz/llm-interceptor)**
Converts OpenAI, Anthropic, and Gemini message formats to a single `NormalizedConversation` before rendering:
```ts
interface NormalizedConversation {
request: NormalizedTurn[]; // messages / contents / prompt from request body
response: NormalizedTurn[]; // assistant response (merged via sseMerger)
contextKey: string | null; // SHA-256 system-prompt fingerprint
}
```
Block types: `text`, `tool_use`, `tool_result`. The Conversation tab uses this shape regardless of provider.
### 4.4 Context key colorization (`src/mitm/inspector/contextKey.ts`)
- Computes `SHA-256` of the system prompt (first `role:system` message, or `system` field, or Gemini `systemInstruction`)
- Returns a 12-character hex prefix (`"a3f9c2..."`)
- Frontend maps the key to a deterministic HSL color for the left-border bar
- **Filtro "same context"**: clicking the `ctx #a3f` chip adds a filter to show only requests with the same fingerprint
This makes it easy to visually distinguish different "personas" or tasks running in the same agent session.
### 4.5 LLM metadata extraction
For LLM requests, the LLM Details tab extracts:
```ts
interface LlmMetadata {
provider: string | null; // "openai" | "anthropic" | "gemini" | ...
apiKind: string | null; // "chat.completions" | "messages" | "embeddings" | ...
model: string | null; // from request body or response
messages: number; // turn count
tokensIn: number | null; // usage.prompt_tokens / usage.input_tokens
tokensOut: number | null; // usage.completion_tokens / usage.output_tokens
streamed: boolean; // true if SSE response
mappedTo: string | null; // x-omniroute-mapped header
costEstimateUsd: number | null; // estimated cost based on OmniRoute pricing
}
```
### 4.6 Live in-flight request filter
The request `status` field is `number | "in-flight" | "error"` — an entry is
pushed as `"in-flight"` the moment the request starts and **updated in place**
when the response (or error) arrives. The toolbar's **"Live"** toggle
(`liveOnly`, i18n key `trafficInspector.liveOnly`) restricts the list to entries
whose `status === "in-flight"`, letting you watch open connections in real time.
The filter is a pure, client-side predicate in
`src/lib/inspector/matchesTrafficFilter.ts`:
```ts
if (f.liveOnly && req.status !== "in-flight") return false;
```
The toggle state lives in `useTrafficFilters` (the inspector dashboard hooks) and
combines with the other filters (profile, host, agent, source, status, context).
### 4.7 Process attribution (Linux)
On Linux, each intercepted request can be attributed to the **originating local
process**. Two optional fields are added to `InterceptedRequest`:
```ts
pid?: number; // originating process id (Linux only)
processName?: string; // originating process name (Linux only)
```
`src/mitm/inspector/processAttribution.ts` maps the connection's _client_
ephemeral port to a PID + name by:
1. Reading `/proc/net/tcp` and `/proc/net/tcp6` to find the socket inode for the
port (`parseProcNetTcpForInode`, a pure fixture-testable parser).
2. Scanning `/proc/<pid>/fd/` for a symlink to `socket:[<inode>]`.
3. Reading the process name from `/proc/<pid>/comm`.
A 1-second TTL cache bounds the procfs scan cost under load. Attribution is
**best-effort** — any failure resolves to `null` and never blocks capture. On
macOS/Windows the function returns `null` (stub; `lsof`/`GetExtendedTcpTable`
support is a follow-up).
---
## §5 Sessions
### 5.1 Recording a session
1. Click **"● Record session"** in the toolbar → enter a name (optional)
2. Live tail continues normally; a red pulsing indicator shows `◉ REC · <name> · 00:42 · 23 reqs`
3. Click **"⏹ Stop"** → the session snapshot is saved to `inspector_sessions` + `inspector_session_requests`
### 5.2 Viewing a recorded session
The **Sessions** dropdown in the toolbar lists saved sessions. Selecting one:
- Loads the session's snapshot (frozen state)
- A banner shows: `Viewing recorded session "<name>" — [Back to live]`
- The Stats tab becomes available with Recharts aggregates
### 5.3 Export formats
Each session can be exported as:
| Format | Use |
| -------------------------- | ------------------------------------------------------------------------------- |
| **HAR** (HTTP Archive 1.2) | Compatible with Chrome DevTools, Charles, Fiddler — import for offline analysis |
| **JSONL** | One `InterceptedRequest` per line — compatible with `llm-interceptor` format |
Export via `GET /api/tools/traffic-inspector/sessions/{id}/export.har` or the ⬇ button in the Sessions dropdown.
---
## §6 Security
Traffic Inspector shows **all intercepted HTTPS traffic**, including authorization headers and request bodies. The following controls are in place:
| Control | Details |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| **LOCAL_ONLY** | All routes and the WebSocket endpoint are loopback-only (enforced in `routeGuard.ts` before auth) |
| **Secret masking** | `maskSecrets()` applied to all headers and bodies before `TrafficBuffer.push()` — enabled by default (`INSPECTOR_MASK_SECRETS=true`) |
| **Body size cap** | Bodies > `INSPECTOR_MAX_BODY_KB` (default 1024 KB) are truncated with `"(truncated for performance)"` notice |
| **Sensitive header masking** | `authorization`, `cookie`, `api-key`, `x-api-key`, `proxy-authorization` → `Bearer ***` in Headers tab; "Show secrets" toggle |
| **CSP** | Strict Content Security Policy on Traffic Inspector pages to prevent XSS via injected response bodies |
| **No persistence by default** | The `TrafficBuffer` is in-memory and lost on server restart. Sessions are persisted only when explicitly recorded |
### Hard Rules applied
| Rule | Application |
| --------------------------------- | ------------------------------------------------------------------------------------- |
| **#12** `sanitizeErrorMessage` | All HTTP error responses from Traffic Inspector routes are sanitized |
| **#15 + #17** `isLocalOnlyPath()` | `/api/tools/traffic-inspector/` is LOCAL_ONLY + SPAWN_CAPABLE (system proxy commands) |
### Known limitations
- **System-wide proxy mode** affects all applications on the machine, including VPN clients and SSO. Always use with the auto-disable timer. Do not use on shared machines.
- **CONNECT tunnel HTTPS**: Mode 3 (HTTP_PROXY) captures only tunnel metadata for HTTPS destinations unless TLS interception is enabled. This is by design — transparent capture without the AgentBridge cert being trusted would break TLS verification for those apps.
- **Hardcoded strings in some components**: Some UI components (F7/F8) have a small number of hardcoded strings not yet covered by i18n keys. These are documented as a Known Limitation in the i18n gap report; they will be migrated in a follow-up pass. Affected strings are UI decorative labels that don't require translation for functional use.
---
## §7 Troubleshooting
### WebSocket disconnection
If the live tail shows "Disconnected":
1. Check the server is still running: `GET /api/tools/traffic-inspector/capture-modes`
2. Reload the page — the WebSocket reconnects and receives a fresh snapshot
3. If the server was restarted, the in-memory buffer was cleared — old entries are gone unless a session was recorded
### Port 8080 conflict
If HTTP_PROXY mode fails to start:
```bash
lsof -i :8080 # find the process
```
Change the port:
```bash
# .env
INSPECTOR_HTTP_PROXY_PORT=8888
```
### System proxy not reverted
If OmniRoute crashes while system-wide proxy mode is active:
**macOS:**
```bash
networksetup -setwebproxystate Wi-Fi off
networksetup -setsecurewebproxystate Wi-Fi off
```
**Linux (GNOME):**
```bash
gsettings set org.gnome.system.proxy mode 'none'
```
**Windows:**
```cmd
netsh winhttp reset proxy
```
The dashboard will also offer "Revert system proxy" on next load if it detects the DB state indicates proxy was active.
### Buffer full
When the buffer reaches `INSPECTOR_BUFFER_SIZE` (default 1000), new entries rotate out the oldest. If important requests are being lost:
- Increase `INSPECTOR_BUFFER_SIZE` (e.g., 5000) — trades memory for retention
- Record a session to persist the relevant window to DB
---
## §8 API reference
All routes are `LOCAL_ONLY` (loopback-only) and `SPAWN_CAPABLE` (system proxy commands). See `src/server/authz/routeGuard.ts`.
Base path: `/api/tools/traffic-inspector/`
### Request management
| Method | Path | Description |
| ------ | --------------------------- | ---------------------------------------------------------------------------------- |
| GET | `/requests` | List requests (filterable: `?profile=llm&host=&agent=&status=&source=&sessionId=`) |
| GET | `/requests/{id}` | Single request details |
| DELETE | `/requests` | Clear the in-memory buffer |
| POST | `/requests/{id}/replay` | Re-execute the same request through OmniRoute router |
| PUT | `/requests/{id}/annotation` | Save or update a note on a request |
### WebSocket
| Method | Path | Description |
| ------ | ----- | -------------------------------------------------------------------------------------- |
| GET | `/ws` | Live WebSocket stream. Sends `snapshot` on connect, then `new`/`update`/`clear` events |
### Export
| Method | Path | Description |
| ------ | ------------- | --------------------------------------- |
| GET | `/export.har` | Export current filtered list as HAR 1.2 |
### Custom hosts
| Method | Path | Description |
| ------ | --------------- | ---------------------------------- |
| GET | `/hosts` | List custom hosts |
| POST | `/hosts` | Add host (auto-edits `/etc/hosts`) |
| DELETE | `/hosts/{host}` | Remove host |
| PATCH | `/hosts/{host}` | Toggle `enabled` |
### Capture modes
| Method | Path | Description |
| ------ | ------------------------------ | ------------------------------------------------------------------------------------------------------ |
| GET | `/capture-modes` | State of the AgentBridge / custom-hosts / HTTP_PROXY / system-proxy modes + the `tls-intercept` toggle |
| POST | `/capture-modes/http-proxy` | Start/stop HTTP_PROXY listener (`{action: "start"\|"stop"}`) |
| POST | `/capture-modes/system-proxy` | Apply/revert system-wide proxy (`{action: "apply"\|"revert"}`) |
| POST | `/capture-modes/tls-intercept` | Toggle HTTPS body decryption in proxy mode (`{enabled: boolean}`) |
> **TPROXY decrypt** (capture mode 5) is driven by a **separate** route under the
> AgentBridge prefix — `GET / POST / DELETE /api/tools/agent-bridge/tproxy` — not
> under `/api/tools/traffic-inspector/`. See
> [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md).
### Sessions
| Method | Path | Description |
| ------ | --------------------------- | ------------------------------------------------------------ |
| POST | `/sessions` | Start recording (`{name?: string}`) |
| PATCH | `/sessions/{id}` | Stop or rename (`{action: "stop"\|"rename", name?: string}`) |
| GET | `/sessions` | List all saved sessions |
| GET | `/sessions/{id}` | Session snapshot (all requests) |
| DELETE | `/sessions/{id}` | Delete session |
| GET | `/sessions/{id}/export.har` | Export session as HAR 1.2 |
### Internal ingest (D4 fallback)
| Method | Path | Description |
| ------ | ------------------ | ----------------------------------------------------------------------------------------------------------------- |
| POST | `/internal/ingest` | Accepts intercepted request from `server.cjs` passthrough path; requires `INSPECTOR_INTERNAL_INGEST_TOKEN` header |
Full OpenAPI schemas: `docs/openapi.yaml` → tag `Traffic Inspector`.
docs/frameworks/WEBHOOKS.md
+259
View File
@@ -0,0 +1,259 @@
---
title: "Webhooks"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Webhooks
> **Source of truth:** `src/lib/webhookDispatcher.ts`, `src/lib/db/webhooks.ts`, `src/app/api/webhooks/`
> **Last updated:** 2026-06-28 — v3.8.40
OmniRoute can fire HTTP webhooks on platform events. Use them to integrate with
Slack, PagerDuty, Datadog, internal alerting services, or any HTTP receiver.
The dispatcher signs each delivery with HMAC-SHA256, retries on transient
failures, tracks delivery health per webhook, and auto-disables endpoints that
keep failing.
## Supported Events
The `WebhookEvent` type (`src/lib/webhookDispatcher.ts`) currently models:
| Event | Fires when |
| -------------------- | --------------------------------------------------------- |
| `request.completed` | A proxied request completes successfully |
| `request.failed` | A proxied request fails after all retries/fallback |
| `provider.error` | A provider returns an error eligible for circuit-breaking |
| `provider.recovered` | A previously failing provider returns to a healthy state |
| `quota.exceeded` | An API key crosses a budget/quota threshold |
| `combo.switched` | A combo strategy switches its primary target |
| `test.ping` | Synthetic event used by the test endpoint |
Subscriptions accept the literal `"*"` to receive every event. Unknown event
names in `events` are ignored at dispatch time.
> Note: the dispatcher API is wired, but production call sites for some of the
> non-`test.ping` events are still landing. Check `grep dispatchEvent` to see
> which paths currently invoke the dispatcher in your release.
## Architecture
```
Caller (handler, service, monitor)
dispatchEvent(event, data) [src/lib/webhookDispatcher.ts]
-> getEnabledWebhooks() [src/lib/db/webhooks.ts]
-> filter by webhook.events
-> for each match (in parallel):
deliverWebhook(url, payload, secret)
build payload { event, timestamp, data }
sign body with HMAC-SHA256 (if secret present)
POST with 10s timeout
retry up to 3 times on 5xx / network error
recordWebhookDelivery(id, status, success)
-> disableWebhooksWithHighFailures(10)
```
Dispatch is fire-and-forget for the caller: `Promise.allSettled` swallows
per-webhook errors so one bad receiver cannot block the others.
## HMAC Signing
When a webhook has a `secret`, OmniRoute signs the JSON body and sends:
```
Content-Type: application/json
User-Agent: OmniRoute-Webhook/1.0
X-Webhook-Event: <event>
X-Webhook-Timestamp: <ISO-8601>
X-Webhook-Signature: sha256=<hex HMAC-SHA256(secret, body)>
```
> Header names use the `X-Webhook-*` prefix (not `X-OmniRoute-*`). The signature
> value is `sha256=<hex>` — verify the full prefix.
If `createWebhook` is called without a secret, the DB module generates one
(`whsec_<48 hex>`) so all webhooks are signed by default.
### Verifying on the receiver
```typescript
import { createHmac, timingSafeEqual } from "node:crypto";
function verify(rawBody: string, signature: string, secret: string) {
const expected = "sha256=" + createHmac("sha256", secret).update(rawBody).digest("hex");
const a = Buffer.from(expected);
const b = Buffer.from(signature);
return a.length === b.length && timingSafeEqual(a, b);
}
```
Always verify against the **raw** request body, before any JSON parsing.
## Retry & Failure Policy
`deliverWebhook(url, payload, secret, maxRetries = 3)`:
- 10 second timeout per attempt (`AbortController`).
- HTTP 2xx counts as success.
- HTTP 3xx/4xx counts as a non-retryable final status — recorded as delivered
with `success = res.ok`.
- HTTP 5xx and network errors are retried with exponential backoff:
`2^attempt * 1000 ms` (1s, 2s, 4s).
- After `maxRetries`, the delivery is recorded as failed.
- Each delivery updates `last_triggered_at`, `last_status`, and either resets
or increments `failure_count`.
- The dispatcher calls `disableWebhooksWithHighFailures(10)` after each fan-out,
so any webhook with `failure_count >= 10` is automatically disabled.
## Database
Table `webhooks` (migration `011_webhooks.sql`):
| Column | Type | Notes |
| ------------------- | ------- | --------------------------------------------- |
| `id` | TEXT PK | UUID |
| `url` | TEXT | Destination URL |
| `events` | TEXT | JSON array; default `["*"]` |
| `secret` | TEXT | HMAC secret (auto-generated if not given) |
| `enabled` | INT | 0/1; defaults to 1 |
| `description` | TEXT | Optional human label |
| `created_at` | TEXT | `datetime('now')` |
| `last_triggered_at` | TEXT | Updated on every delivery attempt |
| `last_status` | INT | HTTP status of the last attempt (0 = network) |
| `failure_count` | INT | Resets to 0 on success, +1 on failure |
There is **no separate `webhook_deliveries` table** in the current schema —
delivery history is aggregated on the `webhooks` row. If you need full audit
history, consume `request.completed` / `audit` style events from a downstream
log store.
## REST API
All endpoints require management auth (`requireManagementAuth`).
| Endpoint | Method | Description |
| ------------------------- | ------ | ------------------------------- |
| `/api/webhooks` | GET | List webhooks (secrets masked) |
| `/api/webhooks` | POST | Create webhook |
| `/api/webhooks/[id]` | GET | Webhook detail (full secret) |
| `/api/webhooks/[id]` | PUT | Update fields |
| `/api/webhooks/[id]` | DELETE | Remove |
| `/api/webhooks/[id]/test` | POST | Fire a `test.ping` (no retries) |
`GET /api/webhooks` masks the secret to `<first 10 chars>...` to avoid leaking
on listing pages. Use the `[id]` GET when you actually need the secret.
### Create webhook
```bash
curl -X POST http://localhost:20128/api/webhooks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"url": "https://hooks.slack.com/services/...",
"secret": "whsec_my_shared_secret",
"events": ["quota.exceeded", "provider.error"],
"description": "Slack alerts"
}'
```
If `secret` is omitted, the server generates a `whsec_<hex>` secret and returns
it in the response.
### Test webhook
```bash
curl -X POST http://localhost:20128/api/webhooks/<id>/test \
-H "Cookie: auth_token=..."
```
Returns `{ delivered, status, error }`. No retries are attempted — useful for
quickly validating that the receiver accepts the payload and signature.
## Dashboard
The dashboard page at `/dashboard/webhooks` (see
`src/app/(dashboard)/dashboard/webhooks/page.tsx`) provides:
- Create/edit webhooks with an event picker
- Status indicator (active / inactive / errored) based on `enabled`,
`failure_count`, and `last_status`
- One-click test delivery
- Manual enable/disable toggle
## Payload Examples
### request.completed
```json
{
"event": "request.completed",
"timestamp": "2026-05-13T20:30:00.123Z",
"data": {
"trace_id": "...",
"api_key_id": "...",
"provider": "openai",
"model": "gpt-5",
"status": 200,
"tokens_in": 142,
"tokens_out": 350,
"cost_usd": 0.0042
}
}
```
### provider.error
```json
{
"event": "provider.error",
"timestamp": "2026-05-13T20:31:00.000Z",
"data": {
"provider": "anthropic",
"status": 503,
"consecutive_failures": 5,
"circuit_state": "open"
}
}
```
### test.ping
```json
{
"event": "test.ping",
"timestamp": "2026-05-13T20:32:00.000Z",
"data": {
"message": "Test webhook delivery from OmniRoute",
"webhookId": "<uuid>"
}
}
```
Field shapes for non-`test.ping` events are defined by the call sites that emit
them; treat the `data` object as forward-compatible (add fields, don't depend on
absence).
## Best Practices
- **Verify the signature on every delivery** against the raw body — prevents
spoofed POSTs from anyone who guesses your webhook URL.
- **Respond 2xx within ~5 seconds** — the dispatcher times out at 10 s. Slow
receivers will eat retries and inflate `failure_count`.
- **Make handlers idempotent** — retries and at-least-once delivery semantics
mean duplicates are possible.
- **Subscribe minimally** — list only events you actually consume; `"*"` will
add cost on receivers you do not control.
- **Watch `failure_count`** — endpoints are auto-disabled at 10 consecutive
failures; reset by calling `PUT /api/webhooks/[id]` with `enabled: true`
after fixing the receiver.
- **Rotate secrets periodically** — `PUT` a new `secret`, deploy the new value
to the receiver, and confirm via the test endpoint.
## See Also
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — full management API surface
- [RESILIENCE_GUIDE.md](../architecture/RESILIENCE_GUIDE.md) — circuit breaker / cooldown
semantics that drive `provider.error` / `provider.recovered`
- Source: `src/lib/webhookDispatcher.ts`, `src/lib/db/webhooks.ts`
docs/frameworks/meta.json
+21
View File
@@ -0,0 +1,21 @@
{
"title": "Frameworks",
"pages": [
"MCP-SERVER",
"A2A-SERVER",
"AGENT_PROTOCOLS_GUIDE",
"ACP",
"CLOUD_AGENT",
"EVALS",
"GAMIFICATION",
"MEMORY",
"NOTION_CONTEXT",
"OBSIDIAN_CONTEXT",
"OPENCODE",
"PLUGIN_MARKETPLACE",
"PLUGINS",
"PLUGIN_SDK",
"SKILLS",
"WEBHOOKS"
]
}