wehub/diegosouzapw--omniroute
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:12 +08:00
commit d8dcd5f6d1
8604 changed files with 2479390 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/API_REFERENCE.md
+1378
View File
File diff suppressed because it is too large Load Diff
docs/reference/CLI-TOOLS.md
+741
View File
@@ -0,0 +1,741 @@
---
title: "CLI Tools — OmniRoute"
version: 3.8.40
lastUpdated: 2026-06-28
---
# CLI Tools — OmniRoute
Last updated: 2026-06-28
OmniRoute integrates with three categories of CLI tools spread across three dedicated dashboard pages:
| Page | Route | Concept | Count |
| -------------- | ----------------------- | ------------------------------------------------------------------------- | ------------ |
| **CLI Code's** | `/dashboard/cli-code` | Coding tools you point at OmniRoute (Client → CLI → OmniRoute → Provider) | 20 |
| **CLI Agents** | `/dashboard/cli-agents` | Autonomous agents you point at OmniRoute (same flow, broader scope) | 6 |
| **ACP Agents** | `/dashboard/acp-agents` | CLIs that OmniRoute spawns as backend via stdio/ACP (reverse flow) | see registry |
Legacy routes redirect via 308: `/dashboard/cli-tools` → `/dashboard/cli-code`, `/dashboard/agents` → `/dashboard/acp-agents`.
---
## How It Works
```
CLI Code's / CLI Agents (consumption flow):
Claude / Codex / OpenCode / Cline / KiloCode / Continue / Hermes Agent / Goose / ...
│
▼ (all point to OmniRoute)
http://YOUR_SERVER:20128/v1
│
▼ (OmniRoute routes to the right provider)
Anthropic / OpenAI / Gemini / DeepSeek / Groq / Mistral / ...
ACP Agents (reverse spawn flow):
Client request → OmniRoute → spawns CLI via stdio/ACP → response
```
**Benefits:**
- One API key to manage all tools
- Cost tracking across all CLIs in the dashboard
- Model switching without reconfiguring every tool
- Works locally and on remote servers (VPS, Docker, Akamai, Cloudflare Tunnel)
---
## Auto-configure with `setup-*`
You do not have to write each tool's config by hand. OmniRoute ships a `setup-*`
command per supported CLI that reads the **live** model catalog from a running
OmniRoute (local or remote) and writes the tool's own config on your machine:
```bash
omniroute setup-codex omniroute setup-claude omniroute setup-opencode
omniroute setup-cline omniroute setup-kilo omniroute setup-continue
omniroute setup-cursor omniroute setup-roo omniroute setup-crush
omniroute setup-goose omniroute setup-qwen omniroute setup-aider
```
Each accepts `--remote <url> --api-key <key>` (configure a local tool against a
remote OmniRoute), `--dry-run` (preview without writing), and `--port`. Tools
without model auto-discovery (Cline, Kilo, Roo, Goose, Qwen, Aider, Gemini) take
`--model <id>` (and `--yes` for non-interactive runs). The launchers
`omniroute launch` (Claude Code) and `omniroute launch-codex` (Codex) spawn the CLI
with the right env injected and write no config at all.
> **Full reference:** the master table — what each command writes, every flag,
> local vs remote, and which tools want a `/v1` suffix — lives in
> **[CLI Integrations](../guides/CLI-INTEGRATIONS.md)**.
---
## Source of Truth
The unified catalog lives in `src/shared/constants/cliTools.ts` as `CLI_TOOLS: Record<string, CliCatalogEntry>`.
Each entry has these fields (defined in `src/shared/schemas/cliCatalog.ts`):
| Field | Type | Description |
| ----------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------ |
| `category` | `"code" \| "agent"` | Which page the tool appears on |
| `vendor` | `string` | Tool origin ("Anthropic", "OSS (P. Gauthier)") |
| `acpSpawnable` | `boolean` | Also usable as an ACP Agent (badge shown) |
| `baseUrlSupport` | `"full" \| "partial" \| "none"` | Custom endpoint support level. `"none"` = MITM backlog |
| `configType` | `"env" \| "custom" \| "guide" \| "custom-builder" \| "mitm"` | Configuration mechanism |
| `id`, `name`, `color`, `description`, `docsUrl` | standard | Core display fields |
Entries with `baseUrlSupport: "none"` are **not shown** in the dashboard pages — they are registered in the MITM backlog for plan 11 (see `_tasks/features-v3.8.6/refactorpages/_orchestration/_plan11-mitm-backlog.md`).
---
## 1. CLI Code's Catalog (20 tools)
Tools that support custom base URL and appear in `/dashboard/cli-code`:
| id | name | vendor | baseUrlSupport | configType | acpSpawnable |
|----|------|--------|---------------|-----------|-------------|
| claude | Claude Code | Anthropic | full | env | true |
| codex | OpenAI Codex CLI | OpenAI | full | custom | true |
| cline | Cline | OSS (ex-Claude Dev) | full | custom | true |
| kilo | Kilo Code | Kilo-Org | full | custom | false |
| roo | Roo Code | Roo (OSS) | full | guide | false |
| continue | Continue | continue.dev | full | guide | false |
| qwen | Qwen Code | Alibaba | full | guide | true |
| aider | Aider | OSS (P. Gauthier) | full | guide | true |
| forge | ForgeCode | Antinomy HQ | full | custom | true |
| jcode | jcode | 1jehuang (OSS) | full | custom | false |
| deepseek-tui | DeepSeek TUI | Hunter Bown (OSS) | full | custom | false |
| codewhale | CodeWhale | Hmbown (OSS) | full | custom | false |
| opencode | OpenCode | Anomaly (ex-SST) | full | guide | true |
| droid | Factory Droid | Factory AI | partial | guide | false |
| copilot | GitHub Copilot CLI | GitHub/MS | full | custom | false |
| cursor-cli | Cursor CLI | Anysphere | partial | guide | true |
| smelt | Smelt | leonardcser (OSS) | full | custom | false |
| pi | Pi (pi-coding-agent) | M. Zechner (OSS) | full | custom | false |
| custom | Custom CLI | — | full | custom-builder | false |
Tools with `baseUrlSupport: "partial"` show a badge "⚠ Base URL parcial" in the dashboard card.
---
## 2. CLI Agents Catalog (6 tools)
Autonomous agents that appear in `/dashboard/cli-agents`:
| id | name | vendor | baseUrlSupport | acpSpawnable |
| ------------ | ---------------- | ------------------------ | -------------- | ------------ |
| hermes-agent | Hermes Agent | Nous Research | full | false |
| openclaw | OpenClaw | OSS (P. Steinberger) | full | true |
| goose | Goose | Block / Linux Foundation | full | true |
| interpreter | Open Interpreter | OSS | full | true |
| warp | Warp AI | Warp Inc. | partial | true |
| agent-deck | Agent Deck | asheshgoplani (OSS) | full | false |
---
## 3. ACP Agents (/dashboard/acp-agents)
This page (renamed from `/dashboard/agents`) shows CLIs that OmniRoute can **spawn** as backend execution engines via stdio/ACP protocol. The catalog is maintained separately in `src/lib/acp/registry.ts` and is **not** the same as `CLI_TOOLS`.
---
## 4. MITM Backlog (not shown in dashboard)
The following CLIs do not support custom base URL natively and are **not listed** in CLI Code's or CLI Agents pages. They are candidates for MITM interception in plan 11:
| CLI | Reason |
| ------------------- | ---------------------------------------------------------- |
| windsurf | BYOK limited to select Claude models + corporate URL/token |
| amp | Closed ecosystem (Sourcegraph) |
| amazon-q / kiro-cli | AWS SSO auth, no custom URL |
| cowork | Anthropic Desktop, no configurable endpoint |
See `_tasks/features-v3.8.6/refactorpages/_orchestration/_plan11-mitm-backlog.md` for the full cross-reference.
---
## 5. Batch Detection API
All tool detection is aggregated via a single endpoint:
**`GET /api/cli-tools/all-statuses`**
- Auth: `requireCliToolsAuth(request)` (same as other `/api/cli-tools/` routes)
- Returns: `Record<toolId, ToolBatchStatus>` (type: `src/shared/types/cliBatchStatus.ts`)
- Strategy: `Promise.all` over all tools, 5s timeout per tool
- Cache: in-memory LRU indexed by config file `mtime`. Cache invalidated when mtime changes. Reset on server restart.
Response shape per tool:
```ts
interface ToolBatchStatus {
detection: {
installed: boolean;
runnable: boolean;
version?: string;
command?: string;
commandPath?: string;
reason?: string;
};
config: {
status: "configured" | "not_configured" | "not_installed" | "unknown" | "other";
endpoint?: string | null;
lastConfiguredAt?: string | null;
};
error?: string; // sanitized, no stack traces
}
```
---
## 6. Settings Handlers for New Tools
New tools with `configType: "custom"` have dedicated settings API routes:
| Route | Tool |
| ------------------------------------------- | ------------------------------ |
| `POST /api/cli-tools/forge-settings` | ForgeCode (.forge.toml) |
| `POST /api/cli-tools/jcode-settings` | jcode (--base-url flag) |
| `POST /api/cli-tools/deepseek-tui-settings` | DeepSeek TUI (OPENAI_BASE_URL, legacy) |
| `POST /api/cli-tools/codewhale-settings` | CodeWhale (OPENAI_BASE_URL, primary + legacy `~/.deepseek` sync) |
| `POST /api/cli-tools/smelt-settings` | Smelt |
| `POST /api/cli-tools/pi-settings` | Pi coding agent |
All routes use `sanitizeErrorMessage()` for error responses (Hard Rule #12).
---
## 7. Dashboard Pages Architecture
### CLI Code's (`/dashboard/cli-code`)
- `src/app/(dashboard)/dashboard/cli-code/page.tsx` — server component
- `src/app/(dashboard)/dashboard/cli-code/CliCodePageClient.tsx` — client grid
- `src/app/(dashboard)/dashboard/cli-code/[id]/page.tsx` — tool detail page
- `src/app/(dashboard)/dashboard/cli-code/components/` — 12 specialized tool cards + `ToolDetailClient.tsx`
### CLI Agents (`/dashboard/cli-agents`)
- `src/app/(dashboard)/dashboard/cli-agents/page.tsx` — server component
- `src/app/(dashboard)/dashboard/cli-agents/CliAgentsPageClient.tsx` — client grid
- `src/app/(dashboard)/dashboard/cli-agents/[id]/page.tsx` — reuses `ToolDetailClient`
### ACP Agents (`/dashboard/acp-agents`)
- `src/app/(dashboard)/dashboard/acp-agents/page.tsx` — server component (moved from `agents/`)
### Shared UI Components (`src/shared/components/cli/`)
| File | Purpose |
| ----------------------- | ------------------------------------------------- |
| `CliToolCard.tsx` | Smart status card (detection + config + endpoint) |
| `CliConceptCard.tsx` | Per-page concept explanation card |
| `CliComparisonCard.tsx` | Three-column comparison across CLI types |
| `BaseUrlSelect.tsx` | Endpoint dropdown (Local/Cloud/Custom) |
| `ApiKeySelect.tsx` | API key selector |
| `ManualConfigModal.tsx` | Copiable config snippet modal |
### Shared Hook (`src/shared/hooks/cli/`)
| File | Purpose |
| ------------------------- | -------------------------------------------------------------------- |
| `useToolBatchStatuses.ts` | Fetches `/api/cli-tools/all-statuses`, manages loading/refresh state |
---
## 8. i18n
New namespaces added in plan 14 F9:
| Namespace | Purpose |
| ----------- | -------------------------------------------------------------------------- |
| `cliCommon` | Shared strings (card labels, concept/comparison texts, detail page labels) |
| `cliCode` | CLI Code's page strings |
| `cliAgents` | CLI Agents page strings |
| `acpAgents` | ACP Agents page strings |
Full PT-BR and EN translations are provided. 39 other locales fall back to EN automatically via namespace-level merge in `src/i18n/request.ts`.
---
## 9. Quick Start
### Step 1 — Get an OmniRoute API Key
1. Open `/dashboard/api-manager` → **Create API Key**
2. Give it a name (e.g. `cli-tools`) and select all permissions
3. Copy the key — you'll need it for every CLI below
> Your key looks like: `sk-xxxxxxxxxxxxxxxx-xxxxxxxxx`
---
### Step 2 — Install CLI Tools
All npm-based tools require Node.js 22.22.2+ or 24.x:
```bash
# Claude Code (Anthropic)
npm install -g @anthropic-ai/claude-code
# OpenAI Codex
npm install -g @openai/codex
# OpenCode
npm install -g opencode-ai
# Cline
npm install -g cline
# KiloCode
npm install -g kilocode
# Qwen Code (Alibaba)
npm install -g @qwen-code/qwen-code
# Aider
pip install aider-chat
# Smelt
cargo install smelt # Rust-based
# Pi coding agent
# see https://github.com/zechnerj/pi-coding-agent for install
# jcode
# see https://github.com/1jehuang/jcode for install
```
---
### Step 3 — Configure via Dashboard
1. Go to `http://localhost:20128/dashboard/cli-code`
2. Find your tool in the grid
3. Click the card to open the tool detail page
4. Select your API key and base URL
5. Click **Apply Config** or copy the manual config snippet
---
### Step 4 — Set Global Environment Variables
```bash
# OmniRoute Universal Endpoint
export OPENAI_BASE_URL="http://localhost:20128/v1"
export OPENAI_API_KEY="sk-your-omniroute-key"
export ANTHROPIC_BASE_URL="http://localhost:20128"
export ANTHROPIC_AUTH_TOKEN="sk-your-omniroute-key"
export GEMINI_BASE_URL="http://localhost:20128/v1"
export GEMINI_API_KEY="sk-your-omniroute-key"
```
> For a **remote server** replace `localhost:20128` with the server IP or domain,
> e.g. `http://<your-server-ip>:20128`.
---
### Step 4 — Configure Each Tool
#### Claude Code
```bash
# Create ~/.claude/settings.json:
mkdir -p ~/.claude && cat > ~/.claude/settings.json << EOF
{
"env": {
"ANTHROPIC_BASE_URL": "http://localhost:20128",
"ANTHROPIC_AUTH_TOKEN": "sk-your-omniroute-key"
}
}
EOF
```
Use the unified Anthropic gateway root for Claude Code. Do not append `/v1` here.
**Test:** `claude "say hello"`
---
#### OpenAI Codex
```bash
mkdir -p ~/.codex && cat > ~/.codex/config.yaml << EOF
model: auto
apiKey: sk-your-omniroute-key
apiBaseUrl: http://localhost:20128/v1
EOF
```
**Test:** `codex "what is 2+2?"`
---
#### OpenCode
```bash
mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json << EOF
{
"\$schema": "https://opencode.ai/config.json",
"provider": {
"omniroute": {
"npm": "@ai-sdk/openai-compatible",
"name": "OmniRoute",
"options": {
"baseURL": "http://localhost:20128/v1",
"apiKey": "sk-your-omniroute-key"
},
"models": {
"claude-sonnet-4-5": { "name": "claude-sonnet-4-5" },
"claude-sonnet-4-5-thinking": { "name": "claude-sonnet-4-5-thinking" },
"gemini-3-flash": { "name": "gemini-3-flash" }
}
}
}
}
EOF
```
**Test:** `opencode`
> Use `opencode run "your prompt" --model omniroute/claude-sonnet-4-5-thinking --variant high`
> to send thinking variants.
---
#### Cline (CLI or VS Code)
**CLI mode:**
```bash
mkdir -p ~/.cline/data && cat > ~/.cline/data/globalState.json << EOF
{
"apiProvider": "openai",
"openAiBaseUrl": "http://localhost:20128/v1",
"openAiApiKey": "sk-your-omniroute-key"
}
EOF
```
**VS Code mode:**
Cline extension settings → API Provider: `OpenAI Compatible` → Base URL: `http://localhost:20128/v1`
Or use the OmniRoute dashboard → **CLI Tools → Cline → Apply Config**.
---
#### KiloCode (CLI or VS Code)
**CLI mode:**
```bash
kilocode --api-base http://localhost:20128/v1 --api-key sk-your-omniroute-key
```
**VS Code settings:**
```json
{
"kilo-code.openAiBaseUrl": "http://localhost:20128/v1",
"kilo-code.apiKey": "sk-your-omniroute-key"
}
```
Or use the OmniRoute dashboard → **CLI Tools → KiloCode → Apply Config**.
---
#### Continue (VS Code Extension)
Edit `~/.continue/config.yaml`:
```yaml
models:
- name: OmniRoute
provider: openai
model: auto
apiBase: http://localhost:20128/v1
apiKey: sk-your-omniroute-key
default: true
```
Restart VS Code after editing.
---
#### VS Code Insiders (`chatLanguageModels.json`)
Use this when VS Code Insiders is configured for custom endpoint models and you want OmniRoute to work without a custom header field.
**Recommended location:**
- Linux: `~/.config/Code - Insiders/User/chatLanguageModels.json`
- Windows: `%APPDATA%/Code - Insiders/User/chatLanguageModels.json`
**Example using the tokenized OmniRoute alias:**
```json
[
{
"vendor": "customendpoint",
"id": "auto",
"name": "OmniRoute Auto",
"family": "gpt-4",
"version": "1.0.0",
"url": "http://localhost:20128/api/v1/vscode/sk-your-omniroute-key/chat/completions",
"modelsUrl": "http://localhost:20128/api/v1/vscode/sk-your-omniroute-key/models",
"requestFormat": "openai-chat-completions",
"contextWindow": 256000,
"maxOutputTokens": 32768,
"auth": {
"type": "none"
}
}
]
```
**Notes:**
- Replace `sk-your-omniroute-key` with an API key created in OmniRoute.
- The `url` field should point to `/api/v1/vscode/{token}/chat/completions`.
- The `modelsUrl` field should point to `/api/v1/vscode/{token}/models`.
- Prefer the normal `/v1` + Bearer header flow when the client supports custom headers.
- URL-embedded tokens are a compatibility fallback and may appear in editor logs or proxy history.
---
#### Kiro CLI (Amazon)
```bash
# Login to your AWS/Kiro account:
kiro-cli login
# The CLI uses its own auth — OmniRoute is not needed as backend for Kiro CLI itself.
# Use kiro-cli alongside OmniRoute for other tools.
kiro-cli status
```
For the **Kiro IDE** desktop app, use the MITM endpoint exposed by OmniRoute
under `/dashboard/cli-tools → Kiro`.
---
#### Qwen Code (Alibaba)
Qwen Code supports OpenAI-compatible API endpoints via environment variables or `settings.json`.
> Qwen OAuth free tier was discontinued on 2026-04-15. Use OmniRoute with
> `bailian-coding-plan` / `alibaba` / `alibaba-cn` / `openrouter` / `anthropic` /
> `gemini` providers instead.
**Option 1: Environment variables (`~/.qwen/.env`)**
```bash
mkdir -p ~/.qwen && cat > ~/.qwen/.env << EOF
OPENAI_API_KEY="sk-your-omniroute-key"
OPENAI_BASE_URL="http://localhost:20128/v1"
OPENAI_MODEL="auto"
EOF
```
**Option 2: `settings.json` with `security.auth`**
```json
// ~/.qwen/settings.json
{
"security": {
"auth": {
"selectedType": "openai",
"apiKey": "sk-your-omniroute-key",
"baseUrl": "http://localhost:20128/v1"
}
},
"model": {
"name": "claude-sonnet-4-6"
}
}
```
**Option 3: Inline CLI flags**
```bash
OPENAI_BASE_URL="http://localhost:20128/v1" \
OPENAI_API_KEY="sk-your-omniroute-key" \
OPENAI_MODEL="auto" \
qwen
```
> For a **remote server** replace `localhost:20128` with the server IP or domain.
---
## 10. Internal OmniRoute CLI
The `omniroute` binary provides commands for server lifecycle, setup, diagnostics, and provider management. Entry point: `bin/omniroute.mjs`.
```bash
omniroute # Start server (default port 20128)
omniroute setup # Interactive setup wizard
omniroute doctor # Check config, DB, ports, runtime
omniroute providers list # Configured provider connections
omniroute providers test-all # Test every active connection
omniroute reset-password # Reset the admin password
omniroute logs # Stream request logs
omniroute health # Detailed health (breakers, cache, memory)
omniroute --version # Print version
omniroute --help # Show all commands
```
### Setup & Initialization
```bash
omniroute setup # Interactive setup wizard
omniroute setup --non-interactive # CI/automation mode (reads env vars + flags)
omniroute setup --password '<value>' # Set admin password directly
omniroute setup --add-provider \
--provider openai \
--api-key '<value>' \
--test-provider # Add and test a provider in one shot
```
Recognized environment variables for non-interactive setup:
| Var | Purpose |
| ------------------- | -------------------------------------------------------------- |
| `OMNIROUTE_API_KEY` | Provider API key (bound to `--api-key` via Commander `.env()`) |
| `DATA_DIR` | Override the OmniRoute data directory |
All other non-interactive inputs are passed as flags, not environment variables:
`--password`, `--provider`, `--provider-name`, `--provider-base-url`, `--default-model`
(see the `omniroute setup` options above).
### Diagnostics
```bash
omniroute doctor # Check config, DB, ports, runtime, memory, liveness
omniroute doctor --json # Machine-readable JSON
omniroute doctor --no-liveness # Skip the HTTP health probe
omniroute doctor --host 0.0.0.0 # Override liveness host
omniroute doctor --liveness-url <url> # Full health endpoint URL override
```
The doctor runs these checks: `Config`, `Database`, `Storage/encryption`,
`Port availability`, `Node runtime`, `Native binary` (better-sqlite3),
`Memory`, and `Server liveness`. It exits non-zero if any check is `fail`.
### Provider Management
```bash
omniroute providers available # OmniRoute provider catalog
omniroute providers available --search openai # Filter catalog by id/name/alias/category
omniroute providers available --category api-key # Filter by category (api-key, oauth, free, ...)
omniroute providers available --json # Machine-readable JSON
omniroute providers list # Configured provider connections
omniroute providers list --json
omniroute providers test <id|name> # Test one configured connection
omniroute providers test-all # Test every active connection
omniroute providers validate # Local-only structural validation
```
> `providers available` reads the OmniRoute catalog; `providers list/test/test-all/validate`
> read the local SQLite database directly and do not require the server to be running.
### Recovery & Reset
```bash
omniroute reset-password # Reset the admin password (also: omniroute-reset-password)
omniroute reset-encrypted-columns # Show warning + dry-run for encrypted credential reset
omniroute reset-encrypted-columns --force # Actually null out encrypted credentials in SQLite
```
### Other subcommands
These assume a running OmniRoute server, unless noted otherwise:
```bash
omniroute status # Comprehensive runtime status
omniroute logs # Stream request logs (--json, --search, --follow)
omniroute config show # Display current configuration
omniroute provider list # List available providers (alias of providers list)
omniroute provider add # Register OmniRoute as a provider on a tool
omniroute keys add | list | remove # Manage API keys
omniroute models [provider] # List models (--json, --search)
omniroute combo list | switch | create | delete
omniroute backup # Snapshot config + DB
omniroute restore # Restore from a previous snapshot
omniroute health # Detailed health (breakers, cache, memory)
omniroute quota # Provider quota usage
omniroute cache # Cache status
omniroute cache clear # Clear semantic + signature caches
omniroute mcp status | restart # MCP server status / restart
omniroute a2a status | card # A2A server status / agent card
omniroute tunnel list | create | stop # Manage tunnels (cloudflare/tailscale/ngrok)
omniroute env show | get <k> | set <k> <v> # Inspect / set env vars (temporary)
omniroute test # Provider connectivity smoke test
omniroute update # Check for updates
omniroute completion # Generate shell completion
```
### Common flags
| Flag | Description |
| ------------------- | ------------------------------------------------------ |
| `--no-open` | Don't auto-open the browser on start |
| `--port <n>` | Override the API port (default 20128) |
| `--mcp` | Run as MCP server over stdio (for IDEs) |
| `--non-interactive` | CI mode (no prompts; reads from env/flags) |
| `--json` | Machine-readable JSON output (doctor, providers, etc.) |
| `--help`, `-h` | Show command-specific help |
| `--version`, `-v` | Print the installed version |
---
## Available API Endpoints
| Endpoint | Description | Use For |
| -------------------------- | ----------------------------- | --------------------------- |
| `/v1/chat/completions` | Standard chat (all providers) | All modern tools |
| `/v1/responses` | Responses API (OpenAI format) | Codex, agentic workflows |
| `/v1/completions` | Legacy text completions | Older tools using `prompt:` |
| `/v1/embeddings` | Text embeddings | RAG, search |
| `/v1/images/generations` | Image generation | GPT-Image, Flux, etc. |
| `/v1/audio/speech` | Text-to-speech | ElevenLabs, OpenAI TTS |
| `/v1/audio/transcriptions` | Speech-to-text | Deepgram, AssemblyAI |
Ready-to-paste examples with a tokenized OmniRoute URL:
```txt
Token example: sk-a3ab3c080beaee3a-69f4a4-070d71af
Standard OpenAI base: http://localhost:20128/v1
VS Code models: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/models
VS Code chat: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/chat/completions
VS Code responses: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/responses
Ollama tags: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/api/tags
Ollama chat: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/api/chat
```
---
## Troubleshooting
| Error | Cause | Fix |
| -------------------------------------------- | ----------------------- | ------------------------------------------------ |
| `Connection refused` | OmniRoute not running | `omniroute serve` |
| `401 Unauthorized` | Wrong API key | Check in `/dashboard/api-manager` |
| `No combo configured` | No active routing combo | Set up in `/dashboard/combos` |
| CLI shows "not installed" | Binary not in PATH | Check `which <command>` |
| Dashboard shows "not detected" after install | Cache stale | Click "⟳ Refresh detection" in dashboard |
| Old link `/dashboard/cli-tools` | Pre-v3.8.6 bookmark | Auto-redirected to `/dashboard/cli-code` (308) |
| Old link `/dashboard/agents` | Pre-v3.8.6 bookmark | Auto-redirected to `/dashboard/acp-agents` (308) |
docs/reference/ENVIRONMENT.md
+1183
View File
File diff suppressed because it is too large Load Diff
docs/reference/FEATURE_FLAGS.md
+238
View File
@@ -0,0 +1,238 @@
---
title: "Feature Flags"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Feature Flags
> Runtime toggles that change OmniRoute's behavior **without a redeploy**.
> Every flag listed here is defined in
> [`src/shared/constants/featureFlagDefinitions.ts`](../../src/shared/constants/featureFlagDefinitions.ts)
> — the single source of truth. The dashboard and the REST API both read from
> that file, so the table below is generated to match it 1:1.
---
## What Feature Flags Are
A feature flag is a named toggle (boolean or enum) whose value can be changed at
runtime and persisted in the database, with no process redeploy required. Each
flag is described by a `FeatureFlagDefinition` with a `key`, `label`,
`description`, `category`, `defaultValue`, `type`, and a `requiresRestart` hint.
### Resolution Order
The **effective value** of a flag is resolved by
[`resolveFeatureFlag()`](../../src/shared/utils/featureFlags.ts) with this
precedence (highest wins):
1. **DB override** — a value stored in the `key_value` table under the
`feature_flags` namespace (set via the dashboard or the REST API).
2. **Environment variable** — `process.env[<KEY>]`, if set and non-empty.
3. **Definition default** — the `defaultValue` from `featureFlagDefinitions.ts`.
A boolean flag is considered **enabled** when its effective value is `"true"`,
`"1"`, or `"yes"` (see `isFeatureFlagEnabled()`).
> [!NOTE]
> Most flags also have a matching environment variable of the **same name**
> documented in [`ENVIRONMENT.md`](./ENVIRONMENT.md). The flag's DB override
> takes precedence over that environment variable. A flag with
> `requiresRestart: true` is persisted immediately but only re-read at process
> startup — toggling it surfaces a **"Restart Server"** banner in the dashboard.
---
## Flag Catalog
38 flags across 6 categories. **Default** is the definition default — the value
used when neither a DB override nor an environment variable is present.
### Security (7)
| Key | Type | Default | Description |
| -------------------------------- | ------- | -------- | ----------------------------------------------------------------------------- |
| `REQUIRE_API_KEY` | boolean | `false` | Require an API key for all incoming requests. |
| `INPUT_SANITIZER_ENABLED` | boolean | `true` | Enable input sanitization for all requests. |
| `INJECTION_GUARD_MODE` | enum | `off` | Prompt injection guard mode. Values: `off`, `warn`, `block`, `redact`. |
| `PII_REDACTION_ENABLED` | boolean | `false` | Redact personally identifiable information from requests. |
| `PII_RESPONSE_SANITIZATION` | boolean | `false` | Sanitize PII from provider responses. |
| `PII_RESPONSE_SANITIZATION_MODE` | enum | `redact` | Mode for PII response sanitization. Values: `redact`, `warn`, `block`, `off`. |
| `OUTBOUND_SSRF_GUARD_ENABLED` | boolean | `true` | Block outbound requests to private/internal IP ranges. |
### Network (8)
| Key | Type | Default | Restart | Description |
| ----------------------------------------------- | ------- | ------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENABLE_TLS_FINGERPRINT` | boolean | `false` | ✓ | Enable TLS fingerprint stealth mode. |
| `ONEPROXY_ENABLED` | boolean | `true` | | Enable 1proxy request proxying. |
| `PROXY_AUTO_SELECT_ENABLED` | boolean | `false` | | When no proxy is assigned to a connection, auto-select the first working proxy from the registry. Off by default (otherwise any registry proxy becomes a global fallback — #3332). |
| `OMNIROUTE_CONTROL_PLANE_PROXY_DIRECT_FALLBACK` | boolean | `false` | | Allow OAuth and provider validation flows to bypass a pinned proxy and connect directly when proxy reachability pre-checks fail. Off by default because this can change egress IP. |
| `MITM_DISABLE_TLS_VERIFY` | boolean | `false` | ✓ | Disable TLS certificate verification for the MITM proxy. **Danger.** |
| `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS` | boolean | `false` | | Allow provider URLs pointing to private/internal networks. |
| `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS` | boolean | `true` | | Allow adding/validating providers on local/private addresses (127.0.0.1, localhost, LAN). On by default (local-first); disable for strict public-only blocking. Cloud-metadata stays blocked. |
| `ENABLE_CC_COMPATIBLE_PROVIDER` | boolean | `false` | ✓ | Enable Claude Code compatible provider mode. |
### Policies (3)
| Key | Type | Default | Restart | Description |
| ----------------------------------------- | ------- | ---------- | ------- | ---------------------------------------------------------------------- |
| `TOOL_POLICY_MODE` | enum | `disabled` | | Tool-use policy enforcement mode. Values: `disabled`, `warn`, `block`. |
| `RATE_LIMIT_AUTO_ENABLE` | boolean | `false` | | Automatically enable rate limiting based on usage patterns. |
| `ALLOW_MULTI_CONNECTIONS_PER_COMPAT_NODE` | boolean | `false` | ✓ | Allow multiple connections per compatibility node. |
### Runtime (10)
| Key | Type | Default | Restart | Description |
| ------------------------------------------- | ------- | ------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_MCP_ENFORCE_SCOPES` | boolean | `true` | | Enforce scope restrictions on MCP tool access. |
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | boolean | `false` | | Compress MCP tool descriptions to reduce token usage. |
| `OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS` | boolean | `false` | | Enable background task processing at runtime. |
| `OMNIROUTE_DISABLE_BACKGROUND_SERVICES` | boolean | `false` | ✓ | Disable all background services (quota refresh, sync, etc). |
| `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS` | boolean | `false` | | Trust project-level RTK filters without validation. |
| `OMNIROUTE_ENABLE_LIVE_WS` | boolean | `true` | ✓ | Start the real-time dashboard WebSocket server on import (port 20129 by default). |
| `OMNIROUTE_CODEX_WS_ENABLED` | boolean | `true` | | Allow Codex to use the Responses-over-WebSocket transport. When off, Codex falls back to HTTP Responses. |
| `OMNIROUTE_EMERGENCY_FALLBACK` | boolean | `true` | | Route budget-exhausted requests to the emergency free fallback provider/model. (See [Emergency Budget Fallback](#emergency-budget-fallback) below.) |
| `MODEL_CATALOG_INCLUDE_NAMES` | boolean | `true` | | Include display-friendly name fields in `/v1/models` responses. Disable for clients that expect model IDs only. |
| `ARENA_ELO_SYNC_ENABLED` | boolean | `true` | | Enable periodic Arena AI leaderboard ELO sync for model intelligence rankings. |
### CLI (3)
| Key | Type | Default | Restart | Description |
| ---------------------------- | ------- | ------- | ------- | -------------------------------------------------------------------------------------------------------------- |
| `CLI_COMPAT_ALL` | boolean | `false` | ✓ | Enable compatibility mode for all CLI clients. |
| `MODEL_ALIAS_COMPAT_ENABLED` | boolean | `false` | | Enable model alias compatibility layer. |
| `PRICING_SYNC_ENABLED` | boolean | `false` | | Enable automatic pricing data synchronization (also requires the `PRICING_SYNC_ENABLED` environment variable). |
### Health (3)
| Key | Type | Default | Description |
| ------------------------------------- | ------- | ------- | -------------------------------------------------------- |
| `OMNIROUTE_DISABLE_LOCAL_HEALTHCHECK` | boolean | `false` | Disable the local instance health check endpoint. |
| `OMNIROUTE_DISABLE_TOKEN_HEALTHCHECK` | boolean | `false` | Disable the token validation health check. |
| `SKILLS_SANDBOX_NETWORK_ENABLED` | boolean | `false` | Enable network access in the skills sandbox environment. |
> [!NOTE]
> The `Restart` column marks flags with `requiresRestart: true` — the value is
> persisted instantly but only takes effect after the process reloads. Enum
> flags reject any value outside their allowed set (validated server-side in
> both `setFeatureFlagOverride()` and the REST `PUT` handler).
---
## Toggling Flags
### Dashboard
Navigate to **Dashboard → Settings → Feature Flags**
(`/dashboard/settings/feature-flags`). The grid
(`src/app/(dashboard)/dashboard/settings/components/FeatureFlagsGrid.tsx`)
supports:
- **Search** by key or description, and **filter** by category (plus a synthetic
**Requires Restart** view).
- A **toggle** for boolean flags and a **dropdown** for enum flags
(`src/app/(dashboard)/dashboard/settings/components/FeatureFlagCard.tsx`).
- A **source badge** per flag — `DB`, `ENV`, or `DEF` — showing where the
effective value came from.
- A **Reset** button (shown only for `DB`-sourced flags) to drop the override,
and a **Reset All Overrides** button at the bottom.
- A **Restart Server** banner when a `requiresRestart` flag is changed.
### REST API
All operations go through a single route:
[`src/app/api/settings/feature-flags/route.ts`](../../src/app/api/settings/feature-flags/route.ts).
Every method requires an authenticated dashboard session (`401` otherwise).
#### `GET /api/settings/feature-flags`
Returns every flag with its effective value, source, and a summary.
```jsonc
{
"flags": [
{
"key": "REQUIRE_API_KEY",
"label": "Require API Key",
"description": "Require an API key for all incoming requests",
"category": "security",
"type": "boolean",
"enumValues": null,
"defaultValue": "false",
"effectiveValue": "false",
"source": "default", // "db" | "env" | "default"
"requiresRestart": false,
"warningLevel": "caution",
},
// ... all 33 flags
],
"summary": {
"total": 33,
"active": 0,
"inactive": 0,
"overriddenByDb": 0,
"overriddenByEnv": 0,
},
}
```
#### `PUT /api/settings/feature-flags`
Set or remove a single override. Body: `{ key: string; value?: string }`.
Omitting `value` removes the override (restoring env / default).
```bash
# Set a DB override
curl -X PUT http://localhost:20128/api/settings/feature-flags \
-H "Content-Type: application/json" \
-d '{"key":"REQUIRE_API_KEY","value":"true"}'
# Remove the override (no "value")
curl -X PUT http://localhost:20128/api/settings/feature-flags \
-H "Content-Type: application/json" \
-d '{"key":"REQUIRE_API_KEY"}'
```
The response echoes the new `effectiveValue`/`source`, the `previousValue`/
`previousSource`, and `requiresRestart`. Unknown keys and out-of-range enum
values are rejected with `400`.
#### `DELETE /api/settings/feature-flags`
Clears **all** DB overrides at once, restoring every flag to its env / default
value. Returns `{ cleared: <count>, message: "..." }`.
> [!NOTE]
> Flags with `requiresRestart: true` only take effect after a process reload.
> The dashboard's restart flow calls `POST /api/restart` and then polls
> `GET /api/health/ping` until the server is back up.
---
## Emergency Budget Fallback
`OMNIROUTE_EMERGENCY_FALLBACK` (category `runtime`, default `true`) controls the
emergency free-fallback path in
[`open-sse/services/emergencyFallback.ts`](../../open-sse/services/emergencyFallback.ts).
When enabled, requests that exhaust their budget are routed to a free fallback
provider/model instead of failing outright. Set it to `false` (or `0`) — via the
dashboard toggle, a DB override, or the `OMNIROUTE_EMERGENCY_FALLBACK`
environment variable — to disable the behavior and let budget-exhausted requests
fail. (Surfaced as a dashboard toggle in PRs #3741 / #3752.)
---
## See Also
- [Environment Variables Reference](./ENVIRONMENT.md) — most flags have a
same-named environment variable documented there (the DB override takes
precedence over it).
- [`src/shared/constants/featureFlagDefinitions.ts`](../../src/shared/constants/featureFlagDefinitions.ts)
— source of truth for every flag.
- [`src/shared/utils/featureFlags.ts`](../../src/shared/utils/featureFlags.ts)
— resolution logic (`resolveFeatureFlag`, `isFeatureFlagEnabled`,
`resolveAllFeatureFlags`).
- [`src/lib/db/featureFlags.ts`](../../src/lib/db/featureFlags.ts) — DB override
persistence in the `feature_flags` namespace of the `key_value` table.
docs/reference/FREE_TIERS.md
+345
View File
@@ -0,0 +1,345 @@
---
title: "Free Tiers & Free-Token Budget"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Free Tiers & Free-Token Budget
> **For Users**: Looking for a simple guide? See the [Free Tiers Guide](../getting-started/FREE-TIERS-GUIDE.md) for step-by-step instructions on getting free AI.
> **Last researched:** 2026-06-17 — per-provider web research (official docs + last-7-days news, 50-agent pass with adversarial verification) refreshing every free-tier quota + ToS.
> **Source of truth (catalog):** `open-sse/config/freeModelCatalog.ts` (per-MODEL budgets, pool-deduped). The token-budget numbers below come from live web research and are an **approximation** — see [Methodology & caveats](#methodology--caveats).
## TL;DR — how much free inference does OmniRoute actually aggregate?
| Metric | Tokens / month | Meaning |
| ------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Documented recurring grant (steady)** | **~1.54B** | Free-tier **pools** (per-model catalog), each shared pool counted **once**. The live source behind `/api/free-tier/summary` and the dashboard's Free-Tier Budget page. **Use this number.** |
| **+ first month with signup credits** | **~2.15B** | Steady + one-time signup credits (Together $25, Z.AI 20M, DeepSeek 5M, …), deduped per account. **First month only** — does not recur. |
| **+ permanently free, no published cap** | _un-quantifiable_ | `siliconflow`, `glm-cn` (GLM-4-Flash), `tencent`, `baidu`, `kilo-gateway`, `opencode-zen` — real recurring access, rate/concurrency-limited, **no token cap to count**. Listed, never summed (counting them at `RPM×24/7` is the inflation we reject). |
| **+ deposit-unlock boost** | **+~24M** | A one-time **$10** OpenRouter top-up raises its free pool from 50 → 1000 req/day. Reported separately so it never inflates the steady number. |
| Theoretical ceiling (all rate limits, 24/7) | ~10B | Sum of every provider rate limit extrapolated to non-stop use. **Not a guarantee** — do not headline this. |
**Honest headline:** _OmniRoute aggregates **~1.6B documented free tokens per month** (up to ~2.1B in your first month with signup credits) across 40+ free-tier pools — plus a long tail of permanently-free, no-cap providers — and RTK + Caveman compression (15–95% token savings) stretches that further._
> **Why this dropped from the previous ~1.94B.** The 2026-06-17 refresh is an honesty correction, not a loss: `gemini` is now pool-deduped (was inflated by counting each Flash variant separately, 462M → 60M), `cloudflare-ai` corrected to its real 10k-Neurons/day (122M → 30M), `doubao` reclassified as a one-time signup credit (not recurring), and shut-down tiers removed (`github-models` closed to new signups, `chutes`/`phind`/`kluster`/`glhf` discontinued). Partly offset by `llm7` (correct 5M/day → 150M) and new free providers (Kilo, OpenCode Zen, Z.AI GLM-Flash).
Biggest **documented** contributors: `mistral` 1.00B, `llm7` 150M, `groq` 117M, `gemini` 60M, `cerebras` 30M, `cloudflare-ai` 30M, `sambanova` 30M. (`longcat` is excluded — its 10M LongCat-2.0 grant is a one-time, KYC-gated signup credit, not a recurring monthly budget.)
> ⚠️ The theoretical ceiling (~10B) is inflated by rate-limit-only providers with **no published token cap** (`tencent`, `siliconflow`, `nvidia`, `baidu`, `glm-cn`, `sparkdesk`) whose figures would be `RPM/TPM × 24/7 × 30d` — a theoretical maximum no single account will sustain. They are **excluded** from the defensible number (shown in the "permanently free, no cap" row instead). This is the same inflation that makes competitors' multi-billion claims unreliable.
---
## 2026-06-17 refresh — what changed since 2026-06-05
A 50-agent web-research pass (official docs + last-7-days news, adversarially verified) refreshed the whole catalog. Highlights:
- **Removed / no free tier (2026):** `chutes` (free tier ended 2026-03), `phind` (company shut down 2026-01), `kluster` (sunset 2026-06-09 → MITO), `glhf` (beta ended), `gitlawb` + `gitlawb-gmi` (MiMo free revoked 2026-05-24, Nemotron promo ended 2026-06 — re-verified 2026-06-18), `aimlapi` (free tier paused — re-verified 2026-06-18), `yi` (Yi-Light retired, pay-as-you-go — re-verified 2026-06-18), `theoldllm` / `featherless-ai` (no current free tier). `iflytek` / `sparkdesk` stay listed but carry a ToS-caution note (Spark Lite is free; the ToS restricts proxy/relay use).
- **GitHub Models** — closed to **new** customers on 2026-06-16; existing accounts keep API/playground access, so it stays in the catalog with a note (not removed).
- **Gemini** — `2.0 Flash` / `2.0 Flash-Lite` shut down 2026-06-01 and `2.5 Pro` left the free tier (2026-04); free tier is now **Flash-family only** (2.5/3/3.1/3.5 Flash + Gemma). The catalog now **pools** the Flash family (was inflated by counting each variant separately: 462M → 60M).
- **Corrected numbers:** `cloudflare-ai` 122M → **30M** (real 10k-Neurons/day), `doubao` reclassified as a one-time signup credit (not recurring), `llm7` 4M → **150M** (documented 5M tokens/day), `together` "-Free" endpoints discontinued → only the **$25** signup credit remains, `longcat` Preview ended + Flash models retired → **LongCat-2.0** only, reclassified as a one-time **10M**-token signup credit (KYC-gated, not recurring).
- **New free providers discovered:** ⭐ **Kilo Code** (`kilo-gateway` — rotating "Auto Free" set: NVIDIA Nemotron 3 family, StepFun, Poolside, Nex-N2-Pro), ⭐ **OpenCode Zen** (`opencode-zen` — 6 rotating free coding models), ⭐ **Z.AI / Zhipu** (`glm-cn` — GLM-4-Flash / 4.5-Flash / 4.7-Flash permanently free + 20M signup bonus), and `arcee-ai` Trinity Large Preview.
- **New honest tiers** (see Methodology): a _permanently-free-but-uncapped_ category (real recurring access, no token cap to count) and a _deposit-unlock boost_ (OpenRouter $10 → +24M/mo), both surfaced **separately** so they never inflate the headline.
> The detailed per-provider table further down is the **2026-06-05 snapshot**; the deltas above supersede it. The live, canonical source is the per-model catalog `open-sse/config/freeModelCatalog.ts`.
---
## Methodology & caveats
- Numbers are **upper-bound estimates** from each provider's documented free-tier limits as of **2026-06-17**, gathered by web research (confidence tagged per row). Free tiers change constantly — re-verify before relying on a figure.
- `estMonthlyFreeTokens` = recurring monthly tokens only. **One-time signup credits do not recur** and count as 0. Discontinued tiers are also 0.
- Daily token cap → `monthly = daily × 30`. Only RPD documented → `RPD × ~800 output tokens × 30`. Only RPM/TPM (no daily cap) → **uncapped** (see below).
- **Permanently free, but no published token cap** (`siliconflow`, `glm-cn`, `tencent`, `baidu`, `kilo-gateway`, `opencode-zen`): these are real recurring free access, rate/concurrency-limited. We classify them `recurring-uncapped` and **never sum them** — multiplying `RPM × 24/7 × 30d` would produce a fantasy ceiling (the inflation we reject). They are listed so you know they exist.
- **Deposit-unlock boost:** a one-time small top-up that permanently raises a free quota (OpenRouter: $10 → 1000 req/day ≈ +24M/mo). Reported as a separate figure, kept out of the steady headline.
---
## ToS attention table
> A quick read on each provider's terms for a self-hosted, single-user personal proxy. `caution` = a personal-use or proxy clause worth checking; `ambiguous` = unclear; `ok` = explicitly permitted. Informational, not legal advice — you decide.
### ⚠️ Caution — personal-use / proxy clauses worth checking (19)
> Their free access is real and OmniRoute can route to them; the clauses below are just worth knowing. The OAuth/keyless ones aren't token-quantifiable, so they're not in the headline number (not because they're unusable).
| Provider | Note |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `agy` | Google Antigravity ToS explicitly prohibits using third-party software, tools, or services (including proxies) to access the service via OAuth; doing… |
| `ai21` | ToS §4.2/§8.2 prohibits sublicensing or distributing API access to third parties; §3.3 restricts trial/evaluation products to "internal evaluation on… |
| `amazon-q` | Product is discontinued for new signups; existing users are subject to AWS Customer Agreement which governs use of managed services — self-hosted pro… |
| `blackbox` | ToS explicitly prohibits sublicensing, reselling, making the service available to third parties, and building derivative services — a self-hosted per… |
| `coze` | Coze ToS explicitly restricts use to "personal and non-commercial use" and prohibits renting, distributing, sublicensing, or reselling the service; a… |
| `duckduckgo-web` | Duck.ai ToS (duckduckgo.com/duckai/privacy-terms) explicitly prohibits "automated querying and developing or offering AI services" and circumventing … |
| `featherless-ai` | Individual plans explicitly restricted to "interactive use or proto-typing and experimentation by the purchaser" — inference resale and proxy use req… |
| `fireworks` | ToS explicitly prohibits proxy/intermediary use, API key transfers, and sublicensing (Sections 2.1 and 2.2(i)(j)); self-hosted personal proxies are n… |
| `friendliai` | ToS Section 8(e) and 8(f) explicitly prohibit using FriendliAI as a proxy or allowing third-party access on a standalone basis, and forbid reselling/… |
| `iflytek` | Section 2.4(3) of the iFlytek Spark LLM Service Agreement explicitly prohibits "using any automated or programmatic methods to extract data or output… |
| `kiro` | Kiro FAQ explicitly prohibits use with "OpenClaw and similar tools that leverage third-party harnesses" — a self-hosted AI proxy (like OmniRoute) rou… |
| `modal` | ToS Section 1.3 explicitly prohibits "rent, resell or otherwise allow any third party direct access to or use of the Service" — building a self-hoste… |
| `muse-spark-web` | Meta ToS explicitly prohibits automated access without prior permission, reverse engineering without written permission, and circumventing technologi… |
| `nlpcloud` | ToS explicitly prohibits "setting up a proxy or other device that allows others to access the Service through it" and grants only a non-transferable,… |
| `opencode` | ToS (Anomaly Innovations, Inc.) explicitly restricts use to "your own internal use, and not on behalf of or for the benefit of any third party" — ope… |
| `qwen-web` | The free OAuth tier is discontinued; no ToS permits a self-hosted proxy using session tokens against chat.qwen.ai. Even before shutdown, automated/pr… |
| `t3-web` | ToS explicitly restricts accounts to personal use only, prohibits credential sharing with third parties, and bans automated/bot/scraping access — a s… |
### ✅ Generally permissive — caution / ambiguous / ok (the rest)
| Provider | ToS | Note |
|---|---|---|
| `aimlapi` | ambiguous | ToS grants a non-exclusive use license but does not explicitly permit or prohibit self-hosted proxy or resale; no "pers… |
| `baichuan` | ambiguous | No explicit prohibition on self-hosted personal proxies found in publicly accessible docs; however, the M3 Plus free pl… |
| `bluesminds` | ambiguous | No explicit ToS clauses found regarding self-hosted proxying or resale; the pricing page focuses on feature/rate limits… |
| `bytez` | ambiguous | No explicit ToS page was accessible (404); no public evaluation-only or no-proxy clauses found in docs, but the platfor… |
| `doubao` | ambiguous | No explicit proxy/resale prohibition found in publicly indexed documentation; Volcengine is a developer-oriented cloud … |
| `gitlawb-gmi` | ambiguous | No explicit ToS clause found prohibiting self-hosted personal proxy use; the free Nemotron model carries an NVIDIA disc… |
| `inclusionai` | ambiguous | No explicit ToS found prohibiting proxy/self-hosted use, but the platform is operated by Ant Group (Chinese company) an… |
| `kluster` | ambiguous | ToS primarily covers website content rights and does not specifically address API proxy use, resale, or self-hosted pro… |
| `monsterapi` | ambiguous | MonsterAPI's ToS page (monsterapi.ai/terms-of-service) was unreachable during research; no specific proxy/resale/person… |
| `nous-research` | ambiguous | Nous Portal itself is an aggregator/proxy service; using it as a backend for another self-hosted proxy creates a proxy-… |
| `ollama-cloud` | ambiguous | ToS prohibits using the service "to develop competing products" but has no explicit ban on self-hosted personal proxies… |
| `stepfun` | ambiguous | No explicit prohibition on self-hosted personal proxy found, but the Step Plan ToS targets developers using specific co… |
| `api-airforce` | caution | ToS explicitly prohibits "building competing services without permission" and "credential sharing" — a self-hosted pers… |
| `arcee-ai` | caution | Free access is via OpenRouter's :free routing layer (not Arcee's direct API terms); OpenRouter ToS permits personal dev… |
| `baidu` | caution | ToS not explicitly reviewed for proxy/resale clauses, but platform requires real-name authentication (Chinese ID typica… |
| `baseten` | caution | ToS restricts use to "Customer's internal business purposes" and explicitly prohibits sublicensing, reselling, or allow… |
| `bazaarlink` | caution | ToS explicitly prohibits reselling or sublicensing API keys to third parties; a self-hosted personal proxy for personal… |
| `brave-search` | caution | ToS prohibits redistribution, resale, and sublicensing of search results; using the API to "replicate or attempt to rep… |
| `byteplus` | caution | Tokens are non-transferable and single-account only; no explicit proxy prohibition, but BytePlus reserves the right to … |
| `cerebras` | caution | ToS grants a non-exclusive, non-transferable, non-sublicensable right for personal or business use; prohibits resale, s… |
| `cloudflare-ai` | caution | Cloudflare Self-Serve ToS §2.2.1(j) prohibits using Services to "provide a virtual private network or other similar pro… |
| `cohere` | caution | Cohere explicitly prohibits trial keys for "production or commercial purposes"; a self-hosted personal proxy routing re… |
| `deepinfra` | caution | ToS allows legal commercial use broadly, but prohibits use "directly or indirectly competitive with any business of the… |
| `deepseek` | caution | Open Platform ToS (effective 2026-04-29) permits broad use including "derivative product development" and personal/comm… |
| `dify` | caution | Self-hosted single-user personal proxy is permitted under the modified Apache 2.0 license; however, multi-tenant deploy… |
| `exa-search` | caution | No explicit "no proxy" or "evaluation only" clauses found; Exa actively offers a reseller partner program allowing API … |
| `firecrawl` | caution | Cloud API ToS has no explicit personal-proxy prohibition found, but the open-source self-hosted version is AGPL-3.0 (re… |
| `gemini` | caution | ToS explicitly states the free tier is for "developers building with Google AI models for professional or business purp… |
| `github-models` | caution | GitHub's Acceptable Use Policy prohibits reselling/proxying the service; GitHub Models ToS delegates to each model's ho… |
| `glhf` | caution | ToS explicitly prohibits sharing account credentials or making the account available to any third party, which makes a … |
| `groq` | caution | Services Agreement §6.3 prohibits reselling, sublicensing, or distributing API access; §3.2 bars reselling/leasing acco… |
| `hackclub` | caution | Service is explicitly scoped to Hack Club teen members building projects/learning; no public ToS found explicitly permi… |
| `huggingchat` | caution | Hugging Face ToS does not explicitly ban personal self-hosted proxies, but supplemental terms (referenced but not fully… |
| `huggingface` | caution | ToS grants a limited license to access/use the service; the document does not explicitly permit or forbid a single-user… |
| `hyperbolic` | caution | ToS grants API access "solely for your own personal or internal business purposes" and explicitly prohibits licensing, … |
| `inference-net` | caution | ToS explicitly prohibits "sublicense, resell, distribute" and transferring API keys without written consent; a single-u… |
| `jina-ai` | caution | Free 10M tokens are explicitly non-commercial (CC-BY-NC 4.0 model license); a single-user personal proxy for personal L… |
| `jina-reader` | caution | ToS prohibits using outputs to build competing services and bans "automated methods to extract information via scraping… |
| `llm7` | caution | ToS positions the service as for "experimentation, development, and research"; no explicit ban on self-hosted personal … |
| `longcat` | caution | The API Platform Service Agreement (longcat.chat/platform/private/) permits commercial integration and self-hosted apps… |
| `mistral` | caution | Consumer ToS explicitly states APIs may only be used for "personal needs" and prohibits making API keys available to th… |
| `morph` | caution | ToS allows commercial use generally; self-hosted proxy deployments require explicit arrangement with sales. Section 18.… |
| `nebius` | caution | ToS (Section 5f) explicitly prohibits resale, redistribution, or offering the service "on a standalone basis" — a self-… |
| `nomic` | caution | ToS grants a non-exclusive, non-transferable API license; Section 6.b prohibits building a competitive service. Using t… |
| `novita` | caution | ToS prohibits resale and competing services but does not explicitly address personal self-hosted proxies; personal use … |
| `nscale` | caution | AUP prohibits "copy, modify, duplicate... frame, mirror, republish... distribute all or any part of the Nscale Platform… |
| `nvidia` | caution | Free tier is explicitly for prototyping/dev/research/evaluation only — production use (serving real end-users) requires… |
| `openrouter` | caution | ToS explicitly prohibits reselling API access or developing a competing service; single-user self-hosted personal proxy… |
| `pollinations` | caution | MIT License cited in API docs suggests liberal reuse; no explicit prohibition on self-hosted proxying found. However, u… |
| `predibase` | caution | Predibase is positioned as an enterprise fine-tuning/serving platform; the free trial is explicitly for exploration and… |
| `publicai` | caution | ToS (publicai.co/tc) designates services as "primarily for research and educational use"; no explicit proxy or resale p… |
| `puter` | caution | Puter ToS forbids using services for "commercial purpose" without written consent; a self-hosted personal proxy consumi… |
| `qoder` | caution | ToS page returned no readable content; Qoder is a coding IDE client (not a public API), and third-party proxy wrappers … |
| `reka` | caution | Business Terms prohibit sublicensing or distributing access to third parties; a personal single-user proxy is likely fi… |
| `sambanova` | caution | ToS Section 1.5(c) explicitly prohibits reselling, sublicensing, or making the service available to third parties; a se… |
| `sensenova` | caution | No explicit proxy or resale prohibition found in reviewed ToS, but the free tier is a promotional beta with no SLA, Sen… |
| `serper-search` | caution | ToS explicitly prohibits "mirroring materials on any other server as-is with no-value-added" — a simple pass-through pr… |
| `siliconflow` | caution | ToS (Clause 3.4(e)(f)(p)) explicitly prohibits making the service available to any third party, reselling/sublicensing,… |
| `sparkdesk` | caution | SparkDesk User Agreement grants only personal, non-commercial use rights; API Interface Policy prohibits automated data… |
| `tavily-search` | caution | ToS explicitly states the API "may not be transferred, assigned, shared, or otherwise made available to any third party… |
| `tencent` | caution | Tencent Cloud ToS explicitly prohibits sublicensing or reselling API access; a self-hosted personal proxy for personal … |
| `together` | caution | ToS Section 4.3(d) explicitly prohibits transferring, distributing, reselling, leasing, or offering the Services on a s… |
| `uncloseai` | caution | Personal proxy use is plausible but not explicitly permitted; ToS bans building "competing machine learning services wi… |
| `veoaifree-web` | caution | ToS explicitly bans automated bots or scripts running at "inhuman speeds" and prohibits copying the platform to create … |
| `vertex` | caution | Google Cloud Service Terms restrict resale to authorized resellers only (Section 14 requires a Reseller Agreement); a s… |
| `voyage-ai` | caution | ToS grants "personal, non-commercial use" for site content and prohibits credential/account sharing with third parties;… |
| `360ai` | unknown | ToS for developer API not publicly accessible without registration; access requires application approval which implies … |
| `chutes` | unknown | ToS page exists at chutes.ai/terms but content was not accessible via fetch; no explicit proxy/resale clauses found in … |
| `freemodel-dev` | unknown | The Terms of Service page (freemodel.dev/terms) returned only a header with no readable content via WebFetch; no clause… |
| `gitlawb` | unknown | No ToS or acceptable-use policy found; proxy/resale restrictions unknown — assume caution for self-hosted proxy use. |
| `liquid` | unknown | No hosted API exists to proxy; open-source model commercial use is free for orgs under $10M annual revenue. No self-hos… |
| `theoldllm` | unknown | No terms of service document was found on the site; proxying, resale, or self-hosted use policy is entirely undocumente… |
| `yi` | unknown | ToS not publicly accessible without login; no proxy/resale clauses could be reviewed. Self-hosted personal proxy use st… |
| `comfyui` | ok | GPL-3.0 open-source license explicitly permits self-hosted personal proxy use; Comfy Org ToS confirms commercial use of… |
| `scaleway` | ok | Scaleway's General Terms of Services are a standard commercial cloud agreement with no explicit prohibition on self-hos… |
| `sdwebui` | ok | AGPL-3.0 license: free to self-host for personal use with no restrictions on usage volume; a personal proxy using this … |
| `searxng-search` | ok | AGPL-3.0 open-source license explicitly permits self-hosted personal proxy use with no restriction on usage type, resal… |
---
## Per-provider free-tier (refreshed 2026-06-17)
> Regenerated from the per-model catalog (`open-sse/config/freeModelCatalog.ts`), pool-deduped. Sorted by recurring steady tokens/mo. `uncapped*` = permanently free but no published token cap (rate/concurrency-limited) — real access, **not** summed into the headline. `—` = credit-only / keyless / not token-quantifiable.
| Provider | Free type | Steady tokens/mo | First-month credit | ToS | Models |
|---|---|---|---|---|---|
| `mistral` | recurring | ~1.00B | — | caution | 5 |
| `llm7` | recurring | ~150M | — | caution | 4 |
| `longcat` | one-time | — | 10M | caution | 1 |
| `gemini` | recurring | ~60M | — | caution | 6 |
| `cerebras` | recurring | ~30M | — | caution | 2 |
| `cloudflare-ai` | recurring | ~30M | — | caution | 6 |
| `api-airforce` | recurring | ~24M | — | caution | 7 |
| `ollama-cloud` | recurring | ~20M | — | ambiguous | 8 |
| `github-models` | recurring | ~18M | — | caution | 14 |
| `groq` | recurring | ~15M | — | caution | 5 |
| `inclusionai` | recurring | ~15M | — | ambiguous | 1 |
| `bluesminds` | recurring | ~7M | — | ambiguous | 22 |
| `sambanova` | recurring | ~6M | — | caution | 5 |
| `arcee-ai` | recurring | ~5M | — | caution | 1 |
| `bazaarlink` | recurring | ~4M | — | caution | 32 |
| `openrouter` | recurring | ~1M | — | caution | 1 |
| `cohere` | recurring | ~800K | — | caution | 6 |
| `huggingchat` | recurring | ~500K | — | caution | 4 |
| `morph` | recurring | ~400K | — | ok | 2 |
| `huggingface` | recurring | ~200K | — | caution | 6 |
| `kiro` | recurring | ~25K | — | avoid | 12 |
| `glm-cn` | uncapped | uncapped\* | ~20M | ok | 4 |
| `baidu` | uncapped | uncapped\* | — | caution | 1 |
| `kilo-gateway` | uncapped | uncapped\* | — | caution | 7 |
| `opencode-zen` | uncapped | uncapped\* | — | caution | 6 |
| `siliconflow` | uncapped | uncapped\* | — | caution | 10 |
| `tencent` | uncapped | uncapped\* | — | caution | 1 |
| `vertex` | signup credit | — | ~300M | caution | 10 |
| `agentrouter` | signup credit | — | ~200M | caution | 4 |
| `predibase` | signup credit | — | ~25M | caution | 1 |
| `together` | signup credit | — | ~25M | caution | 1 |
| `doubao` | signup credit | — | ~15M | ambiguous | 1 |
| `ai21` | signup credit | — | ~10M | avoid | 2 |
| `deepseek` | signup credit | — | ~5M | ok | 2 |
| `hyperbolic` | signup credit | — | ~5M | ok | 8 |
| `nscale` | signup credit | — | ~5M | caution | 6 |
| `bytez` | signup credit | — | ~1M | ambiguous | 3 |
| `deepinfra` | signup credit | — | ~1M | caution | 22 |
| `fireworks` | signup credit | — | ~1M | avoid | 10 |
| `nebius` | signup credit | — | ~1M | caution | 1 |
| `qoder` | signup credit | — | ~1M | caution | 14 |
| `scaleway` | signup credit | — | ~1M | ok | 6 |
| `novita` | signup credit | — | ~500K | caution | 1 |
| `agy` | keyless | — | — | avoid | 16 |
| `baichuan` | keyless | — | — | ambiguous | 1 |
| `blackbox` | keyless | — | — | avoid | 6 |
| `coze` | keyless | — | — | avoid | 1 |
| `duckduckgo-web` | keyless | — | — | avoid | 6 |
| `freemodel-dev` | keyless | — | — | unknown | 4 |
| `friendliai` | keyless | — | — | avoid | 2 |
| `hackclub` | keyless | — | — | caution | 3 |
| `iflytek` | keyless | — | — | avoid | 1 |
| `inference-net` | keyless | — | — | caution | 3 |
| `liquid` | keyless | — | — | unknown | 1 |
| `monsterapi` | keyless | — | — | ambiguous | 1 |
| `muse-spark-web` | keyless | — | — | avoid | 3 |
| `nlpcloud` | keyless | — | — | avoid | 1 |
| `nous-research` | keyless | — | — | ambiguous | 2 |
| `nvidia` | keyless | — | — | caution | 13 |
| `opencode` | keyless | — | — | avoid | 7 |
| `pollinations` | keyless | — | — | caution | 31 |
| `publicai` | keyless | — | — | caution | 3 |
| `puter` | keyless | — | — | caution | 33 |
| `qwen-web` | keyless | — | — | avoid | 3 |
| `reka` | keyless | — | — | caution | 2 |
| `sensenova` | keyless | — | — | caution | 1 |
| `sparkdesk` | keyless | — | — | caution | 1 |
| `stepfun` | keyless | — | — | ok | 1 |
| `t3-web` | keyless | — | — | avoid | 23 |
| `uncloseai` | keyless | — | — | caution | 3 |
---
## What changed since the shipped catalog (`freeNote`)
> The v3.8.0-era `freeNote` strings are stale. Corrections found by this research (these drive the catalog update in `_tasks/features-v3.8.12`):
- **`360ai`** — The shipped freeNote "Free 360 AI Brain models" appears outdated. Current access is application-gated and paid. The 2023 launch-era promotional tokens (100M–250M one-time) may have been the basis for…
- **`agentrouter`** — Our shipped freeNote says "$200 free credits on signup." Current reality shows standard (non-referral) signups receive only $100; referral signups may get $200 but a community comment from April 2026…
- **`agy`** — Our shipped freeNote says "(none)" implying no free tier, but Antigravity does have a free OAuth-gated tier. However, the ToS explicitly prohibits using this free tier through a proxy like OmniRoute …
- **`ai21`** — Tightened: trial window shrunk from "3 months" to 7 days. The $10 credit amount remains the same, but validity dropped sharply from ~90 days to 7 days.
- **`aimlapi`** — Changed significantly. Shipped freeNote advertised "$0.025/day free credits — 200+ models" but the free tier is now paused/discontinued. The $0.025/day credit allocation (50,000 credits/day, 10 req/d…
- **`amazon-q`** — Our shipped freeNote says "(none)" — the reality is worse: the product is now discontinued for new signups (May 15, 2026). Previously the free tier offered 50 agentic requests/month + unlimited inlin…
- **`api-airforce`** — Catalog ships freeNote "(none)" but a documented free tier exists: 1 RPM / 1,000 RPD recurring, account signup required, limited to basic models.
- **`arcee-ai`** — The shipped freeNote ("Free Trinity Large Thinking model (262K context)") is partially accurate — Trinity Large Thinking is indeed free via OpenRouter with 262K context — but the note omits that this…
- **`baichuan`** — Our shipped freeNote says "Free Baichuan models" which implies ongoing free access, but current reality is only a one-time 80 CNY trial credit for new users (valid 3 months). There are no permanently…
- **`baidu`** — The catalog says "Free ERNIE Speed/Lite models" which is broadly accurate, but understates the scope: ERNIE-Tiny and multiple context-window variants (8K and 128K) are also free. The free tier appear…
- **`bazaarlink`** — Broadly matches — the shipped freeNote accurately describes auto:free routing for zero-cost inference. However, the current reality includes explicit rate limits (10-20 RPM, ~150 RPD) not mentioned i…
- **`blackbox`** — Our shipped freeNote claims "unlimited basic chat plus Minimax-M2.5." In reality, unlimited Minimax-M2.5 agent requests are a paid-plan feature (Pro+), not part of the free tier. The free tier has li…
- **`bluesminds`** — Our shipped freeNote was "(none)" — but BluesMinds does have a documented free tier: 500 pi credits, 20 RPM, 300 RPD, permanent free plan. The catalog significantly understates the offering.
- **`brave-search`** — The catalog notes "(none)" suggesting no free tier was tracked, but in reality there was a free 5,000 queries/month tier (no card) until February 12, 2026, which has since been replaced by a $5/month…
- **`byteplus`** — Our catalog shipped "(none)" but BytePlus ModelArk does have a free tier: a one-time trial credit of 500k tokens per LLM model for new accounts. The catalog underreports this.
- **`cerebras`** — TPM appears tightened from 60K to 30K on current documented models (gpt-oss-120b, zai-glm-4.7). RPM of 5 is now explicitly documented (was not in our shipped note). Daily token cap of 1M/day is uncha…
- **`chutes`** — The shipped freeNote says "Free tier available" but as of March 15, 2026, the free tier has been officially discontinued. The catalog note is stale and should be updated to reflect that there is no r…
- **`coze`** — The shipped note "Free ByteDance agent platform" is directionally accurate but omits that the free tier is now tightly credit-capped (10 credits/day ≈ 5–100 messages depending on model), a constraint…
- **`deepinfra`** — Our shipped freeNote says "Free signup credits for API testing" — this appears stale. The official pricing page now requires card/prepayment with no documented general free signup credit. The free ti…
- **`deepseek`** — Our shipped note says "5M free tokens on signup - no credit card required" — this is still accurate for the one-time grant, but importantly the credits expire after 30 days (not mentioned in the ship…
- **`dify`** — The shipped freeNote ("Free open-source AI app builder + RAG") is directionally accurate but incomplete. The cloud free tier is more constrained than implied: 200 message credits appear to be a one-t…
- **`doubao`** — The shipped freeNote "Free Doubao models (ByteDance)" is directionally correct but underspecified. Current reality is more structured: there is a quantified recurring daily free tier (2M tokens/day v…
- **`duckduckgo-web`** — The core "free anonymous access" description still holds, but the service has matured significantly: it now has explicit paid tiers (Plus/Pro) with higher limits implying the free tier is rate-constr…
- **`exa-search`** — Catalog ships freeNote "(none)" but Exa has a documented recurring free tier of 1,000 requests/month. This is a significant gap — the free tier exists and is permanent.
- **`featherless-ai`** — Our shipped freeNote says "Free tier available" but there is no general free tier. The only free access is through an invitation/application-based Builder Series program, which is not a standard free…
- **`firecrawl`** — Our catalog shipped freeNote "(none)", implying no free tier. In reality Firecrawl has a documented recurring free plan with 1,000 credits/month — the catalog entry is incorrect.
- **`freemodel-dev`** — Our shipped freeNote is "(none)" — this was likely a placeholder meaning the provider was not yet cataloged. In reality the provider does have a $300 one-time trial credit offer. However, this is a o…
- **`friendliai`** — The shipped freeNote ("Free tier for serverless inference") is partially accurate but misleading. There is free access via Tier 0 and free-designated models, but the rate limits are undefined and ada…
- **`gemini`** — The shipped freeNote says "1,500 req/day for Gemini 2.5 Flash" — this was accurate before December 2025. Google cut free-tier limits by 50-80% in December 2025, reducing Gemini 2.5 Flash from 1,500 R…
- **`github-models`** — Catalog note "Free GPT-5, o-series, DeepSeek-R1, Llama 4, Grok 3" is directionally correct about model availability but omits the daily rate limits (50 RPD for high-tier models, 150 RPD for low-tier)…
- **`gitlawb`** — The shipped freeNote "Free tier available" is effectively stale. The original free MiMo access was removed in May 2026; the only remaining "free" option is a temporary promotional model (Nemotron 3 U…
- **`gitlawb-gmi`** — Partially still accurate — free tier exists but is now narrowed to a single model (Nemotron 3 Ultra) after MiMo free access was revoked in late May 2026. The shipped note "Free tier available" unders…
- **`glhf`** — The shipped freeNote ("Free tier for open-source model inference") is now stale. The free beta ended in January 2025; GLHF Chat is now a paid pay-as-you-go service. There is no ongoing recurring free…
- **`groq`** — The shipped freeNote "30 RPM / 14.4K RPD" is accurate only for llama-3.1-8b-instant. Most other models (including llama-3.3-70b-versatile) have a much lower 1K RPD cap. The note omits model-specific …
- **`hackclub`** — The "30+ models" count appears accurate and still matches. The core offering remains free for Hack Club members. No evidence of tightening — still "$0 ALWAYS FREE" per the homepage. The freeNote omit…
- **`huggingchat`** — The shipped freeNote ("Free LLM chat — no subscription required. Rate limits apply.") is partially accurate but significantly understates the restrictions. The free tier now operates on a hard $0.10/…
- **`huggingface`** — Significantly tightened. The shipped freeNote ("Free Inference API for thousands of models") implied unlimited/generous free access, but as of mid-2025 the free tier is capped at $0.10/month in recur…
- **`hyperbolic`** — Our shipped freeNote says "$1-5 trial credits on signup" — the $1 trial credit portion is accurate, but the "$5" figure refers to the minimum deposit required to unlock GPU rental (not free credits g…
- **`iflytek`** — Catalog says "Free Spark Lite models" — this is broadly accurate. However the current reality is more nuanced: only Spark Lite is free (the Max 100M token offer was a one-time promo, not recurring); …
- **`inclusionai`** — Our shipped freeNote says "Free Ling-2.6-flash model (262K context)" without specifying token limits. Reality is more specific: the free tier is 500K tokens/day (shared across all models), with a 2 Q…
- **`inference-net`** — The shipped freeNote states "$25 free credits on signup plus research grants." The current pricing page shows only $1 recurring monthly credits with no mention of a $25 signup bonus or research grant…
- **`jina-reader`** — Our shipped freeNote was "(none)", which is incorrect. Jina Reader has had a publicly documented free tier since launch: keyless access at 20 RPM plus a 10M one-time token grant with a free API key. …
- **`kiro`** — Catalog shipped freeNote "(none)" — but Kiro has a documented, perpetual free tier of 50 credits/month. The free tier existed since Kiro's public launch (pricing formalized ~October 2025). This is a …
- **`kluster`** — The $5 free credits on signup appears to still match. However, there is evidence of an additional permanent free tier (post-credit) with undocumented limits, which may represent an improvement over t…
- **`llm7`** — Rate limits have increased from the shipped freeNote (20 RPM / 100 req/hr → 40 RPM / 200 req/hr). The "no signup required" claim is now outdated — a free token from token.llm7.io is now required (tho…
- **`longcat`** — The public preview/beta ended and the Flash models were retired; only the GA `LongCat-2.0` remains. The free tier is a **one-time 10M-token grant** unlocked after account signup + **KYC verification** — it does **not** reset daily or monthly. Beyond the grant it is pay-as-you-go.
- **`mistral`** — The shipped freeNote ("Free Experiment tier: rate-limited access to all models") is directionally correct but understated. Current reality adds specific documented limits: 2 RPM, 500K TPM, 1B tokens/…
- **`monsterapi`** — The shipped freeNote says "Free credits for decentralized GPU inference" which is partially accurate — there are one-time trial credits on signup. However, the recurring free tier has 0 credits/month…
- **`morph`** — The shipped freeNote mentions "250K credits/month" which matches the current credit allocation; however, the more significant constraint is 200 requests/month which was not captured in the original c…
- **`muse-spark-web`** — The shipped freeNote ("Free with login — Meta AI platform with Llama models") is broadly accurate regarding login requirement and Llama model access. No tightening of the free tier was detected; it r…
- **`nlpcloud`** — The shipped freeNote says "Trial credits for new accounts," which implies a one-time trial. In reality, NLP Cloud's free tier is a recurring monthly free plan (10,000 requests/month), not trial credi…
- **`nomic`** — Our shipped freeNote says "Free Nomic Embed API" with no qualification, implying ongoing free access. Reality is a one-time 1M-token trial credit only — after that token budget is consumed, paid subs…
- **`nous-research`** — The shipped freeNote ("Free tier: 50 RPM, 500,000 TPM") does not match the current Nous Portal product. The portal launched April 27, 2026 and structures its free tier as $0.10/month in recurring cre…
- **`nvidia`** — The "40 RPM, 70+ models" rate limit element matches the catalog, but the freeNote framing as a simple dev-access tier undersells that the old one-time credit pool has been removed — access is now tru…
- **`ollama-cloud`** — Our shipped freeNote is "(none)" — this is stale. Ollama Cloud launched a cloud inference product with a genuine free tier that provides light weekly GPU-time-based access to hosted open models.
- **`openrouter`** — RPD tightened from 200 to 50 for zero-credit accounts (RPM unchanged at 20). The catalog note was accurate on RPM but overstated the RPD by 4x for the no-credits baseline tier.
- **`phind`** — Phind shut down on January 16, 2026. The provider has now been **fully removed** from the catalog (registry, executor, and both the web-cookie and API-key catalog entries) — matching the dead-service-removal precedent (#5246 Gemini CLI).
- **`pollinations`** — Partially matches — the "no API key required" claim is still true for anonymous access, but the catalog freeNote omits that: (1) rate limits do apply (interval throttle of ~1 req/6-15s for anonymous …
- **`predibase`** — The shipped freeNote ($25 free trial credits, 30-day validity) still matches current documentation. However, the catalog omits the concurrent 20,000 tokens/day serverless rate limit that applies duri…
- **`publicai`** — The shipped freeNote ("Free community inference tier") is broadly accurate but understates the specificity: the 20 RPM rate limit is now documented. No major tightening found; the service remains fre…
- **`puter`** — Partially matches: the "500+ models" count is still accurate. However "users pay via Puter account" understates the reality — free accounts receive an undocumented starting credit that can be exhaust…
- **`qoder`** — Our catalog ships freeNote "(none)", but Qoder does have a free tier: a Community Edition with unlimited basic-model completions (daily-capped, unspecified limit) plus a one-time 14-day/300-credit Pr…
- **`qwen-web`** — The shipped freeNote ("Free — Qwen models via chat.qwen.ai with login token") is now stale. The login-token/OAuth free API path was terminated on 2026-04-15. The qwen-web executor will receive 401 er…
- **`sambanova`** — Our shipped note only described the one-time $5 credit (30-day validity). The current reality includes a permanent recurring free tier with documented rate limits (20 RPM, 20 RPD, 200k TPD) that pers…
- **`sensenova`** — Our shipped freeNote says "Free SenseTime models" which is vague but directionally correct — free access does exist. However, reality is more nuanced: free access is a time-limited public beta (Token…
- **`serper-search`** — The shipped freeNote says "(none)" which is partially accurate — there is no recurring free plan — but Serper does offer 2,500 one-time trial credits on signup. The catalog note could be more precise…
- **`siliconflow`** — Partially matches but more nuanced: the $1 free credits are a one-time trial (not recurring), while the "permanently free models" component still holds — free $0 models continue to exist (Qwen3-8B, D…
- **`sparkdesk`** — Partially matches — the shipped freeNote "Free iFlytek Spark models" is accurate in that Spark Lite is permanently free, but understates the constraint (2 QPS per App ID) and overstates scope (only S…
- **`stepfun`** — The shipped freeNote "Free Step-2 models" is stale. Step-2 LLM free access is no longer offered; the platform has transitioned to Step 3.x models on a paid-per-token basis with no free LLM tier. Only…
- **`t3-web`** — The shipped freeNote is broadly accurate (limited model access, Pro unlocks 50+ models for $8/month), but misses two key updates: (1) the free tier now resets daily instead of monthly (changed around…
- **`tavily-search`** — Catalog ships freeNote "(none)" implying no free tier, but Tavily does in fact offer a documented recurring free tier of 1,000 credits/month with no credit card required. This is a significant discre…
- **`tencent`** — Largely matches — the shipped freeNote ("Free Hunyuan Lite models") is accurate. Hunyuan-lite has been permanently free since May 2024 and remains so as of 2026. The catalog note undersells the detai…
- **`theoldllm`** — Our shipped freeNote was "(none)" — this still matches in the sense that no structured API/free tier offering exists; the service remains a UI-only chat wrapper with no catalogable API tier.
- **`together`** — The shipped note says "$25 signup credits + 3 permanently free models" but reality shows far more permanently free models (~80, not 3). The $25 trial credit figure is contested — official billing doc…
- **`uncloseai`** — Largely matches — still free forever with no signup. However, the ToS (terms-of-use.html) clarifies IP-based throttling exists for excessive use and prohibits building competing ML services without a…
- **`veoaifree-web`** — The shipped freeNote states "6 requests/hour" but no such explicit limit is currently documented anywhere on veoaifree.com. The site claims unlimited free generation with no login. The models listed …
- **`voyage-ai`** — The shipped freeNote "200M free tokens for embeddings and reranking" is directionally correct on the token count but misleading — it omits that this is a one-time per-account allocation, not a recurr…
- **`yi`** — The shipped freeNote "Free Yi-Light models" references a model name ("Yi-Light") that does not appear in any current 01.AI documentation or model catalog — no such model is listed on platform.01.ai, …
---
## Glossary
| Term | Meaning |
| ----------------------- | -------------------------------------------------------------------------- |
| **RPM / RPD / RPH** | Requests per minute / day / hour |
| **TPM / TPD** | Tokens per minute / day |
| **Documented grant** | Provider publishes an explicit daily/monthly token cap (defensible budget) |
| **Theoretical ceiling** | `rate-limit × 24/7 × 30d` — a maximum, not a granted budget |
| **Neuron** | Cloudflare compute unit (~1 output token) |
> Generated from per-provider research on 2026-06-05. Re-run the research workflow (see `_tasks/features-v3.8.12`) to refresh.
docs/reference/PROVIDER_PLUGIN_MANIFEST.md
+81
View File
@@ -0,0 +1,81 @@
---
title: "Provider Plugin Manifest"
version: 3.8.42
lastUpdated: 2026-07-01
---
# Provider Plugin Manifest
`open-sse/config/providerPluginManifest.ts` defines the JSON-safe provider
plugin contract. `open-sse/config/providerPluginManifestRegistry.ts` binds that
contract to the current provider registry for sidecars such as Bifrost,
CLIProxyAPI, or a future Go/Rust router. The TypeScript registry remains the
source of truth, but sidecars can consume the manifest without importing
executor code, OAuth defaults, headers, or process environment state.
The same manifest is available over HTTP at
`GET /api/v1/provider-plugin-manifest` for sidecars that run out-of-process.
OmniRoute advertises that URL to Bifrost and CLIProxyAPI via the
`X-OmniRoute-Provider-Manifest-Url` request header. Set
`OMNIROUTE_PROVIDER_MANIFEST_URL` when the sidecar needs a public or container
network URL instead of the local request origin.
## Goal
Move provider metadata toward a plugin contract so the hot request path can
eventually be owned by a lower-latency sidecar while OmniRoute keeps the
TypeScript route as the policy gate and fallback. The manifest is additive: it
does not change request routing by itself.
## Contract
The manifest contains:
- provider id and alias
- upstream format and executor name
- auth type, auth header, and optional auth prefix
- static endpoint metadata
- sidecar eligibility and explicit reasons when a provider should stay on TS
- JSON-safe model metadata such as context length, vision/reasoning flags, and
unsupported params
- capability tags including `apikey`, `oauth`, `custom-executor`,
`passthrough-models`, `responses`, and `sidecar-candidate`
The manifest intentionally excludes:
- OAuth client secrets and default secret values
- runtime environment resolution
- request headers and public credential helpers
- dynamic URL builders
- executor functions
- session pool internals
## Sidecar Use
Sidecars should treat `sidecar.eligible` as a conservative candidate signal, not
as an unconditional routing decision. The first import target should be
API-key, static-endpoint providers using the default executor. Providers with
custom web executors, OAuth/session flows, dynamic URL builders, or pool config
stay on the TypeScript fallback path until a sidecar implements equivalent
behavior and telemetry proves parity.
Suggested migration phases:
1. Generate and validate the provider plugin manifest from the TS registry.
2. Teach Bifrost or CLIProxyAPI to import the manifest for API-key/static
providers.
3. Route eligible providers through the sidecar behind `OMNIROUTE_RELAY_BACKEND`
while keeping TS fallback enabled.
4. Promote providers only when success rate, p99 latency, streaming behavior,
and unsupported-param handling match the TS path.
5. Add sidecar-native plugins for custom executors one provider family at a
time.
## Why Not Embed Providers Directly In Next
The Next frontend should not own provider execution. It should call the API
boundary. The backend can then decide whether to use the TypeScript executor,
Bifrost, CLIProxyAPI, or a future native sidecar. This keeps request signing,
allowlist checks, DB policy, and fallback behavior centralized before any
sidecar handoff.
docs/reference/PROVIDER_REFERENCE.md
+327
View File
@@ -0,0 +1,327 @@
---
title: "Provider Reference"
version: 3.8.43
lastUpdated: 2026-07-02
---
# Provider Reference
> **Auto-generated** from `src/shared/constants/providers.ts` — do not edit by hand.
> Regenerate with: `npm run gen:provider-reference`
> **Last generated:** 2026-07-02
Total providers: **237**. See category breakdown below.
## Categories
- **Free** — free tier with API key (configured via dashboard)
- **OAuth** — sign-in flow handled by OmniRoute, no API key needed
- **Web cookie** — wraps the provider's web app via cookie auth
- **API key** — paid provider configured via API key (free credits may apply)
- **Local** — runs on the user's machine (Ollama, LM Studio, vLLM, etc.)
- **Search** — web search providers
- **Audio** — audio-only providers (TTS/STT)
- **Upstream proxy** — providers that proxy to other providers
- **Cloud agent** — long-running coding agents (Codex Cloud, Devin, Jules)
- **System** — OmniRoute-internal providers (loopback, etc.)
Additional tags: `image`, `video`, `aggregator`, `enterprise`, `embed/rerank`, `self-hosted`.
Use the dashboard at `/dashboard/providers` to enable, configure, and test each provider.
---
## OAuth Providers (20)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `agy` | `agy` | Antigravity CLI | OAuth | [link](https://antigravity.google) | Import your Antigravity CLI (`agy`) login (paste/upload its token file), auto-detect a local CLI login, or sign in with Google. Shares the Antigravity backend (incl. Claude models). |
| `amazon-q` | `aq` | Amazon Q | OAuth | [link](https://aws.amazon.com/q/developer/) | Uses the same AWS Builder ID or imported refresh-token flow as Kiro, but keeps Amazon Q connections separate. |
| `antigravity` | — | Antigravity | OAuth | — | — |
| `claude` | `cc` | Claude Code | OAuth | — | — |
| `cline` | `cl` | Cline | OAuth | — | — |
| `codebuddy-cn` | `cbcn` | CodeBuddy CN | OAuth | [link](https://copilot.tencent.com) | Tencent CodeBuddy CN (copilot.tencent.com). Sign in via the official CLI device-code flow, or paste a direct API key (sent as Authorization: Bearer). Catalog: GLM / Kimi / MiniMax / DeepSeek / Hunyuan. |
| `codex` | `cx` | OpenAI Codex | OAuth | — | — |
| `cursor` | `cu` | Cursor IDE | OAuth | — | — |
| `devin-cli` | `dv` | Devin CLI (Official) | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai |
| `github` | `gh` | GitHub Copilot | OAuth | — | — |
| `gitlab-duo` | `gitlab-duo` | GitLab Duo | OAuth | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | OAuth application with ai_features + read_user scopes. Configure GITLAB_DUO_OAUTH_CLIENT_ID and optionally GITLAB_DUO_OAUTH_CLIENT_SECRET on this OmniRoute instance. |
| `grok-cli` | `gc` | Grok Build | OAuth | — | Paste your ~/.grok/auth.json (or the JWT access token) from the Grok Build CLI; refresh_token is rotated automatically. |
| `kilocode` | `kc` | Kilo Code | OAuth | — | — |
| `kimi-coding` | `kmc` | Kimi Coding | OAuth | — | — |
| `kiro` | `kr` | Kiro AI | OAuth | — | Free tier: 50 credits/month (~25K–100K tokens). ⚠️ Kiro ToS prohibits third-party proxy/harness use. |
| `qoder` | `if` | Qoder | OAuth | — | — |
| `qwen` | `qw` | Qwen Code | OAuth | — | ⚠️ **DEPRECATED.** Qwen OAuth free tier was discontinued on 2026-04-15. Use 'bailian-coding-plan', 'alibaba', 'alibaba-cn', or 'openrouter' provider with API key instead. |
| `trae` | `tr` | Trae | OAuth | [link](https://trae.ai) | Trae is an AI-native IDE by ByteDance (SOLO remote agent). Authorize via trae.ai in the popup, or sign in at solo.trae.ai and paste the Cloud-IDE-JWT (sent as 'Authorization: Cloud-IDE-JWT <token>', ~14-day lifetime) as the access token; web_id/biz_user_id/user_unique_id/scope/tenant/region propagate via providerSpecificData. No headless refresh for pasted tokens — re-paste on expiry. |
| `windsurf` | `ws` | Windsurf (Devin CLI) | OAuth | [link](https://windsurf.com) | In the Windsurf / VS Code IDE, open the command palette and run `Windsurf: Provide Auth Token` (or click the Jupyter "Get Windsurf Authentication Token" button), then copy the shown token and paste it here. Note: opening windsurf.com/show-auth-token directly only renders a "Redirecting" page — the IDE must initiate the flow (it adds a `?state=...` param) for the token to appear. |
| `zed` | `zd` | Zed IDE | OAuth | [link](https://zed.dev) | Zed stores LLM provider credentials (OpenAI, Anthropic, Google, Mistral, xAI) in the OS keychain. Use the Import button below to discover and import them automatically. |
## Web Cookie Providers (23)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `adapta-web` | `adp-web` | Adapta.org (Adapta One Web) | Web cookie | [link](https://agent.adapta.one) | Paste your __client cookie value from .clerk.agent.adapta.one (DevTools → Application → Cookies) |
| `blackbox-web` | `bb-web` | Blackbox Web (Subscription) | Web cookie | [link](https://app.blackbox.ai) | Paste your __Secure-authjs.session-token value or full cookie header from app.blackbox.ai |
| `chatgpt-web` | `cgpt-web` | ChatGPT Web (Plus/Pro) | Web cookie | [link](https://chatgpt.com) | Paste your __Secure-next-auth.session-token cookie value from chatgpt.com |
| `claude-web` | `cw` | Claude Web | Web cookie | [link](https://claude.ai) | Paste your session cookie from claude.ai |
| `copilot-m365-web` | `m365copilot` | Microsoft 365 Copilot (BizChat) | Web cookie | [link](https://m365.cloud.microsoft/chat) | Sign in at m365.cloud.microsoft/chat, then open DevTools → Network → filter 'WS' → click the Chathub WebSocket connection. Copy both the access_token query parameter AND the account-specific Chathub path segment from its request URL (wss://…/Chathub/<path>?…&access_token=…). It is NOT an Authorization: Bearer header on an XHR/Fetch request. The token is short-lived; this is an unofficial integration. |
| `copilot-web` | `copilot` | Microsoft Copilot Web | Web cookie | [link](https://copilot.microsoft.com) | Paste your access_token from copilot.microsoft.com (or export a .har file from DevTools while logged in) |
| `deepseek-web` | `ds-web` | DeepSeek Web | Web cookie | [link](https://chat.deepseek.com) | Paste your userToken from chat.deepseek.com — DevTools → Application → Local Storage → userToken |
| `doubao-web` | `db` | Doubao Web (ByteDance) | Web cookie | [link](https://www.doubao.com) | Paste your session cookie from doubao.com (DevTools → Application → Cookies) |
| `gemini-business` | `gembiz` | Gemini Business (Enterprise) | Web cookie | [link](https://business.gemini.google) | From your enterprise account: open business.gemini.google/home/cid/{your-cid}, then copy __Secure-1PSID and __Secure-1PSIDTS cookies from DevTools → Application → Cookies. Paste as a cookie header below. |
| `gemini-web` | `gweb` | Gemini Web (Free) | Web cookie | [link](https://gemini.google.com) | Paste your __Secure-1PSID cookie value from gemini.google.com. Optionally add __Secure-1PSIDTS separated by semicolon. |
| `grok-web` | `gw` | Grok Web (Subscription) | Web cookie | [link](https://grok.com) | Paste the full grok.com cookie line from DevTools → Application → Cookies. Include both `sso` and `sso-rw` (e.g. `sso=...; sso-rw=...`) — Grok's anti-bot rejects `sso` on its own. |
| `huggingchat` | `huggingchat` | HuggingChat (Free) | Web cookie | [link](https://huggingface.co/chat) | Paste the full Cookie header from huggingface.co/chat (DevTools → Network → /chat/conversation → Request Headers → Cookie). It should include hf-chat and may also include token / aws-waf-token. |
| `inner-ai` | `in-ai` | Inner.ai (Subscription) | Web cookie | [link](https://app.innerai.com) | Paste your token cookie and email separated by a space: open DevTools → Application → Cookies → .innerai.com, copy the token value, then append a space and your Inner.ai login email. Example: eyJhbG... user@example.com |
| `kimi-web` | `kimi-web` | Kimi Web (Moonshot AI) | Web cookie | [link](https://www.kimi.com) | Paste your Cookie header from www.kimi.com (must contain kimi-auth=...). Find it via DevTools → Network → request → Cookie. |
| `lmarena` | `lma` | LMArena (Free) | Web cookie | [link](https://lmarena.ai) | Paste the full Cookie header from lmarena.ai (DevTools → Network → request → Cookie). The session is now split across arena-auth-prod-v1.0, .1, … — copy the whole header. Optional — works with free tier for basic comparisons. |
| `muse-spark-web` | `ms-web` | Muse Spark Web (Meta AI) | Web cookie | [link](https://www.meta.ai) | Paste your ecto_1_sess value or full cookie header from meta.ai |
| `perplexity-web` | `pplx-web` | Perplexity Web (Pro/Max) | Web cookie | [link](https://www.perplexity.ai) | Paste your __Secure-next-auth.session-token cookie value from perplexity.ai |
| `poe-web` | `poe` | Poe Web (Subscription) | Web cookie | [link](https://poe.com) | Paste your p-b cookie value from poe.com (DevTools → Application → Cookies → p-b) |
| `qwen-web` | `qwen-web` | Qwen Web (Free) | Web cookie | [link](https://chat.qwen.ai) | Open chat.qwen.ai, log in, then open DevTools → Application → Local Storage → copy the "token" value (or use tongyi_sso_ticket cookie as Bearer token). |
| `t3-web` | `t3chat` | t3.chat (Pro/Free) | Web cookie | [link](https://t3.chat) | Open t3.chat in your browser, log in, then open DevTools → Application → Local Storage → https://t3.chat. Copy the value of 'convex-session-id'. Also open DevTools → Network, copy the Cookie header from any request. Paste both values here. See provider setup docs for a step-by-step guide. |
| `v0-vercel-web` | `v0` | v0 Vercel Web (Code Gen) | Web cookie | [link](https://v0.dev) | Paste your session cookie from v0.dev (DevTools → Application → Cookies) |
| `venice-web` | `ven` | Venice Web (Privacy) | Web cookie | [link](https://venice.ai) | Paste your session cookie from venice.ai (DevTools → Application → Cookies) |
| `zenmux-free` | `zmf` | ZenMux Free (Web) | Web cookie | [link](https://zenmux.ai) | Login at zenmux.ai, then export all cookies using EditThisCookie or Cookie-Editor and paste the full Cookie header string here. Refresh every ~30 days. |
## API Key Providers (paid / paid-with-free-credits) (158)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `360ai` | `360ai` | 360 AI | API key | [link](https://ai.360.cn) | Get API key at ai.360.cn |
| `agentrouter` | `agentrouter` | AgentRouter | API key, aggregator | [link](https://agentrouter.org) | $200 free credits on signup - multi-model routing gateway |
| `ai21` | `ai21` | AI21 Labs | API key | [link](https://www.ai21.com) | $10 trial credits on signup (valid 3 months), no credit card required |
| `aimlapi` | `aiml` | AI/ML API | API key, aggregator | [link](https://aimlapi.com) | Free tier paused (2026) — AI/ML API is now pay-as-you-go only (min $20 top-up); no recurring free credits. |
| `alibaba` | `ali` | Alibaba | API key | [link](https://bailian.console.alibabacloud.com/) | — |
| `alibaba-cn` | `ali-cn` | Alibaba (China) | API key | [link](https://dashscope.console.aliyun.com/) | — |
| `anthropic` | `anthropic` | Anthropic | API key | [link](https://platform.claude.com) | — |
| `api-airforce` | `af` | Api.airforce | API key | [link](https://api.airforce) | 55 free tier models including Grok-3, Claude 3.7, Qwen3, Kimi-K2, Gemini 2.5 Flash, DeepSeek-V3 |
| `arcee-ai` | `arcee` | Arcee AI | API key | [link](https://arcee.ai) | Get API key at arcee.ai |
| `azure-ai` | `azure-ai` | Azure AI Foundry | API key, enterprise | [link](https://learn.microsoft.com/azure/ai-foundry) | Use your Azure AI Foundry key. Base URL can be https://<resource>.services.ai.azure.com/openai/v1/ or https://<resource>.openai.azure.com/openai/v1/. |
| `azure-openai` | `azure` | Azure OpenAI | API key, enterprise | [link](https://azure.microsoft.com/products/ai-services/openai-service) | Use your Azure OpenAI API key. Base URL should be your resource endpoint, for example https://my-resource.openai.azure.com. |
| `baichuan` | `baichuan` | Baichuan | API key | [link](https://baichuan.com) | Get API key at platform.baichuan-ai.com |
| `baidu` | `baidu` | Baidu (ERNIE) | API key | [link](https://yiyan.baidu.com) | Get API key at console.bce.baidu.com |
| `bailian-coding-plan` | `bcp` | Alibaba Coding Plan | API key | [link](https://www.alibabacloud.com/help/en/model-studio/coding-plan) | — |
| `baseten` | `baseten` | Baseten | API key | [link](https://baseten.co) | $30 free trial credits for GPU inference |
| `bazaarlink` | `bzl` | BazaarLink | API key | [link](https://bazaarlink.ai) | Use your BazaarLink API key (starts with sk-bl-) in Authorization: Bearer <key>. OpenAI SDK works with base URL https://bazaarlink.ai/api/v1. Models use provider/model-name format. |
| `bedrock` | `bedrock` | Amazon Bedrock | API key, enterprise | [link](https://aws.amazon.com/bedrock) | Use your Amazon Bedrock API key and configure the AWS region where your models are enabled (for example eu-west-2). OmniRoute calls Bedrock's native Converse API directly. |
| `black-forest-labs` | `bfl` | Black Forest Labs | API key, image | [link](https://blackforestlabs.ai) | — |
| `blackbox` | `bb` | Blackbox AI | API key | [link](https://blackbox.ai) | Free tier: unlimited basic chat plus Minimax-M2.5, no credit card required |
| `bluesminds` | `bm` | BluesMinds | API key | [link](https://www.bluesminds.com) | Free daily pi credits — supports 200+ models including GPT-4o, GPT-4.1, Claude Sonnet 4.5, Gemini 2.0 Flash, DeepSeek V4, Qwen, Kimi K2 |
| `byteplus` | `bpm` | BytePlus ModelArk | API key | [link](https://console.byteplus.com/ark) | — |
| `bytez` | `bytez` | Bytez | API key | [link](https://bytez.com) | $1 free credits, refreshes every 4 weeks |
| `cablyai` | `cablyai` | CablyAI | API key, aggregator | [link](https://cablyai.com) | ⚠️ **DEPRECATED.** cablyai.com no longer resolves (DNS NXDOMAIN, verified 2026-06-30) — the domain is gone and every request fails with a DNS error (#5568). |
| `cerebras` | `cerebras` | Cerebras | API key | [link](https://inference.cerebras.ai) | Free Trial: 1M tokens/day, 30K TPM, 5 RPM — no credit card. |
| `chutes` | `chutes` | Chutes.ai | API key, aggregator | [link](https://chutes.ai) | Bearer API key for the Chutes OpenAI-compatible gateway. |
| `clarifai` | `clarifai` | Clarifai | API key, enterprise | [link](https://docs.clarifai.com) | Use your Clarifai PAT or app-specific API key. OmniRoute targets the OpenAI-compatible endpoint at https://api.clarifai.com/v2/ext/openai/v1 and authenticates with Authorization: Key <token>. |
| `cloudflare-ai` | `cf` | Cloudflare Workers AI | API key | [link](https://developers.cloudflare.com/workers-ai) | Requires API Token AND Account ID (found at dash.cloudflare.com) |
| `codestral` | `codestral` | Codestral | API key | [link](https://mistral.ai) | — |
| `cohere` | `cohere` | Cohere | API key | [link](https://cohere.com) | Free Trial: 1,000 API calls/month for testing, no credit card required |
| `command-code` | `cmd` | Command Code | API key | [link](https://commandcode.ai/) | Use a Command Code API key. Requests are sent to Command Code's /alpha/generate endpoint. |
| `coze` | `coze` | Coze | API key | [link](https://coze.com) | Get API key at coze.com/open/api |
| `crof` | `crof` | CrofAI | API key | [link](https://crof.ai) | — |
| `databricks` | `databricks` | Databricks | API key, enterprise | [link](https://www.databricks.com) | — |
| `datarobot` | `datarobot` | DataRobot | API key, enterprise | [link](https://docs.datarobot.com) | Use your DataRobot API token. Optional Base URL can be the account root (for LLM Gateway) or a deployment URL under /api/v2/deployments/<id>. |
| `deepinfra` | `deepinfra` | DeepInfra | API key | [link](https://deepinfra.com) | Free signup credits for API testing and model exploration |
| `deepseek` | `ds` | DeepSeek | API key | [link](https://platform.deepseek.com) | 5M free tokens on signup - no credit card required |
| `dgrid` | `dgrid` | DGrid | API key | [link](https://dgrid.ai) | DGrid Free Models Router: 10 requests/minute and 100 requests/day. A $5 lifetime top-up unlocks up to 20 requests/minute and 1,000 requests/day. |
| `dify` | `dify` | Dify | API key | [link](https://dify.ai) | Get API key from your Dify instance. |
| `dit` | `dai` | DIT.ai | API key | [link](https://dit.ai) | Use your dit.ai API key in Authorization: Bearer <key>. Fully OpenAI-compatible — a drop-in replacement, just change the base URL to https://api.dit.ai/v1. |
| `doubao` | `doubao` | Doubao | API key | [link](https://doubao.com) | Get API key at console.volcengine.com |
| `empower` | `empower` | Empower | API key, aggregator | [link](https://docs.empower.dev) | Bearer API key for the Empower OpenAI-compatible endpoint. |
| `factory` | `factory` | Factory | API key | [link](https://factory.ai) | Bearer API key for the Factory OpenAI-compatible gateway. |
| `fal-ai` | `fal` | Fal.ai | API key, image | [link](https://fal.ai) | — |
| `featherless-ai` | `featherless` | Featherless AI | API key | [link](https://featherless.ai) | Free tier available — no credit card required |
| `fenayai` | `fenayai` | FenayAI | API key, aggregator | [link](https://fenayai.com) | Bearer API key for the FenayAI OpenAI-compatible gateway. |
| `firecrawl` | `fc` | Firecrawl | API key | [link](https://firecrawl.dev) | — |
| `fireworks` | `fireworks` | Fireworks AI | API key | [link](https://fireworks.ai) | $1 free starter credits on signup for API testing |
| `freeaiapikey` | `faik` | FreeAIAPIKey | API key | [link](https://freeaiapikey.com) | — |
| `freemodel-dev` | `fmd` | FreeModel.dev | API key | [link](https://freemodel.dev) | $300 free credits on signup — no credit card required. Access GPT-5.4 and GPT-5.5 (OpenAI's latest flagship models) through an OpenAI-compatible API. |
| `friendliai` | `friendli` | FriendliAI | API key | [link](https://friendli.ai) | Free tier for serverless inference — no credit card required |
| `galadriel` | `galadriel` | Galadriel | API key | [link](https://galadriel.com) | ⚠️ **DEPRECATED.** api.galadriel.ai no longer resolves (sweep 2026-06-19); the inference API appears discontinued. |
| `gemini` | `gemini` | Gemini (Google AI Studio) | API key | [link](https://aistudio.google.com) | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card, get key at aistudio.google.com |
| `getgoapi` | `ggo` | GoAPI | API key, aggregator | [link](https://api.getgoapi.com) | — |
| `gigachat` | `gigachat` | GigaChat (Sber) | API key | [link](https://developers.sber.ru) | — |
| `github-models` | `ghm` | GitHub Models | API key | [link](https://github.com/marketplace/models) | Create a GitHub PAT with 'models: read' scope at github.com/settings/tokens |
| `gitlab` | `gitlab` | GitLab Duo PAT | API key | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | GitLab personal access token for the public Code Suggestions API. Configure a self-hosted base URL when not using gitlab.com. |
| `gitlawb` | `glb` | Gitlawb Opengateway (MiMo) | API key | [link](https://opengateway.gitlawb.com) | Free MiMo (xiaomi/mimo-v2.5) revoked 2026-05 — Opengateway is now a pay-as-you-go credit gateway; no recurring free model. |
| `gitlawb-gmi` | `glb-gmi` | Gitlawb Opengateway (GMI Cloud) | API key | [link](https://opengateway.gitlawb.com) | Free Nemotron promo ended 2026-06 — the GMI Cloud route is now pay-as-you-go credit only. |
| `glhf` | `glhf` | GLHF Chat | API key, aggregator | [link](https://glhf.chat) | ⚠️ **DEPRECATED.** glhf.chat shut down (2026); its api.laf.run gateway no longer serves the catalog (sweep 2026-06-19). |
| `glm` | `glm` | GLM Coding | API key | [link](https://z.ai/subscribe) | — |
| `glm-cn` | `glmcn` | GLM Coding (China) | API key | [link](https://open.bigmodel.cn) | — |
| `glmt` | `glmt` | GLM Thinking | API key | [link](https://open.bigmodel.cn) | — |
| `groq` | `groq` | Groq | API key | [link](https://groq.com) | Free tier: 30 RPM / 14.4K RPD — no credit card |
| `hackclub` | `hc` | Hackclub AI | API key, aggregator | [link](https://ai.hackclub.com) | Sign in with your Hack Club account at ai.hackclub.com. |
| `haiper` | `hp` | Haiper | API key, video | [link](https://haiper.ai) | Get API key at haiper.ai/haiper-api |
| `heroku` | `heroku` | Heroku AI | API key, enterprise | [link](https://www.heroku.com) | — |
| `huggingface` | `hf` | HuggingFace | API key | [link](https://huggingface.co) | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
| `hyperbolic` | `hyp` | Hyperbolic | API key | [link](https://hyperbolic.xyz) | $1-5 trial credits on signup for serverless inference |
| `ideogram` | `ideo` | Ideogram | API key | [link](https://ideogram.ai) | Get API key at ideogram.ai/docs/api |
| `iflytek` | `iflytek` | iFlytek Spark | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
| `inclusionai` | `inclusion` | InclusionAI | API key | [link](https://inclusionai.com) | ⚠️ **DEPRECATED.** api.inclusionai.tech no longer resolves (sweep 2026-06-19); the inference API appears discontinued. |
| `inference-net` | `inet` | Inference.net | API key | [link](https://inference.net) | $25 free credits on signup plus research grants available |
| `jina-ai` | `jina` | Jina AI | API key, embed/rerank | [link](https://jina.ai) | Bearer API key for the Jina AI rerank API. |
| `jina-reader` | `jr` | Jina Reader | API key | [link](https://jina.ai/reader) | — |
| `kie` | `kie` | KIE.AI | API key | [link](https://kie.ai) | — |
| `kilo-gateway` | `kg` | Kilo Gateway | API key, aggregator | [link](https://kilo.ai) | — |
| `kimi` | `kimi` | Kimi | API key | [link](https://platform.moonshot.ai) | — |
| `kimi-coding-apikey` | `kmca` | Kimi Coding (API Key) | API key | [link](https://www.kimi.com/code) | — |
| `kluster` | `kluster` | Kluster AI | API key | [link](https://kluster.ai) | ⚠️ **DEPRECATED.** kluster.ai shut down (2026-06-09); api.kluster.ai no longer resolves (sweep 2026-06-19). Use another OpenAI-compatible provider. |
| `lambda-ai` | `lambda` | Lambda AI | API key | [link](https://lambda.ai) | — |
| `laozhang` | `lz` | LaoZhang AI | API key, aggregator | [link](https://api.laozhang.ai) | — |
| `leonardo` | `leo` | Leonardo AI | API key, video | [link](https://leonardo.ai) | Get API key at leonardo.ai/developer |
| `liquid` | `liquid` | Liquid AI | API key | [link](https://liquid.ai) | Get API key at liquid.ai |
| `llamagate` | `llamagate` | LlamaGate | API key | [link](https://llamagate.ai) | — |
| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | No signup required - 2 req/s, 20 RPM, 100 req/hr free tier |
| `longcat` | `lc` | LongCat AI | API key | [link](https://longcat.chat/platform/docs) | Free: one-time 10M-token grant after account signup + KYC verification (LongCat-2.0). One-time only — not a recurring daily/monthly allowance. |
| `maritalk` | `maritalk` | Maritalk | API key | [link](https://www.maritaca.ai) | — |
| `meta-llama` | `meta` | Meta Llama API | API key | [link](https://llama.developer.meta.com) | — |
| `minimax` | `minimax` | Minimax Coding | API key, video | [link](https://www.minimax.io) | — |
| `minimax-cn` | `minimax-cn` | Minimax (China) | API key | [link](https://www.minimaxi.com) | — |
| `mistral` | `mistral` | Mistral | API key | [link](https://mistral.ai) | Free Experiment tier: rate-limited access to all models, no credit card required |
| `modal` | `mdl` | Modal | API key, enterprise | [link](https://modal.com/docs) | Use the bearer token that protects your Modal deployment, if enabled. Base URL should point to your OpenAI-compatible Modal app, for example https://<workspace>--<app>.modal.run/v1. |
| `monsterapi` | `monster` | MonsterAPI | API key | [link](https://monsterapi.ai) | Get API key at monsterapi.ai |
| `moonshot` | `moonshot` | Moonshot AI | API key | [link](https://platform.moonshot.ai) | — |
| `morph` | `morph` | Morph | API key | [link](https://morphllm.com) | Free tier: 250K credits/month, $0 |
| `nanogpt` | `nanogpt` | NanoGPT | API key | [link](https://nano-gpt.com) | — |
| `nebius` | `nebius` | Nebius AI | API key | [link](https://nebius.com) | ~$1 trial credits on signup for API testing |
| `nlpcloud` | `nlpc` | NLP Cloud | API key | [link](https://docs.nlpcloud.com) | Use your NLP Cloud API key in Authorization: Token <key>. OmniRoute targets the chatbot endpoint on https://api.nlpcloud.io/v1/gpu/<model>/chatbot by default. |
| `nomic` | `nomic` | Nomic | API key | [link](https://nomic.ai) | Get API key at atlas.nomic.ai |
| `nous-research` | `nous` | Nous Research | API key | [link](https://portal.nousresearch.com/help) | Use your Nous Portal API key. OmniRoute targets the official OpenAI-compatible inference endpoint at https://inference-api.nousresearch.com/v1. |
| `novita` | `novita` | Novita AI | API key, aggregator | [link](https://novita.ai) | $0.50 trial credits on signup (valid about 1 year) |
| `nscale` | `nscale` | nScale | API key | [link](https://nscale.com) | $5 free credits on signup for inference testing |
| `nvidia` | `nvidia` | NVIDIA NIM | API key | [link](https://build.nvidia.com) | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2...) |
| `oci` | `oci` | OCI Generative AI | API key, enterprise | [link](https://www.oracle.com/artificial-intelligence/generative-ai) | Use your OCI Generative AI API key or IAM bearer token. Base URL can be https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1/. |
| `ollama-cloud` | `ollamacloud` | Ollama Cloud | API key | [link](https://ollama.com/settings/keys) | — |
| `openadapter` | `oad` | OpenAdapter | API key | [link](https://openadapter.dev) | Use your OpenAdapter API key in Authorization: Bearer sk-cv-<key>. Fully OpenAI-compatible. API base URL: https://api.openadapter.in/v1. |
| `openai` | `openai` | OpenAI | API key | [link](https://platform.openai.com) | — |
| `opencode-go` | `opencode-go` | OpenCode Go | API key | [link](https://opencode.ai/go) | — |
| `opencode-zen` | `opencode-zen` | OpenCode Zen | API key | [link](https://opencode.ai/zen) | — |
| `openrouter` | `openrouter` | OpenRouter | API key, aggregator | [link](https://openrouter.ai) | Free models at $0/token with :free suffix - 20 RPM / 200 RPD |
| `orcarouter` | `orcarouter` | OrcaRouter | API key | [link](https://www.orcarouter.ai) | — |
| `ovhcloud` | `ovh` | OVHcloud AI | API key | [link](https://www.ovhcloud.com) | — |
| `perplexity` | `pplx` | Perplexity | API key | [link](https://www.perplexity.ai) | — |
| `piapi` | `pi` | PiAPI | API key, aggregator | [link](https://piapi.ai) | — |
| `pioneer` | `pn` | Pioneer AI | API key | [link](https://pioneer.ai) | $75 free usage credits — no credit card required |
| `poe` | `poe` | Poe | API key, aggregator | [link](https://creator.poe.com/api-reference) | Bearer API key for the Poe OpenAI-compatible API. |
| `pollinations` | `pol` | Pollinations AI | API key, video | [link](https://pollinations.ai) | Free keyless tier: openai, openai-fast, openai-large, qwen-coder, mistral, deepseek, grok, gemini-flash-lite-3.1, perplexity-fast, perplexity-reasoning. Premium models (claude, gemini, midijourney) require a Pollinations API key from enter.pollinations.ai. |
| `predibase` | `predibase` | Predibase | API key | [link](https://predibase.com) | ⚠️ **DEPRECATED.** serving.app.predibase.com no longer resolves (sweep 2026-06-19); the managed serving API appears discontinued. |
| `publicai` | `publicai` | PublicAI | API key | [link](https://publicai.co) | Requires an API key — one-time signup credit, then paid |
| `puter` | `pu` | Puter AI | API key | [link](https://puter.com) | Get token at puter.com/dashboard → Copy Auth Token |
| `qianfan` | `qianfan` | Baidu Qianfan | API key | [link](https://cloud.baidu.com/product/wenxinworkshop) | — |
| `recraft` | `recraft` | Recraft | API key, image | [link](https://recraft.ai) | — |
| `reka` | `reka` | Reka | API key | [link](https://docs.reka.ai/chat/overview) | Use your Reka API key. OmniRoute supports the OpenAI-compatible base URL https://api.reka.ai/v1 and sends both Authorization and X-Api-Key headers for compatibility. |
| `runwayml` | `runway` | Runway | API key, video | [link](https://docs.dev.runwayml.com) | Use your Runway API key in Authorization: Bearer <key>. OmniRoute targets the current Runway API at https://api.dev.runwayml.com/v1 and sends the required X-Runway-Version header automatically. |
| `sambanova` | `samba` | SambaNova | API key | [link](https://sambanova.ai) | $5 free credits on signup (30-day validity), no credit card required |
| `sap` | `sap` | SAP Generative AI Hub | API key, enterprise | [link](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/generative-ai-hub-in-sap-ai-core) | Use your SAP AI Core bearer token. Base URL can be your AI_API_URL root or a deploymentUrl from Generative AI Hub. |
| `scaleway` | `scw` | Scaleway AI | API key | [link](https://www.scaleway.com/en/docs/ai-data/generative-apis/) | 1M free tokens for new accounts — EU/GDPR compliant (Paris), Qwen3 235B & Llama 70B |
| `sensenova` | `sensenova` | SenseNova | API key | [link](https://platform.sensenova.cn) | Get API key at platform.sensenova.cn |
| `siliconflow` | `siliconflow` | SiliconFlow | API key | [link](https://cloud.siliconflow.com) | $1 free credits plus permanently free models after identity verification |
| `snowflake` | `snowflake` | Snowflake Cortex | API key, enterprise | [link](https://www.snowflake.com) | — |
| `sparkdesk` | `sparkdesk` | SparkDesk | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
| `stability-ai` | `stability` | Stability AI | API key, image | [link](https://stability.ai) | — |
| `stepfun` | `stepfun` | StepFun | API key | [link](https://stepfun.com) | Get API key at platform.stepfun.com |
| `suno` | `suno` | Suno | API key | [link](https://suno.ai) | Paste session cookie from suno.ai (Clerk auth) |
| `synthetic` | `synthetic` | Synthetic | API key, aggregator | [link](https://synthetic.new) | — |
| `tencent` | `tencent` | Tencent Hunyuan | API key | [link](https://hunyuan.tencent.com) | Get API key at console.cloud.tencent.com |
| `thebai` | `thebai` | TheB.AI | API key, aggregator | [link](https://theb.ai) | Bearer API key for the TheB.AI OpenAI-compatible gateway. |
| `together` | `together` | Together AI | API key, video | [link](https://www.together.ai) | — |
| `tokenrouter` | `trk` | TokenRouter | API key | [link](https://tokenrouter.com) | Use your TokenRouter API key in Authorization: Bearer <key>. Fully OpenAI-compatible. API base URL: https://api.tokenrouter.com/v1. |
| `topaz` | `topaz` | Topaz | API key, image | [link](https://topazlabs.com) | — |
| `udio` | `udio` | Udio | API key | [link](https://udio.com) | Paste session cookie from udio.com (Supabase auth) |
| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. |
| `upstage` | `upstage` | Upstage | API key | [link](https://www.upstage.ai) | — |
| `v0-vercel` | `v0` | v0 (Vercel) | API key | [link](https://v0.dev) | — |
| `venice` | `venice` | Venice.ai | API key | [link](https://venice.ai) | — |
| `vercel-ai-gateway` | `vag` | Vercel AI Gateway | API key, aggregator | [link](https://vercel.com/docs/ai-gateway) | — |
| `vertex` | `vertex` | Vertex AI | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide Service Account JSON or OAuth access_token |
| `vertex-partner` | `vp` | Vertex AI Partners | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide the same Service Account JSON used for Vertex AI partner models. |
| `volcengine` | `volcengine` | Volcengine | API key | [link](https://www.volcengine.com) | — |
| `voyage-ai` | `voyage` | Voyage AI | API key, embed/rerank | [link](https://www.voyageai.com) | Bearer API key for Voyage AI embeddings and rerank APIs. |
| `wafer` | `wafer` | Wafer AI | API key | [link](https://wafer.ai) | — |
| `wandb` | `wandb` | Weights & Biases Inference | API key | [link](https://wandb.ai) | — |
| `watsonx` | `watsonx` | IBM watsonx.ai Gateway | API key, enterprise | [link](https://www.ibm.com/products/watsonx-ai) | Use your watsonx bearer token. Base URL can be https://<region>.ml.cloud.ibm.com/ml/gateway/v1/ or a self-managed /ml/gateway/v1 endpoint. |
| `xai` | `xai` | xAI (Grok) | API key | [link](https://x.ai) | — |
| `xiaomi-mimo` | `mimo` | Xiaomi MiMo | API key | [link](https://mimo.mi.com) | — |
| `yi` | `yi` | Yi (01.AI) | API key | [link](https://01.ai) | Get API key at platform.lingyiwanwu.com |
| `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — |
| `zenmux` | `zm` | ZenMux | API key | [link](https://zenmux.ai) | Use your ZenMux API key in Authorization: Bearer <key>. ZenMux is fully OpenAI-compatible. Base URL: https://zenmux.ai/api/v1. |
## Local Providers (12)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `comfyui` | `comfyui` | ComfyUI | Local | [link](https://github.com/comfyanonymous/ComfyUI) | No API key required. Configure the local ComfyUI base URL (default: http://localhost:8188). |
| `docker-model-runner` | `dmr` | Docker Model Runner | Local, self-hosted | [link](https://docs.docker.com/ai/model-runner/) | API key optional. Configure the local Docker Model Runner OpenAI-compatible base URL (default: http://localhost:12434/v1). |
| `lemonade` | `lemonade` | Lemonade Server | Local, self-hosted | [link](https://lemonade-server.ai) | API key optional. Configure the local Lemonade OpenAI-compatible base URL (default: http://localhost:13305/api/v1). |
| `llama-cpp` | `llamacpp` | llama.cpp | Local, self-hosted | [link](https://github.com/ggml-org/llama.cpp) | API key optional (use any value, e.g. sk-no-key-required). Configure the llama-server OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). Note: if Llamafile is also installed, both default to port 8080 — run only one at a time or override the port. |
| `llamafile` | `llamafile` | Llamafile | Local, self-hosted | [link](https://github.com/Mozilla-Ocho/llamafile) | API key optional. Configure the local Llamafile OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). |
| `lm-studio` | `lmstudio` | LM Studio | Local, self-hosted | [link](https://lmstudio.ai) | API key optional. Configure the local LM Studio OpenAI-compatible base URL (default: http://localhost:1234/v1). |
| `ollama-local` | `ollama` | Ollama | Local, self-hosted | [link](https://ollama.com) | No API key required. Ollama runs locally — configure its OpenAI-compatible base URL (default: http://localhost:11434/v1) and make sure Ollama is running before connecting. |
| `oobabooga` | `ooba` | oobabooga | Local, self-hosted | [link](https://github.com/oobabooga/text-generation-webui) | API key optional. Configure the local oobabooga OpenAI-compatible base URL (default: http://localhost:5000/v1). |
| `sdwebui` | `sdwebui` | SD WebUI | Local | [link](https://github.com/AUTOMATIC1111/stable-diffusion-webui) | No API key required. Configure the local WebUI base URL (default: http://localhost:7860). |
| `triton` | `triton` | NVIDIA Triton | Local, self-hosted | [link](https://developer.nvidia.com/triton-inference-server) | API key optional. Configure the Triton OpenAI-compatible base URL (default: http://localhost:8000/v1). |
| `vllm` | `vllm` | vLLM | Local, self-hosted | [link](https://github.com/vllm-project/vllm) | API key optional. Configure the local vLLM OpenAI-compatible base URL (default: http://localhost:8000/v1). |
| `xinference` | `xinference` | XInference | Local, self-hosted | [link](https://inference.readthedocs.io) | API key optional. Configure the local XInference OpenAI-compatible base URL (default: http://localhost:9997/v1). |
## Search Providers (11)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `brave-search` | `brave-search` | Brave Search | Search | [link](https://brave.com/search/api) | Subscription token from Brave Search API dashboard |
| `exa-search` | `exa-search` | Exa Search | Search | [link](https://exa.ai) | API key from dashboard.exa.ai |
| `google-pse-search` | `google-pse` | Google Programmable Search | Search | [link](https://developers.google.com/custom-search/v1/overview) | Requires a Google API key and your Programmable Search Engine ID (cx) |
| `linkup-search` | `linkup` | Linkup Search | Search | [link](https://docs.linkup.so) | Bearer API key from the Linkup dashboard |
| `ollama-search` | `ollama-search` | Ollama Search | Search | [link](https://ollama.com/settings/keys) | Same API key as Ollama Cloud (from ollama.com/settings/keys) |
| `perplexity-search` | `pplx-search` | Perplexity Search | Search | [link](https://docs.perplexity.ai/guides/search-quickstart) | Same API key as Perplexity (pplx-...) |
| `searchapi-search` | `searchapi` | SearchAPI | Search | [link](https://www.searchapi.io/docs/google) | API key from SearchAPI (query param or Bearer auth) |
| `searxng-search` | `searxng` | SearXNG Search | Search | [link](https://docs.searxng.org) | API key is optional. Set your SearXNG base URL. Some instances may require a bearer token for access. |
| `serper-search` | `serper-search` | Serper Search | Search | [link](https://serper.dev) | API key from serper.dev dashboard |
| `tavily-search` | `tavily-search` | Tavily Search | Search | [link](https://tavily.com) | API key from app.tavily.com (format: tvly-...) |
| `youcom-search` | `youcom-search` | You.com Search | Search | [link](https://you.com/business/api/) | X-API-Key from the You.com platform dashboard |
## Audio-only Providers (7)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `assemblyai` | `aai` | AssemblyAI | Audio | [link](https://assemblyai.com) | — |
| `aws-polly` | `polly` | AWS Polly | Audio | [link](https://aws.amazon.com/polly/) | Use AWS Secret Access Key as API key; set providerSpecificData.accessKeyId and optional region. |
| `cartesia` | `cartesia` | Cartesia | Audio | [link](https://cartesia.ai) | — |
| `deepgram` | `dg` | Deepgram | Audio | [link](https://deepgram.com) | — |
| `elevenlabs` | `el` | ElevenLabs | Audio | [link](https://elevenlabs.io) | — |
| `inworld` | `inworld` | Inworld | Audio | [link](https://inworld.ai) | — |
| `playht` | `playht` | PlayHT | Audio | [link](https://play.ht) | — |
## Upstream Proxy Providers (2)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `9router` | `nr` | 9router | Upstream proxy | [link](https://www.npmjs.com/package/9router) | — |
| `cliproxyapi` | `cpa` | CLIProxyAPI | Upstream proxy | [link](https://github.com/router-for-me/CLIProxyAPI) | — |
## Cloud Agent Providers (3)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `codex-cloud` | `codex-cloud` | Codex Cloud | Cloud agent | [link](https://openai.com/codex) | OpenAI API key with Codex Cloud task access. |
| `devin` | `devin` | Devin | Cloud agent | [link](https://devin.ai) | Devin API key for cloud agent sessions. |
| `jules` | `jules` | Google Jules | Cloud agent | [link](https://jules.google) | Jules API key for creating and managing cloud coding tasks. |
## System Providers (1)
| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
| `auto` | `auto` | Auto (Zero-Config) | System | — | — |
## Sources of truth
- Catalog: [`src/shared/constants/providers.ts`](../../src/shared/constants/providers.ts)
- Registry (per-model details): [`open-sse/config/providerRegistry.ts`](../../open-sse/config/providerRegistry.ts)
- Executors: [`open-sse/executors/`](../../open-sse/executors/) (31 files)
- Translators: [`open-sse/translator/`](../../open-sse/translator/)
## See Also
- [FREE_TIERS.md](./FREE_TIERS.md) — curated free-tier guide
- [USER_GUIDE.md](../guides/USER_GUIDE.md) — provider setup walkthrough
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — overall architecture
docs/reference/RELAY_BACKEND_STRATEGY.md
+84
View File
@@ -0,0 +1,84 @@
---
title: "Relay Backend Strategy"
version: 3.8.42
lastUpdated: 2026-06-30
---
# Relay Backend Strategy
## Summary
OmniRoute now supports three relay modes for `/api/v1/relay/chat/completions`:
- `ts`: Use the TypeScript relay in-process.
- `bifrost`: Force the Bifrost gateway.
- `auto`: Prefer Bifrost when available, fall back to TypeScript on failure.
When you are running at high request rate (large number of tokens/day, near-constant throughput), the best strategy is to keep the fallback path explicit and fast so the hot path never blocks on a dead sidecar.
## Mode behavior
- `ts`
- Lowest operational complexity.
- All routing and validation runs in Node.
- No sidecar dependency for availability.
- `bifrost`
- Force all requests through the sidecar gateway.
- No automatic fallback.
- Useful only when sidecar health and latency are guaranteed.
- `auto`
- Sidecar is used when it can be reached and is enabled.
- Failed attempts trigger fallback headers and return traffic to TS so request success remains bounded.
- This mode is the safest choice for production when uptime matters more than strict sidecar-only routing.
## 9router vs CLIPROXYAPI today
9router and CLIPROXYAPI are both integrations that historically exposed compatibility paths for upstream providers.
- 9router is an embedded path for upstream orchestration and compatibility behavior.
- CLIPROXYAPI is a proxy API bridge for CLI / SDK style traffic.
- Bifrost is being stabilized as the externalized path when you need a dedicated sidecar-like hop and low-latency local dispatch.
If you are currently comparing 9router/CLIPROXYAPI:
- Keep request signing, allowlist checks, and DB policy gates in the API route before handoff.
- If a workflow needs strict sidecar behavior and lower per-request variance, use `OMNIROUTE_RELAY_BACKEND=bifrost`.
- If you need sidecar resilience with graceful degradation under incident conditions, use `OMNIROUTE_RELAY_BACKEND=auto`.
## Backend boundary contract
The stable product boundary is the OmniRoute relay API, not the dashboard implementation. The Next.js dashboard may install, configure, and supervise local services, but request routing should enter through the relay API and hand off behind that boundary.
Long-term backend choices:
- Keep the TypeScript relay as the in-process policy and fallback path. Auth, allowlists, request normalization, accounting, and safety checks stay here before any backend handoff.
- Use Bifrost as the preferred high-throughput Tier-1 sidecar when the deployment needs lower routing variance, centralized provider rotation, or scale-out across multiple OmniRoute replicas.
- Keep 9router and CLIPROXYAPI as embedded compatibility services. They run as supervised local processes and are useful when their provider/CLI behavior is the desired adapter, but they should not become the default Tier-1 routing engine.
- Do not make the dashboard pass arbitrary service URLs into the hot path. UI-provided URLs are configuration inputs; routing code should resolve registered, health-checked backends from server-side settings and supervisor state.
- Prefer loopback HTTP for supervised services today because the managed processes already expose HTTP-compatible APIs, the route guard can audit the boundary, and failure/fallback behavior is visible in request logs. A future SDK or socket transport is only worth adding if it measurably reduces p99 routing latency without weakening isolation or fallback semantics.
For a very high-throughput deployment, the default answer is therefore `auto` with Bifrost enabled: use the Go sidecar on the hot path while preserving the TypeScript fallback for success rate. Use `bifrost` only when strict sidecar-only behavior is more important than graceful degradation.
## High-throughput guidance
For sustained high RPM/RPS and strict success SLO:
1. Use `auto` with a practical cooldown and failure telemetry.
2. Keep upstream validation and API-key checks on the TypeScript route boundary.
3. Enable explicit headers/counters so your alerting sees fallback frequency and reasons.
4. Tune sidecar timeouts to fail fast, not to wait forever.
5. Keep service auto-restart and health telemetry loop healthy so fallback is truly exceptional.
## Suggested baseline
- `OMNIROUTE_RELAY_BACKEND=auto`
- `BIFROST_ENABLED=1`
- Keep API keys, allowlist, sanitizer, and rate-limit checks enabled in route handlers (they always run before downstream forwarding).
- Export fallback metrics from your reverse proxy and request logs so sidecar outages are visible within one minute.
## Provider plugin contract
Sidecars should import provider metadata through the JSON-safe provider plugin
manifest instead of depending on TypeScript executor internals. See
[Provider Plugin Manifest](./PROVIDER_PLUGIN_MANIFEST.md) for the sidecar
eligibility contract and migration phases.
docs/reference/meta.json
+11
View File
@@ -0,0 +1,11 @@
{
"title": "Reference",
"pages": [
"API_REFERENCE",
"CLI-TOOLS",
"ENVIRONMENT",
"FEATURE_FLAGS",
"FREE_TIERS",
"PROVIDER_REFERENCE"
]
}