wehub-resource-sync
370 lines
16 KiB
Markdown
370 lines
16 KiB
Markdown
# Contributing to OmniRoute
|
|
|
|
Thank you for your interest in contributing! This guide covers everything you need to get started.
|
|
|
|
---
|
|
|
|
## Development Setup
|
|
|
|
### Prerequisites
|
|
|
|
- **Node.js** `>=22.22.3 <23`, or `>=24.0.0 <27` (recommended: 24 LTS)
|
|
- **npm** 10+
|
|
- **Git**
|
|
|
|
### Clone & Install
|
|
|
|
```bash
|
|
git clone https://github.com/diegosouzapw/OmniRoute.git
|
|
cd OmniRoute
|
|
npm install
|
|
```
|
|
|
|
### Environment Variables
|
|
|
|
```bash
|
|
# Create your .env from the template
|
|
cp .env.example .env
|
|
|
|
# Generate required secrets
|
|
echo "JWT_SECRET=$(openssl rand -base64 48)" >> .env
|
|
echo "API_KEY_SECRET=$(openssl rand -hex 32)" >> .env
|
|
```
|
|
|
|
Key variables for development:
|
|
|
|
| Variable | Development Default | Description |
|
|
| ---------------------- | ------------------------ | --------------------- |
|
|
| `PORT` | `20128` | Server port |
|
|
| `NEXT_PUBLIC_BASE_URL` | `http://localhost:20128` | Base URL for frontend |
|
|
| `JWT_SECRET` | (generate above) | JWT signing secret |
|
|
| `INITIAL_PASSWORD` | `CHANGEME` | First login password |
|
|
| `APP_LOG_LEVEL` | `info` | Log verbosity level |
|
|
|
|
### Dashboard Settings
|
|
|
|
The dashboard provides UI toggles for features that can also be configured via environment variables:
|
|
|
|
| Setting Location | Toggle | Description |
|
|
| ------------------- | ------------------ | ------------------------------ |
|
|
| Settings → Advanced | Debug Mode | Enable debug request logs (UI) |
|
|
| Settings → General | Sidebar Visibility | Show/hide sidebar sections |
|
|
|
|
These settings are stored in the database and persist across restarts, overriding env var defaults when set.
|
|
|
|
### Running Locally
|
|
|
|
```bash
|
|
# Development mode (hot reload)
|
|
npm run dev
|
|
|
|
# Production build
|
|
npm run build # next build → .build/next/ then assembleStandalone → dist/
|
|
npm run start
|
|
|
|
# Release build (clean rebuild + HEAD sentinel — required for deploy)
|
|
npm run build:release # rm -rf .build dist && build + writes dist/BUILD_SHA
|
|
|
|
# Common port configuration
|
|
PORT=20128 NEXT_PUBLIC_BASE_URL=http://localhost:20128 npm run dev
|
|
```
|
|
|
|
### Build Output Layout
|
|
|
|
| Directory | Contents | Tracked |
|
|
| --------- | ------------------------------------------------------------------------- | ------- |
|
|
| `src/` | Application source (TypeScript / TSX) | Yes |
|
|
| `.build/` | Intermediates — `next build` output (gitignored, `distDir = .build/next`) | No |
|
|
| `dist/` | Shippable bundle — assembled by `assembleStandalone` (gitignored) | No |
|
|
|
|
The build pipeline is a single pass:
|
|
|
|
```
|
|
npm run build
|
|
└─ next build → .build/next/standalone (Next.js output)
|
|
└─ assembleStandalone() (copies standalone + static + public + native assets)
|
|
└─ output: dist/ (server.js, .next/static/, public/, node_modules/)
|
|
```
|
|
|
|
`npm run build:release` additionally cleans both directories first and writes
|
|
`dist/BUILD_SHA` (= `git rev-parse --short HEAD`) as a deploy integrity sentinel.
|
|
|
|
> **VPS deploy note:** the remote image directory `/usr/lib/node_modules/omniroute/app/`
|
|
> is unchanged. The deploy skills rsync the contents of `dist/` into it.
|
|
> Only the in-repo build output path moved (`app/` → `dist/`).
|
|
|
|
Default URLs:
|
|
|
|
- **Dashboard**: `http://localhost:20128/dashboard`
|
|
- **API**: `http://localhost:20128/v1`
|
|
|
|
---
|
|
|
|
## Git Workflow
|
|
|
|
> ⚠️ **NEVER commit directly to `main`.** Always use feature branches.
|
|
|
|
```bash
|
|
git checkout -b feat/your-feature-name
|
|
# ... make changes ...
|
|
git commit -m "feat: describe your change"
|
|
git push -u origin feat/your-feature-name
|
|
# Open a Pull Request on GitHub
|
|
```
|
|
|
|
### Branch Naming
|
|
|
|
| Prefix | Purpose |
|
|
| ----------- | ------------------------- |
|
|
| `feat/` | New features |
|
|
| `fix/` | Bug fixes |
|
|
| `refactor/` | Code restructuring |
|
|
| `docs/` | Documentation changes |
|
|
| `test/` | Test additions/fixes |
|
|
| `chore/` | Tooling, CI, dependencies |
|
|
|
|
### Commit Messages
|
|
|
|
Follow [Conventional Commits](https://www.conventionalcommits.org/):
|
|
|
|
```
|
|
feat: add circuit breaker for provider calls
|
|
fix: resolve JWT secret validation edge case
|
|
docs: update SECURITY.md with PII protection
|
|
test: add observability unit tests
|
|
refactor(db): consolidate rate limit tables
|
|
```
|
|
|
|
Scopes (v3.8): `db`, `sse`, `oauth`, `dashboard`, `api`, `cli`, `docker`, `ci`, `mcp`, `a2a`, `memory`, `skills`, `cloud-agent`, `guardrails`, `compression`, `auto-combo`, `resilience`, `providers`, `executors`, `translator`, `domain`, `authz`.
|
|
|
|
---
|
|
|
|
## Running Tests
|
|
|
|
```bash
|
|
# All tests (unit + vitest + ecosystem + e2e)
|
|
npm run test:all
|
|
|
|
# Single test file (Node.js native test runner — most tests use this)
|
|
node --import tsx/esm --test tests/unit/your-file.test.ts
|
|
|
|
# Vitest (MCP server, autoCombo, cache)
|
|
npm run test:vitest
|
|
|
|
# E2E tests (requires Playwright)
|
|
npm run test:e2e
|
|
|
|
# Protocol clients E2E (MCP transports, A2A)
|
|
npm run test:protocols:e2e
|
|
|
|
# Ecosystem compatibility tests
|
|
npm run test:ecosystem
|
|
|
|
# Coverage gate: 60% statements/lines/functions/branches
|
|
npm run test:coverage
|
|
npm run coverage:report
|
|
|
|
# Lint + format check
|
|
npm run lint
|
|
npm run check
|
|
|
|
# Gated real-upstream combo smoke (requires VPS access + real provider credits)
|
|
# Hits REAL providers — costs a little. NEVER runs in CI. Skips cleanly without the gate.
|
|
# Needs: ssh root@192.168.0.15 access (sources a read-only DB snapshot from the VPS).
|
|
RUN_COMBO_LIVE=1 npm run test:combo:live
|
|
|
|
# Phase-3 VPS live smoke — plain Node ESM scripts, hit the live .15 server directly.
|
|
# Requires: ssh root@192.168.0.15 access (combos created/torn down via SSH sqlite).
|
|
# Hits REAL providers (small cost). Creates/deletes only __live_test__* combos. NEVER runs in CI.
|
|
# REQUIRE_API_KEY=false on .15 so no API key needed, but honors COMBO_LIVE_BASE_URL / COMBO_LIVE_API_KEY if set.
|
|
npm run test:combo:live:vps # 7 HTTP scenarios (priority/round-robin/weighted/cost/fusion/auto + health)
|
|
npm run test:combo:live:vps:failover # adds a real cross-provider failover scenario (8 total)
|
|
```
|
|
|
|
Coverage notes:
|
|
|
|
- `npm run test:coverage` measures source coverage for the main unit test suite, excludes `tests/**`, and includes `open-sse/**`
|
|
- Pull requests must keep the coverage gate at **60%+** statements/lines/functions/branches
|
|
- If a PR changes production code in `src/`, `open-sse/`, `electron/`, or `bin/`, it must add or update automated tests in the same PR
|
|
- `npm run coverage:report` prints the detailed file-by-file report from the latest coverage run
|
|
- `npm run test:coverage:legacy` preserves the older metric for historical comparison
|
|
- See `docs/ops/COVERAGE_PLAN.md` for the phased coverage improvement roadmap
|
|
|
|
### Pull Request Requirements
|
|
|
|
Before opening or merging a PR:
|
|
|
|
- Run `npm run test:unit`
|
|
- Run `npm run test:coverage`
|
|
- Ensure the coverage gate stays at **60%+** statements/lines/functions/branches
|
|
- Include the changed or added test files in the PR description when production code changed
|
|
- Check the SonarQube result on the PR when the project secrets are configured in CI
|
|
|
|
Current test status: **122 unit test files** covering:
|
|
|
|
- Provider translators and format conversion
|
|
- Rate limiting, circuit breaker, and resilience
|
|
- Semantic cache, idempotency, progress tracking
|
|
- Database operations and schema (21 DB modules)
|
|
- OAuth flows and authentication
|
|
- API endpoint validation (Zod v4)
|
|
- MCP server tools and scope enforcement
|
|
- Memory and Skills systems
|
|
|
|
---
|
|
|
|
## Code Style
|
|
|
|
- **ESLint** — Run `npm run lint` before committing
|
|
- **Prettier** — Auto-formatted via `lint-staged` on commit (2 spaces, semicolons, double quotes, 100 char width, es5 trailing commas)
|
|
- **TypeScript** — All `src/` code uses `.ts`/`.tsx`; `open-sse/` uses `.ts`/`.js`; document with TSDoc (`@param`, `@returns`, `@throws`)
|
|
- **No `eval()`** — ESLint enforces `no-eval`, `no-implied-eval`, `no-new-func`
|
|
- **Zod validation** — Use Zod v4 schemas for all API input validation
|
|
- **Naming**: Files = camelCase/kebab-case, components = PascalCase, constants = UPPER_SNAKE
|
|
|
|
---
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
src/ # TypeScript (.ts / .tsx)
|
|
├── app/ # Next.js 16 App Router
|
|
│ ├── (dashboard)/ # Dashboard pages (23 sections)
|
|
│ ├── api/ # API routes (51 directories)
|
|
│ └── login/ # Auth pages (.tsx)
|
|
├── domain/ # Policy engine (policyEngine, comboResolver, costRules, etc.)
|
|
├── lib/ # Core business logic (.ts)
|
|
│ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
|
│ ├── acp/ # Agent Communication Protocol registry
|
|
│ ├── compliance/ # Compliance policy engine
|
|
│ ├── db/ # SQLite database layer (21 modules + 16 migrations)
|
|
│ ├── memory/ # Persistent conversational memory
|
|
│ ├── oauth/ # OAuth providers, services, and utilities
|
|
│ ├── skills/ # Extensible skill framework
|
|
│ ├── usage/ # Usage tracking and cost calculation
|
|
│ └── localDb.ts # Re-export layer only — never add logic here
|
|
├── middleware/ # Request middleware (promptInjectionGuard)
|
|
├── mitm/ # MITM proxy (cert, DNS, target routing)
|
|
├── shared/
|
|
│ ├── components/ # React components (.tsx)
|
|
│ ├── constants/ # Provider definitions (177), MCP scopes, 14 routing strategies
|
|
│ ├── utils/ # Circuit breaker, sanitizer, auth helpers
|
|
│ └── validation/ # Zod v4 schemas
|
|
└── sse/ # SSE proxy pipeline
|
|
|
|
open-sse/ # @omniroute/open-sse workspace
|
|
├── executors/ # 14 provider-specific request executors
|
|
├── handlers/ # 11 request handlers (chat, responses, embeddings, images, etc.)
|
|
├── mcp-server/ # MCP server (25 tools, 3 transports, 10 scopes)
|
|
├── services/ # 36+ services (combo, autoCombo, rateLimitManager, etc.)
|
|
├── translator/ # Format translators (OpenAI ↔ Claude ↔ Gemini ↔ Responses ↔ Ollama)
|
|
├── transformer/ # Responses API transformer
|
|
└── utils/ # 22 utility modules (stream, TLS, proxy, logging)
|
|
|
|
electron/ # Electron desktop app (cross-platform)
|
|
|
|
tests/
|
|
├── unit/ # Node.js test runner (1,574 test files)
|
|
├── integration/ # Integration tests
|
|
├── e2e/ # Playwright tests
|
|
├── security/ # Security tests
|
|
├── translator/ # Translator-specific tests
|
|
└── load/ # Load tests
|
|
|
|
docs/
|
|
├── adr/ # Architecture Decision Records
|
|
├── architecture/ # System architecture & resilience
|
|
├── comparison/ # OmniRoute vs alternatives
|
|
├── compression/ # Compression guides & rules
|
|
├── dev/ # Development guides
|
|
├── diagrams/ # Architecture diagrams
|
|
├── frameworks/ # MCP, A2A, OpenCode, Memory, Skills
|
|
├── guides/ # User guide, Docker, setup, troubleshooting
|
|
├── i18n/ # Internationalized README translations
|
|
├── marketing/ # Marketing materials
|
|
├── ops/ # Deployment, proxy, coverage, releases
|
|
├── providers/ # Provider-specific docs
|
|
├── reference/ # API reference, env vars, CLI tools, free tiers
|
|
├── releases/ # Release notes
|
|
├── routing/ # Auto-combo engine, reasoning replay
|
|
├── screenshots/ # Dashboard screenshots
|
|
├── security/ # Guardrails, compliance, stealth, tokens
|
|
└── specs/ # Design specs
|
|
```
|
|
|
|
---
|
|
|
|
## Adding a New Provider
|
|
|
|
### Step 1: Register Provider Constants
|
|
|
|
Add to `src/shared/constants/providers.ts` — Zod-validated at module load.
|
|
|
|
### Step 2: Add Executor (if custom logic needed)
|
|
|
|
Create executor in `open-sse/executors/your-provider.ts` extending the base executor.
|
|
|
|
### Step 3: Add Translator (if non-OpenAI format)
|
|
|
|
Create request/response translators in `open-sse/translator/`.
|
|
|
|
### Step 4: Add OAuth Config (if OAuth-based)
|
|
|
|
Add OAuth credentials in `src/lib/oauth/constants/oauth.ts` and service in `src/lib/oauth/services/`.
|
|
|
|
If the upstream provider distributes a public OAuth client_id/secret or Firebase Web API key inside its public CLI / browser bundle, **do not** embed it as a string literal. Use `resolvePublicCred()` from `open-sse/utils/publicCreds.ts` and add a masked byte entry to `EMBEDDED_DEFAULTS`. The full mandatory workflow is documented in [`docs/security/PUBLIC_CREDS.md`](./docs/security/PUBLIC_CREDS.md).
|
|
|
|
Inside handlers/executors, error messages reaching the client must go through `buildErrorBody()` / `sanitizeErrorMessage()` from `open-sse/utils/error.ts` — never put raw `err.stack` or `err.message` in a Response body. See [`docs/security/ERROR_SANITIZATION.md`](./docs/security/ERROR_SANITIZATION.md).
|
|
|
|
### Step 5: Register Models
|
|
|
|
Add model definitions in `open-sse/config/providerRegistry.ts`.
|
|
|
|
### Step 6: Add Tests
|
|
|
|
Write unit tests in `tests/unit/` covering at minimum:
|
|
|
|
- Provider registration
|
|
- Request/response translation
|
|
- Error handling
|
|
|
|
---
|
|
|
|
## Pull Request Checklist
|
|
|
|
- [ ] Tests pass (`npm test`)
|
|
- [ ] Linting passes (`npm run lint`)
|
|
- [ ] Build succeeds (`npm run build`)
|
|
- [ ] TypeScript types added for new public functions and interfaces
|
|
- [ ] No hardcoded secrets or fallback values
|
|
- [ ] Public upstream credentials embedded via `resolvePublicCred()` (see [`docs/security/PUBLIC_CREDS.md`](./docs/security/PUBLIC_CREDS.md)), never as literals
|
|
- [ ] Error responses route through `buildErrorBody()` / `sanitizeErrorMessage()` — no raw stack traces in response bodies (see [`docs/security/ERROR_SANITIZATION.md`](./docs/security/ERROR_SANITIZATION.md))
|
|
- [ ] Shell commands (`exec` / `spawn`) pass runtime values via `env`, not via string interpolation
|
|
- [ ] All inputs validated with Zod schemas
|
|
- [ ] CHANGELOG updated (if user-facing change)
|
|
- [ ] Documentation updated (if applicable)
|
|
- [ ] No new CodeQL / Secret-Scanning alerts opened, or each one dismissed with technical justification referencing the relevant `docs/security/` doc
|
|
- [ ] Routes that spawn child processes (`/api/mcp/`, `/api/cli-tools/runtime/`) classified as `isLocalOnlyPath()` in `src/server/authz/routeGuard.ts` — see [Hard Rule #15](docs/security/ROUTE_GUARD_TIERS.md)
|
|
- [ ] No `Co-Authored-By` trailers in commit messages — commits must appear solely under the repository owner's Git identity (Hard Rule #16)
|
|
|
|
---
|
|
|
|
## Releasing
|
|
|
|
Releases are managed via the `/generate-release` workflow. When a new GitHub Release is created, the package is **automatically published to npm** via GitHub Actions.
|
|
|
|
For VPS deploys, use `npm run build:release` (not `npm run build`) — it performs a clean
|
|
rebuild, assembles the bundle into `dist/`, and writes the `dist/BUILD_SHA` sentinel.
|
|
Then use the `/deploy-vps-*-cc` skills which rsync `dist/` to the remote `app/` directory.
|
|
|
|
---
|
|
|
|
## Getting Help
|
|
|
|
- **Architecture**: See [`docs/architecture/ARCHITECTURE.md`](docs/architecture/ARCHITECTURE.md)
|
|
- **API Reference**: See [`docs/reference/API_REFERENCE.md`](docs/reference/API_REFERENCE.md)
|
|
- **Security docs**: [`docs/security/CLI_TOKEN.md`](docs/security/CLI_TOKEN.md), [`docs/security/ROUTE_GUARD_TIERS.md`](docs/security/ROUTE_GUARD_TIERS.md), [`docs/security/ERROR_SANITIZATION.md`](docs/security/ERROR_SANITIZATION.md), [`docs/security/PUBLIC_CREDS.md`](docs/security/PUBLIC_CREDS.md)
|
|
- **Ops docs**: [`docs/ops/SQLITE_RUNTIME.md`](docs/ops/SQLITE_RUNTIME.md)
|
|
- **Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
|
|
- **ADRs**: See `docs/adr/` for architectural decision records
|