chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,370 @@
|
||||
---
|
||||
title: "Quota Sharing Engine"
|
||||
version: 3.8.40
|
||||
lastUpdated: 2026-06-28
|
||||
---
|
||||
|
||||
# Quota Sharing Engine
|
||||
|
||||
> **Doc reference**: `docs/routing/QUOTA_SHARE.md`
|
||||
> Part of Group B (plans 16 + 22).
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The Quota Sharing Engine distributes a provider's time-based quota (e.g. Codex
|
||||
5-hour window, Kimi 1500 req/h) fairly across multiple API keys that share the
|
||||
same connection.
|
||||
|
||||
**Problem it solves:** OmniRoute proxies many API keys against the same upstream
|
||||
provider account. Without sharing logic, a burst from key A can exhaust the
|
||||
provider quota for the hour, leaving keys B and C blocked until the window resets.
|
||||
The engine prevents this by:
|
||||
|
||||
1. Tracking each key's rolling consumption per dimension (%, requests, tokens, $).
|
||||
2. Applying a work-conserving fair-share algorithm: a key may borrow from idle
|
||||
shares while the global pool is not saturated.
|
||||
3. Enforcing the result in the hot path (`chatCore.ts`) before the request
|
||||
reaches the upstream executor.
|
||||
|
||||
---
|
||||
|
||||
## Algorithm: Fair-Share Work-Conserving
|
||||
|
||||
Implemented in `src/lib/quota/fairShare.ts`.
|
||||
|
||||
### Modes
|
||||
|
||||
| Condition | Mode | Behaviour |
|
||||
| ------------------------------------------ | ------------ | ------------------------------------------------------ |
|
||||
| `globalUsedPercent < saturationThreshold` | **Generous** | Key may borrow up to global limit minus consumed-total |
|
||||
| `globalUsedPercent >= saturationThreshold` | **Strict** | Enforce individual fair share strictly |
|
||||
|
||||
Default `saturationThreshold = 0.5` (env `QUOTA_SATURATION_THRESHOLD`).
|
||||
|
||||
### Per-dimension decision
|
||||
|
||||
For each active dimension in the pool, the engine computes:
|
||||
|
||||
```
|
||||
fairShareAllowed = poolLimit × (allocationWeight / 100)
|
||||
consumed = current rolling value for this key (from QuotaStore.peek)
|
||||
remaining = fairShareAllowed - consumed
|
||||
```
|
||||
|
||||
Then:
|
||||
|
||||
- **`policy = hard`**: if `consumed > fairShareAllowed` and mode is strict → **block**.
|
||||
- **`policy = soft`**: if `consumed > fairShareAllowed` and mode is strict → **penalize** (deprioritize in combo; never hard-block).
|
||||
- **`policy = burst`**: allow while global headroom exists regardless of fair share.
|
||||
|
||||
### Cap absoluto
|
||||
|
||||
`capValue` + `capUnit` on an allocation is a hard ceiling independent of mode or
|
||||
policy. Any dimension where `consumed >= capValue` always **blocks** the request.
|
||||
|
||||
### Multi-dimension check
|
||||
|
||||
A request is blocked if **any** dimension in the pool would block it. Dimensions
|
||||
are independent — a 5h% exhaustion does not affect the weekly% dimension.
|
||||
|
||||
### Borrowing
|
||||
|
||||
In generous mode, a key whose allocation is under-consumed can use surplus from
|
||||
other keys' unallocated shares. The formula is:
|
||||
|
||||
```
|
||||
maxAllowed = globalLimit - consumedByOtherKeys
|
||||
```
|
||||
|
||||
where `consumedByOtherKeys = consumedTotal - consumedByThisKey`. The teto global
|
||||
(pool `limit` for that dimension) is always the hard ceiling.
|
||||
|
||||
---
|
||||
|
||||
## Sliding Window Counter
|
||||
|
||||
Implemented in `src/lib/quota/sqliteQuotaStore.ts` and `redisQuotaStore.ts`.
|
||||
|
||||
Two buckets per `(apiKeyId, dimensionKey)`:
|
||||
|
||||
- `curr`: current bucket (`floor(nowMs / windowMs)`)
|
||||
- `prev`: previous bucket (`curr - 1`)
|
||||
|
||||
Effective rolling value:
|
||||
|
||||
```
|
||||
effectiveBucketIndex = floor(nowMs / windowMs)
|
||||
bucketStartMs = effectiveBucketIndex × windowMs
|
||||
elapsed = nowMs - bucketStartMs
|
||||
weight = 1 - elapsed / windowMs
|
||||
|
||||
effective = prev × weight + curr
|
||||
```
|
||||
|
||||
**Precision**: ~99% accurate. The error is at most 1% of the window size at the
|
||||
boundary between buckets (inherent to the 2-bucket approximation).
|
||||
|
||||
### Concurrency
|
||||
|
||||
SQLite driver: in-memory mutex per `(apiKeyId | dimensionKey)` key prevents the
|
||||
read-modify-write race. Pattern mirrors `src/sse/services/auth.ts` anti-thundering-herd.
|
||||
|
||||
Redis driver: Lua EVAL script for atomic increment — runs as a single Redis command.
|
||||
|
||||
---
|
||||
|
||||
## Drivers
|
||||
|
||||
### SQLite (default, 0-install)
|
||||
|
||||
- Table: `quota_consumption` (see migration `073_quota_pools.sql` / `074_quota_consumption.sql`).
|
||||
- Best for single-instance deployments.
|
||||
- All persistence is in the existing OmniRoute SQLite DB (`DATA_DIR/storage.sqlite`).
|
||||
|
||||
### Redis (optional, multi-instance)
|
||||
|
||||
- Requires `ioredis` npm package.
|
||||
- Counters stored in Redis; metadata (pools/allocations) still in SQLite.
|
||||
- Best for multi-replica deployments where counters must be shared.
|
||||
|
||||
### Switching drivers
|
||||
|
||||
Via settings UI (`/dashboard/settings` → Quota Store), or via env vars:
|
||||
|
||||
```bash
|
||||
QUOTA_STORE_DRIVER=redis
|
||||
QUOTA_STORE_REDIS_URL=redis://localhost:6379
|
||||
```
|
||||
|
||||
DB setting has precedence over env. If `driver=redis` but URL is absent or
|
||||
`ioredis` is not installed, the factory falls back to SQLite and logs a warning.
|
||||
|
||||
Driver selection order:
|
||||
|
||||
1. DB setting `quotaStore.driver`
|
||||
2. Env `QUOTA_STORE_DRIVER`
|
||||
3. Default: `sqlite`
|
||||
|
||||
---
|
||||
|
||||
## Multi-Dimension
|
||||
|
||||
A pool can have multiple dimensions. Each dimension is independent:
|
||||
|
||||
```ts
|
||||
QuotaDimension {
|
||||
unit: "percent" | "requests" | "tokens" | "usd",
|
||||
window: "5h" | "hourly" | "daily" | "weekly" | "monthly",
|
||||
limit: number, // global pool ceiling for this dimension
|
||||
}
|
||||
```
|
||||
|
||||
**Example: Codex plan** (5h% + weekly%):
|
||||
|
||||
```json
|
||||
[
|
||||
{ "unit": "percent", "window": "5h", "limit": 100 },
|
||||
{ "unit": "percent", "window": "weekly", "limit": 100 }
|
||||
]
|
||||
```
|
||||
|
||||
A request must satisfy all dimensions to be allowed.
|
||||
|
||||
---
|
||||
|
||||
## Plan Resolver
|
||||
|
||||
Implemented in `src/lib/quota/planResolver.ts`.
|
||||
|
||||
Precedence (highest to lowest):
|
||||
|
||||
1. **Manual DB override** — `provider_plans` table, per `connectionId`.
|
||||
2. **Known catalog** — `src/lib/quota/planRegistry.ts` (data-only).
|
||||
3. **Empty plan** — no dimensions, manual configuration required.
|
||||
|
||||
### Known catalog
|
||||
|
||||
| Provider | Dimensions |
|
||||
| --------------------- | ------------------------------------------------------------- |
|
||||
| `codex` | `percent/5h/100`, `percent/weekly/100` |
|
||||
| `glm` | `tokens/5h` (limit=0, unknown), `tokens/weekly` |
|
||||
| `minimax` | `tokens/5h`, `tokens/weekly` |
|
||||
| `bailian` | `percent/5h/100`, `percent/weekly/100`, `percent/monthly/100` |
|
||||
| `kimi` | `requests/hourly/1500` |
|
||||
| `alibaba` | `requests/monthly/90000` |
|
||||
| `openai`, `anthropic` | No default — manual configuration required |
|
||||
|
||||
---
|
||||
|
||||
## Pipeline Integration
|
||||
|
||||
### PRE hook (`open-sse/handlers/chatCore.ts`)
|
||||
|
||||
Runs before the upstream executor, after auth and policy checks:
|
||||
|
||||
```
|
||||
resolveComboTargets / handleSingleModel
|
||||
→ enforceQuotaShare(apiKeyId, connectionId, provider, estimatedCost)
|
||||
→ getQuotaStore().peek() per dimension
|
||||
→ fairShare.decideFairShare()
|
||||
→ if block → return 429 (buildErrorBody, Hard Rule #12)
|
||||
→ if allow + deprioritize → set quotaSoftPenalty=true on candidate
|
||||
→ executor.execute()
|
||||
```
|
||||
|
||||
**Fail-open**: if `enforceQuotaShare` throws, the request is allowed through
|
||||
with a `pino.warn` log. This prevents a quota-engine bug from blocking all
|
||||
traffic.
|
||||
|
||||
### POST hook (record consumption)
|
||||
|
||||
After a successful response:
|
||||
|
||||
```
|
||||
executor returns success
|
||||
→ spendRecorder.recordConsumption(apiKeyId, connectionId, provider, actualCost)
|
||||
→ getQuotaStore().consume() per dimension
|
||||
→ fail-open: errors logged as pino.warn, never propagated to client
|
||||
```
|
||||
|
||||
**Drift note**: if `consume` fails post-response, the rolling counter under-counts.
|
||||
The saturation signal from the provider (e.g. `anthropic-ratelimit-unified-5h-utilization`)
|
||||
corrects the global estimate on the next request.
|
||||
|
||||
### Combo soft penalty (`open-sse/services/combo.ts`)
|
||||
|
||||
When `decision.deprioritize === true`:
|
||||
|
||||
```ts
|
||||
if (candidate.quotaSoftPenalty) {
|
||||
score *= QUOTA_SOFT_DEPRIORITIZE_FACTOR; // default 0.7
|
||||
}
|
||||
```
|
||||
|
||||
The penalty is applied after all other scoring factors. It lowers the auto-combo
|
||||
probability of selecting a saturated key without hard-blocking it.
|
||||
|
||||
---
|
||||
|
||||
## UI Walkthrough
|
||||
|
||||
### `/dashboard/costs/quota-share` — Main pools page
|
||||
|
||||
Components (all in `src/app/(dashboard)/dashboard/costs/quota-share/`):
|
||||
|
||||
| Component | Purpose |
|
||||
| ---------------------- | ----------------------------------------------------------------- |
|
||||
| `QuotaConceptCard` | Introductory card explaining quota sharing to new users |
|
||||
| `CreatePoolModal` | Create a new quota pool (connection + name + initial allocations) |
|
||||
| `PoolCard` | Per-pool summary: name, connection, allocation count |
|
||||
| `DimensionBar` | Per-dimension stacked bar: each key's share + global usage |
|
||||
| `AllocationTable` | Table with consumed, fair share, deficit/surplus, borrowing flag |
|
||||
| `BurnRateChart` | EMA burn-rate line chart (lazy Recharts via `dynamic()`) |
|
||||
| `EditAllocationsModal` | Edit allocation weights, caps, and policies for a pool |
|
||||
|
||||
The page hooks:
|
||||
|
||||
- `usePools` — fetches `GET /api/quota/pools` every 30s.
|
||||
- `usePoolUsage` — fetches `GET /api/quota/pools/[id]/usage` on demand.
|
||||
- `useLocalStoragePoolMigration` — runs once on mount to migrate legacy LS data.
|
||||
|
||||
### `/dashboard/costs/quota-share/plans` — Provider plan config
|
||||
|
||||
- `ProviderPlanConfigClient.tsx`: dropdown to select a provider, view resolved
|
||||
plan (auto from catalog or manual override), and edit dimensions.
|
||||
- Changes write to `PUT /api/quota/plans/[connectionId]`.
|
||||
- Deletion reverts to catalog or empty plan.
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ---------------------------------- | --------- | ------------------------------------------------------ |
|
||||
| `QUOTA_STORE_DRIVER` | `sqlite` | Driver to use: `sqlite` or `redis` |
|
||||
| `QUOTA_STORE_REDIS_URL` | _(empty)_ | Redis URL, e.g. `redis://localhost:6379` |
|
||||
| `QUOTA_SATURATION_THRESHOLD` | `0.5` | 0..1; `>= threshold` activates strict mode |
|
||||
| `QUOTA_SOFT_DEPRIORITIZE_FACTOR` | `0.7` | 0..1; multiplier for soft-policy combo score |
|
||||
| `QUOTA_CONSUMPTION_RETENTION_DAYS` | `14` | Days before GC removes old `quota_consumption` buckets |
|
||||
|
||||
DB settings (`quotaStore.*`) override env vars.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Redis configured but not connecting
|
||||
|
||||
Check that `ioredis` is installed (`npm ls ioredis`) and `QUOTA_STORE_REDIS_URL`
|
||||
is reachable. On connection failure the factory falls back to SQLite (logged at
|
||||
`warn`).
|
||||
|
||||
### `peek` returns stale / fail-open
|
||||
|
||||
If `peek` throws, `enforceQuotaShare` treats the result as "allow" (fail-open).
|
||||
Check `pino` logs for `quota:enforce` and `quota:factory` entries to identify
|
||||
the root cause.
|
||||
|
||||
### Consumption counter drift
|
||||
|
||||
If the actual provider usage differs from the counters, it is expected — the
|
||||
2-bucket sliding window has ~1% error at window boundaries, and `consume` is
|
||||
fire-and-forget post-response. The saturation signal (`saturationSignals.ts`)
|
||||
reads the real provider utilization with a 30s TTL and adjusts `globalUsedPercent`
|
||||
accordingly.
|
||||
|
||||
### Pool shows "no data" for burn rate
|
||||
|
||||
`computeBurnRate` requires at least 2 historical samples. New pools without prior
|
||||
`consume` calls will show `tokensPerSecond: 0` and `timeToExhaustionMs: null`.
|
||||
|
||||
---
|
||||
|
||||
## Migration from localStorage
|
||||
|
||||
When `/dashboard/costs/quota-share` first loads, the hook `useLocalStoragePoolMigration`
|
||||
checks:
|
||||
|
||||
1. `localStorage.getItem("omniroute:quota-share:pools")` is non-empty.
|
||||
2. `GET /api/quota/pools` returns `[]` (DB is empty).
|
||||
|
||||
If both are true, it posts each legacy pool to `POST /api/quota/pools` in batch,
|
||||
then removes the localStorage key. The migration is idempotent: condition 2 prevents
|
||||
re-migration.
|
||||
|
||||
---
|
||||
|
||||
## Internal Strategy Classification
|
||||
|
||||
`quota-share` is an **internal-only** routing strategy (`INTERNAL_ROUTING_STRATEGY_VALUES` in
|
||||
`src/shared/constants/routingStrategies.ts`). It is used exclusively by system-minted
|
||||
`qtSd/` pool combos and is deliberately excluded from `ROUTING_STRATEGY_VALUES` so it never
|
||||
appears as a user-selectable option in the UI or API.
|
||||
|
||||
---
|
||||
|
||||
## Test Coverage
|
||||
|
||||
Two layers of automated coverage ship with the quota-share engine:
|
||||
|
||||
| Suite | Command | What it covers |
|
||||
| :----------------- | :--------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Unit (29 tests) | `node --import tsx/esm --test tests/unit/quota-share-strategy.test.ts` | DRR scheduler, saturation gating, concurrency caps, fairShare math, backlog queueing |
|
||||
| Integration matrix | `npm run test:combo:matrix` | End-to-end routing decision through the real combo pipeline; DRR fairness + saturation deprioritization via live seams (`registerQuotaFetcher`, `setLKGP`, `__setHeadroomSaturationFetcherForTests`) |
|
||||
|
||||
The integration matrix runs in CI alongside the other 17 public strategies. The unit suite
|
||||
can be run standalone.
|
||||
|
||||
---
|
||||
|
||||
## DB Schema Summary
|
||||
|
||||
Three tables added by migrations `073–075`:
|
||||
|
||||
- `quota_pools` + `quota_allocations` — pool definitions and per-key allocations.
|
||||
- `quota_consumption` — rolling 2-bucket counters per `(apiKeyId, dimensionKey)`.
|
||||
- `provider_plans` — manual provider plan overrides (dimensions JSON per connectionId).
|
||||
|
||||
All tables added via idempotent `CREATE TABLE IF NOT EXISTS` migrations.
|
||||
Reference in New Issue
Block a user