wehub-resource-sync
371 lines
14 KiB
Markdown
371 lines
14 KiB
Markdown
---
|
||
title: "Quota Sharing Engine"
|
||
version: 3.8.40
|
||
lastUpdated: 2026-06-28
|
||
---
|
||
|
||
# Quota Sharing Engine
|
||
|
||
> **Doc reference**: `docs/routing/QUOTA_SHARE.md`
|
||
> Part of Group B (plans 16 + 22).
|
||
|
||
---
|
||
|
||
## Overview
|
||
|
||
The Quota Sharing Engine distributes a provider's time-based quota (e.g. Codex
|
||
5-hour window, Kimi 1500 req/h) fairly across multiple API keys that share the
|
||
same connection.
|
||
|
||
**Problem it solves:** OmniRoute proxies many API keys against the same upstream
|
||
provider account. Without sharing logic, a burst from key A can exhaust the
|
||
provider quota for the hour, leaving keys B and C blocked until the window resets.
|
||
The engine prevents this by:
|
||
|
||
1. Tracking each key's rolling consumption per dimension (%, requests, tokens, $).
|
||
2. Applying a work-conserving fair-share algorithm: a key may borrow from idle
|
||
shares while the global pool is not saturated.
|
||
3. Enforcing the result in the hot path (`chatCore.ts`) before the request
|
||
reaches the upstream executor.
|
||
|
||
---
|
||
|
||
## Algorithm: Fair-Share Work-Conserving
|
||
|
||
Implemented in `src/lib/quota/fairShare.ts`.
|
||
|
||
### Modes
|
||
|
||
| Condition | Mode | Behaviour |
|
||
| ------------------------------------------ | ------------ | ------------------------------------------------------ |
|
||
| `globalUsedPercent < saturationThreshold` | **Generous** | Key may borrow up to global limit minus consumed-total |
|
||
| `globalUsedPercent >= saturationThreshold` | **Strict** | Enforce individual fair share strictly |
|
||
|
||
Default `saturationThreshold = 0.5` (env `QUOTA_SATURATION_THRESHOLD`).
|
||
|
||
### Per-dimension decision
|
||
|
||
For each active dimension in the pool, the engine computes:
|
||
|
||
```
|
||
fairShareAllowed = poolLimit × (allocationWeight / 100)
|
||
consumed = current rolling value for this key (from QuotaStore.peek)
|
||
remaining = fairShareAllowed - consumed
|
||
```
|
||
|
||
Then:
|
||
|
||
- **`policy = hard`**: if `consumed > fairShareAllowed` and mode is strict → **block**.
|
||
- **`policy = soft`**: if `consumed > fairShareAllowed` and mode is strict → **penalize** (deprioritize in combo; never hard-block).
|
||
- **`policy = burst`**: allow while global headroom exists regardless of fair share.
|
||
|
||
### Cap absoluto
|
||
|
||
`capValue` + `capUnit` on an allocation is a hard ceiling independent of mode or
|
||
policy. Any dimension where `consumed >= capValue` always **blocks** the request.
|
||
|
||
### Multi-dimension check
|
||
|
||
A request is blocked if **any** dimension in the pool would block it. Dimensions
|
||
are independent — a 5h% exhaustion does not affect the weekly% dimension.
|
||
|
||
### Borrowing
|
||
|
||
In generous mode, a key whose allocation is under-consumed can use surplus from
|
||
other keys' unallocated shares. The formula is:
|
||
|
||
```
|
||
maxAllowed = globalLimit - consumedByOtherKeys
|
||
```
|
||
|
||
where `consumedByOtherKeys = consumedTotal - consumedByThisKey`. The teto global
|
||
(pool `limit` for that dimension) is always the hard ceiling.
|
||
|
||
---
|
||
|
||
## Sliding Window Counter
|
||
|
||
Implemented in `src/lib/quota/sqliteQuotaStore.ts` and `redisQuotaStore.ts`.
|
||
|
||
Two buckets per `(apiKeyId, dimensionKey)`:
|
||
|
||
- `curr`: current bucket (`floor(nowMs / windowMs)`)
|
||
- `prev`: previous bucket (`curr - 1`)
|
||
|
||
Effective rolling value:
|
||
|
||
```
|
||
effectiveBucketIndex = floor(nowMs / windowMs)
|
||
bucketStartMs = effectiveBucketIndex × windowMs
|
||
elapsed = nowMs - bucketStartMs
|
||
weight = 1 - elapsed / windowMs
|
||
|
||
effective = prev × weight + curr
|
||
```
|
||
|
||
**Precision**: ~99% accurate. The error is at most 1% of the window size at the
|
||
boundary between buckets (inherent to the 2-bucket approximation).
|
||
|
||
### Concurrency
|
||
|
||
SQLite driver: in-memory mutex per `(apiKeyId | dimensionKey)` key prevents the
|
||
read-modify-write race. Pattern mirrors `src/sse/services/auth.ts` anti-thundering-herd.
|
||
|
||
Redis driver: Lua EVAL script for atomic increment — runs as a single Redis command.
|
||
|
||
---
|
||
|
||
## Drivers
|
||
|
||
### SQLite (default, 0-install)
|
||
|
||
- Table: `quota_consumption` (see migration `073_quota_pools.sql` / `074_quota_consumption.sql`).
|
||
- Best for single-instance deployments.
|
||
- All persistence is in the existing OmniRoute SQLite DB (`DATA_DIR/storage.sqlite`).
|
||
|
||
### Redis (optional, multi-instance)
|
||
|
||
- Requires `ioredis` npm package.
|
||
- Counters stored in Redis; metadata (pools/allocations) still in SQLite.
|
||
- Best for multi-replica deployments where counters must be shared.
|
||
|
||
### Switching drivers
|
||
|
||
Via settings UI (`/dashboard/settings` → Quota Store), or via env vars:
|
||
|
||
```bash
|
||
QUOTA_STORE_DRIVER=redis
|
||
QUOTA_STORE_REDIS_URL=redis://localhost:6379
|
||
```
|
||
|
||
DB setting has precedence over env. If `driver=redis` but URL is absent or
|
||
`ioredis` is not installed, the factory falls back to SQLite and logs a warning.
|
||
|
||
Driver selection order:
|
||
|
||
1. DB setting `quotaStore.driver`
|
||
2. Env `QUOTA_STORE_DRIVER`
|
||
3. Default: `sqlite`
|
||
|
||
---
|
||
|
||
## Multi-Dimension
|
||
|
||
A pool can have multiple dimensions. Each dimension is independent:
|
||
|
||
```ts
|
||
QuotaDimension {
|
||
unit: "percent" | "requests" | "tokens" | "usd",
|
||
window: "5h" | "hourly" | "daily" | "weekly" | "monthly",
|
||
limit: number, // global pool ceiling for this dimension
|
||
}
|
||
```
|
||
|
||
**Example: Codex plan** (5h% + weekly%):
|
||
|
||
```json
|
||
[
|
||
{ "unit": "percent", "window": "5h", "limit": 100 },
|
||
{ "unit": "percent", "window": "weekly", "limit": 100 }
|
||
]
|
||
```
|
||
|
||
A request must satisfy all dimensions to be allowed.
|
||
|
||
---
|
||
|
||
## Plan Resolver
|
||
|
||
Implemented in `src/lib/quota/planResolver.ts`.
|
||
|
||
Precedence (highest to lowest):
|
||
|
||
1. **Manual DB override** — `provider_plans` table, per `connectionId`.
|
||
2. **Known catalog** — `src/lib/quota/planRegistry.ts` (data-only).
|
||
3. **Empty plan** — no dimensions, manual configuration required.
|
||
|
||
### Known catalog
|
||
|
||
| Provider | Dimensions |
|
||
| --------------------- | ------------------------------------------------------------- |
|
||
| `codex` | `percent/5h/100`, `percent/weekly/100` |
|
||
| `glm` | `tokens/5h` (limit=0, unknown), `tokens/weekly` |
|
||
| `minimax` | `tokens/5h`, `tokens/weekly` |
|
||
| `bailian` | `percent/5h/100`, `percent/weekly/100`, `percent/monthly/100` |
|
||
| `kimi` | `requests/hourly/1500` |
|
||
| `alibaba` | `requests/monthly/90000` |
|
||
| `openai`, `anthropic` | No default — manual configuration required |
|
||
|
||
---
|
||
|
||
## Pipeline Integration
|
||
|
||
### PRE hook (`open-sse/handlers/chatCore.ts`)
|
||
|
||
Runs before the upstream executor, after auth and policy checks:
|
||
|
||
```
|
||
resolveComboTargets / handleSingleModel
|
||
→ enforceQuotaShare(apiKeyId, connectionId, provider, estimatedCost)
|
||
→ getQuotaStore().peek() per dimension
|
||
→ fairShare.decideFairShare()
|
||
→ if block → return 429 (buildErrorBody, Hard Rule #12)
|
||
→ if allow + deprioritize → set quotaSoftPenalty=true on candidate
|
||
→ executor.execute()
|
||
```
|
||
|
||
**Fail-open**: if `enforceQuotaShare` throws, the request is allowed through
|
||
with a `pino.warn` log. This prevents a quota-engine bug from blocking all
|
||
traffic.
|
||
|
||
### POST hook (record consumption)
|
||
|
||
After a successful response:
|
||
|
||
```
|
||
executor returns success
|
||
→ spendRecorder.recordConsumption(apiKeyId, connectionId, provider, actualCost)
|
||
→ getQuotaStore().consume() per dimension
|
||
→ fail-open: errors logged as pino.warn, never propagated to client
|
||
```
|
||
|
||
**Drift note**: if `consume` fails post-response, the rolling counter under-counts.
|
||
The saturation signal from the provider (e.g. `anthropic-ratelimit-unified-5h-utilization`)
|
||
corrects the global estimate on the next request.
|
||
|
||
### Combo soft penalty (`open-sse/services/combo.ts`)
|
||
|
||
When `decision.deprioritize === true`:
|
||
|
||
```ts
|
||
if (candidate.quotaSoftPenalty) {
|
||
score *= QUOTA_SOFT_DEPRIORITIZE_FACTOR; // default 0.7
|
||
}
|
||
```
|
||
|
||
The penalty is applied after all other scoring factors. It lowers the auto-combo
|
||
probability of selecting a saturated key without hard-blocking it.
|
||
|
||
---
|
||
|
||
## UI Walkthrough
|
||
|
||
### `/dashboard/costs/quota-share` — Main pools page
|
||
|
||
Components (all in `src/app/(dashboard)/dashboard/costs/quota-share/`):
|
||
|
||
| Component | Purpose |
|
||
| ---------------------- | ----------------------------------------------------------------- |
|
||
| `QuotaConceptCard` | Introductory card explaining quota sharing to new users |
|
||
| `CreatePoolModal` | Create a new quota pool (connection + name + initial allocations) |
|
||
| `PoolCard` | Per-pool summary: name, connection, allocation count |
|
||
| `DimensionBar` | Per-dimension stacked bar: each key's share + global usage |
|
||
| `AllocationTable` | Table with consumed, fair share, deficit/surplus, borrowing flag |
|
||
| `BurnRateChart` | EMA burn-rate line chart (lazy Recharts via `dynamic()`) |
|
||
| `EditAllocationsModal` | Edit allocation weights, caps, and policies for a pool |
|
||
|
||
The page hooks:
|
||
|
||
- `usePools` — fetches `GET /api/quota/pools` every 30s.
|
||
- `usePoolUsage` — fetches `GET /api/quota/pools/[id]/usage` on demand.
|
||
- `useLocalStoragePoolMigration` — runs once on mount to migrate legacy LS data.
|
||
|
||
### `/dashboard/costs/quota-share/plans` — Provider plan config
|
||
|
||
- `ProviderPlanConfigClient.tsx`: dropdown to select a provider, view resolved
|
||
plan (auto from catalog or manual override), and edit dimensions.
|
||
- Changes write to `PUT /api/quota/plans/[connectionId]`.
|
||
- Deletion reverts to catalog or empty plan.
|
||
|
||
---
|
||
|
||
## Environment Variables
|
||
|
||
| Variable | Default | Description |
|
||
| ---------------------------------- | --------- | ------------------------------------------------------ |
|
||
| `QUOTA_STORE_DRIVER` | `sqlite` | Driver to use: `sqlite` or `redis` |
|
||
| `QUOTA_STORE_REDIS_URL` | _(empty)_ | Redis URL, e.g. `redis://localhost:6379` |
|
||
| `QUOTA_SATURATION_THRESHOLD` | `0.5` | 0..1; `>= threshold` activates strict mode |
|
||
| `QUOTA_SOFT_DEPRIORITIZE_FACTOR` | `0.7` | 0..1; multiplier for soft-policy combo score |
|
||
| `QUOTA_CONSUMPTION_RETENTION_DAYS` | `14` | Days before GC removes old `quota_consumption` buckets |
|
||
|
||
DB settings (`quotaStore.*`) override env vars.
|
||
|
||
---
|
||
|
||
## Troubleshooting
|
||
|
||
### Redis configured but not connecting
|
||
|
||
Check that `ioredis` is installed (`npm ls ioredis`) and `QUOTA_STORE_REDIS_URL`
|
||
is reachable. On connection failure the factory falls back to SQLite (logged at
|
||
`warn`).
|
||
|
||
### `peek` returns stale / fail-open
|
||
|
||
If `peek` throws, `enforceQuotaShare` treats the result as "allow" (fail-open).
|
||
Check `pino` logs for `quota:enforce` and `quota:factory` entries to identify
|
||
the root cause.
|
||
|
||
### Consumption counter drift
|
||
|
||
If the actual provider usage differs from the counters, it is expected — the
|
||
2-bucket sliding window has ~1% error at window boundaries, and `consume` is
|
||
fire-and-forget post-response. The saturation signal (`saturationSignals.ts`)
|
||
reads the real provider utilization with a 30s TTL and adjusts `globalUsedPercent`
|
||
accordingly.
|
||
|
||
### Pool shows "no data" for burn rate
|
||
|
||
`computeBurnRate` requires at least 2 historical samples. New pools without prior
|
||
`consume` calls will show `tokensPerSecond: 0` and `timeToExhaustionMs: null`.
|
||
|
||
---
|
||
|
||
## Migration from localStorage
|
||
|
||
When `/dashboard/costs/quota-share` first loads, the hook `useLocalStoragePoolMigration`
|
||
checks:
|
||
|
||
1. `localStorage.getItem("omniroute:quota-share:pools")` is non-empty.
|
||
2. `GET /api/quota/pools` returns `[]` (DB is empty).
|
||
|
||
If both are true, it posts each legacy pool to `POST /api/quota/pools` in batch,
|
||
then removes the localStorage key. The migration is idempotent: condition 2 prevents
|
||
re-migration.
|
||
|
||
---
|
||
|
||
## Internal Strategy Classification
|
||
|
||
`quota-share` is an **internal-only** routing strategy (`INTERNAL_ROUTING_STRATEGY_VALUES` in
|
||
`src/shared/constants/routingStrategies.ts`). It is used exclusively by system-minted
|
||
`qtSd/` pool combos and is deliberately excluded from `ROUTING_STRATEGY_VALUES` so it never
|
||
appears as a user-selectable option in the UI or API.
|
||
|
||
---
|
||
|
||
## Test Coverage
|
||
|
||
Two layers of automated coverage ship with the quota-share engine:
|
||
|
||
| Suite | Command | What it covers |
|
||
| :----------------- | :--------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| Unit (29 tests) | `node --import tsx/esm --test tests/unit/quota-share-strategy.test.ts` | DRR scheduler, saturation gating, concurrency caps, fairShare math, backlog queueing |
|
||
| Integration matrix | `npm run test:combo:matrix` | End-to-end routing decision through the real combo pipeline; DRR fairness + saturation deprioritization via live seams (`registerQuotaFetcher`, `setLKGP`, `__setHeadroomSaturationFetcherForTests`) |
|
||
|
||
The integration matrix runs in CI alongside the other 17 public strategies. The unit suite
|
||
can be run standalone.
|
||
|
||
---
|
||
|
||
## DB Schema Summary
|
||
|
||
Three tables added by migrations `073–075`:
|
||
|
||
- `quota_pools` + `quota_allocations` — pool definitions and per-key allocations.
|
||
- `quota_consumption` — rolling 2-bucket counters per `(apiKeyId, dimensionKey)`.
|
||
- `provider_plans` — manual provider plan overrides (dimensions JSON per connectionId).
|
||
|
||
All tables added via idempotent `CREATE TABLE IF NOT EXISTS` migrations.
|