chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,234 @@
|
||||
TODO: delete
|
||||
|
||||
# Model Sync Scripts
|
||||
|
||||
Model syncs are centralized in `packages/core/src/sync/index.ts`. The runner owns file IO, TOML formatting, validation, reporting, dry runs, and deletion behavior. `packages/core/script/sync-models.ts` is only the CLI wrapper for `bun models:sync`. Individual provider sync modules only fetch source data, parse it, and translate each source model into the catalog schema.
|
||||
|
||||
The grouped sync targets are available for local convenience, but CI syncs each provider separately so every provider gets its own reusable automation PR.
|
||||
|
||||
## Commands
|
||||
|
||||
- `bun models:sync aggregators` syncs every provider in the `aggregators` group.
|
||||
- `bun models:sync openrouter` syncs only OpenRouter.
|
||||
- `bun models:sync cloudflare-workers-ai` syncs only Cloudflare Workers AI.
|
||||
- `bun models:sync cloudflare` syncs the Cloudflare sync group.
|
||||
- `bun models:sync direct` syncs every provider in the `direct` group.
|
||||
- `bun models:sync google` syncs only Google.
|
||||
- `bun models:sync digitalocean` syncs only DigitalOcean.
|
||||
- `bun models:sync xai` syncs only xAI.
|
||||
- `bun models:sync kilo` syncs only Kilo.
|
||||
- `bun models:sync openai` syncs only OpenAI catalog availability.
|
||||
- `bun models:sync aggregators --dry-run` prints changes without writing model files.
|
||||
- `bun models:sync aggregators --new-only` creates new model files but skips updates and removals.
|
||||
- `bun validate` validates the generated catalog after a sync.
|
||||
|
||||
Sync runs also write `.sync/model-sync-report.md` for the automation workflow PR body. Do not commit that report from local runs.
|
||||
|
||||
## Runner Responsibilities
|
||||
|
||||
`packages/core/src/sync/index.ts` handles the shared behavior:
|
||||
|
||||
- Reads existing TOML files from the provider `modelsDir`.
|
||||
- Parses existing files with `Bun.TOML.parse` and `AuthoredModelShape.partial()`.
|
||||
- Resolves existing `base_model` / `base_model_omit` metadata before passing local metadata to provider modules.
|
||||
- Calls the provider module to fetch, parse, and translate source models.
|
||||
- Validates translated models with `AuthoredModel` before writing.
|
||||
- Formats TOML consistently for all synced providers.
|
||||
- Compares authored TOML shapes before writing so existing factored TOMLs stay factored instead of being expanded.
|
||||
- Replaces symlinked files safely by removing the symlink before writing.
|
||||
- Removes existing files that are no longer present in the desired synced set.
|
||||
- Writes `.sync/model-sync-report.md` for GitHub Actions.
|
||||
|
||||
Because the runner removes files missing from the desired set, a provider module should only skip source models when deleting existing local files for those skipped IDs is intentional.
|
||||
|
||||
## Provider Modules
|
||||
|
||||
Provider modules live in `packages/core/src/sync/providers/`. A provider exports an object satisfying `SyncProvider<SourceModel>`:
|
||||
|
||||
```ts
|
||||
export const provider = {
|
||||
id: "provider-id",
|
||||
name: "Provider Name",
|
||||
modelsDir: "providers/provider-id/models",
|
||||
async fetchModels() {
|
||||
return fetch("https://example.com/models").then((response) => response.json());
|
||||
},
|
||||
parseModels(raw) {
|
||||
return ProviderResponse.parse(raw).data;
|
||||
},
|
||||
translateModel(model, context) {
|
||||
return {
|
||||
id: model.id,
|
||||
model: buildModel(model, context.existing(model.id)),
|
||||
};
|
||||
},
|
||||
} satisfies SyncProvider<ProviderModel>;
|
||||
```
|
||||
|
||||
Keep provider modules focused on provider-specific logic:
|
||||
|
||||
- Define Zod schemas for the provider API response.
|
||||
- Fetch from the provider API, including auth headers when needed.
|
||||
- Convert provider pricing units to per-1M-token catalog prices.
|
||||
- Convert dates, modalities, limits, capabilities, and model IDs into catalog fields.
|
||||
- Preserve existing hand-authored fields only when the provider API is not authoritative for that field.
|
||||
- Preserve `base_model` and `base_model_omit` from existing TOMLs when updating a factored provider model.
|
||||
- Return `undefined` from `translateModel` only when skipped source models should be treated as absent from the synced catalog.
|
||||
|
||||
Do not put TOML scanning, writing, deletion, reporting, or generic comparison logic in provider modules.
|
||||
|
||||
Provider sync code must use `base_model` and `base_model_omit`; do not write legacy `[extends]` tables. If a sync or generator updates a provider file that already uses `base_model`, it should keep that pointer and only write provider-specific overrides.
|
||||
|
||||
## Adding A Provider
|
||||
|
||||
1. Create `packages/core/src/sync/providers/<provider>.ts`.
|
||||
2. Define strict-enough Zod schemas for the provider response.
|
||||
3. Export a `SyncProvider` implementation with `fetchModels`, `parseModels`, and `translateModel`.
|
||||
4. Add the provider to `providers` in `packages/core/src/sync/index.ts`.
|
||||
5. Add the provider ID to an existing group or create a new group in `groups`.
|
||||
6. Add any required API secrets to `.github/workflows/sync-models.yml` if the provider needs new credentials.
|
||||
7. Run `bun models:sync <provider> --dry-run` to inspect the first diff.
|
||||
8. Run `bun models:sync <provider>` to write files.
|
||||
9. Run `bun models:sync <provider> --dry-run` again and expect a clean result.
|
||||
10. Run `bun validate`.
|
||||
|
||||
Prefer small, provider-specific PRs when adding a provider. If the provider has ambiguous source data, keep it out of shared groups until the source-of-truth behavior is clear.
|
||||
|
||||
## Automation
|
||||
|
||||
`.github/workflows/sync-models.yml` runs on an hourly schedule and manually through `workflow_dispatch`.
|
||||
|
||||
The workflow:
|
||||
|
||||
- Checks out `dev`.
|
||||
- Installs dependencies with Bun.
|
||||
- Discovers sync providers with `bun models:sync --list-providers`.
|
||||
- Runs one provider per matrix job with `bun models:sync ${{ matrix.provider }}`.
|
||||
- Runs `bun validate`.
|
||||
- Creates or updates a provider-specific sync PR only when `providers` changed.
|
||||
- Uses `.sync/model-sync-report.md` as the PR body.
|
||||
|
||||
Each provider job checks out `dev` and writes to a fixed provider branch like `automation/sync-models-openrouter`. If that provider's sync PR is already open, later scheduled runs force-update the same branch and edit the existing PR instead of creating another one. Provider jobs do not share unmerged changes with each other; OpenRouter only uses `base_model` for model metadata entries already present on `dev`.
|
||||
|
||||
CI automatically picks up providers registered in `providers` in `packages/core/src/sync/index.ts`. Adding a new sync provider there is enough to get an hourly provider-specific sync job, branch, labels, title, and PR naming convention. The workflow only needs manual updates when a new provider requires new secrets or other environment variables.
|
||||
|
||||
Actions are pinned by commit SHA. Keep new workflow actions pinned the same way.
|
||||
|
||||
## OpenRouter Notes
|
||||
|
||||
OpenRouter is implemented in `packages/core/src/sync/providers/openrouter.ts`.
|
||||
|
||||
- Source endpoint: `https://openrouter.ai/api/v1/models`.
|
||||
- Optional auth: `OPENROUTER_API_KEY`.
|
||||
- Model IDs map directly to TOML paths under `providers/openrouter/models`.
|
||||
- API prices are per-token strings and are converted to per-1M-token numbers.
|
||||
- `structured_output` comes from `supported_parameters.includes("structured_outputs")` only.
|
||||
- Existing `status`, `interleaved`, `knowledge`, `limit.input`, and `cost.tiers` may be preserved when OpenRouter is not authoritative enough for those fields.
|
||||
- Canonical OpenRouter model IDs should emit `base_model` references to model metadata when a matching `models/` entry exists.
|
||||
|
||||
## Kilo Gateway Notes
|
||||
|
||||
Kilo Gateway is implemented in `packages/core/src/sync/providers/kilo.ts`.
|
||||
|
||||
- Source endpoint: `https://api.kilo.ai/api/gateway/models`.
|
||||
- Optional auth: `KILO_API_KEY`.
|
||||
- Model IDs map directly to TOML paths under `providers/kilo/models`.
|
||||
- API prices are per-token strings and are converted to per-1M-token numbers.
|
||||
- `structured_output` comes from `supported_parameters.includes("structured_outputs")` only.
|
||||
- Existing `status`, `interleaved`, `knowledge`, `limit.input`, and `cost.tiers` may be preserved when Kilo is not authoritative enough for those fields.
|
||||
- Canonical Kilo model IDs should emit `base_model` references to model metadata when a matching `models/` entry exists.
|
||||
- `reasoning_options` is derived from `opencode.variants` when present.
|
||||
|
||||
## Cloudflare Workers AI Notes
|
||||
|
||||
Cloudflare Workers AI is implemented in `packages/core/src/sync/providers/cloudflare-workers-ai.ts`.
|
||||
|
||||
- Source endpoint: `https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_WORKERS_AI_SYNC_ACCOUNT_ID/ai/models/search?format=openrouter`.
|
||||
- Required auth: `CLOUDFLARE_WORKERS_AI_SYNC_ACCOUNT_ID` and `CLOUDFLARE_WORKERS_AI_SYNC_API_TOKEN`.
|
||||
- Use a dedicated token scoped to Workers AI read access so sync automation does not share deploy credentials.
|
||||
- The endpoint is parsed as Cloudflare's OpenRouter-like Workers AI metadata.
|
||||
- Model IDs map directly to TOML paths under `providers/cloudflare-workers-ai/models`.
|
||||
- This sync target does not manage `providers/cloudflare-ai-gateway`, because the AI Gateway `/compat/models` endpoint does not support `format=openrouter` and does not provide enough model metadata for authoritative catalog sync.
|
||||
|
||||
## Google Notes
|
||||
|
||||
Google is implemented in `packages/core/src/sync/providers/google.ts`.
|
||||
|
||||
- Source endpoint: `https://generativelanguage.googleapis.com/v1beta/models`.
|
||||
- Required auth: `GOOGLE_API_KEY`, `GEMINI_API_KEY`, or `GOOGLE_GENERATIVE_AI_API_KEY`.
|
||||
- Model IDs are derived from the `models/{model}` resource names.
|
||||
- The API is authoritative for display names, token limits, temperature metadata, and the `thinking` flag when present.
|
||||
- Local Google models missing from the API response are removed.
|
||||
- New Google API models are reported in `.sync/model-sync-report.md` but not created automatically because the API does not provide authoritative modalities, pricing, knowledge cutoff, release date, tool calling, or structured output metadata.
|
||||
|
||||
## xAI Notes
|
||||
|
||||
xAI is implemented in `packages/core/src/sync/providers/xai.ts`.
|
||||
|
||||
- Source endpoints: `https://api.x.ai/v1/language-models`, `https://api.x.ai/v1/image-generation-models`, and `https://api.x.ai/v1/video-generation-models`.
|
||||
- Required auth: `XAI_API_KEY`.
|
||||
- The richer typed endpoints provide model IDs, creation timestamps, modalities, pricing for language models, and prompt/input limits where available.
|
||||
- Existing xAI models are updated from API-authoritative fields while local metadata is preserved for fields the API does not expose, especially output token limits and some feature/capability flags.
|
||||
- New xAI API models are reported in `.sync/model-sync-report.md` but not created automatically because the API does not provide enough authoritative metadata for complete catalog entries.
|
||||
|
||||
## OpenAI Notes
|
||||
|
||||
- OpenAI is implemented in `packages/core/src/sync/providers/openai.ts`.
|
||||
- Source endpoint: `https://api.openai.com/v1/models`.
|
||||
- Required auth: `OPENAI_API_KEY` from an automation account with access to the full first-party catalog.
|
||||
- The endpoint is used only to monitor catalog availability. Existing TOMLs are preserved byte-for-byte, including models absent from the response, because model access can be scoped to the API project.
|
||||
- Fine-tuned and customer-owned models are excluded. Unknown first-party models are reported for manual review without changing the catalog.
|
||||
|
||||
## OVHcloud Notes
|
||||
|
||||
OVHcloud AI Endpoints is implemented in `packages/core/src/sync/providers/ovhcloud.ts`.
|
||||
|
||||
- Source endpoint: `https://catalog.endpoints.ai.ovh.net/rest/v2/openrouter`.
|
||||
- No auth required: the catalog is public.
|
||||
- Model IDs are lowercased from the catalog `id` to match the existing TOML paths under `providers/ovhcloud/models`.
|
||||
- API prices are per-token strings and are converted to per-1M-token numbers; free models (price `0`) get no `[cost]` section.
|
||||
- `reasoning`, `tool_call`, and `structured_output` come from `supported_features`; `temperature` comes from `supported_sampling_parameters`.
|
||||
- Authored `reasoning_options` are preserved for reasoning models. `Qwen3-32B` supports toggling reasoning through OVHcloud's documented `/no_think` prompt control. Both gpt-oss models support `low`, `medium`, and `high` reasoning effort. The Qwen3.5 models support `none`, `low`, `medium`, and `high`; Qwen3.6-27B additionally supports `minimal`.
|
||||
- `attachment` is derived from non-text `input_modalities`, and `open_weights` from the presence of `hugging_face_id`.
|
||||
- `release_date`/`last_updated` default to the catalog `created` timestamp but preserve any existing hand-authored dates; `knowledge`, `family`, `status`, `interleaved`, and `limit.input` are preserved when present.
|
||||
|
||||
## DigitalOcean Notes
|
||||
|
||||
- DigitalOcean is implemented in `packages/core/src/sync/providers/digitalocean.ts`.
|
||||
- Source endpoints: `https://api.digitalocean.com/v2/gen-ai/models` for lifecycle and reasoning metadata, and the public `https://api.digitalocean.com/v2/gen-ai/models/catalog` for availability, modalities, limits, and pricing.
|
||||
- Required auth: `DIGITALOCEAN_API_TOKEN` or `DIGITALOCEAN_ACCESS_TOKEN` for the control-plane model endpoint; the model catalog is public.
|
||||
- The sync manages serverless text-output models. Other model types, dedicated-only models, and local models absent from the API are retained for manual lifecycle review.
|
||||
- Catalog pricing updates standard, cache-read, cache-write, and extended-context rates while preserving authored reasoning and audio prices.
|
||||
|
||||
## Vercel Status
|
||||
|
||||
Vercel is intentionally not wired into `bun models:sync` right now. Keep using the existing `vercel:generate` script until Vercel sync behavior is redesigned and reviewed separately.
|
||||
|
||||
Do not add Vercel model changes to OpenRouter sync PRs.
|
||||
|
||||
## Chutes Notes
|
||||
|
||||
Chutes is implemented in `packages/core/src/sync/providers/chutes.ts`.
|
||||
|
||||
- Run it with `bun models:sync chutes` or `bun chutes:sync`.
|
||||
- Source endpoint: `https://llm.chutes.ai/v1/models`; no auth required (the model list is public).
|
||||
- Model IDs map directly to TOML paths under `providers/chutes/models`.
|
||||
- `reasoning`, `tool_call`, and `structured_output` come from `supported_features`; `temperature` comes from `supported_sampling_parameters`.
|
||||
- `reasoning_options` is always an empty array: the API advertises a `reasoning` capability but exposes no toggle or effort parameter, so there is no provider evidence for a reasoning option.
|
||||
- TEE model IDs emit `base_model` references to matching `models/` metadata; checkpoints without a canonical entry (e.g. `Qwen3-235B-A22B-Thinking-2507`, `DeepSeek-V3.2`) are written inline.
|
||||
- `attachment` is derived from non-text `input_modalities`, and all models are `open_weights`.
|
||||
- `release_date`/`last_updated` default to the API `created` timestamp but preserve existing hand-authored dates; `knowledge`, `family`, `status`, `interleaved`, and `limit.input` are preserved when present.
|
||||
|
||||
## Venice Notes
|
||||
|
||||
Venice is implemented in `packages/core/src/sync/providers/venice.ts`.
|
||||
|
||||
- Run it with `bun models:sync venice` or `bun venice:sync`.
|
||||
- `VENICE_API_KEY` is optional locally and includes models visible to that account when set.
|
||||
- Models missing from the API response are removed from the Venice catalog.
|
||||
- Every Venice model uses `base_model`; flattened IDs are matched to provider-agnostic metadata before provider-specific overrides are written.
|
||||
- Every Venice model declares `reasoning_options`; models without API-provided effort levels use an empty array.
|
||||
|
||||
## Standalone Generators
|
||||
|
||||
Some provider scripts in `packages/core/script/generate-*.ts` are not wired into `bun models:sync`. When updating those scripts, preserve existing `base_model` and `base_model_omit` fields for generated TOMLs that already use model metadata inheritance. New inheritance-aware output should use `base_model`; do not reintroduce legacy `[extends]` syntax.
|
||||
Reference in New Issue
Block a user