# Model catalogs Each `*.yaml` file in this directory declares one provider's model catalog. The registry loads every YAML at boot and joins it to the matching provider plugin under `application/llm/providers/`. To add or edit models, you almost always only touch a YAML here — no Python code required. ## Add a model to an existing provider Open the provider's YAML (e.g. `anthropic.yaml`) and append two lines under `models:`: ```yaml models: - id: claude-3-7-sonnet display_name: Claude 3.7 Sonnet ``` Capabilities default to the provider's `defaults:` block. Override per-model only when needed: ```yaml - id: claude-3-7-sonnet display_name: Claude 3.7 Sonnet context_window: 500000 ``` Restart the app. The new model appears in `/api/models`. > The model `id` is what gets stored in agent / workflow records. Once > users start picking the model, **don't rename it** — agent and > workflow rows reference it as a free-form string and silently fall > back to the system default if the id disappears. ## Add an OpenAI-compatible provider (zero Python) Drop a YAML in this directory (or in your `MODELS_CONFIG_DIR`) that uses the `openai_compatible` plugin. Set the env var named in `api_key_env` and you're done — no Python, no settings.py edit, no LLMCreator change: ```yaml # mistral.yaml provider: openai_compatible display_provider: mistral # shown in /api/models response api_key_env: MISTRAL_API_KEY # env var the plugin reads at boot base_url: https://api.mistral.ai/v1 defaults: supports_tools: true context_window: 128000 models: - id: mistral-large-latest display_name: Mistral Large - id: mistral-small-latest display_name: Mistral Small ``` `MISTRAL_API_KEY=sk-... ; restart` — Mistral models appear in `/api/models` with `provider: "mistral"`. They route through the OpenAI wire format (it's `OpenAILLM` under the hood) but with Mistral's endpoint and key. Multiple `openai_compatible` YAMLs coexist: each file is one logical endpoint with its own `api_key_env` and `base_url`. Drop in `together.yaml`, `fireworks.yaml`, etc. side by side. If an env var isn't set, that catalog is silently skipped at boot (logged at INFO) — no error. Working example: `examples/mistral.yaml.example`. Files inside `examples/` aren't loaded by the registry; the glob only picks up `*.yaml` at the top level. ## Add a provider with its own SDK For a provider that doesn't speak OpenAI's wire format, add one Python file to `application/llm/providers/.py`: ```python from application.llm.providers.base import Provider from application.llm.my_provider import MyLLM class MyProvider(Provider): name = "my_provider" llm_class = MyLLM def get_api_key(self, settings): return settings.MY_PROVIDER_API_KEY ``` Register it in `application/llm/providers/__init__.py` (one line in `ALL_PROVIDERS`), add `MY_PROVIDER_API_KEY` to `settings.py`, and create `my_provider.yaml` here with the model catalog. ## Schema reference ```yaml provider: # matches the Provider plugin's `name` # openai_compatible only — required for that provider, ignored for others display_provider: # label shown in /api/models response api_key_env: # name of the env var carrying the key base_url: # endpoint URL defaults: # optional, applied to every model below supports_tools: bool # default false supports_structured_output: bool # default false supports_streaming: bool # default true attachments: [, ...] # default [] context_window: int # default 128000 input_cost_per_token: float # default null output_cost_per_token: float # default null reasoning_effort: # default null; none|minimal|low|medium|high|xhigh (subset is model-dependent) models: # required - id: # unique registry key; persisted in agent records display_name: # default: id description: # default: "" enabled: bool # default true; false hides from /api/models base_url: # optional custom endpoint for this model upstream_model_id: # default: id; the name actually sent to the provider # All `defaults:` fields above can be overridden here per-model. ``` ### Reasoning effort, and one model at multiple efforts `reasoning_effort` is forwarded to the provider for OpenAI reasoning models. Accepted values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`, but the subset each model accepts varies (older o-series take only `low`/`medium`/`high`; GPT-5.5 adds `xhigh`) — check the model page. Set it per-model; sending it to a non-reasoning model is rejected by the API: ```yaml - id: gpt-5.4-mini display_name: GPT-5.4 Mini reasoning_effort: medium ``` To expose the *same* upstream model at two efforts, give each entry a distinct `id` and point both at one `upstream_model_id`. The `id` is the unique registry key (and what's stored in agent records); the `upstream_model_id` is the name actually sent to the provider, defaulting to `id` when omitted: ```yaml - id: gpt-5.4-mini-low display_name: GPT-5.4 Mini (Low Reasoning) upstream_model_id: gpt-5.4-mini reasoning_effort: low - id: gpt-5.4-mini-high display_name: GPT-5.4 Mini (High Reasoning) upstream_model_id: gpt-5.4-mini reasoning_effort: high ``` Both call `gpt-5.4-mini` on the wire; token usage is attributed to the distinct `id`s, so cost dashboards split by reasoning level. ### Attachment aliases The `attachments:` list can mix human-readable aliases with raw MIME types. Aliases are defined in `_defaults.yaml`: | Alias | Expands to | |---|---| | `image` | `image/png`, `image/jpeg`, `image/jpg`, `image/webp`, `image/gif` | | `pdf` | `application/pdf` | | `audio` | `audio/mpeg`, `audio/wav`, `audio/ogg` | Use raw MIME types when you need surgical control: ```yaml attachments: [image/png, image/webp] # only these two ``` ## Operator-supplied YAMLs (`MODELS_CONFIG_DIR`) Set the `MODELS_CONFIG_DIR` env var (or `.env` entry) to a directory path. Every `*.yaml` in that directory is loaded **after** the built-in catalog under `application/core/models/`. Operators use this to: - Add new `openai_compatible` providers (Mistral, Together, Fireworks, Ollama, ...) without forking the repo. - Extend an existing provider's catalog with extra models — append models under `provider: anthropic` and they show up alongside the built-ins. - Override a built-in model's capabilities — declare the same `id` with different fields (e.g. a higher `context_window`). Later wins; the override is logged as a `WARNING` so you can audit it. Things you cannot do via `MODELS_CONFIG_DIR`: - Add a brand-new non-OpenAI provider — that needs a Python plugin under `application/llm/providers/` (see "Add a provider with its own SDK" above). Operator YAMLs may only target a `provider:` value that already has a registered plugin. ### Example: Docker Mount your model YAMLs into the container and point the env var at the mount path: ```yaml # docker-compose.yml services: app: image: arc53/docsgpt environment: MODELS_CONFIG_DIR: /etc/docsgpt/models MISTRAL_API_KEY: ${MISTRAL_API_KEY} volumes: - ./my-models:/etc/docsgpt/models:ro ``` Then `./my-models/mistral.yaml` (the file from `examples/mistral.yaml.example`) gets picked up at boot. ### Example: Kubernetes Mount a `ConfigMap` containing your YAMLs at a known path and set `MODELS_CONFIG_DIR` on the deployment. The same `examples/mistral.yaml.example` becomes a key in the ConfigMap. ### Misconfiguration If `MODELS_CONFIG_DIR` is set but the path doesn't exist (or isn't a directory), the app logs a `WARNING` at boot and continues with just the built-in catalog. The app does *not* fail to start — operators can ship config drift without taking down the service — but the warning is loud enough to surface in any reasonable log aggregator. ## Validation YAMLs are parsed with Pydantic at boot. The app fails to start with a clear error message if: - a top-level key is unknown - a model is missing `id` - an attachment alias isn't defined - the `provider:` value isn't registered as a plugin This is intentional — silent fallbacks would mean users don't notice their model picks broke until they hit the API. ## Reserved fields (not yet implemented) - `aliases:` on a model — old IDs that resolve to this model. Reserved for future renames; the schema accepts the field but it is not yet acted on.