arc53--docsgpt/docs/content/Models/cloud-providers.mdx

---
title: Connecting DocsGPT to Cloud LLM Providers
description: Connect DocsGPT to various Cloud Large Language Model (LLM) providers to power your document Q&A.
---

# Connecting DocsGPT to Cloud LLM Providers

DocsGPT is designed to seamlessly integrate with a variety of Cloud Large Language Model (LLM) providers, giving you access to state-of-the-art AI models for document question answering.

## Configuration via `.env` file

The primary method for configuring your LLM provider in DocsGPT is through the `.env` file. For a comprehensive understanding of all available settings, please refer to the detailed [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings).

To connect to a cloud LLM provider, you will typically need to configure the following basic settings in your `.env` file:

*   **`LLM_PROVIDER`**:  This setting is essential and identifies the specific cloud provider you wish to use (e.g., `openai`, `google`, `anthropic`).
*   **`LLM_NAME`**:  Specifies the exact model you want to utilize from your chosen provider (e.g., `gpt-5.1`, `gemini-3.5-flash`, `claude-3-5-sonnet-20241022`).  Refer to your provider's documentation for a list of available models.
*   **`API_KEY`**:  Almost all cloud LLM providers require an API key for authentication. Obtain your API key from your chosen provider's platform and securely store it in your `.env` file.

## Explicitly Supported Cloud Providers

DocsGPT offers direct, streamlined support for the following cloud LLM providers, making configuration straightforward.  The table below outlines the `LLM_PROVIDER` and example `LLM_NAME` values to use for each provider in your `.env` file.

| Provider                     | `LLM_PROVIDER` | Example `LLM_NAME`          |
| :--------------------------- | :------------- | :-------------------------- |
| DocsGPT Public API           | `docsgpt`      | `None`                      |
| OpenAI                       | `openai`       | `gpt-5.1`                   |
| OpenAI-compatible (BYOM)     | `openai_compatible` | (any; with per-model `base_url`/`api_key`) |
| Google (Vertex AI, Gemini)   | `google`       | `gemini-3.5-flash`          |
| Anthropic (Claude)           | `anthropic`    | `claude-3-5-sonnet-20241022`|
| Groq                         | `groq`         | `llama-3.3-70b-versatile`   |
| OpenRouter                   | `openrouter`   | (See OpenRouter docs)       |
| Novita AI                    | `novita`       | (See Novita docs)           |
| HuggingFace Inference API    | `huggingface`  | `meta-llama/Llama-3.1-8B-Instruct` |
| Prem AI                      | `premai`       | (See Prem AI docs)          |
| AWS SageMaker                | `sagemaker`    | (See SageMaker docs)        |

DocsGPT also ships a **model catalog** (`application/core/models/*.yaml`) that the in-app model picker reads, so common models from these providers — including DeepSeek — appear ready to select once the matching API key is set.

## Connecting to OpenAI-Compatible Cloud APIs

DocsGPT's flexible architecture allows you to connect to any cloud provider that offers an API compatible with the OpenAI API standard. This opens up a vast ecosystem of LLM services.

To connect to an OpenAI-compatible cloud provider, you will still use `LLM_PROVIDER=openai` in your `.env` file.  However, you will also need to specify the API endpoint of your chosen provider using the `OPENAI_BASE_URL` setting.  You will also likely need to provide an `API_KEY` and `LLM_NAME` as required by that provider.

**Example for DeepSeek (OpenAI-Compatible API):**

To connect to DeepSeek, which offers an OpenAI-compatible API, your `.env` file could be configured as follows:

```
LLM_PROVIDER=openai
API_KEY=YOUR_API_KEY # Your DeepSeek API key
LLM_NAME=deepseek-chat # Or your desired DeepSeek model name
OPENAI_BASE_URL=https://api.deepseek.com/v1 # DeepSeek's OpenAI API URL
```

Remember to consult the documentation of your chosen OpenAI-compatible cloud provider for their specific API endpoint, required model names, and authentication methods.

### Dedicated `openai_compatible` provider (bring-your-own-model)

Beyond the global `OPENAI_BASE_URL`, DocsGPT has a first-class `openai_compatible` provider. It lets a model carry its **own** `base_url` and `api_key`, which is how per-user "bring your own model" (BYOM) endpoints work — each model can point at a different OpenAI-compatible server without changing instance-wide settings. Outbound requests use an SSRF-pinned HTTP client for safety.

This is the mechanism behind catalog entries like DeepSeek, which declare their own `base_url` and API key environment variable rather than relying on `OPENAI_BASE_URL`.

## OpenAI Responses API and reasoning

For OpenAI models that support it, DocsGPT can call the newer **Responses API** (`/v1/responses`) instead of Chat Completions. This is selected per model in the catalog via an `api_flavor: responses` capability and enables features like server-side reasoning. Related settings:

- `reasoning_effort` — per-model reasoning effort hint (for example `medium`) declared in the model catalog.
- `OPENAI_RESPONSES_STORE` (default `false`) — when `true`, lets OpenAI persist Responses API state server-side.

See [App Configuration](/Deploying/DocsGPT-Settings) for the full settings reference.

## Adding Support for Other Cloud Providers

If you wish to connect to a cloud provider that is not explicitly listed above or doesn't offer OpenAI API compatibility, you can extend DocsGPT to support it. Within the DocsGPT repository, navigate to the `application/llm` directory. Here, you will find Python files defining the existing LLM integrations.  You can use these files as examples to create a new module for your desired cloud provider.  After creating your new LLM module, you will need to register it within the `llm_creator.py` file. This process involves some coding, but it allows for virtually unlimited extensibility to connect to any cloud-based LLM service with an accessible API.